HDR module in 1.8.515 Public Alpha is extremely slow

[BrendanC]

Hi Ivo,

I love it when a new module or upgrade is released. However, I am a bit concerned by the new HDR module in the latest alpha release.

I just tested it on my machines - a Lenovo T440P running an i5-4300M CPU @ 2.6GHz. and a custom-built PC primarily for music production, running a slightly faster i5-4590M CPU @ 3.3GHz, both Windows 10 with 8GB RAM, Intel 4600 GPU and 20 compute units. Admittedly neither of them are necessarily high-end, especially for graphics, but they've been fine for StarTools and other graphics processing.

However, on opening the HDR module, for a linear image, around 1,800 x 1,100 pixels in size (binned and cropped), with the standard initial 50 pixels setings for the Context Size, it's taking between 3.5 and 4 minutes to process, before I can do anything. If I go up to 101 pixels, well, it's reached 5 minutes now and the progress indicator hasn't crept beyond the first bar!

So, whereas I could sort of use this within certain bounds, it's inconvenient not to be able to use it to its full capacity, and even within those bounds, so slow that it's disruptive to any workflow. Hence my concern! The previous HDR worked fine.

I notice that with SVDecon I can just use the synthetic, non-sampled approach and it works pretty much as before. So, is there any way HDR could do something similar? If not within the same screen, then perhaps have two options in the sidebar, in which the old HDR is still available but called something like HDR Lite, or HDR Global, and the new one is called HDR Full, or HDR Local?

I completely appreciate my machines aren't the last word in latest graphics tech, but I'd imagine I'm not alone in this. I'd very much rather not have to invest in new hardware in order to run StarTools, especially if there's any way it could be adapted to cater for less-than-high-end users.

I hope this feedback is useful. I think StarTools is great, and you do a fantastic job. This is why I'd like to keep using it!

Happy to provide any more feedback/stats if it helps.

Thanks
Brendan

[admin]

Hi Brendan,

When using a new module, it is always enlightening to peruse the documentation, which amongst other things addresses this. If any concerns or questions remain, do let me know.

[BrendanC]

Thanks Ivo. I did read that. It says the initial processing may take some time, but surely five minutes without moving beyond the first progress bar isn't viable?

[Guy]

Hi,

I've done some performance tests which hopefully will help.
I used STReplay and ran one of the demo sessions in a couple of environments to compare performance of the new HDR module on a version of STReplay (which is still being tested and not yet publicly available).
Now STReplay does not try to be the quickest it can - and in some cases to ensure repeatability it can be slow (e.g. in AutoDev) but it does provide a repeatable processing sequence for comparison purposes.

I ran the demo 'M8 by Kurt M Demo - 64bit' in two environments:
A) An HP ProBook 440 G7 laptop 8G RAM with Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz
B) A Virtual Machine with 6G RAM running under Hyper-V running on PC with Intel(R) Core(TM) i5-7600K CPU @ 3.80GHz

Results with no GPU:
HDR Module Time:
A) 3m12s
B) 12m14s
Which is a factor of nearly 4.

Whilst other modules were also slower in B than in A the worst difference was the Unified De-Noise module which was just over a factor of 2.
Unified De-Noise Module Time:
A) 1m10s
B) 2m28s

So it seems that HDR is much more sensitive to processing environment than other modules.

I ran the tests on A) with the StarTools GPU version and could see no discernable GPU activity even though the processor was at >90%.
I checked other modules and they are using the GPU OK - although the test session used is not particularly demanding.
Ivo, should HDR be using the GPU?

Regards

Guy

[EG1]

Guy,

The new HDR module is very nice and does use a lot of processing power.

Thanks Ivo for improving this…

It runs well on my AMD Ryzen 4750G 8 cores 16 Threads Processor …

Eric

If you are on a slower machine, you can just wait a bit for the magic…

[BrendanC]

I don't doubt it's a brilliant new module, I'd just like to be able to run it!

I'd say that 4 mins is very disruptive when you want to try different settings, and that's at the default value. Go higher, and it's even slower. So, it seems I basically cannot use the HDR module now, because the lower settings might not be suitable for the image.

As I said, surely there's room to accommodate less than high-end systems with a HDR Lite module, in exactly the same way SVDecon can run a less processor-intensive way?

If not then this is pretty much a disaster for me, to be honest. I simply cannot afford to upgrade my hardware. I don't quite understand how a new module can ramp up the processing requirements so very much.

[EG1]

You can select a small preview

[BrendanC]

That does improve things, I agree, but even selecting a fairly small ROI only allows me to go up to around the mid-60 pixels mark before it becomes so slow it's unusable again.

I'm perfectly prepared to put my hands up and say fair enough, my machines aren't up to the task, and think about sticking with an older version of StarTools, or skipping the HDR module, or upgrading my hardware, or moving on from StarTools.

I'm just suggesting solutions that mean people like me can continue to benefit from enhancements.

Having said which, if they're all going to be as processor-intensive as HDR, and that's the roadmap for StarTools, then sooner or later I guess I'll be priced out anyway. In which case, the processor specs page is going to have to change quite radically: https://www.startools.org/downloads/tec ... imum-specs

Out of interest, what would a minimum spec be to run the HDR module at the highest Context setting?

[LuckyEddie]

I have to agree - I'm actually finding this HDR module a backwards step from the previous. I have to wait multiple minutes even on low resolution images (eg. 1k x1k). This is for the default 50 pixels context size, I don't dare vary the context size to see what effect this has.

I know the benefit is meant to be quicker parameter modifications after the context size has finished calculating but this is not my experience. In the previous version, I'd typically wait maybe 20-30 seconds on entry to the module and then parameter changes would take 5-10 seconds. Now I wait multiple minutes and parameter changes take 3-4 seconds. I tried a good, complex data set - the latest IKO processing challenge on SGL - and I had to wait 20 minutes for the default module load calcs.

Sorry Ivo, I have some idea of the amount of hard work that you put into each update but for me this one is a miss - I'm rolling back to 1.8.512.

[BrendanC]

I'm also rolling back.

I agree about the time and care Ivo puts into these changes. It would be wonderful if somehow we could have two flavours of HDR that work across more hardware. I'm not sure I could justify buying a whole new rig just for the HDR module.