Mac 1.9.542alpha: Crash when Sharp Module is loaded

[cytan299]

HI Ivo,
Just tried 1.9.546 alpha and still crashing with my 25 MB file in Sharp module. Crash is when it is "Initializing ...".

I'm not sure whether I should continue reporting or wait for a message from you that I should test a a build on this large file ...

cytan

[admin]

I've audited the code again, but (un)fortunately cannot find see any problems.
As it stands, it appears the problem is an underpowered graphics solution.

As opposed to other operating systems, macOS unfortunately does not provide a means (that I know of) to change the graphics driver watchdog timer. I can change the code to allow more back and forth to "cut up the task" in smaller chunks, but that would mean performance degradation across the board for everyone else.

The best solution right now is to force StarTools to use your CPU instead of your GPU for bigger datasets. You can do so by opening up the 'config' file in a text editor of your choice and changing the following line;

opencl_forcecpu=false

to

opencl_forcecpu=true

Please let me know if that solves the issue.

[cytan299]

I've audited the code again, but (un)fortunately cannot find see any problems.
As it stands, it appears the problem is an underpowered graphics solution.

As opposed to other operating systems, macOS unfortunately does not provide a means (that I know of) to change the graphics driver watchdog timer. I can change the code to allow more back and forth to "cut up the task" in smaller chunks, but that would mean performance degradation across the board for everyone else.

The best solution right now is to force StarTools to use your CPU instead of your GPU for bigger datasets. You can do so by opening up the 'config' file in a text editor of your choice and changing the following line;

opencl_forcecpu=false

to

opencl_forcecpu=true

Please let me know if that solves the issue.

ugghh, it's worse now. With (see attached config)

"opencl_forcecpu=true"

StarTools 1.9.546alpha crashes in the Wipe Module. See StarTools.log and the crash log.

cytan

[admin]

Well this is getting "interesting" Again a crash in the OpenCL code.

A couple of questions;

• Does the crash happen if you bin first?
• I just noticed that you should have an AMD Radeon Pro 5500M 8GB at your disposal. With StarTools configured to use the GPU, it does not appear your system is using that GPU. Are you able to force this GPU on? (see also here)
• Can you confirm for me the 'resources' file has been extracted as well along with the application?
• Is your system otherwise stable with other applications that use lots of memory?

Thank you,

[cytan299]

Well this is getting "interesting" Again a crash in the OpenCL code.

A couple of questions;

• Does the crash happen if you bin first?
• I just noticed that you should have an AMD Radeon Pro 5500M 8GB at your disposal. With StarTools configured to use the GPU, it does not appear your system is using that GPU. Are you able to force this GPU on? (see also here)
• Can you confirm for me the 'resources' file has been extracted as well along with the application?
• Is your system otherwise stable with other applications that use lots of memory?

Thank you,

Binning 50% doesn't stop the crash in the wipe module.
See attached zip file.

The resources file is in the StarTools_alpha directory:

Screenshot 2023-03-05 at 6.17.35 PM.jpg (161.32 KiB) Viewed 16230 times

I did a checksum that you can check that it is the correct resources file.

Note: I don't understand what you mean that the GPU is not used because I thought by setting
opencl_forcecpu=true
ST will not use the GPU. Isn't this the expected behaviour?

I've loaded gSwitch and it says that I am using the GPU:

Screenshot 2023-03-05 at 6.32.27 PM.jpg (47.83 KiB) Viewed 16225 times

cytan

[cytan299]

And forcing the GPU to be used (but with "opencl_forcecpu=true") , ST still crashes in Wipe module+ 50% bin:

Screenshot 2023-03-05 at 6.40.19 PM.jpg (420.05 KiB) Viewed 16224 times

cytan

[admin]

And forcing the GPU to be used (but with "opencl_forcecpu=true") , ST still crashes in Wipe module+ 50% bin:

Screenshot 2023-03-05 at 6.40.19 PM.jpg

cytan

Hi Cytan,

You will want to have gSwitch use the Discrete GPU (not automatically switch).

opencl_forcecpu=true forces StarTools to use the CPU, not GPU

so, setting opencl_forcecpu=false will disable that and allow StarTools to use your GPU again (e.g. the default behavior).

If despite using gSwitch, StarTools keeps defaulting to the internal GPU (the Intel one, whose drivers generated the original crash) and not the discrete GPU (the AMD one, which is much more capable), then can you please run the command 'clinfo' and give me the output?

We can make StarTools use any secondary enumerated GPU if needed.

[cytan299]

Hi Cytan,

You will want to have gSwitch use the Discrete GPU (not automatically switch).

opencl_forcecpu=true forces StarTools to use the CPU, not GPU

so, setting opencl_forcecpu=false will disable that and allow StarTools to use your GPU again (e.g. the default behavior).

If despite using gSwitch, StarTools keeps defaulting to the internal GPU (the Intel one, whose drivers generated the original crash) and not the discrete GPU (the AMD one, which is much more capable), then can you please run the command 'clinfo' and give me the output?

We can make StarTools use any secondary enumerated GPU if needed.

Good news!

I set gSwitch to use the Discrete CPU and set opencl_forcecpu=false and I went through the entire workflow and NO CRASH!!!

Where in the Startools.log can you tell whether the GPU is used or not?

cytan

[cytan299]

And when I didn't do bin 50% which I did previously for these tests, now SVDdecon doesn't find any good stars for me to sample in *full* resolution: in fact what ST found aren't stars at all:

Screenshot 2023-03-05 at 8.19.17 PM.jpg (341.71 KiB) Viewed 16209 times

Attached is the StarTools.log file (log.zip) just in case I messed something up in SVDdecon.

cytan

[admin]

Good news!

I set gSwitch to use the Discrete CPU and set opencl_forcecpu=false and I went through the entire workflow and NO CRASH!!!

Excellent! You should hopefully notice a fairly big speed up as well, as the AMD card is far more capable.

Where in the Startools.log can you tell whether the GPU is used or not?

You cannot tell in the log, however you can see on the splash screen, as well as in the about dialog which resources were detected and are being used (it should show both CPU and GPU).

As for the SVDecon star detection, I cannot be 100% sure, but from the screenshot it appears your dataset is severely oversampled and may also exhibit blotchy pattern noise (both not entirely surprising for the resolution you are processing at). As a result, the apodization mask generator is latching on the blotches, rather than the oversampled stars.

If you could post a small crop of the stretched image's background, so we can try to verify if this is indeed the case.

You really should not be processing at this high resolution with this dataset - it will only work to your disadvantage. Instead, binning would be highly recommended.

If you really don't want to reduce the resolution and you cannot find the cause of the pattern noise, then you could try to clean up pattern noise in the Wipe module. using the Correlation Filtering parameter. It is more useful for zipper-artefact-like pattern noise, but it may work.