StarTools Crash doing AutoDev

[DesertSky]

This file will crash StartTools when doing AutoDev or Develop: https://www.dropbox.com/s/44b4xttt7wqu742/M33.fit?dl=0
Load as linear with debayer.

[admin]

Hi,

Thank you for reporting this as well as uploading the FTS file.
I'm having trouble replicating this issue.
Can you tell me what operating system (Windows, MacOSX, Linux distro, 32-bit or 64-bit) you're running and which version of StarTools is exhibiting this behaviour?
Can you tell me how much space you have available on your hard drive and whether StarTools is able to create the .trk files ok (in the StarTools folder where the executable is located on Windows, or in the /tmp folder on MacOSX/Linux)?

Thank you!

[DesertSky]

I have StarTools in "c:\Program Files" which is a protected directory (only admin can write there) so the .trk file can not be created there. I moved StarTools to another location and confirmed that this is the problem. For Windows, the recommended location to store files like this is:
c:\users\<UserName>\Appdata\Local\StarTools where <UserName> is the current user name

"Program Files" is the standard location to store 64 bit exe's on windows so this is a genuine issue.

[DesertSky]

An additional thought, StarTools needs better error handling if this file can't be created. That is, an appropriate error message rather than crashing.

[admin]

An additional thought, StarTools needs better error handling if this file can't be created. That is, an appropriate error message rather than crashing.

Agreed.

I have StarTools in "c:\Program Files" which is a protected directory (only admin can write there) so the .trk file can not be created there. I moved StarTools to another location and confirmed that this is the problem. For Windows, the recommended location to store files like this is:
c:\users\<UserName>\Appdata\Local\StarTools where <UserName> is the current user name

The reason why this crashing behaviour has up until now been regarded as a "fringe case" is that other files (such as the resources and license files) have the same read/write requirement (they will throw errors). That doesn't excuse the crashing behavior obviously!

"Program Files" is the standard location to store 64 bit exe's on windows so this is a genuine issue.

StarTools is self-contained and distributed like this for 2 main reasons.

1. portability and platform independence; you should be able to put your full StarTools folder on another machine (regardless of operating system, Windows, Mac, Linux, 32-bit or 64-bit) and it should just magically work. Moreover, the above mentioned Windows Users folder is unfortunately only applicable to newer versions of Windows (e.g. Win 7 and up).
2. RAM disk acceleration; by putting the folder on a RAM disk, StarTools can be made to operate a great deal faster (Linux and MacOS systems can mount a RAM disk for the /tmp folder, but WIndows is trickier).

Thanks again for reporting this!

[DesertSky]

Anything before Windows 7 is unsupported so the issue is not going away. Interesting thought about a single directory, however, consider having two directories - a static (read only) directory and a data directory. It does mean the user needs to understand where each directory is located on each system. At the very least, you need to make users aware of this issue.

I should start another thread for this, however, try to process the supplied file in StarTools. Using AutoDev and Wipe, I could not get any good results. I had better results using Develop, however, the contrast was poor and I was unable to fix it. After that, the other tools seemed to work well.

[admin]

Anything before Windows 7 is unsupported so the issue is not going away.

StarTools officially supports Windows Vista and XP (and ST will likely run on even earlier versions). If you mean that anything before Vista is no longer supported by Microsoft, then know that there are a great number of (mostly Internet disconnected) XP machines still in service in many observatories for legacy reasons. This is also the reason why 32-bit a version is still available.

Interesting thought about a single directory, however, consider having two directories - a static (read only) directory and a data directory. It does mean the user needs to understand where each directory is located on each system. At the very least, you need to make users aware of this issue.

It is my contention that if a user can successfully get StarTools to work in the Program Files folder, then that user has already modified and/or granted read/write permissions to the application. But you may of course disagree.

I should start another thread for this, however, try to process the supplied file in StarTools. Using AutoDev and Wipe, I could not get any good results. I had better results using Develop, however, the contrast was poor and I was unable to fix it. After that, the other tools seemed to work well.

No problem!

One thing I noticed when opening it, is that there seems to be some sort of artificial pattern in the data (StarTools' scaling code intentionally causes aliasing at every scale, so any repeating patterns cause patterns at every scale).
Did you happen to debayer with a debayering algorithm other than Bilinear interpolation or VNG? It appears that something has been creating artefacts from the background noise (possibly a debayering algorithm from the AHD family).
Such artefacts in turn make it harder for noise reduction/detection routines to do their job (virtually all modules make use of some sort of noise analysis or mitigation scheme), as they no longer appear to be random noise.

As for your original question, Wipe and AutoDev should definitely give good results with this data. The noise and presence of dead pixels (aka "dark anomalies") do call for increasing the Dark Anomaly Filter somewhat. You may also want to mask M33 out as a precaution so its halo is not accidentally mistaken for a gradient (I was lazy and didn't do this). This is the Wipe module after loading the data (and cropping it for better framing) with Temporary AutoDev set to "Yes" (so we can see what we're doing);

scr2.jpg (287.83 KiB) Viewed 10827 times

Then launch AutoDev and click & drag a Region of Interest over M33 (the ROI being the area that AutoDev should optimize the dynamic range allocation for);

scr3.jpg (500.21 KiB) Viewed 10827 times

Tweak the "Ignore Fine Detail" (to make AutoDev ignore noise in low SNR datasets such as these) and "Outside ROI influence" (to allocate some dynamic range to the area outside the ROI) as you see fit.

From there you can indeed go nuts (or not ).

This was a simple HDR optimisation in 1.4.324 (Reveal DSO core, with very high detail size range) + trivial Color balancing.;

M33.jpg (377.72 KiB) Viewed 10827 times

Does this help?

[DesertSky]

I need to study what you supplied, however, to answer your question the debayer is VNG. I've only seen this pattern with Startools and not other programs. When I tried Develop with is image there is no pattern so there is something about this image that is causing a problem for AutoDev.

[admin]

I need to study what you supplied, however, to answer your question the debayer is VNG. I've only seen this pattern with Startools and not other programs. When I tried Develop with is image there is no pattern so there is something about this image that is causing a problem for AutoDev.

Interesting... AFAIK a straight VNG algorithm should not be causing these mazing artefacts like these. Could you tell me what stacking software you're using?

The pattern is not causing a problem for AutoDev - it merely highlights that this is what it found in your data (that's why, combined with the intentionally aliasing scaler, it is so useful for analysis). StarTools propensity to highlight these issues has already helped a few people iron out kinks in their processing flow (see also this thread).

Zoom to 100% and you will notice the pattern (visually) disappears. It's still latent in your data however.