StarTools

Hi - I've been using StarTools for about a year, and overall I love it.

In that time, I've encountered a few issues (or just, behaviours) and I'm interested to see if anyone has any feedback about them.

For context, I shoot using an unmodified Nikon DSLR under reasonably heavy light pollution.

1. In more recent versions of StarTools, processing is effectively in monochrome up until the point of using the colour module. That's fine for some things, but with a lot of steps the true impact can't really be assessed without colour. So:

(a) Is it bad practice to jump straight to colour, do a rough adjustment and then return to processing using other modules?

(b) Is it possible to add a feature to give you a 'colour preview' throughout processing?

I find this is particularly problematic with stretching. Without colour, the impact of a given stretch is very hard to assess. I often find I stretch, go and do colour tweaking, and then have to restretch with the benefit of colour in the image at which point the colour work is not necessarily consistent with what I wanted to achieve and so I have to go and do colour again.

2. I have literally never had a good result from automatic stretching. I'm not sure if this is a factor of the level of light pollution I deal with, but invariably the image is way, way overexposed. Am I doing something wrong, or is this just a fact of life using Bortle 6/7 data? I find I get vastly better results using manual stretching 100% of the time. I have tried using ROI etc with no good results.

3. Is there any way to set a blackpoint or to do an 'inverse stretch' where you modify black levels without messing with the light end of your data? This is one key feature that StarTools seems to lack. I find at the end of working in ST I need to jump into Gimp and use the levels/curves tools to push down the very bottom end (like, last fraction of a percent or so) of dark parts of the image closer to true black, yielding much more pleasing results and much less noticeable noise.

4. Is there any way to get noise reduction to work harder on darker areas and do less processing on light areas? As above, this is typically the noisiest part of the image for me, especially stretching the hell out of light polluted data, but when I crank up noise reduction it tends to obliterate reasonably (perceptually) non-noisy parts of the image too much.

5. I understand the point of binning, but I have found that working at full resolution yields consistently better results. Things like wipe, sharpen, deconvolution and final NR just seem to work better without binning. Is this expected behaviour?

Indeed, it seems to me that many features of ST operate without reference to the resolution of the image, so the effects of a given module seem to change quite a lot depending on whether you've binned first. For example, anything that works on a 256x256 sample seems to do so irrespective of the total size of the image. Does ST take into account total image size, or average star size, or any factor like that when processing?

6. StarTools consistently seems to underutilise my PC - I get spurts of 100% CPU usage, but on the whole it sits there using far less than that while nothing in particular seems to be happening. I know there's a lot going on under the hood but the software really doesn't seem to be pushing my CPU all that hard. I have checked and this is not a case of storage access slowing it down, either (because storage read/write is also not high). Is there something I need to fiddle with to improve CPU usage?

7. Is there a good tutorial somewhere about how to separately process stars? I am finding that to get good nebulosity under my light conditions stretching invariably results in somewhat blown out stars. See e.g. here: https://www.astrobin.com/7slyfr/
I think this would look much better with smaller stars.

None of that is a complaint - just me wanting to understand more about how to use this excellent software. I am no pro, but I think it's incredible that using ST someone like me with no prior astrophotography background can pull that image out of just over an hour's data under light polluted skies. Now I just want to get better.

Hi,
1) Indeed there is a Color Preview in the Wipe Module. I wish there would also be one in AutoDev - maybe a feature request for Ivo?
2) This overexposing is intentional. To get familiar with the concept, check USER NOTES by guy or the Tutorial section of the inofficial manual on the download section. Long story short, AutoDev is used for 2 different tasks: Diagnostic Stretch (default) and Final Stretch. The diagnostic will crank up everything to make any potential flaw of your image visible. To do the Final Strech (after Wipe), just define an ROI to tell ST what dynamic range you are interested in. This will give you a stretch that may be further refined in a few very easy steps.
3) You can reach this goal differently in StarTools: Use the Shadow Dynamic range Parameter in Contrast module. It will squeeze the bottom end into fewer dynamic range, leaving more for your signal.
4) Denoise works harder on areas with low SNR by design. If it eats too much detail you may control this by
- reducing the grain size in the first screen - do not use values higher than absolutely needed!
- reducing the larger scales in second screen.
- Another trick is to reduce Brightness Detail loss, while keeping Color Detail loss.
- to mitigate the 'too clean' look, you may add a slight dose of 'Grain Equalization' which creates a film-like look by adding grain
5) This is depending on the pixel resolution (Arcsec/pixel) of your telescope/cam combo. If the seeing limit has more arcsec than one pixel, binning makes sense. Else you'll get more out of full resolution.
6) Ivo has answered this in regards to GPU usage on the FAQ (see homepage) - probably some similar applies to GPU usage.
7) YES! There is - Again feel encouraged to check User Notes / Inofficial manual for getting more tips. This has quick and easy step-by step tutorials for basic stuff up to special techniques. Meanwhile that manual is available in English, German and Spanish.
Clear Skies,
jochen

Thank you for your post and @hixx for answering it already.
Some further addendums/clarifications;

caitsith01 wrote: ↑Tue Apr 06, 2021 6:31 am 1. In more recent versions of StarTools, processing is effectively in monochrome up until the point of using the colour module. That's fine for some things, but with a lot of steps the true impact can't really be assessed without colour. So:

(a) Is it bad practice to jump straight to colour, do a rough adjustment and then return to processing using other modules?

(b) Is it possible to add a feature to give you a 'colour preview' throughout processing?

Until you hit the Color module, your processing does not impact color at all, except in the Wipe module.
Nothing you do to luminance impacts your coloring. The impact of a stretch on color is exactly nil. The impact of deconvolution, sharpening, Contrast, HDR, etc. likewise nil. Your actual color information is not stretched or manipulated at any point. There are no previews, because there is simply nothing to preview.

On a well calibrated screen, you will see this evidenced in the Color module; as long as you use a CIELAB Luminance Retention mode for LRGB Method Emulation, nothing you do to color should perceptually(!) impact luminance.

This is the whole premise behind color constancy; a stretch or detail enhancement procedure should - wherever possible - not impact the hue or saturation. O-III should remain perceptually the same teal green in a dimly lit area and a brightly lit area. The premise is that areas in outer space don't magically change color because an earthling chose a different exposure time, stretched his/her image this-or-that way, or enhanced detail in a particular way.

2. I have literally never had a good result from automatic stretching. I'm not sure if this is a factor of the level of light pollution I deal with, but invariably the image is way, way overexposed. Am I doing something wrong, or is this just a fact of life using Bortle 6/7 data? I find I get vastly better results using manual stretching 100% of the time. I have tried using ROI etc with no good results.

It should really be the other way around; AutoDev should be yielding the better result 100% of the time, so something is going wrong.
The documentation for AutoDev was updated a few months ago, with a more in-depth explanation of how it works, what its goals are and how it should be operated under different scenarios.

If that doesn't help thing move along, then there may be something strange going on with your dataset(s), so having a look at one of those would help. It may also help seeing what you deem to be a 100% vastly better result for such a dataset so we have a baseline.

Cheers!

Which Nikon are you using?
There is a known problem with some and there is a detailed post about this on cloudynights. I think the posts was particularly relevant to the Nikon 5300D

Ah yes your using a D5300 same as me here is probably the issue please read this thread and download marks ring fix and Adobe dng converter https://www.cloudynights.com/topic/7461 ... ric-rings/
If you are unit a d5300 best too dither between frames , using the required settings for the stacker you use, see Startools notes on st website, use bias as darks , but also make sure to calibrate flats with bias , but essentially it’s the lossy compression on the Nikon that may be causing colour issues .

Regards Dave

Burly wrote: ↑Thu Apr 08, 2021 5:39 am Ah yes your using a D5300 same as me here is probably the issue please read this thread and download marks ring fix and Adobe dng converter https://www.cloudynights.com/topic/7461 ... ric-rings/
If you are unit a d5300 best too dither between frames , using the required settings for the stacker you use, see Startools notes on st website, use bias as darks , but also make sure to calibrate flats with bias , but essentially it’s the lossy compression on the Nikon that may be causing colour issues .

Regards Dave

Hi - thanks for the tip, I had actually come across that discussion before. I have personally never noticed this problem with my gear, and I note that a number of people discussing it have also used a D5300 with no apparent problem.

hixx wrote: ↑Tue Apr 06, 2021 4:04 pm Hi,
1) Indeed there is a Color Preview in the Wipe Module. I wish there would also be one in AutoDev - maybe a feature request for Ivo?

The colour preview in wipe isn't (I assume) intended to give any real sense of how the image will actually look, though? As I understand it, it's there to absolutely max out your colour data so you can see what's going on before and after a wipe. I'm referring more to the fact that for me, processing without colour is unintuitive, I suppose. As you say, maybe a feature to be added one day...

hixx wrote: ↑Tue Apr 06, 2021 4:04 pm 2) This overexposing is intentional. To get familiar with the concept, check USER NOTES by guy or the Tutorial section of the inofficial manual on the download section. Long story short, AutoDev is used for 2 different tasks: Diagnostic Stretch (default) and Final Stretch. The diagnostic will crank up everything to make any potential flaw of your image visible. To do the Final Strech (after Wipe), just define an ROI to tell ST what dynamic range you are interested in. This will give you a stretch that may be further refined in a few very easy steps.

I understand the purpose of the 'diagnostic' stretch, but I have never, ever been able to achieve even an acceptable final stretch using AutoDev. I suspect the problem is that I'm typically dealing with quite 'fragile' data (because I'm under horrible urban skies) and autodev just isn't subtle enough to deal with the resulting noise levels satisfactorily.

hixx wrote: ↑Tue Apr 06, 2021 4:04 pm 3) You can reach this goal differently in StarTools: Use the Shadow Dynamic range Parameter in Contrast module. It will squeeze the bottom end into fewer dynamic range, leaving more for your signal.

Interesting. I find this quite counter-intuitive, as to me (and every image editing software program I've ever used) this fits more naturally into what Startools calls 'develop', i.e., tweaking gamma/exposure levels. But perhaps I'm making assumptions about what the contrast module represents - I'll play around with the setting you mention.

hixx wrote: ↑Tue Apr 06, 2021 4:04 pm 4) Denoise works harder on areas with low SNR by design. If it eats too much detail you may control this by
- reducing the grain size in the first screen - do not use values higher than absolutely needed!
- reducing the larger scales in second screen.
- Another trick is to reduce Brightness Detail loss, while keeping Color Detail loss.
- to mitigate the 'too clean' look, you may add a slight dose of 'Grain Equalization' which creates a film-like look by adding grain
5) This is depending on the pixel resolution (Arcsec/pixel) of your telescope/cam combo. If the seeing limit has more arcsec than one pixel, binning makes sense. Else you'll get more out of full resolution.

On both of these, I think as above that one of my problems is probably that I'm dealing with very light polluted skies and some of the ways these modules work may not fully accomodate such data. I'll try those suggestions, though.

Thanks for your detailed replies. Sorry for the delay in my reply - life gets in the way of this hobby...

admin wrote: ↑Wed Apr 07, 2021 1:28 am Thank you for your post and @hixx for answering it already.
Some further addendums/clarifications;

caitsith01 wrote: ↑Tue Apr 06, 2021 6:31 am 1. In more recent versions of StarTools, processing is effectively in monochrome up until the point of using the colour module. That's fine for some things, but with a lot of steps the true impact can't really be assessed without colour. So:

(a) Is it bad practice to jump straight to colour, do a rough adjustment and then return to processing using other modules?

(b) Is it possible to add a feature to give you a 'colour preview' throughout processing?

Until you hit the Color module, your processing does not impact color at all, except in the Wipe module.
Nothing you do to luminance impacts your coloring. The impact of a stretch on color is exactly nil. The impact of deconvolution, sharpening, Contrast, HDR, etc. likewise nil. Your actual color information is not stretched or manipulated at any point. There are no previews, because there is simply nothing to preview.

On a well calibrated screen, you will see this evidenced in the Color module; as long as you use a CIELAB Luminance Retention mode for LRGB Method Emulation, nothing you do to color should perceptually(!) impact luminance.

This is the whole premise behind color constancy; a stretch or detail enhancement procedure should - wherever possible - not impact the hue or saturation. O-III should remain perceptually the same teal green in a dimly lit area and a brightly lit area. The premise is that areas in outer space don't magically change color because an earthling chose a different exposure time, stretched his/her image this-or-that way, or enhanced detail in a particular way.

As with my reply above, I think the missing factor here is that for my conditions the biggest battle is typically noise in darker areas due to significant light pollution. I understand your point that colour does not affect luminance. But that doesn't mean colour is irrelevant when adjusting luminance for this reason: colour noise will not be apparent until colour info is added (obviously), but luminance adjustments might be necessary to minimise the impact of colour noise. So for example, I typically work to drive genuinely empty space down as close to 0 luminance as I can to eliminate as much dark area colour noise as I can without losing detail from lighter areas. Usually I'm fighting residual red/brown noise in dark areas arising from light pollution that wipe doesn't fully eliminate. But because I can't see colour during the initial run through the other modules, I can't see the impact of my other choices on this colour noise without then effectively having to do another run through after I've used the colour module.

I hope that makes sense - I may be using some terminology wrong. But basically I have to use the colour settings before I can use other ('earlier') modules to crush as much low luminance noise as I can acceptably. So in my view it would be extremely helpful to allow users to have some sort of 'colour preview' in earlier modules, even if that's just presenting a very rought/vanilla colour application.

As per my original post, I'm curious as to whether it actually matters if I jump straight down to the colour module first and then work back through the other modules?

admin wrote: ↑Wed Apr 07, 2021 1:28 am

2. I have literally never had a good result from automatic stretching. I'm not sure if this is a factor of the level of light pollution I deal with, but invariably the image is way, way overexposed. Am I doing something wrong, or is this just a fact of life using Bortle 6/7 data? I find I get vastly better results using manual stretching 100% of the time. I have tried using ROI etc with no good results.

It should really be the other way around; AutoDev should be yielding the better result 100% of the time, so something is going wrong.
The documentation for AutoDev was updated a few months ago, with a more in-depth explanation of how it works, what its goals are and how it should be operated under different scenarios.

If that doesn't help thing move along, then there may be something strange going on with your dataset(s), so having a look at one of those would help. It may also help seeing what you deem to be a 100% vastly better result for such a dataset so we have a baseline.

Cheers!

I'll have another play with AutoDev based on the manual. If I was going to upload a dataset, what's the best way to go about doing that?

Thanks for your responses so far!

Usually I'm fighting residual red/brown noise in dark areas arising from light pollution that wipe doesn't fully eliminate.

Me too! Just for clarification: That noise is normal and expected in such scenario. The excessive color noise described here is actually the Shot noise of the light pollution. Wipe (as other gradient removal tools of other software) is designed to remove the light pollution gradient, but NOT to remove its Shot Noise. Therefore after the second stretch so much noise is being revealed. That should be handled by different modules at a later point. If you have severe light pollution in your images (like I do have) the , here's a list of measures:
#1) more integration time (more/longer Lightframes)
#2) even more integration time
#3) Light Pollution Filter for L will not only reduce unwanted light but also its shot noise amount. For RGB shots use Optolong extreme, Idas NB-1 or NB-X etc. or consider going narrowband entirely.

Here's some hints what can be done using ST modules:

1) See the AutoDev steps already mentioned in this thread above, to darken background as much as possible (Ignore Fine Detail, ROI etc)

2) Flux Module's Noise preset is awesome - use before SuperStructure (after Color). You need to tweak some of the parameters but can get rid of a lot of noise without eating detail:
- Launch the 'Flux' Module
- Select the 'Noise' preset
- Increase or decrease the 'Positive Flux' parameter. This will process the bright area. Stars get a bit rounded by decreasing this.
- Increase or decrease the 'Negative Flux' parameter. This will process the dark area. Stars get compressed and a bit rounded by increasing this.
- 'Filter Amount' will regulate the intensity of the filter, lower it until the brightness does not make the star core too fat.
- 'Filter Radius' determines the radius up to which the filter will act. The larger, the larger the affected radius of the star.
- 'Filter Fuzz' eliminates noise according to the size of the pixel that we put.
The user elpajare did a thread in this forum on this method with examples

3) SuperStructure module's Isolate and Dim presets help to push back stars and noise a lot without impacting larger objects. This is absolutely brilliant! Use as last module prior switching off tracking and performing Denoise
Hope this helps
Clear skies,
jochen