StarTools 1.9 preview

General discussion about StarTools.
decay
Posts: 493
Joined: Sat Apr 10, 2021 12:28 pm
Location: Germany, NRW

Re: StarTools 1.9 preview

Post by decay »

admin wrote: Sun May 14, 2023 8:20 am A fair bit of work has gone into SV decon in general.
I tried 550 with my latest dataset and indeed it seems to be a great improvement. A lot of usable starfishies in the whole field, no empty ones and no good candidates left out. Sampling is smooth, easy and well performing. SVD had a good impact not only on the stars, but on clearity and detail overall. De-ringing worked great with defaults, there was no need to adjust anything.

Thanks, Ivo! Best regards, Dietmar.
decay
Posts: 493
Joined: Sat Apr 10, 2021 12:28 pm
Location: Germany, NRW

Re: StarTools 1.9 preview

Post by decay »

P.S.: I just noticed, that there was a file 'maskstf.tiff' created during my latest run. It seems to contain the SVD mask with the starfishies. I've no idea what I did to cause this ... ?! :think:
Mike in Rancho
Posts: 1166
Joined: Sun Jun 20, 2021 10:05 pm
Location: Alta Loma, CA

Re: StarTools 1.9 preview

Post by Mike in Rancho »

I'm definitely excited to see where this ends up going. :thumbsup:

But, I wasn't as lucky as Dietmar. :( Though I too did get a mask tiff in my save folder. Not a big deal, that's where I save my Sharp masks and whatnot also...

So...hmm I haven't figured out how to make bullets here yet, so will just do numbers:

1. First go through I got a mess of a sampling mask. Few samples, a lot of stars turned red, and what white outlines existed were broken up. I tried selecting one or two but the result was no good, either missing the centroid and/or shifting whole objects by many pixels. I restored, restretched, and tried again, at which point SVD just crashed out. I could not later replicate any of this. Maybe some errant click got me there? I took screenshots but if I never see this again I'll just chalk it up to some one-off aberration. Subsequent runs were much more normal.

2. There are some blue pixels within some white outlines. Any meaning?

3. Am I noticing a difference in results depending on how I process the data on the way to SVD, or am I just seeing things?

4. Deringing detect seems to act in the opposite manner from what I would expect, once one gets above 15% or so. Thus, I seemed to get a better deringing appearance when I dropped from the default 50% down to say 20. If I tried up to 70, ringing increased. Turning detect completely off causes kind of a harsh mess, but from 1 to again maybe 15 it seems to dering, before going backwards.

5. Is deringing acting differently on my stars that end up outline sampled versus those that aren't? Maybe those are just bigger brighter stars, but they do ring more. :confusion-shrug:

6. In general with those larger stars, I seem to be getting a bigger diameter central white core than in 536. And in fact they seem less pinpointed than just from the OptiDev and pre-SVD. Granted, this may just be the redistribution of energy inward, and something I need to get used to and/or use increase dynamic range to see what that does. I know you said 536 had some sort of flaw, but I still think it gives a better overall deringed appearance across the board.

So far I've just tested with one widefield galaxy dataset of limited integration (can provide if helpful), but tonight will try to start experimenting and comparing with more and different datasets.
User avatar
admin
Site Admin
Posts: 3382
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: StarTools 1.9 preview

Post by admin »

Thank you all for the feeback! :thumbsup:

The TIFF file was a debugging remnant. I removed it for the latest upload; alpha "lucky" 13. :)
You can now find it in the download section.

I feel I'm finally nearing the end of this development cycle.
Mike in Rancho wrote: Sun May 14, 2023 7:46 pm 1. First go through I got a mess of a sampling mask. Few samples, a lot of stars turned red, and what white outlines existed were broken up. I tried selecting one or two but the result was no good, either missing the centroid and/or shifting whole objects by many pixels. I restored, restretched, and tried again, at which point SVD just crashed out. I could not later replicate any of this. Maybe some errant click got me there? I took screenshots but if I never see this again I'll just chalk it up to some one-off aberration. Subsequent runs were much more normal.
Did you happen to have another version of StarTools running in parallel? It's usually a clash of tracking files.
2. There are some blue pixels within some white outlines. Any meaning?
They do have a meaning indeed - I'm still deciding on this visualization. They show a couple of different quality aspects of the sample. Usually, samples with pure white borders are the best ones.
3. Am I noticing a difference in results depending on how I process the data on the way to SVD, or am I just seeing things?
Aside from Tracking of course having an influence, if you mean in terms of the samples it provides, then - for challenging datasets - it indeed it now takes into account your earlier estimates of noise that you made in other modules. E.g. if you used a high Dark Anomaly filter settings in Wipe or AutoDev, it will take this into account as an extra cue when trying to discern between stars and correlated background noise (because DAF settings target the same type of noise).
4. Deringing detect seems to act in the opposite manner from what I would expect, once one gets above 15% or so. Thus, I seemed to get a better deringing appearance when I dropped from the default 50% down to say 20. If I tried up to 70, ringing increased. Turning detect completely off causes kind of a harsh mess, but from 1 to again maybe 15 it seems to dering, before going backwards.
Indeed - it's definitley not a "strength" parameter any more. In my earlier post, I mentioned the nature of the slider has changed; it is balancing a number of different parameters. For challenging datasets (noisy and/or undersampled), experimentation on what works best may be in order.

In all cases, you should be able to achieve superior results in 550+ vs 536 (particularly in the visibility of darker structural detail), and in all cases touching the Dering Detect parameter on 536 will make darker detail/structures harder to see, while on 550+ the de-ringing will just look "different" but should impact structure much less.
5. Is deringing acting differently on my stars that end up outline sampled versus those that aren't? Maybe those are just bigger brighter stars, but they do ring more. :confusion-shrug:
De-ringing targets stars (and structures) beyond the sample outlines. They are decoupled.

The approach to de-ringing in StarTools two-pronged;
  • Detect and prevent ringing during deconvolution; this is the preferred cause of action (prevention!) and has yielded things like PSF Resampling. It involves actively adjusting the PSF during deconvolution, as well as estimating (and pulling back on) areas that may start to become "compromised".
  • Detect and "repair" ringing after deconvolution; this is the "catch-all" fallback and the traditional way of countering ringing artifacts. It is more about restoring pixels that are deemed "too far gone" to their non-deconvolved state.
Good, active PSF estimation is key to preventing ringing. Gracefully/intelligently dealing with singularities (e.g. overexposed ares) and destabilisation is a close second. Cleaning up any remnants is third, but can still make or break an image.
6. In general with those larger stars, I seem to be getting a bigger diameter central white core than in 536. And in fact they seem less pinpointed than just from the OptiDev and pre-SVD.
I just made a slight tweak in alpha "lucky" 13 that reduces sizes a little.
Granted, this may just be the redistribution of energy inward, and something I need to get used to and/or use increase dynamic range to see what that does. I know you said 536 had some sort of flaw, but I still think it gives a better overall deringed appearance across the board.
:( I have been testing against 536 with various datasets, with various (but identical) samples and settings (including intentionally sub-optimal or very aggressive), and have consistently found quite the opposite; you should see much less ringing across the board, while you should also observe slightly more dark detail being recovered. You may even see some recoverable stars appearing ever so slightly more pin-point.

At worst, when given a challenging dataset, deringing performance should be a wash between the two versions, but 550+ will always recover more dark detail.
ST_1_9_536vs1.jpg
ST_1_9_536vs1.jpg (74.2 KiB) Viewed 22655 times
The 1.9.550+ image (bottom) shows the sort of improvement you should generally be seeing; e.g. the image exhibits less ringing around a number of stars, while at the same time showing more structural detail/clarity in the darker areas in the nebulosity/filaments.

It is important, however, not to confuse pin-point with acuity; with 536 being more prone to ringing and "shadows", the local darkening around some stars would provide increased contrast (exactly like old school Unsharp mask). Acuity is obviously not the goal or purpose of deconvolution.

I'd be very interested to get my hands on a dataset where you feel there is a substantial regression.
So far I've just tested with one widefield galaxy dataset of limited integration (can provide if helpful), but tonight will try to start experimenting and comparing with more and different datasets.
If you'd like to share that dataset, it may indeed prove useful!
Ivo Jager
StarTools creator and astronomy enthusiast
Mike in Rancho
Posts: 1166
Joined: Sun Jun 20, 2021 10:05 pm
Location: Alta Loma, CA

Re: StarTools 1.9 preview

Post by Mike in Rancho »

Great info, will be useful for experimenting and testing. :thumbsup:

551 may be a keeper. Beta time? :bow-yellow:

It's going to take some seat time to get used to the new stuff, and maybe re-work on my stretches whilst keeping an eye on IFD and DAF. I've actually tended to push IFD as a quick way to alter the stretch, typically target brightening, so will likely have to stop that.

I must also confess that while I blink to check detail recovery, I usually just trust that SVD is getting the best it can from the data, and look to the stars. They can make or break (usually break) an image to me even if not the target. And local darkening (is that different from "ringing"?) is what drives me up a wall, especially when in front of nebulosity or the face of a galaxy. Grrr. Hence my obsession with the deringing. :lol:

All seems pretty good here though, so far, though I'm still feeling out the settings. :D Only remnant ringing I think I've seen was around a handful of 4-pixel stars, and that was readily fixed with targeted deringing in Shrink.

Sample selection seems more important than ever, and on the ST M42 Tutorial data (good population of both round and eggy stars), simply removing a dud sample nearby made a bright star shadow go away.

I also like the blue outline segments. I ran some data that really showed off what it seemed to be looking for, from unexpected shapes to being in the midst of nebula.

Detail recovery and clarity seemed top-notch, even at mere defaults and only adding intra-iteration centroid and adjusting deringing. I ran through Eyal's NGC2070 SHO data from that old practice thread, and....wow. :shock:
happy-kat
Posts: 373
Joined: Sun Feb 01, 2015 11:31 am

Re: StarTools 1.9 preview

Post by happy-kat »

1.9.551 GPU win
trying SVD to see if my stars are better liked but when I view sampling the stars are all starfish (except for the comet).
From the startools main menu outside of any module went to mask/clear/keep > back to mask > auto >
auto generated a star mask of starfish
selected 'stars' mask regenerated and was better star shapes
selected 'fat stars' mask improved
selected 'stars' switching between fat stars and stars some stars get bigger mask area but then if select fat stars the mask correct, fat star mask bigger mask area appears to only be on 'stars' rather then 'fat stars'
every time select 'all stars' get the starfish stars as seen in SVD
Assume it is noise being selected in the starfish mask but I have no user control of the mask type used in SVD
I'll try different data to see how that is later, and I don't expect perfect as my data isn't great it's always noisy with the DSLR
mask-1ds.jpg
mask-1ds.jpg (75.3 KiB) Viewed 22475 times
User avatar
admin
Site Admin
Posts: 3382
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: StarTools 1.9 preview

Post by admin »

Thank you happy-kat,
happy-kat wrote: Sun May 21, 2023 6:58 pm 1.9.551 GPU win
trying SVD to see if my stars are better liked but when I view sampling the stars are all starfish (except for the comet).
That's actually a good thing! :)
The more pixels from the tapering-off stellar profile are include in the sample, the better. The more pixels from a single star can be sampled, the better SV Decon can model the blur that is affecting that stars.
From the startools main menu outside of any module went to mask/clear/keep > back to mask > auto >
auto generated a star mask of starfish
selected 'stars' mask regenerated and was better star shapes
selected 'fat stars' mask improved
selected 'stars' switching between fat stars and stars some stars get bigger mask area but then if select fat stars the mask correct, fat star mask bigger mask area appears to only be on 'stars' rather then 'fat stars'
every time select 'all stars' get the starfish stars as seen in SVD
These mask serve different purposes. The Alt Stars (Alt as in alternative) mask includes as much of the stellar profiles as possible. The other mask modes may be useful in other ways.
Ivo Jager
StarTools creator and astronomy enthusiast
Carles
Posts: 214
Joined: Thu Jul 30, 2020 5:06 pm

Re: StarTools 1.9 preview

Post by Carles »

@admin

Thanks Ivo, indeed the new 1,9 decon did get my datasets stars :) both from the old QHY8L and the planetary camera also.
It gets a bit slower than previous though specially when showing the selected mask and beetween selecting the samples. But result seems good

but in some bright stars creats this artifact
imagen_2023-05-22_134333393.png
imagen_2023-05-22_134333393.png (710.67 KiB) Viewed 22440 times
decay
Posts: 493
Joined: Sat Apr 10, 2021 12:28 pm
Location: Germany, NRW

Re: StarTools 1.9 preview

Post by decay »

Carles wrote: Mon May 22, 2023 11:34 am but in some bright stars creats this artifact
Hi Carles,

this looks a bit like the artefacts we discussed a few days or weeks ago:
viewtopic.php?f=4&t=2821&p=14291&hilit=centroid#p14291
(There have been some more posts regarding this issue in the "StarTools 1.9 preview" and other threads.)
Have you switched on the 'PSF Resampling' 'Centroid Tracking' option and/or checked for dud PSF samples?

Best regards, Dietmar.
happy-kat
Posts: 373
Joined: Sun Feb 01, 2015 11:31 am

Re: StarTools 1.9 preview

Post by happy-kat »

Thank you Ivo, the explanation helps a lot.
Post Reply