Donut stars

[Mike in Rancho]

Much wow here.

Not diving too deeply into things for a Friday night after an awful commute. But a few thoughts nonetheless...

I guess perception really is everything! It's almost as if our brains are creating their own extra acutance at the points of change, even though what we see doesn't match the numbers or the side view graph.

A few things about the plateau, or in some cases plateau with a spike! ( ) I imagine it effectively creates a transition, where there probably really shouldn't be one. And as we learned from digging down in the other thread, that transition could possibly be detected as edges by Sharp or whatever else, and thus end up enhancing. Recall all the blur-subtract-add stuff. And in which case all sorts of things might go haywire, as we propagate it into further modules.

Sharp has a protective mask, if it works right or is touched up properly, but I'm not sure everything does.

Blowing up the zoom shows that the plateau isn't necessarily totally flat, but close enough that I think it can be deemed effectively so. And depending on the OptiDev settings, the level of the plateaus all seem to be about the same place.

I wonder if any of this might be some kind of clipping protection kicking in, leading to many stars having the same shoulder level of plateaus. Some are just plain flat-topped, but the bigger brighter ones seem to be able to burst through and show that central spike.

It's hard to get good profiles when trying crazier stretches, as the background just gets raised so much, but I do want to see if non-overexposing stars can be brought to a plateau-with-spike shape also.

It all kind of reminds me of the testing we did on Ron's M13 and the 3D profiles, which was last Spring already!

dxron newM31 linear and stretched 3D plots.jpg (419.09 KiB) Viewed 6965 times

Dietmar I was curious what you used to create the profile viewer app? At first I was thinking that adding a checkbox or toggle button to set TopMost on the form would be handy, so that we could watch the profile change as we altered stretching parameters. But, the viewer doesn't react to live changes, and only sets upon window movement. Or toggling the subtract box on/off. So, perhaps not.

[decay]

Yes, it seems only two distinct values are assigned to the core regions of bright stars. The max value to the 'inner core' and then just one slightly lower value out for a distance surrounding it.

The max value of the inner core is caused by saturation in your and in my example. But for non-saturated stars there's a well-formed inner peak as we can see at Mike's example.

It feels weird to realize how much time I've spent chasing an optical illusion (usually with fuzz in SVD) - yet the optical illusion really does degrade the appearance of the images.

This optical illusion is astounding, but only part or a result of the problem. The main issue is this plateau or shoulder we see. But yes, we had a lot of discussions and never realized the underlying cause of all this symptoms.

that transition could possibly be detected as edges by Sharp or whatever else, and thus end up enhancing. Recall all the blur-subtract-add stuff. And in which case all sorts of things might go haywire, as we propagate it into further modules.

Absolutely! I remember a lot of discussions (Martin, Steve, Andy, aclmd ... ? ) regarding star appearance, rings, problems with SV Decon, HDR, dark cores and whatever. And I'm sure most of this have been problems caused by this plateau.

And depending on the OptiDev settings, the level of the plateaus all seem to be about the same place.
[...]
I wonder if any of this might be some kind of clipping protection kicking in, leading to many stars having the same shoulder level of plateaus.

They are all at the same level, because this shoulders are a direct result of the shape of the global stretching curve. As I tried to explain in my previous post. I'm quite sure and I don't think that this is some kind of clipping protection.

I wonder if we should wait for Ivo to chime in or if we should post a bug report.

[decay]

Dietmar I was curious what you used to create the profile viewer app? At first I was thinking that adding a checkbox or toggle button to set TopMost on the form would be handy, so that we could watch the profile change as we altered stretching parameters. But, the viewer doesn't react to live changes, and only sets upon window movement. Or toggling the subtract box on/off. So, perhaps not.

New version 0.0.3 is online:
https://c.web.de/@334960167135216273/Wk ... ftD_439MzQ

2024-03-23 12_02_55-CN=Microsoft Windows, O=Microsoft Corporation, L=Redmond, S=Washington, C=US.jpg (24.66 KiB) Viewed 6961 times

This is a C# .NET 4.8.1 app. I know, this is the dark side of the force ... but who cares? I've done worse things in my life.

Best regards, Dietmar.

[admin]

The plateauing can be caused by a number of things and would - as already mentioned - be chiefly caused by an area that shows comparatively very little change in co-located pixels (as a proxy for "detail"). It would therefore receive a smaller dynamic range allocation. I've seen this happen as a result of two scenarios;

• stacking of stars that saturate in some subframes, but not in others (due to loss of focus, or deteriorating atmospheric conditions)
• non-linear sensor response as wells fill up (sometimes even switching to different gains)

...or a combination of both.

It may be useful to evaluate a single frame to rule out some scenarios.

[dx_ron]

The plateauing can be caused by a number of things and would - as already mentioned - be chiefly caused by an area that shows comparatively very little change in co-located pixels (as a proxy for "detail"). It would therefore receive a smaller dynamic range allocation. I've seen this happen as a result of two scenarios;

• stacking of stars that saturate in some subframes, but not in others (due to loss of focus, or deteriorating atmospheric conditions)
• non-linear sensor response as wells fill up (sometimes even switching to different gains)

...or a combination of both.

It may be useful to evaluate a single frame to rule out some scenarios.

I don't think the first possibility happens in my case. The star I showed earlier is from a single-night stack where there were no clouds. I refocus automatically every 1° ambient temperature change, so I'm confident there are no poorly focused subs - if there were any they would certainly have been excluded from the stack, as I only stacked the best 50% by FWHM here.

As for the second - I certainly did not switch gains. But it seems quite likely that the sensor could be non-linear at near-saturation. I image with an IMX571c - along with the mono variant it is probably the most common sensor in use today.

I did pull out a single sub. Here is Optidev (nothing else applied) on that same star:

sub339_optidev.jpg (64.61 KiB) Viewed 6947 times

[dx_ron]

Links to download
M3 stack https://www.dropbox.com/scl/fi/19k8e7fv ... 5hwi4&dl=0

Single sub https://www.dropbox.com/scl/fi/yp36jy17 ... mieja&dl=0

[Mike in Rancho]

New version 0.0.3 is online:

This is a C# .NET 4.8.1 app. I know, this is the dark side of the force ... but who cares? I've done worse things in my life.

Best regards, Dietmar.

Awesome, Dietmar. Thanks.

Dark side? Well, what is the good side of the Schwartz then?

I don't know any C. Or, anything much at all. Last summer I downloaded VS 2022 and figured I'd teach myself VB, because how different could BASIC be from what I did 40 years ago? Well, I was wrong there. What is all this object oriented crap?

I still have a little tool I wanted to make, but never got to really starting it. And by now I've probably forgotten whatever little rudimentary VB I managed to learn.

Anyway (unless there was another change) the TopMost toggle permits live reaction to whatever is going on in the line of interest, so very useful to make adjustments and watch the profile change.

[Mike in Rancho]

My thoughts about acquisition and stacking were more in line with things like dithering, and avoiding any kind of cosmetic correction which could alter linear profiles. Now, it's possible we are all doing the same kind of things...saturating at least some stars is a given. And probably unavoidable. But really this pops up across a large cohort, with many different optics, sensors, and stackers.

Not much success today yet. I took my M33 L-only stack and then clone stamped out every saturated star I could see in the linear view. Yes it looks pretty funny! Not the least of which because my clone source had a non-overexposing star (which I couldn't see) in the middle of it. Oh well, I suppose that works.

That gave me a linear file, albeit 16 bit, that had some normal DSO data in it. Need something there taking up the field for ST's image analyzing. Then I started painting in various synthetic overexposing "stars" to see what OptiDev thought of them. First was just a pure white flat-sided cylinder. Then I made a angle-sided tall pyramid (think TransAmerica). Lastly I've been trying to create a synthetic Guassian star.

The cylinder and the pyramid are just ending up as clipped disks. Ugly, but no surprise. The Guassian is where I am maybe starting to get a plateau with a bump (and slight sense of the donut illusion), though I haven't been able to replicate a plateau with a big spike yet.

So no idea yet what it is that OptiDev is latching onto and doing, but it must be something.

I'll keep at it. And once we have a theory, we'll also need to think about what OptiDev should be doing up there in the stellar highlights range, and then how to achieve it. Like a donut or PSF-protection slider.

EDIT/ADD:

Want to include more praise for the Profile Viewer v3.

Was able to throw every type of stretch imaginable at stars in Siril, as the screen stays put and at the same zoom with the Viewer on top and overlaid. Perfect! ST resets the view a lot when changing between modules and whatnot, but still very useful.

In O/D it's hard to avoid a plateau with central bump or spike. It can be somewhat (or fully) suppressed with ROI and no outside influence, though even M33 core and faint stars can engender it, and a funny ROI can make the overall image look bad anyway. With other stretching and even F/D, I can maintain and massage a star's profile and slope, but not O/D. I can't come up with any reasonable settings to avoid it at the moment.

[admin]

Links to download
M3 stack https://www.dropbox.com/scl/fi/19k8e7fv ... 5hwi4&dl=0

Single sub https://www.dropbox.com/scl/fi/yp36jy17 ... mieja&dl=0

Thank you for uploading that.

I was able to replicate the "plateauing" with this dataset. I was also able to "hide" it by increasing Optidev's curve resolution (the amount of points it calculates to interpolate the constructed curve between) 16-fold (from 4096/12-bit points to 65536-bit points), which should not be necessary unless something has stuffed lots of data/"detail" in a tiny (1/4096th) slice of the - in this case - upper dynamic range (giving it very high importance, and thus making it "deserving" of more dynamic range).

In that case spreading it out over 16x more data points will distribute this "noise" across more curve points, reducing their dynamic range importance. Spreading data thin like this is undesirable as there may not be enough to go around to provide a good representation for all curve points (65536 is at the very upper end of useful resolution, as not many cameras have 16-bit ADCs).

I noticed that both the dataset and the single sub were floating point FITS files, which tends to be more amenable to hiding lots of aberrant data in small values. The single sub is encoded as in 32-bit floating point format (and already debayered), which is not something I would expect to come from a sensor. A sensor should produce a limited range of discrete values (ADUs) between 0 and full well capacity. In these datasets, it appears to me we are dealing with a significant non-linear response in the far upper part of the dynamic range.

What sort of intervening steps have been carried out to produce the single sub?

From a pure signal processing point of view, now that more cameras are equipped with 16-bit ADC I might add an option to OptiDev to reflect this, however in practice, it *should* not really yield much visible difference (and may have an adverse impact if data is too sparse or quantized).

EDIT: Running through a number of test datasets that would be the most problematic, sparsity issues may not be as bad as I thought... Will do some more tests, but I may be able to roll out 16-bit OptiDev ADC support without affecting "regular" operation much. There is still the question of whether "hiding" issues like this is desirable or not...

EDIT2: Ok, I've rolled this out. 1.9.565 is now up for download. I'd still like to figure out what's going on with these particular datasets though...

[Mike in Rancho]

Thanks for the work, Ivo!

The highlights stretching of DSO is less aggressive, as probably expected, and so will take some changing of ROI and IFD as well as perhaps kick some more work into the enhancing modules. Will be interesting to see how this all works in SVD and if that needs adjusting.

Ron's single sub appears to be calibrated, so says the FITS header anyway, so must be how it ended up 32-bit.

I was toying around with some pixel levels in a line through the middle of a star, and still can't really see where the plateau shoulders were coming from. I ran through it (from just before the plateau point and over to the other side) in a single sub (16-bit 2600MM), the stack (32-bit but 16-bit read-out), and then OptiDevs.

Levels Sheet.jpg (106.84 KiB) Viewed 6897 times

This isn't a lot of data and just L, but it sure cleaned up the middle of my stars anyway.

PI Donut Collage Sub-Stack-OD1-OD2.jpg (207.37 KiB) Viewed 6897 times