Another histogram clipping question

Questions and answers about processing in StarTools and how to accomplish certain tasks.
almcl
Posts: 265
Joined: Wed Jan 21, 2015 7:15 pm
Location: Shropshire. UK

Re: Another histogram clipping question

Post by almcl »

As ever, Ivo, many thanks for your response.
admin wrote: Tue Oct 11, 2022 7:58 am
Why are you concerned about clipping if I may ask?
My concern is that every dataset I process ends up with a histogram that looks like this:

histogram.jpg
histogram.jpg (28.74 KiB) Viewed 2969 times

it looks as if some faint detail may be being lost and while I can move the peak away from the left hand end using skyglow in FilmDev:

skyglow.jpg
skyglow.jpg (22.66 KiB) Viewed 2969 times
.
this doesn't produce the smooth, more symmetrical curve that I think would be preferable. I also notice that large swathes of the background in the image have RGB levels of zero or one.

I have managed to lift the histogram by combining images after ST processing in other software, but would really like to achieve something similar within ST?
smoother curve.jpg
smoother curve.jpg (21.92 KiB) Viewed 2969 times
Skywatcher 190MN, ASI 2600 or astro modded Canon 700d, guided by OAG, ASI120, PHD2
User avatar
admin
Site Admin
Posts: 3382
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Another histogram clipping question

Post by admin »

Would you be able to share a dataset with me that exhibits this problem, along with a workflow that replicates the problem you are experiencing?

Thank you!
Ivo Jager
StarTools creator and astronomy enthusiast
almcl
Posts: 265
Joined: Wed Jan 21, 2015 7:15 pm
Location: Shropshire. UK

Re: Another histogram clipping question

Post by almcl »

Thanks, Ivo.

PM sent.

Al
Skywatcher 190MN, ASI 2600 or astro modded Canon 700d, guided by OAG, ASI120, PHD2
User avatar
admin
Site Admin
Posts: 3382
Joined: Thu Dec 02, 2010 10:51 pm
Location: Melbourne
Contact:

Re: Another histogram clipping question

Post by admin »

almcl wrote: Tue Oct 11, 2022 11:42 am Thanks, Ivo.

PM sent.

Al
Much apreciated Al, I will respond here so others may benefit from this.

I just had a look at the dataset (a nice M16 widefiled), and all appears in order. All modules work as they are supposed to, and the "issue" is indeed as I surmised in the thread; much of the detail is intentionally squeezed into the lower dynamic range by the user;
Redoing stretch of linear data
--- Auto Develop
Parameter [Ignore Fine Detail <] set to [3.5 pixels]
Parameter [Outside RoI Influence] set to [10 %]
Parameter [RoI X1] set to [1511 pixels]
Parameter [RoI Y1] set to [501 pixels]
Parameter [RoI X2] set to [1655 pixels (-1398)]
Parameter [RoI Y2] set to [1360 pixels (-676)]
Parameter [Detector Gamma] set to [1.00]
Parameter [Shadow Linearity] set to [50 %]
In AutoDev, by specifying an RoI for just that small region, dynamic range is optimised only for that small region, while dynamic range is intentionally withheld from the rest of the image. To reiterate how the RoI works; the stretch will perfectly allocate dynamic range for the RoI, and the rest of the image will be subject to that allocation.

You can see precisely how this works in the AutoDev docs;
Image
^Confining the Region of Interest ("RoI") progressively to the core of this galaxy, the stretch becomes more and more optimised for the core and less and less for the outer regions.

The lower right panel shows what happens if you just select the brightest part of the image (as happened in your M16 image); the rest of the image will almost appear clipped (but is not really when viewed in, say, a 16-bit histogram).

To quote the relevant part of the docs (which explicitly addresses this sort of apparent clipping);
Detail inside the RoI


Let's say our image is of galaxy, neatly situated in the center. Then confining the RoI progressively to the core of the galaxy, the stretch becomes more and more optimised for the core and less and less for the outer rim. Conversely, if we want to show more of the outer regions as well, we would include those regions in the RoI.
Detail outside the RoI


Shrinking or enlarging the RoI, you will notice how the stretch is optimised specifically to show as much as possible of the image inside of the RoI. That is not to say any detail outisde the RoI shall be invisible. It just means that any detail there will not (or much less) have a say in how the stretch is made. For example, if we would have an image of a galaxy, cloned it, put the two image side by side to create a new image, and then specified the RoI perfectly over just one of the cloned galaxies, the other one, outside the RoI would be stretched precisely the same way (as it happens to have exactly the same detail). Whatever detail lies outside the RoI, is simply forced to conform to the stretch that was designed for the RoI.


It is important to note that AutoDev will never clip your blackpoints outside the RoI, unless the 'Outside RoI Influence' parameter is explicitly set to 0% (though it is still not guaranteed to clip even at that setting). Detail outside the RoI may appear very dark (and approach 0/black), but will never be clipped.


Bringing up the 'Outside RoI Influence' parameter will let AutoDev allocate the specified amount of dynamic range to the area outside the RoI as well, at the expense of some dynamic range inside the RoI. If 'Outside RoI Influence' set 100%, then precisely 50% of the dynamic range will be used to show detail inside the RoI and 50% of the dynamic range will be used to show detail outside the RoI. Note that, visually, this behavior is area-size dependent; if the RoI is only a tiny area, the area outside the RoI will have to make do with just 50% of the dynamic range to describe detail for a much larger area (e.g. it has to divide the dynamic range over many more pixels), while the smaller RoI area has much fewer pixels and can therefore allocate each pixel more dynamic range if needed, in turn showing much more detail.
Or in other words, if you do not incorporate everything of interest in your Region of Interest, those things may be hard (or almost impossible) to see. Or in terms of an 8-bit histogram; the pixel values of those things will be bunched together.

In your case you appear to acquire widefields at which scale dynamic range often needs to be optimised for the entire field. Indeed in the case of your M16 dataset, nebulosity extends across the entire image. As such the Region of Interest is really the entire image.

If I were to process that particular dataset, I would not specify a specific RoI at all; all areas contain detail that needs to be allocated dynamic range. If you are worried about the detail in the core (the pillars etc.), this should be recovered using tools that work on increasingly finer levels. To once again quote the AutoDev docs;
Keeping in mind AutoDev's purpose

The purpose of AutoDev is to give you the most optimal global starting point, ready for enhancement and refinement with modules on a more local level. Always keep in the back of your mind that you can use local detail restoration modules such as the Contrast, HDR and Sharp modules to locally bring out detail.
Any questions, do let me know. Hope this helps!
Ivo Jager
StarTools creator and astronomy enthusiast
almcl
Posts: 265
Joined: Wed Jan 21, 2015 7:15 pm
Location: Shropshire. UK

Re: Another histogram clipping question

Post by almcl »

Many thanks for that, Ivo.

I'll have to have a good look at a lot of previous data and see what a larger (or no) RoI does for things.

Outstanding support once again :thumbsup:
Skywatcher 190MN, ASI 2600 or astro modded Canon 700d, guided by OAG, ASI120, PHD2
Post Reply