Huge green bias

General discussion about StarTools.
Mike in Rancho
Posts: 1166
Joined: Sun Jun 20, 2021 10:05 pm
Location: Alta Loma, CA

Re: Huge green bias

Post by Mike in Rancho »

patraip1 wrote: Tue Oct 19, 2021 4:35 am Hi Mike
Thanks a lot for your effort!!!
That looks way better than mine lol!!
At this point I am just experinenting with ST....
I woukd appreciate some help with wipe
I think the tablet I am using for taking flats may be small so I am not illuminating evenlemy the entire entrance of the scope. Will try again

Thanks for the help
Wipe can be hard to learn, especially for a beginner, since beginner data tends to push Wipe to the edge. Mine sure did! M31's can also be weird, and the key to a lot of them is to use a mask to make sure that Wipe doesn't start erasing chunks of the galaxy, mistaking them for gradient. In my limited experience, I think I've seen more M31's need a Wipe mask (in reality an "unmask") than anything else. Check the ST main website for Wipe usage info, as well as the user notes sections here for additional reading.

Practice making your unmask for M31 (the lasso works well) and adjusting parameters. You'll see it if you've gone overboard. On your M31, my parameters ended up being DAF 4, CF 0.7, and Gradient 94%. This is 1.8 beta, so skip the CF if using 1.7.

Still, your processing will improve much if you get the acquisition, calibration, and stacking all sorted out. :D

I tried to make something out of your M33, and failed. Maybe Ivo's skills are needed here. But really, IMHO you are fighting the calibration problems mostly, so even though you kinda-sorta got a fair image out of it (your first one anyway), there's still all sorts of remnant vignetting that's left over and pretending to be part of the outer galaxy.

I looked at your subs, and even gave them a run in DSS as a mini-stack. Better, but at only 2 minutes integration I couldn't pull anything out of it.

I only used the lights, flats, and bias. Generally no need to use both bias and dark flats. And at such a short exposure time for the DF's, they are nearly identical to the bias anyway. Since you don't have darks, and DSS doesn't allow you to create your own calibration formula, just use the bias here. In that way, your lights will be bias-subtracted, as will your flats, resulting in proper calibration for the division.

Thus, when I used DSS with only your lights, flats, and bias loaded, I think it worked out better. Hard to tell without the whole enchilada, of course. I used ST recommendations, as well. Still a big gradient to deal with and I cropped a bunch off, but Wipe handles that far easier than what was there in your stack.

Your flats were well underexposed per the general rule, which is to be generally around 50% of your linear ADU histogram, not the stretched jpg BOC histogram. You may want to try longer flats. On my D5300, I expose them to about 7/8 BOC histogram, also ensuring that all three channels are captured and not clipped.

Finally, to be on the safe side and because Iris showed a really bizzare histogram shape for your flat NEF's, I ran all your files through Adobe DNG converter and then Mark Shelley's Nikon file fixer (ring remover), which smooths out data errors resulting from lossy NEF compression. You may want to look into that eventually, although the D5500 I think has a less harsh compression.

You might consider a light pad (tracing thingy) for flats. I just started using one after taking them with my laptop screen for a year, and I think they are coming out much better.

Let us know what you come up with after making a few fixes!
patraip1
Posts: 25
Joined: Sat Oct 16, 2021 8:15 am

Re: Huge green bias

Post by patraip1 »

Ok Mike thanks a lot!
Pardon my ignorance but how do I check the adu histogram? On the camera?
patraip1
Posts: 25
Joined: Sat Oct 16, 2021 8:15 am

Re: Huge green bias

Post by patraip1 »

Hello guys,
I think I have solved the green cast issue and as Ivo correctly suggested it was down to bad calibration (flats) files.
I took new flats last afternoon pointing the scope at the sky opposite the sun and keeping the histogram in the middle.
I will take even better ones as those I took prior to Mikes suggestion about the histogram
I had another 220 subs of m33 last night even with the almost full moon very close to my target.
I managed to create a fair (in my beginner opinion) picture that I am quite happy with after stacking with ST suggestions in DSS and dealing with the huge light gradient by creating a mask of the galaxy in wipe. Colours came up fine. I used all wavelets in order with pretty much default settings (except wipe where I used DAF 5 ,vingetting and 94% agressiveness to edit the masked background) and came up with this
m33400subsnowhite.jpeg
m33400subsnowhite.jpeg (174.41 KiB) Viewed 5687 times
Here is a sample of the new flats
https://drive.google.com/file/d/12SuXF5 ... sp=sharing

and here is the stack of 400 10sec iso200 subs ,30 flats and 30 bias (I skipped the dark flats as per Mike's suggestions)

https://drive.google.com/file/d/1--7aoq ... sp=sharing

PLease keep in mind that 200 of those subs where taken with the moon at 75% and high in the south and the rest with an almost full moon in the SE
That is in my opinion moonlight and not vingetting in the stacked picture .

Anyway thanks for all the help and suggestions
Mike in Rancho
Posts: 1166
Joined: Sun Jun 20, 2021 10:05 pm
Location: Alta Loma, CA

Re: Huge green bias

Post by Mike in Rancho »

Nice improvement! Things are coming along fast, good to see. :D

You might be a little purple-heavy? :think: It's possible that green bias was reduced too much in color, so a little boost to that might clean it up, as well as make the stuff that should be blue...blue again.

Getting late and was a long day, so sorry I didn't respond more quickly to your previous question. I'll try to take a look at your new stack and the flats tomorrow!

For checking ADU of RAW files, I use Iris. Free program, quite old, but there's not a lot of options and it does the job well enough (though it's a mono histogram). Just load a RAW file and then do view histogram, and change the scaling and axis markings as needed. Remember for your 14-bit NEF's, your range will be 1 to 16384. Or maybe 0 to 16383, I dunno.

Really your flat subs should be fine as long as none of the channels are either clipped, nor too close to the edge in the "non-linear" area of the sensor. Where those cutoffs are, exactly, I have no idea. So many people aim for their flats to have a median value of 50% of their ADU range. There's a lot of wiggle room, but that should be guaranteed to be safe.

Except of course for potential Nikon compression ringing. Or that shooting at things like LCD screens, or sky flats, might shift the color channels in ways that do not match up with the lights. Always something. :lol:

And the moon will be gone soon enough, so you'll be able to test out your vignetting theory. However, lights, flats, and bias should calibrate out just fine in DSS, assuming all proper settings.
patraip1
Posts: 25
Joined: Sat Oct 16, 2021 8:15 am

Re: Huge green bias

Post by patraip1 »

here is my second attempt...
m33SECONDATTEMPT.JPEG
m33SECONDATTEMPT.JPEG (165.17 KiB) Viewed 5651 times
I have to say these are waaayyyy better than what I could do in Gimp. So happy I found startools!!! It is far easier for me to process my data with ST

PS Iris does not work on my laptop sadly...
Mike in Rancho
Posts: 1166
Joined: Sun Jun 20, 2021 10:05 pm
Location: Alta Loma, CA

Re: Huge green bias

Post by Mike in Rancho »

Cool. I'll try it out and see what's there. Color balance seems better. :thumbsup:

I looked at the new flat, and to me it would still be pretty underexposed. But again, it may be good enough. I'm not sure where the non-linear range of the D5500 would be considered.

However, to show you the difference between a stretched perceptual histogram (on the BOC screen it is of the internal jpg, not the linear RAW) and the full ADU range of your sensor, I made some graphs showing both old and new flats in a Gimp histogram (similar to BOC) and in Iris.

Pardon the crudity of this model.
Flats histograms.jpg
Flats histograms.jpg (56.36 KiB) Viewed 5627 times
patraip1
Posts: 25
Joined: Sat Oct 16, 2021 8:15 am

Re: Huge green bias

Post by patraip1 »

Next time I will try flats with the peak in the BOC display at 3/4 and pay attention not to clip any of the channels. I will try with a white t shirt pointed at the morning sky.
Mike in Rancho
Posts: 1166
Joined: Sun Jun 20, 2021 10:05 pm
Location: Alta Loma, CA

Re: Huge green bias

Post by Mike in Rancho »

That might come out better. Again there's wiggle room as long as you are away from the sides. I'm usually at 7/8 BOC or better (for full RGB), and that lands me around halfway ADU.

It will be interesting to see how your flats pan out and if it helps with those circular gradients and/or vignetting that's holding back the data (along with integration time). If you think it's moon glow, have you considered attaching a DIY shield?

I've used one before, just a baseball cap bill of sorts cut out of a manila folder and taped to the dewshield. This blocks moonlight from coming across the top of the telescope opening - where it shines into the opposite side and creates some gnarly glow down to the sensor. Others I have seen have actually made a more permanent version, with a little elastic band so they can just slip it on when needed.
patraip1
Posts: 25
Joined: Sat Oct 16, 2021 8:15 am

Re: Huge green bias

Post by patraip1 »

Hi Mike!
I am having a terrible vingetting issue after all....
I shot M31 again and as I had collimated the scope and fixed various optical path issues with my honemade light shroud and refocused I took brand new calibration frames. I tried flats at 1/30 and at 1 sec with a white tshirt and my laptop screen and got a histogram at 3/4 BOC.
The subs where 10sec and I took 200 before the moon rose and 100 after.

No matter how I try to stack them in DSS (I have tried nunerous settings) I get a stacked image with reverse vingetting that gets worse the more subs I intergrate.

The stacked subs alone have a classic vingetting image that gets worse over more subs.

I think it may have sonething to do with my truss tube being retracted 5cm to achieve prime focus with my D5500.

My flats are overcorrecting and I get a reverse vingetted image which eventually (after 150subs) starts clipping my data at the center.

I will post images later

Any thoughts?

Here is a sample of an autodev of a stacked just lights fts
justlights.jpg
justlights.jpg (632.26 KiB) Viewed 5490 times
I will try to upload a folder with all my data so anyone can try and stack them

I am getting ready for a trip to a nearby (1 hour drive) bortle 2 location this Friday evening, so I will get new data......

I am also considering turning vignette control to normal from off at my camera.
Mike in Rancho
Posts: 1166
Joined: Sun Jun 20, 2021 10:05 pm
Location: Alta Loma, CA

Re: Huge green bias

Post by Mike in Rancho »

Wow.

Well, you are doing difficult and interesting things. :D

Your D5500 has vignette control? Likely meant for terrestrial shots though, uncertain how it might pan out for extreme low light astrophotos. One can always experiment, I suppose (but not during a dark site trip!)

I assume you have some kind of very dark shroud/trap for that truss area? I looked up the telescope, boy, a lot of stuff to figure out.

Anyway I was thinking of some kind of light leak for the flats.

You are doing lights, flats, bias? And the bias was taken with the camera body cap on, like in a dark closet, with the built-in intervalometer doing the work?

Overcorrection would usually mean some kind of math problem in the calibration, like one side was bias subtracted and the other wasn't, or completely wrong exposure on the flats. But DSS should easily handle a simple calibration like this. :think:

Some sample files might be interesting.

If we can't figure it out from those, honestly I would just forget about specific targets for now and just start doing some experiments for vignetting. Any old patch of sky will do, and it probably doesn't even need to be tracked well. Just enough for you to pick up the vignetting in the skyglow. And then try to figure out how to make some flats that match up and correct it out properly.

Theoretically you could use shots already taken, although I think you said you changed the optics so maybe not.

Edit: Did you take a picture of the screen? I guess it does the job. :lol: But really you should be able to just hit Shift+Print Screen, Paste it into Gimp or something, then scale and export it out as a jpg.
Post Reply