StarTools

Hi Ivo,

It's been awhile since I've posted. I have discovered the joy of ending an imaging session to discover my corrector plate dewed over. I've been shut down waiting for a Dew-Buster. I expect it today in time for a full moon.

In your description of your processing of the Pacman data in the recent thread you said, "The data appears to be oversampled at native resolution (e.g. 1 unit of real detail is 'smeared out' over multiple pixels), so really it's probably a good idea to Bin the data a little. I didn't do that in this case, just to see if we could use deconvolution to bring back the detail to 1 unit-to-1 pixel."

Can you elaborate on exactly what you look at and your thought process to determine if binning is appropriate? I've been reading about sampling rate but I haven't gotten my brain wrapped around it yet. What would you describe as "1 unit" of data which should correspond to 1 pixel of image?

I've been binning by default on the thinking that I need all the noise suppression help I can get using an uncooled DSLR in the summer. I'd like to be able to make a data driven decision about whether I'm giving up any/too much detail.

Hi,

That's an awesome question on a pretty important subject.

Data binning is a data pre-processing technique used to reduce the effects of minor observation errors.

Many astrophotographers are familiar with the virtues of hardware binning. The latter pools the value of 4 (or more) CCD pixels before the final value is read. Because reading introduces noise by itself, pooling the value of 4 or more pixels reduces this 'read noise' also by a factor of 4 (one read is now sufficient, instead of having to do 4). Ofcourse, by pooling 4 pixels, the final resolution is also reduced by a factor of 4. There are many, many factors that influence hardware binning and Steve Cannistra has done a wonderful write-up on the subject on his starrywonders.com website. It also appears that the merits of hardware binning are heavily dependent on the instrument.

Most OSCs and DSLR do not offer any sort of hardware binning in color, due to the presence of a Bayer matrix; binning adjacent pixels makes no sense, as they alternate in the color that they pick up. The best we can do in that case is create a grayscale blend out of them. So hardware binning is out of the question for these instruments.

So why does StarTools offer software binning?

Firstly, because it allows us to trade resolution for noise reduction. By grouping multiple pixels into 1, a more accurate 'super pixel' is created that pools multiple measurements into one. Note that we are actually free to use any statistical reduction method that we want. Take for example this patch of pixels;

7 7
3 7

A 'super pixel' that uses simple averaging yields (7 + 7 + 3 + 7) / 4 = 6. If we suppose the '3' is anomalous value due to noise and '7' is correct, then we can see here how the other 3 readings 'pull up' the average value to 6; pretty darn close to 7.
We could use a different method (for example taking the median of the 4 values) which would yield 7, etc. The important thing is that grouping values like this tends to filter out outliers and make your super pixel value more precise.

But what about the downside of losing resolution?

That super high resolution may have actually been going to waste! If for example your CCD can resolve detail at 0.5 arcsecs per pixel, but your seeing is at best 2.0 arcsecs, then you effectively have 4 times more pixels than you need to record one 1 unit of real resolvable celestial detail. Your image will be oversampled, meaning that you have allocated more resolution than the signal really will ever require. When that happens, you can zoom in into your data and you will notice that all fine detail looks blurry and smeared out over multiple pixels. And with the latest DSLRS having sensors that count 20 million pixels and up, you can bet that most of this resolution will be going to waste at even the most moderate magnification. Sensor resolution may be going up, but the atmosphere's resolution will forever remain the same - buying a high resolution instrument will do nothing for the resolution of your data in that case! This is also the reason why professional CCDs are typically much lower in resolution; the manufacturers rather use the surface area of the chip for coarser but more precise CDD wells ('pixels') than squeezing in a lot of very imprecise (noisy) CCD wells.

There is one other reason to bin OSC and DSLR data to at least 25% of its original resolution; the presence of a bayer matrix means that (assuming an RGGB matrix) after applying a debayering(aka 'demosaicing') algorithm, 75% of all red pixels, 50% of all green pixels, and another 75% of all blue pixels are completely made up! Yes, your 16MP camera may have a native resolution of 16 million pixels, however it has to divide these 16 million pixels up between the red, green and blue channels! Here is another very good reason why you might not want to keep your image at native resolution. Binning to 25% of native resolution will ensure that each pixel corresponds to one real recorded pixel in the red channel, one real recorded pixel in the blue channel and two pixels in the green channel (the latter yielding a 50% noise reduction in the green channel).

Now StarTools' binning algorithm is a bit special in that it allows you to apply 'fractional' binning; you're not stuck with pre-determined factors (ex. 2x2, 3x3 or 4x4). You can bin exactly the amount that achieves a single unit of celestial detail in a single pixel. In order to see what that limit is, you simply keep reducing resolution until no blurriness can be detected when zooming into the image. Fine detail (not noise!) should look crisp.

I hope this helps explaining software binning a little!

Cheers,

Thanks Ivo. That was very helpful.

I image with a Canon 450D at .8"/pixel. I have been reading articles Craig Stark wrote regarding sampling rates. It seems likely that I am oversampling by at least 2x, if not 3-4x. I will take a much closer look at the Bin module to see if I can find my sweet spot.

Gary

Very informative reply as usual. Thanks for the question and answer.
Che

Forgive me if I missed this point, but should binning be done before or after deconv. thanks Che

Cheman wrote:Forgive me if I missed this point, but should binning be done before or after deconv. thanks Che

Hi Che,

Binning should really be the first thing you do to your data. All other steps that follow will benefit greatly from the reduced noise.
You bring up a good point though; deconvolution is another way you can make use of oversampled data - it can reverse the 'blurring' that was caused by, for example, bad seeing. The extent to which it can reverse this blurring depends on the quality of the data; noisy data cannot be deconvolved as effectively as noiseless data. And this is where you might make another tradeoff. You could bin the data just enough for deconvolution to be viable again (example here), e.g. you bin less than you could, still leaving the data oversampled a little and then use of the rest of the oversampling for deconvolution. Only in StarTools can you do this, as the Bin module allows you to freely specify how much you want to bin (you're not stuck with 2x2, 3x3 or 4x4 fixed values) and how much you want to leave oversampled.

Awesome info You have a real knack of explaining things simply enough to understand, but with enough detail to make me actually have to think about it
Che

Finally, time to read the forum in detail. I've been experimenting with various DSLRs, spectrum mods and cooling. Ivo, your explanation about resolution is similar to JTWs rationale for using the Canon 1100D for its hypercooled DSLRs. Very helpful confirmation of a decision to use an APS-C sized sensor, among other considerations. Having designed built and tested 3 DSLR cooling systems, including a full frame model, I am settling on a design and camera to build a fourth. Happy to share the specs.

Rowland wrote:Finally, time to read the forum in detail. I've been experimenting with various DSLRs, spectrum mods and cooling. Ivo, your explanation about resolution is similar to JTWs rationale for using the Canon 1100D for its hypercooled DSLRs. Very helpful confirmation of a decision to use an APS-C sized sensor, among other considerations. Having designed built and tested 3 DSLR cooling systems, including a full frame model, I am settling on a design and camera to build a fourth. Happy to share the specs.

Very interesting Rowland. Yes, the 1100D to me, on paper, sounds like the better astro DSLR around still. However, much depends on the rest of its CCD/CMOS specs vs what newer models offer. Another area I have been following with great interest are HD video cameras with HD-SDI output. Love to hear more about your exploits in due time!