Help with Auto-dev, and backgound noise management

[grant.pullen]

Hi there,

Firstly I would appreciate some clarification of the use of the auto dev module. When I first do an auto dev everything is red...
I assume its showing the "bad" parts, so I then do a wipe and a "nomal" dev, otherwise if I do an auto dev again I still get major discoloration again, showing a bit less red/or sometimes green.
I am then unable to get the desired colour in the colour module after an auto-dev, however after using a normal dev I am able to fix the colours in the colour module.
I thus never seem to get auto dev on its own to work for me (is this wrong on my side?)... My typical workflow is bin->autodev->wipe->->remove horizonal banding->dev->contrast->sharpen->colour->stop tracking... is this okay?

Secondly I am having some issues processing this image, as soon as I lift the galaxy out I get red "blotches" in the surrounding sky, I tried masking it however also ended up with a bad result as the transition was not too good suggestions, instructions, logs, magic words to process this image would be greatly appreciated.
The link to the image https://www.dropbox.com/s/lg381slzltxcu ... -05-19.TIF

Thanks in advance for help, suggestions, or any other comments...

Grant

[ubertank]

First I just want to state that I'm a total beginner with this software. I wasn't really sure if I should reply or post my attempt at all, but even if it does't help you at all, it doesn't hurt I guess.

Somebody will probably be along to do a proper attempt for you and answer your questions more concisely, but these may be helpful in the meantime:

http://www.youtube.com/watch?v=ql-h2WmE0yc
http://www.youtube.com/watch?v=5DTQKQK6 ... e=youtu.be

These video's will give you a little insight into how to remove heavy noise such as that found in your image. Autodev is always useful for an initial inspection, but often a manual develop is used after a wipe.

As far as the image itself is concerned, using the standard workflow I managed to get the following image. it has lost a lot of quality as I had to screenshot it as a JPEG and upload to photobucket (buying software when I'm paid this week hopefully). This has actually brought back a bit more noise around the galaxy than was there in startools.

Although the noise was hard to reign in, I found that with the life module and final noise reduction it got it to a fairly decent level.

Once again sorry I couldn't be any more help. I just noticed you'd not had a reply so thought I'd spend ten minutes helping out as much as I'm able at this inexperienced stage.

[admin]

Hi Grant,

Firstly I would appreciate some clarification of the use of the auto dev module. When I first do an auto dev everything is red...
I assume its showing the "bad" parts,

It is indeed!

so I then do a wipe and a "nomal" dev, otherwise if I do an auto dev again I still get major discoloration again, showing a bit less red/or sometimes green.

It is possible that Wipe leaves remnants due to extreme noise or some types of anomalous data. There are roughly two types of remnants;

• In the face of dark anomalies such as dead pixels or dust donuts, Wipe can produce halos. If this occurs, the dark anomaly needs to be addressed by either masking it out, or, if the anomalies are small (such as dead pixels) by increasing the dark anomaly filter setting.
• When the data is exceedingly noisy (which seems to be the case here), Wipe will do its best to remove gradient/vignetting/light pollution, but leave the background noise level. It leaves this noise because it refuses to clip the background. If this is noise is very high it will readily show up in a subsequent run of AutoDev.

Indeed, I used the Vignetting preset and a Dark Anomaly Filter of 7 pixels and ended up with the following background noise pattern;

noisest.jpg (339.53 KiB) Viewed 15378 times

I am then unable to get the desired colour in the colour module after an auto-dev, however after using a normal dev I am able to fix the colours in the colour module. I thus never seem to get auto dev on its own to work for me (is this wrong on my side?)... My typical workflow is bin->autodev->wipe->->remove horizonal banding->dev->contrast->sharpen->colour->stop tracking... is this okay?

We can see the random noise is uniform (which is a good thing - no further remnants are visible). We can also see some banding, which you indeed remove using the Band tool. Do note however, that the Band module is, alas, still sequence dependent (which, I admit, sucks - I will fix this!). So perform the banding *after* your final global stretch.
You are also quite right in that AutoDev cannot help us any further here when it comes to stretching the data, as it keeps bringing out the left over noise & issues. So, you rightly resorted to a manual global stretch using the Develop module.
The rest of your workflow is spot-on, Contrast, Sharpen, finish off with Color before stopping your tracking.

Secondly I am having some issues processing this image, as soon as I lift the galaxy out I get red "blotches" in the surrounding sky, I tried masking it however also ended up with a bad result as the transition was not too good suggestions, instructions, logs, magic words to process this image would be greatly appreciated.

I was just writing this as Ubertank wrote a great reply, with a fantatsic rendition of your data - great stuff!
As Ubertank pointed out, the Life module's Isolate preset is your friend if you need to lift/rescue structures from a very, very noisy background. Don't worry too much about noise showing up until you switch tracking off - it should do a good job getting rid of it.
I played a little with your data as well and this is what I came up with;

fc2013.jpg (144.39 KiB) Viewed 15378 times

Open file, indicate it is stil linear.
--- Bin
Parameter [Scale] set to [(scale/noise reduction 25.00%)/(1600.00%)/(+4.00 bits)]
AuotDev to see what we have.
Wipe using Vignetting preset and Parameter [Dark Anomaly Filter] set to [7 pixels]
AutoDev again. Nothing more we can do, so let's manually develop (globally stretch) the image;
Develop with Parameter [Digital Development] set to [99.02 %]
Remove horizontal banding using the Band module.
Let's try some deconvolution as well. StarTools usually knows if/when/where it can improve your image. I just used the Deon preset in the auto mask generator, after which I inverted and Shrank a few times to make sure none of the very bright stars were selected;
Deconvolution with Parameter [Radius] set to [2.0 pixels]
I noticed a vertical streak as well and tried to remove it with the Band module again.
Next I fixed the colors;
Color module with Parameter [Cap Green] set to [To Yellow], Parameter [Bottom End Saturation] set to [2.60], Parameter [Top End Saturation] set to [9.90], Parameter [Saturation] set to [274 %], Parameter [Blue Ratio] set to [1.19], Parameter [Green Ratio] set to [1.14] and Parameter [Red Ratio] set to [1.00].
It's better to use the color module at the end, but I was being cheeky...
I also tried to push back the rest of the noisy bacgrkound versus the galaxy. I put the galaxy in a mask and ran the Life module's Isolate preset.
Addtional parameters were; Parameter [Strength] set to [50 %], Parameter [Mask Fuzz] set to [12.6 pixels].
I did just a little bit wavelet sharpening;
Parameter [Amount] set to [158 %]
Another run of the Life module's Isolate preset (without mask). This time I'm using it for both pushing back the noise a little more and increasing saturation of luminous objects at the same time. Parameter [Saturation] set to [348 %]
Next I switched off tracking and performed noise reduction. Default parameters, except Parameter [Brightness Detail Loss] set to [37 %]

And that's it!

Hope this helps,

[grant.pullen]

Thanks Ubertank and Ivo,

I appreciate the time and effort you guys put into this and I am truly amazed with the miracles you guys managed to perform
I cant wait to try to reprocess it and some others as soon as time permits... unfortunately this frail body demands sleep..
Ivo, I can not complement you enough on startools its a fantastic tool which empowers us all to capture the beauty of the universe
The tool is my all time favorite piece of software, I actually view the site daily to look for any updates (no pressure
( So don't ever stop working on it as I am sure I and many others will be life long users/admirers)

Grant

[gboulton]

Figured I'd take a shot as well.

• --- Bin
  • Parameter [Scale] set to [(scale/noise reduction 50.00%)/(400.00%)/(+2.00 bits)]
  --- Auto Develop - Used a ROI around the object.
  • Parameter [Gamma] set to [0.63]
    Parameter [Outside ROI Influence] set to [5 %]
    Parameter [Ignore Detail <] set to [10.0 pixels]
  --- Wipe - Note DA filter and DAH, as well as the slightly elevated aggressiveness.
  • Parameter [Mode] set to [Correct Color & Brightness]
    Parameter [Top End Treatment] set to [Wipe 2.0]
    Parameter [Cap Green] set to [No]
    Parameter [Output Gradient Only] set to [No]
    Parameter [Precision] set to [256 x 256 pixels]
    Parameter [Dark Anomaly Filter] set to [30 pixels]
    Parameter [Drop Off Point] set to [0 %]
    Parameter [Corner Aggressiveness] set to [100 %]
    Parameter [Aggressiveness] set to [86 %]
    Parameter [Dark Anomaly Headroom] set to [0 %]
  --- Develop - I'm a fan of manual Develop after AutoDev/Wipe. I tend to like to use the Digital Development to really stretch things, and the manage the noise and background as best I can with Gamma. I just flat out FORGOT to use the DAF setting here. oops!
  • Parameter [White Calibration] set to [Use Stars]
    Parameter [Gamma] set to [1.02]
    Parameter [Skyglow] set to [0 %]
    Parameter [Digital Development] set to [98.11 %]
    Parameter [Blue Luminance Contrib.] set to [100 %]
    Parameter [Green Luminance Contrib.] set to [100 %]
    Parameter [Red Luminance Contrib.] set to [100 %]
    Parameter [Dark Anomaly Headroom] set to [5 %]
    Parameter [Dark Anomaly Filter] set to [Off]
  --- Wavelet Sharpen
  • Parameter [Channels] set to [Brightness Only]
    Parameter [Scale 1] set to [0 %]
    Parameter [Scale 2] set to [26 %]
    Parameter [Scale 3] set to [100 %]
    Parameter [Scale 4] set to [100 %]
    Parameter [Scale 5] set to [100 %]
    Parameter [Mask Fuzz] set to [8.0 pixels]
    Parameter [Brightness Mask Power] set to [0.00]
    Parameter [Amount] set to [100 %]
    Parameter [Small Detail Bias] set to [75 %]
  --- Life - Here's where I depart a bit from "the norm". I tend to create a smaller mask, letting mask fuzz draw the edges out for me, and then go with a less aggressive filter...note the low strength here. This creates a more pleasing (for ME...strictly subjective) transition from background to object when dealing with very noisy images, imo.
  • Parameter [Detail Preservation] set to [Linear Brightness Mask]
    Parameter [Compositing Algorithm] set to [Multiply, Gamma Correct]
    Parameter [Inherit Brightness, Color] set to [Off]
    Parameter [Output Glow Only] set to [No]
    Parameter [Airy Disk Sampling] set to [128 x 128 pixels]
    Parameter [Airy Disk Radius] set to [8 pixels]
    Parameter [Glow Threshold] set to [0 %]
    Parameter [Detail Preservation Radius] set to [20.0 pixels]
    Parameter [Saturation] set to [100 %]
    Parameter [Strength] set to [38 %]
    Parameter [Mask Fuzz] set to [40.0 pixels]
  --- Color - Again, a departure from the norm for me. I mask the object first, and work with color on it alone.
  Parameter [Cap Green] set to [To Yellow]
  Parameter [Bottom End Saturation] set to [Full]
  Parameter [Top End Saturation] set to [Full]
  Parameter [Saturation] set to [123 %]
  Parameter [Blue Ratio] set to [1.09]
  Parameter [Green Ratio] set to [1.00]
  Parameter [Red Ratio] set to [1.05]
  Parameter [Mask Fuzz] set to [1.0 pixels]
  --- Color - Now, a star mask, to work on star colors. Note the mask fuzz here.
  Parameter [Cap Green] set to [To Yellow]
  Parameter [Bottom End Saturation] set to [Full]
  Parameter [Top End Saturation] set to [Full]
  Parameter [Saturation] set to [188 %]
  Parameter [Blue Ratio] set to [1.04]
  Parameter [Green Ratio] set to [1.00]
  Parameter [Red Ratio] set to [1.00]
  Parameter [Mask Fuzz] set to [8.0 pixels]
  --- Wavelet De-Noise - Again, use of a mask to allow different de-noise strengths and settings for the object vs the background.
  Parameter [Scale 1] set to [100 %]
  Parameter [Scale 2] set to [100 %]
  Parameter [Scale 3] set to [100 %]
  Parameter [Scale 4] set to [57 %]
  Parameter [Scale 5] set to [0 %]
  Parameter [Mask Fuzz] set to [40.0 pixels]
  Parameter [Brightness Mask Power] set to [0.00]
  Parameter [Color Detail Loss] set to [50 %]
  Parameter [Brightness Detail Loss] set to [50 %]
  Parameter [Structural Emphasis] set to [0 pixels]
  Parameter [Edge Repair Strength] set to [15 %]
  Parameter [Noise Tracking Influence] set to [100 %]
  Parameter [Intelligent Despeckle] set to [48 %]
  --- Wavelet De-Noise - And now the object...
  Parameter [Scale 1] set to [0 %]
  Parameter [Scale 2] set to [36 %]
  Parameter [Scale 3] set to [38 %]
  Parameter [Scale 4] set to [0 %]
  Parameter [Scale 5] set to [0 %]
  Parameter [Mask Fuzz] set to [1.0 pixels]
  Parameter [Brightness Mask Power] set to [0.00]
  Parameter [Color Detail Loss] set to [50 %]
  Parameter [Brightness Detail Loss] set to [50 %]
  Parameter [Structural Emphasis] set to [0 pixels]
  Parameter [Edge Repair Strength] set to [15 %]
  Parameter [Noise Tracking Influence] set to [100 %]
  Parameter [Intelligent Despeckle] set to [0 %]
  --- Bin - A final bin for posting purposes
  Parameter [Scale] set to [(scale/noise reduction 50.00%)/(400.00%)/(+2.00 bits)]

========================

There's obviously some fine tuning left to do here, seasoning colors to taste, repairing some stars, etc. Just figured I'd toss out what I suppose we might call a "masked approach".

Thanks for sharing this data with us...always fun to get a chance to work with data I might not otherwise see or acquire!

[admin]

Thanks Gordon - a very interesting processing flow with very nice results!

[grant.pullen]

Thanks Gordon,

I especially appreciate that you submitted such a detailed log, (which makes me think I have a few suggestions which I will add to the relevant forum...)
On a side note its really great that startools has such a nice community of people helping each other out.

Regards
Grant

[grant.pullen]

Hi there everyone,
After everyones input, spent a few minutes trying to go at it again.
Not the best, but my personal best for now...

fc-2013-05-19-2.jpg (305.23 KiB) Viewed 15339 times

--- Bin
Parameter [Scale] set to [(scale/noise reduction 50.00%)/(400.00%)/(+2.00 bits)]
--- Auto Develop
Parameter [Gamma] set to [1.00]
Parameter [Outside ROI Influence] set to [5 %]
Parameter [Ignore Detail <] set to [Off]
--- Wipe
Parameter [Mode] set to [Correct Color & Brightness]
Parameter [Top End Treatment] set to [Wipe 2.0]
Parameter [Cap Green] set to [No]
Parameter [Output Gradient Only] set to [No]
Parameter [Precision] set to [256 x 256 pixels]
Parameter [Dark Anomaly Filter] set to [8 pixels]
Parameter [Drop Off Point] set to [0 %]
Parameter [Corner Aggressiveness] set to [100 %]
Parameter [Aggressiveness] set to [75 %]
Parameter [Dark Anomaly Headroom] set to [10 %]
--- Band
Parameter [Algorithm] set to [Algorithm 1]
Parameter [Orientation] set to [Horizontal]
--- Develop
Parameter [White Calibration] set to [Use Stars]
Parameter [Gamma] set to [1.00]
Parameter [Skyglow] set to [0 %]
Parameter [Digital Development] set to [99.41 %]
Parameter [Blue Luminance Contrib.] set to [100 %]
Parameter [Green Luminance Contrib.] set to [100 %]
Parameter [Red Luminance Contrib.] set to [100 %]
Parameter [Dark Anomaly Headroom] set to [5 %]
Parameter [Dark Anomaly Filter] set to [3.0 pixels]
--- Band
Parameter [Algorithm] set to [Algorithm 1]
Parameter [Orientation] set to [Horizontal]
--- Contrast
Parameter [Expose Dark Areas] set to [No]
Parameter [Compensate Gamma] set to [No]
Parameter [Precision] set to [256 x 256 pixels]
Parameter [Dark Anomaly Filter] set to [4 pixels]
Parameter [Aggressiveness] set to [75 %]
Parameter [Dark Anomaly Headroom] set to [5 %]
--- Deconvolution
Parameter [Image Type] set to [Deep Space]
Parameter [Mask Behavior] set to [De-ring Mask Gaps, Hide Result]
Parameter [Radius] set to [3.0 pixels]
Parameter [Iterations] set to [6]
Parameter [Regularization] set to [1.00 (optimal noise and detail)]
Parameter [Mask Fuzz] set to [8.0 pixels]
--- Life
Parameter [Detail Preservation] set to [Linear Brightness Mask]
Parameter [Compositing Algorithm] set to [Multiply, Gamma Correct]
Parameter [Inherit Brightness, Color] set to [Off]
Parameter [Output Glow Only] set to [No]
Parameter [Airy Disk Sampling] set to [256 x 256 pixels]
Parameter [Airy Disk Radius] set to [8 pixels]
Parameter [Glow Threshold] set to [0 %]
Parameter [Detail Preservation Radius] set to [20.0 pixels]
Parameter [Saturation] set to [100 %]
Parameter [Strength] set to [56 %]
Parameter [Mask Fuzz] set to [40.0 pixels]
--- Develop
Parameter [White Calibration] set to [Use Stars]
Parameter [Gamma] set to [1.00]
Parameter [Skyglow] set to [0 %]
Parameter [Digital Development] set to [98.50 %]
Parameter [Blue Luminance Contrib.] set to [100 %]
Parameter [Green Luminance Contrib.] set to [100 %]
Parameter [Red Luminance Contrib.] set to [100 %]
Parameter [Dark Anomaly Headroom] set to [5 %]
Parameter [Dark Anomaly Filter] set to [Off]
--- Band
Parameter [Algorithm] set to [Algorithm 1]
Parameter [Orientation] set to [Horizontal]
--- Contrast
Parameter [Expose Dark Areas] set to [No]
Parameter [Compensate Gamma] set to [No]
Parameter [Precision] set to [256 x 256 pixels]
Parameter [Dark Anomaly Filter] set to [1 pixels]
Parameter [Aggressiveness] set to [75 %]
Parameter [Dark Anomaly Headroom] set to [5 %]
--- Wavelet De-Noise
Parameter [Scale 1] set to [100 %]
Parameter [Scale 2] set to [75 %]
Parameter [Scale 3] set to [50 %]
Parameter [Scale 4] set to [25 %]
Parameter [Scale 5] set to [0 %]
Parameter [Mask Fuzz] set to [1.0 pixels]
Parameter [Brightness Mask Power] set to [0.00]
Parameter [Color Detail Loss] set to [50 %]
Parameter [Brightness Detail Loss] set to [50 %]
Parameter [Structural Emphasis] set to [0 pixels]
Parameter [Edge Repair Strength] set to [15 %]
Parameter [Noise Tracking Influence] set to [20 %]
Parameter [Intelligent Despeckle] set to [0 %]
File saved [C:\personal\astro\fc-2013-05-19-1.tiff].
--- Color
Parameter [Cap Green] set to [To Yellow]
Parameter [Bottom End Saturation] set to [3.50]
Parameter [Top End Saturation] set to [Full]
Parameter [Saturation] set to [250 %]
Parameter [Blue Ratio] set to [1.02]
Parameter [Green Ratio] set to [1.02]
Parameter [Red Ratio] set to [1.01]
Parameter [Mask Fuzz] set to [1.0 pixels]
--- Repair
Parameter [Radial Samples] set to [32]
Parameter [Sub Sampling] set to [4x]
Parameter [Algorithm] set to [Warp]
Parameter [Grow Mask] set to [0 pixels]
File saved

Regards
Grant

[gboulton]

On a side note its really great that startools has such a nice community of people helping each other out.

I agree. So much of this stuff is subjective...let's be honest, our human eyes could never see these objects as we photograph them, EVEN if we only photograph in "visible light". And until we can start replicating a few of Geordi's visors, we aren't ever going to, EVEN if we could go there. So...pretty much by definition, our results...anyone's results...are going to be our own interpretation of what the camera has captured.

In, sad to say, a majority of online communities, this fact leads to some fairly contentious behaviour, as members argue and fuss about what image is "right" or whose color is 'right" or whatever. It baffles me how two human beings whose vision stops well short of IR can argue over the "shade" of IR light in their astro-images.

I much prefer communities like this one...where the focus is on helping each other achieve the results we want from our own images, and simply enjoying the work of others. None of us own the Hubble, and none of us can see Ultraviolet light...so why not just raise a pint, say cheers, and have fun, ya know?

[admin]

On a side note its really great that startools has such a nice community of people helping each other out.

I agree. So much of this stuff is subjective...let's be honest, our human eyes could never see these objects as we photograph them, EVEN if we only photograph in "visible light". And until we can start replicating a few of Geordi's visors, we aren't ever going to, EVEN if we could go there. So...pretty much by definition, our results...anyone's results...are going to be our own interpretation of what the camera has captured.

In, sad to say, a majority of online communities, this fact leads to some fairly contentious behaviour, as members argue and fuss about what image is "right" or whose color is 'right" or whatever. It baffles me how two human beings whose vision stops well short of IR can argue over the "shade" of IR light in their astro-images.

I much prefer communities like this one...where the focus is on helping each other achieve the results we want from our own images, and simply enjoying the work of others. None of us own the Hubble, and none of us can see Ultraviolet light...so why not just raise a pint, say cheers, and have fun, ya know?

Could not agree more & this is exactly the spirit in which I created StarTools!
StarTools is about leveling the playing field - taking the difficulty of processing out of the equation as much as possible, while giving you enhanced control over how you want to present your image. That's what this software, forum and community is about. Nothing else. As much as I love StarTools, it is, and always shall remain, a necessary evil. It should be pure a tool of creative expression without learning curve. The ideal processing application would be a mind-reading device that 'just does it'. That said, it is really encouraging how new users like Ubertank can make me - the freakin' creator of the software - sit up and take notice of alternative presentations that are based on taste, not processing prowess or tech-savviness. I love that!

I'm a fairly timid, nerdy guy and I'm not one to seek a confrontation, but in any instance where I see someone use false authority to argue a completely subjective point, I will speak out and engage these people with ferocity. The worst thing you can say on a forum about someone else's work is that 'it doesn't look quite natural'. No one - and mean no one - has a monopoly on what looks natural. As Gordon points out, sadly there are a number of forums (some associated with software products that promote this), where people argue, from a false sense authority, that they can tell you what's natural. This false sense of authority is all based on the amount of money and time they have sunk into the hobby and getting to grips with the cycle of acquisition and processing. They feel like they've done the hard yards. It simply does. Not. Matter. One of the most hilarious examples of this, is an 'award winning' user stuck in the PhotoShop era (I won't mention names) who was adamant that the detail I had brought out in a Nebula was unnatural and strongly suspected it was all artifacts. It took for another user to pull up a Hubble comparison that demonstrated the exact same detail for him to quietly disappear. Tell me you don't like my image - all cool. Tell me my, or someone else's taste/interpretation is 'wrong' and I will eat you!