StarTools

Here are some notes relating to using the Wipe module.
They relate to StarTools version 1.7.420 and later
Please let me know if anyone sees any errors or has any additional advice they think helpful.
I will update this as needed.
To see a full alphabetical list of module topics click here.
For an index of similar notes on the other StarTools modules see StarTools Main Window Use.

Wipe Module

The description below is intended to help in getting good results. Please feel free to experiment - and provide feed back.

Purpose:

• To remove background colour casts and gradients such as unwanted light (light pollution, moon, street lights), vignetting, amp glow and colour casts. It also compensates for sensor column and row defects.

Description
For a general overview see Wipe: Light Pollution, Vignetting and Gradient Removal.
Wipe flattens the field and removes gradients, Wipe also calibrates the background and removes bias in the colour channels. Note that this does not necessarily mean that the background becomes a nice neutral grey. Rather it makes sure that the background is luminance bias-free across all colour channels. Subsequent colour calibration will modify and neutralize the chrominance (colour) information.
Wipe also now compensates for vertical and horizontal bias errors & sensor column and row defects and so completely replaces the Band module.

Useful Sources
The Official StarTools English Manual (pdf) gives a good description of all the modules. It relates to StarTools version 1.6+.
The Unofficial StarTools English Manual (pdf) is a good general source of help. There are versions that relate to StarTools version 1.6 and
version 1.7
There is a good general description of the Wipe module in Wipe Usage.
StarTools links & tutorials
A short processing workflow of a real-world, imperfect dataset StarTools simple image processing workflow tutorial with real-world, imperfect data. Describes the Wipe module. Uses StarTools v1.5, May 08 2019
The video StarTools: Removing vignetting and complex gradients describes a more complicated use of the Wipe module. Uses StarTools v1.4, Mar 03 2018

When to use:

• After using Initial AutoDev, Lens, Crop and Bin as needed.
• Only use Wipe once - although you can use 'Undo' on main screen to immediately go back and redo the Wipe Module if you aren't happy with the results.

Example Workflow (v1.7):
AutoDev-{Lens}-Bin-Crop-Wipe-AutoDev (or FilmDev)-{Contrast/HDR/Sharp/Decon/Flux/Life}-Color-{SuperStr/Entropy/Filter/Shrink}-Track/NR-{Layer/Heal/Repair/Synth/Stereo 3D}
Key: {...} optional modules

Example Workflow (v1.8):
{Compose}-AutoDev-{Lens}-Bin-Crop-Wipe-AutoDev (or FilmDev)-{Contrast/HDR/Sharp/SVDecon}-Color-{Shrink/Filter/Entropy/SuperStr/NBAccent/}-Track/NR(Unified-Denoise)-{Flux/Repair/Heal/Layer/Synth}
Key: {...} optional modules

Method:
This is a way of using the module which should give good results in most cases:

1. When doing the Initial AutoDev (initial global stretch):
   • Look at the image and see what sort of gradients you have. If the brightness drops away at the corners choose 'Vignetting'. If there is a general lightness in the background choose 'Basic'. If there is an intensity in a particular area - especially at the edge - you have 'Amp glow'.
   • Look for any stacking artefacts which need to be cropped out before using the Wipe module.
   • Look for any dust specks, dead pixels, trees, mountains etc. which will need to be masked out.
2. After using the Crop module to remove any stacking artefacts, load the Wipe module.
3. Set the mask to exclude pixels you are sure are not background - e.g. the target, dust specks and other dark anomalies as described below:
   Mask - Clear - Lassoo around Dark Anomalies and Target as needed - Invert - Keep.
   or Mask - Clear - Flood Fill Lighter Pixels - click on an edge pixel on the subject to highlight it - Grow x 4 - Invert - Keep.

   

   KurtM_M8_WipeMask_qs.jpg (22.84 KiB) Viewed 24644 times

4. Adjust the Dark Anomaly Filter to mask the effect of small Dark Anomalies (see below).
5. Choose a preset which best describes what you are trying to achieve (see below). If there are multiple issues choose the most aggressive.
6. Default values often work fine. Wait for the processing to complete.
7. Check the results - Use the 'Color/Luminance' button to check both results if in 'Compose' mode
8. If the result is not right, ensure the subject, dust specks, dead pixels, trees etc. are not included in the analysis by using a combination of Mask and Dark Anomaly Filter controls as described below.
9. If you see banding or other sensor issues, adjust the Synth Dark/bias using the appropriate setting for the issues seen. Wait for the processing to complete.
10. Reduce the Synth/Bias Edge Area from 100% if required.
11. Adjust the 'Gradient Falloff' only when there is Vignetting.
12. Press 'Keep' when done.

What result to look for:

• Wipe automatically applies a temporary stretch which is particularly aggressive to the displayed image to show up problems clearly. The characteristics of the background are greatly emphasised.
• After applying Wipe the stretched image should have a uniform and dark background.
• There should be no remaining background hue or colour cast. If there is check the masking and redo, and try increasing the Gradient Aggressiveness.
• Look at the stretched image background for halos and other casts - this may be due to dark features such as dead pixels, dust, trees etc. which have not been removed. These can be removed using the Mask or Dark Anomaly Filter as described below.
• If some detail has been wiped reduce the Gradient Aggressiveness - or Mask the area affected.
• If casts exist try increasing the Gradient Aggressiveness.

After use:

• The Wipe module should make the background darker. It is important to redo the global stretch (using FilmDev or AutoDev) to take advantage of the extra dynamic range made available.

Ways of getting better results:

• Use Flats when stacking so that the Vignetting, dust specks, etc. will have already been accurately compensated for. This will simplify the Wipe to work specifically on light gradients.
• If Wipe does not reduce the colour cast well it may be that the image has a lot of noise. In this case, increasing the Dark Anomaly filter can make a difference.

Special Techniques:

Masking out flaws (e.g. dust or artefacts) that were hidden before running Wipe
See the full description in the topic Wipe module - editing mask.

• Use the Wipe module as normal and 'Keep' the flawed result.
• Use the 'Mask' button on the main screen to edit the mask to exclude the newly highlighted flaws.
• Crop any remaining stacking artifacts.
• Use the 'Restore' button on the main screen - select 'Original' - this reverts to the Original (Linear, Binned & Cropped) image.
• Use the Wipe module as normal - which now uses the updated mask.

Description of Controls:
Mask
For general instructions on using masks see Mask.

• Mask out (make non-green) the main subject and any dust specks, mountains, trees etc, using the 'Lassoo' Brush Mode.
  This will stop Wipe trying to use them in analysing how to wipe the image - the whole image will still be wiped.
• If using the 'Amp Glow' preset - select (make green) the pixels that are affected by amp glow.
• You must use FilmDev or AutoDev to bring out the image details before using Wipe if you want to see them in the Mask.

Presets:
The following buttons pre-set the control settings to values most suited to the intended purpose:

• Basic (v1.7+) - Simple wipe - gradual changes in background light level across the image - e.g. from light pollution or the moon.
• Vignetting - for removing any background light level that falls away at the corners - use flats instead if you can.
• Narrow Band - optimised wipe for narrow band images - Light pollution and gradients are less of a problem with narrow band data.
• Uncalibrated (v1.7+) - use when no calibration has been done - tries to correct for vignetting etc.
• Uncalib2 (v1.7.443+) - use when no calibration has been done - an alternative to Uncalibrated (now called Uncalib1).
• Amp Glow - a brightness normally at one edge of the image due to an uneven heating effect from the camera electronics.

Color/Luminance
In 'Compose' mode the Wipe module works on the luminance and colour datasets separately but in parallel. You can see the results of each by pressing this button. You will be reminded if you haven't checked the color view.

Correlation Filtering (v1.8)
This setting controls the removal of correlation artefacts. These are features in the image which look like detail but are due to dithering, sensor issues or debayering.

• Sets the size of the correlation artefacts to be removed.
• This filter works best on oversampled data where the artefacts are smaller than the resolved detail.
• Default is Off, Range is Off(0.0) to 3.0 pixels.

Gradient Aggressiveness
Sets how aggressive Wipe should be:

• Wipe searches for a constant background level of brightness around each pixel in the image - the search area is governed by the 'Aggressiveness' setting - the higher the value the smaller the area searched to establish the background level. See the description in the topic Wipe feature - when to use Mask.
• Default is 75% for Basic and Uncalibrated presets, 0% for NrwBand, and not set for Vignetting or Amp Glow.
• Mask out any objects of interest if using a particularly aggressive setting.
• Change if gradients remain - up to 95% in extreme cases. Reduce from the default if detail is being wiped.

Dark Anomaly Filter:
Use this to stop dark spots in the image (such as dead pixels, small dust specks and small scratches) affecting the Wipe.

• Default is 1 pixel. Range is 'Off'(0) to 30 pixels.
• Set to the size of the dark area. Larger dark areas (including larger dust specks) should be deselected using the mask instead.
• Try Increasing to 2-10 pixels - look at the effect - continue until no further benefit.

Gradient Falloff
Sets how gradient undulation changes closer to the corners.

• This can help get a flat field in cases where there is uneven lighting towards the corners but the basic vignetting modelling set by Synthetic Flats can't model light correctly or sufficiently.
• Default is 0% for Basic and NrwBand, 75% for Vignetting and Uncalibrated, and not set for Amp Glow preset.

Synthetic Flats
Specifies what standard vignetting model to try to fit to the dataset.

• Off - no model used
• Vignetting
• Inverse Vignetting (Overcorrection)
• Default is 'Off' for Basic preset, 'Vignetting' for Vignetting and Uncalibrated presets, and not set for Narrow Band and Amp Glow presets.

Gradient Edge Behavior
Defines how Wipe should assume the gradient falls off beyond the edge of the image.

• Grow opposite axis - Best for poorly calibrated datasets and those that have a lot of changes in bias.
• Bounce Back - Best for well calibrated datasets and datasets that contain image-filling nebulosity.
• Absorb 50% - Good choice for datasets that fall between the two types described above.
• Defaults are 'Absorb 50%' for 'Basic' preset, 'Bounce Back' for Narrow Band, 'Grow opposite axis' for Uncalibrated, and not set for Vignetting and Amp Glow presets.

Synthetic Dark/Bias
Sets what darks and bias issues Wipe should try to detect and repair.
Tries to detect and repair the effects of:

• Off - None
• Defective Sensor Columns - One or more adjacent sensor columns giving a strong positive or negative bias.
• Defective Sensor Rows - One or more adjacent sensor rows giving a strong positive or negative bias.
• Vertical Bias - Sensor columns having subtly different biases compared to their neighbours across the entire height of the image.
• Horizontal Bias - Sensor rows having subtly different biases compared to their neighbours across the entire width of the image.
• Adaptive Vertical Bias - Sensor columns having subtly different biases compared to other columns across the height of the image.
• Adaptive Horizontal Bias - Sensor rows having subtly different biases compared to other rows across the entire width of the image.
• Adaptive Multi-Axis Bias Aggressive - Sensor rows and columns having subtly different biases compared to other sensors across the entire width of the image - Aggressive correction.
• Adaptive Multi-Axis Bias Conservative - Sensor rows and columns having subtly different biases compared to other sensors across the entire width of the image - Conservative correction.
• Defaults are 'Off' for 'Basic' preset, 'Adaptive Multi-Axis Bias Conservative' for Uncalibrated, 'Adaptive Multi-Axis Bias Aggressive' for Amp Glow preset, and not set for Vignetting and Narrow Band presets.

Sampling Precision
Defines the image 'resolution' that Wipe uses when analysing the image prior to wiping.

• Range is 128x128, 256x256, 512x512, 1024x1024 or 2048x2048.
• Default is 256x256 pixels for Basic, Vignetting and Uncalibrated presets, 128x128 for Narrow Band, not set for Amp Glow preset.
• Change from the default to a larger number if the image has rapidly changing gradients and some background gradient has not been compensated for.

Synth. Bias Edge Area
Sets how far from the edge towards the image centre Wipe should detect and correct dark/bias issues (i.e those issues defined by the Synthetic Dark/Bias setting).

• Applies only when Synthetic Dark/Bias is used.
• Use this setting to target the correction of issues that happen near the edge of the image (e.g. thermal noise/bias) without affecting the central part of the image.
• Default is 33% for Uncalibrated and Amp Glow presets.

Background Notes

Dark Anomalies
Every night sky image that is still linear has a minimum background light level caused by the sky glow.
This is represented by a certain pixel value. If there are pixels with values below this then they don't represent something in the sky.
These values are either caused by hot or stuck pixels, dust or scratches, noise, or something terrestrial like a tree or mountain.
StarTools calls these Dark Anomalies.

Dark Anomalies stop some modules from working correctly so they need to be identified so they can be managed.

Dark Anomaly Headroom
The Wipe module looks for the global background level and subtracts that from the signal to reveal more detail.
The only time this is a problem is when there is no global background - i.e when you are not looking at the sky (trees & mountains or dust) or when there is a faulty pixel.
In this case the subtraction would produce a negative value and so clip to 0.
To avoid this, Dark Anomalies are detected and some dynamic range is allocated for these.
The background level is raised for the whole image so that when the signal level is subtracted from it it is always greater than 0.
The amount the background level is raised by is called the Dark Anomaly Headroom. In the Wipe module the Dark Anomaly Headroom allows all the dark anomalies to be mapped directly- without modification.
The Contrast module allows some compression of the Dark Anomaly Headroom.

Dark Anomaly Filter and Mask
Wipe tries to find this background level so as to identify the minimum and maximum pixel values that represent light from the sky.
Dark Anomalies are dips below the minimum level which would cause Wipe to set an artificially low background level.
In order for StarTools to find the correct background level it uses two methods to identify and ignore Dark Anomalies

• Mask - Larger Dark Anomalies, such as mountains or trees, as well as dust bunnies can be identified using a Mask.
• Dark Anomaly Filter - The Dark Anomaly Filter lets Wipe automatically ignore small clusters of pixels with very low values when deciding the background level. The value of the Dark Anomaly Filter sets the maximum pixel cluster size that will be ignored. Set to 1 pixel to ignore single pixels such as cold or stuck pixels. Set higher to ignore clusters caused by noise or small dust specks or scratches.

Color/Luminance button
In 'Compose' mode the Wipe module works on the luminance and colour datasets separately but in parallel. You can see the results of each by pressing this button.

Calibration Modelling
The Wipe module models and corrects for three stages of calibration:

1. Synthetic bias/darks modelling and correction (subtraction)
2. Synthetic flats modelling and correction (division)
3. Gradient modelling and correction (subtraction).

Any issues selected in the parameters and/or detected are modelled during the correct stage and feeds into the next.

Synthetic Bias/Darks Modelling
Modelling and correction for Bias and Darks issues is new to Wipe in v1.7 and makes the Band module obsolete.
A number of different sensor defects can be detected and repaired.

Awesome Guy!
Will add it to the Inofficial Manual V 1.7