StarTools parameterization and project (session) save

[joseamirandavelez]

I'm just spewing my mind here... Please don't take this as a complaint. I love this tool!

I was talking with a friend about StarTools. We come from the engineering world. Where stuff is done in a linear fashion (step-by-step)... I know StarTools is designed to be "non-linear" in that you can do any amount of steps and then re-do them again. However, I think StarTools is missing a lot by not keeping a track of the steps performed. I mean, a track the user can see. For example, take the image below:

Thats ANSYS Workbench, which is a huge improvement from ANSYS Classic since it introduced model parameterization (basically you can edit modify or remove anything at any moment). Forget it is a Finite Element Analysis tool and think about the tree view on the left. It would be not only awesome, but game changing, to have something like that in StarTools.

Imagine I open my image, do a crop, develop, wipe... I would get a list of those in that tree:

Crop
Develop
Wipe
Contrast
HDR
...

And then when you click on the items it would open the parameters used. It could allow for duplicate items (develop as is) or keep just one (develop from scratch)...

Crop
Develop
Wipe
Contrast
HDR
Develop (as is)
...

I know StarTools is handling a huge amount of data and keeping track of all of this would be crazy. But if at least we had an editable list of steps done, it would so much improve the processing. I know StarTools is designed to be non-linear, but being able to see the steps you have taken and to modify them and see the effects seems obvious from my point of view. It allows you to develop your workflow (even though its not needed from a software perspective, I still need a list of things to do, and things I've done...

Also, I think we need a way to store our images in progress as a project. I believe that simply dumping everything to a file would be a killer but maybe saving the steps performed along with the original image to a binary file would be really nice. StarTools could then open the image (in the binary file), and re-do all the steps done (kind of a script)... It would allow you to stop a project any time and continue later. Or even sending a project to another person to take a look... Or you can even use the current log file, but add a way to read this log file on the image. It would take a while to run the log but it opens a lot of possibilities (including scripting, as a side effect).

[admin]

Thanks for this!
There are a number of things I'm toying with for 1.4 (which will be mostly about architectural, optimisation, stability and engine upgrades), some are very much along the lines of which you just described (in fantastic clarity and detail - thanks!), including tighter intergation of the StarTools.log with the Restore/Undo functionality, a Project file, as well as tutorial generation. I'd very much like something along these lines as well, though simplicity and userfriendliness should be paramount.

[joseamirandavelez]

Agree. Simplicity of use should be the north, as it has been up to now. Some of my ideas may not be aligned with your design philosophy but I'm looking forward to see stuff such as a project file or even a good implementation of a replay file. Waiting anxiously!

[ecuador]

Is this still in play for the future releases of 1.4?
For example one thing I would really like is to be able to apply a workflow on a different stacking of the same image (which means even the same masks would be applicable), which is along the same lines. Any chance of seeing it in the future?

[admin]

Is this still in play for the future releases of 1.4?

Yep!

For example one thing I would really like is to be able to apply a workflow on a different stacking of the same image (which means even the same masks would be applicable), which is along the same lines. Any chance of seeing it in the future?

This is sort of like recording a macro. I'd have to think about that one and whether that would be appropriate/useful without any human intervention. I can see problems arise if stacking artefacts are different or if noise levels are different. What sort of different stacking did you have in mind?

[ecuador]

For example I am trying to figure out the effect of including some more light frames that are not as good. Or seeing the effect of average/median/kappa sigma. Or seeing the difference between different ISO sessions...
I find it useful to explore such things in order to improve results, however to see the differences you have to process the image, and do the exact same processing as much as possible.
Or, something that has happened to me at least a couple of times. I start working on an image and I find out that at some point some weird artifact or other strangeness is going on in some part of the photo, and it turns out it was a sub-exposure that had a faint satellite or something like that, so I'd better re-stack, but it would be nice to be able to automatically get to the point I was previously in StarTools...

[admin]

For example I am trying to figure out the effect of including some more light frames that are not as good. Or seeing the effect of average/median/kappa sigma. Or seeing the difference between different ISO sessions...
I find it useful to explore such things in order to improve results, however to see the differences you have to process the image, and do the exact same processing as much as possible.
Or, something that has happened to me at least a couple of times. I start working on an image and I find out that at some point some weird artifact or other strangeness is going on in some part of the photo, and it turns out it was a sub-exposure that had a faint satellite or something like that, so I'd better re-stack, but it would be nice to be able to automatically get to the point I was previously in StarTools...

Understood. The issue with this, however, is that different data warrants subtly different parameters and choices during post-processing. Stacking artefacts could - potentially - be different, gradients could - potentially - be different, noise levels could - potentially - be different, dark anomalies could - potentially - crop up, etc. In StarTools, the idea is to have parameters control a deviation from an automatically determined baseline using Tracking data mining, rather than starting from scratch with absolute values for parameters that are (almost always) wildly inappropriate and require major experimenting with. It gets better/more appropriate results quicker without losing flexibility.
The latter also, unfortunately means that the baseline will vary per image stack, and hence, the 'tweak' on top of it will, most likely need some correcting. It'd be a shame to completely ignore the subtly better/different results that you could've achieved thanks to the improved/different characteristics of the data. It'd be a practice I would not want to promote... Does that make sense?

[joseamirandavelez]

I think it would be easier to understand if you posted some sneak peaks of what you are cooking right now...

[xiando]

Presuming this is still an idea being developed... it might be easiest just to log the sequence of actions to a textfile.

A simple text log would require very little in the way of supporting code, and would provide the necessary sequencing for future efforts.

Andrew