Idea: Toolbox project level scenarios - meta scenarios... about spine-toolbox

[jkiviluo]

I think this is bit more generic issue. We have a similar problem when running a workflow where there is some post-processing of results. The scenarios don't yet necessarily exist in the result database when selecting/executing a workflow that contains both model execution and result post-processing.

I think the solution is to have inheritance of scenario selections. That should be an option in the "arrow scenario filter" (when this "inherit scenario choices from upstream" is activated, the scenario filter choices should be greyed out as they become overruled). In your case, the trouble is that DS1 -> "Some Tool" cannot inherit the choice from DS1 -> "Run SpineOpt1", since they are parallel and not serial. The easy solution is to put a Merger right after DS1, so that there is only one "arrow scenario filter" after DS1. All the dependents get their data from the Merger instead of DS1. The downside is that it's not obvious for the user that this should be done when this is needed. The upside is that it could be made to work much faster than more fundamental changes (would just require this inheritance choice).

I believe there is a further issue in your example - how to compare two specific scenarios in "Some Tool" (with_transmission and without_transmission). I think that's a different issue, since they are truly different scenarios. The quick fix is that you provide some information in DS1 that indicates which scenarios to compare and in what way. "Some Tool" should then use the information to deduce what to do. That way you don't need to it manually but once. However, we could also think about supporting something like that more natively. If so, it needs it's own issue and is further down the road.

[DillonJ]

The inheritance is not a bad idea... I wouldn't take my example too literally as the real workflow is more complex and I was trying to identify some generic functionality that would be useful. As you identify though, the trouble is that scenarios are being used for different things - 1. to identify a type of run and 2 to identify an actual scenario... being able to "tie" a scenario to a project level scenario would be nice in this regard. Maybe also, something higher level than scenario is needed which would offer quite a lot of flexibility.

Definitely not expecting anything in a timeframe that allows us to solve this issue - more trying to provide some feedback from some real use cases.

[soininen]

One workaround is to not select any scenarios in the arrows pointing to Some Tool, but instead give the scenario as a command line argument to Some Tool which then manually applies the correct scenario filters to the database mappings. This way there would be only one place where the post-processing scenario is defined.

[DillonJ]

Not a bad idea at all @soininen