stepEnv not setable about bob

[jkloetzke]

Manipulating the environment of a step from the pathFmt plugin is risky. Normally the environment is a central part in the Variant-/Build-Id calculation. Do you change the environment from the state tracker or from one of the *NameFormatter hooks?

[rhubert]

from the NameFormatter.

Why I'm doing this:

Normally the CMAKE_BUILD_TYPE comes from the root recipe and is passed to downstream recipes. But in case I want to build certain modules with another buildtype I need to modify the recipe to set this new BUILD_TYPE. This is passed to it's dependencies which is also not desired.
Actual solution is to parse another environment variable, lets call it BUILD_TYPE_OVERWRITE in the nameFormatter and modify the existing BUILD_TYPE. I also add the new buildtype to the name of dist/build directory.

Since BUILD_TYPE_OVERWRITE is taken into account for build-ID calculation it should be save?

Another possible solution is to parse the BUILD_TYPE_OVERWRITE in a class to setup the Cmake variable and to parse it again in the NameFormatter to get the directoryName. But this doubles the code. ( Not really, because it'll need a bash and a python implementation...)

[jkloetzke]

This sounds like it just works only because the build type is appended to the directory. BUILD_TYPE_OVERWRITE would only have an influence if this variable is actually used in a step, e.g. by buildVars.

Currently the design of Bob assumes that the Steps, once they have been generated, are constant. The path name calculation runs afterwards and could therefore contradict this assumption. That's also why there are public "set" functions on a Step at all.

It seems to me that you just need a way to set the environment for the recipe alone without propagating that to the downstream recipes. Would something like a privateEnvironment keyword fit your use case? Basically this keyword would allow you to set environment variables right after the dependencies have been traversed but before the steps are processed.

[rhubert]

privateEnvironment sounds useful to me to reduce the number of environment variables in the downstream recipies, but I'm not sure if it fits to this specific problem. Normally I want to use the Buildtype from the upstream recipe - so I can't set BUILD_TYPE as a privateEnvironment.

Nevertheless privateEnvironment should be added to the TODO List. Variables like PKG_VERSION can be private.

[jkloetzke]

What if you could set the privateEnvironment the following way?

privateEnvironment:
   BUILD_TYPE: "${BUILD_TYPE_OVERWRITE:-${BUILD_TYPE}}"

That would set the BUILD_TYPE to BUILD_TYPE_OVERWRITE if set. Otherwise BUILD_TYPE would be kept as is...

What I don't understand yet is how you set BUILD_TYPE_OVERWRITE?

[rhubert]

BUILD_TYPE_OVERWRITE is defined as: "buildtype_a:module1,module2#buildtype_b:module3,module3" and it's just a variable passed to bob by -DBUILD_TYPE_OVERWRITE

privateEnvironment:
   BUILD_TYPE: "${ $(parse $BUILD_TYPE_OVERWRITE):-${BUILD_TYPE}}"

maybe something like this can do the trick?

[jkloetzke]

That's exactly what I had in mind but what would take some time to implement. In the end a plugin could provide custom functions ("parse" in this case) that could be provided by a plugin.

[jkloetzke]

I have hacked something to allow advanced string processing: https://github.com/jkloetzke/bob/tree/string-processing

It's far from finished but I would like to have some feedback if this is heading in the right direction. The idea is that there are string processing functions called in the following way: $(function,arg1,arg2,...). Such function can be defined by a plugin. I can think of a number of built-in functions too but this is not implemented yet. There is an example test case in test/blackbox/string-plugin that should show the principle operation.

The privateEnvironment is still missing...

[rhubert]

👍 I did some basic tests and played around with this new feature. It's exactly what I need in this case.

privateEnvironment isn't that important since I can use something like:

environment:
   BUILD_TYPE: "$(override, BUILD_TYPE_OVERRIDE, ${BUILD_TYPE_DEFAULT})"`

[jkloetzke]

Perfect. 😄 I will add some default functions and the missing documentation. Strictly speaking this is an backwards-incompatible change because previously $ had to be escaped by $$ whereas now it is done by a backslash. I hope that is not a problem.

At the same time I would also like to use the same functions for the if clauses. That would easily enable to test for an enabled sandbox or available tools. Something like that:

depends:
   -
      name: foobar-toolchain
      if: "$(not,$(is-tool-defined,toolchain))"
   -
      name: host-toolchain
      if: "$(is-sandbox-enabled)"

[jkloetzke]

In order to use the new string substitutions you have to add 'bobMinimumVersion: "0.3"' to config.yaml. Note that this also changes the behavior of the 'if' keyword in the way described above.