merge in https://github.com/ashcrow/filetranspiler about butane

[ashcrow]

I am OK with this. One thing to keep in mind is that this will become something used in the field enough there should be stable builds produced and available.

cc @dustymabe @e-minguez @mrguitar @christianh814

[jlebon]

Could also be part of the YAML file itself as sugar, e.g.:

storage:
  local:
    - path: relpath/to/local/dir
      target: /path/to/extract  # or default to / if omitted?
systemd:
  local:
  - relpath/to/dir/of/units

That way the YAML stays self-contained and is easier to use from a config git repo containing all those source files.

[bgilbert]

Proposal:

variant: fcos
version: 1.1.0-experimental
storage:
  trees:
    # Can have more than one of these
    - local: etc             # local base directory to walk, relative to files-dir
      path: /etc             # root of the destination tree; defaults to /
      follow_symlinks: true  # defaults to false, which copies symlinks as symlinks

Semantics:

• This would walk the local tree and create file nodes embedding the contents of each file it finds (or link nodes, for links when follow_symlinks is false or missing). Mode bits would be copied, ownership would not.
• Any existing file node with a matching path and existing contents would be an error.
• Any existing file node without contents would be merged in. The node's mode, if set, would override the tree's file mode.
• directory nodes would not be created; we'd rely on Ignition's existing behavior of creating parent directories automatically. If the user wants to override the default directory mode, they can create directory entries by hand.

Possible future extensions:

• Explicitly support a tree of systemd units. For now, users can write into /etc/systemd/system directly.
• Add an option to copy directory modes.

Thoughts?

[cgwalters]

Yeah, one thing I am omitting here is the whole spec2 vs spec3 as well as another elephant 🐘 in the room which is MachineConfig vs Ignition. We'll need a debate about that. Plus the fact that parts of fcct high level may not apply to RHCOS (though ideally that's minimized).

I think tactically short term, we're not removing the filetranspiler project, just adding functionality to fcct.

[cgwalters]

(I would probably say that we should have something like oc adm generate-machineconfig)

[jlebon]

Also worth cross-posting a similar-ish request for having fcct read files from the filesystem directly at coreos/fedora-coreos-tracker#375.

[ashcrow]

Could also be part of the YAML file itself as sugar, e.g.:

storage:
  local:
    - path: relpath/to/local/dir
      target: /path/to/extract  # or default to / if omitted?
systemd:
  local:
  - relpath/to/dir/of/units

That way the YAML stays self-contained and is easier to use from a config git repo containing all those source files.

I like this pattern. It doesn't seem like it would be helpful (yet) in RHCOS land but it's still a much nicer UX.

[cgwalters]

Also related here is whether fcct exposes conversion functionality too: coreos/mantle#1190

[cgwalters]

Also from OpenShift architects one feedback on things here is a desire to avoid handing lots of binaries over to users. So we may end up e.g. adding Ignition functionality either to oc or openshift-installer. (Probably the former, but it needs an enhancement)

[dustymabe]

seems reasonable to me

[cgwalters]

What defines "files-dir"?

[dustymabe]

does files-dir default to $PWD?

[bgilbert]

-d/--files-dir is added in #99, and does not have a default. The idea is that we should never inject files from the filesystem without the user explicitly requesting it. That matters for hosted FCCT services, and arguably also for users pasting malicious FCCs from the Internet.

[lucab]

@bgilbert in your proposal, how is user/group ownership handled?

[bgilbert]

They wouldn't be set by default (so, root/root), but could be overridden on a per-file basis by creating a file node without contents but with the user/group set.