coreos / butane Goto Github PK

fcct does only support source URLs. The data schema has the drawback that everything needs to be urlencoded.
ct provides a handy inline option:
https://github.com/coreos/container-linux-config-transpiler/blob/50afd07d6e9cb73fbeb0e9c7b7f34c4d29829ffb/doc/examples.md#files

If including file data with inline or local (#83), automatically compress it if that produces a smaller Ignition config.

I have a local patch for this; waiting for #83 to be resolved first.

Depends on coreos/ignition#938 landing before Ignition spec 3.1.0 is stabilized.

I'd like to use the Docker image to generate the Ignition Config with Gitlab CI.

Currently my configuration (common.fcc) looks like this:

variant: fcos
version: 1.0.0
passwd:
  users:
    - name: core
      password_hash: "<removed>"

And my .gitlab-ci.yml looks like this:

stages:
  - build

generate_config:
  stage: build
  image: quay.io/coreos/fcct:v0.2.0
  script:
    - mkdir config
    - /usr/local/bin/fcct --strict --pretty --input common.fcc --output config/common.ign
  artifacts:
    name: "config-${CI_JOB_STAGE}_${CI_COMMIT_REF_NAME}"
    paths:
      - config/*

Every push in my repo triggers a CI build. The build always fails with the following error message:

...
Skipping Git submodules setup

Error translating config: Error unmarshaling yaml: yaml: unmarshal errors:
  line 3: cannot unmarshal !!str `set -eo...` into common.Common
Error translating config: Error unmarshaling yaml: yaml: unmarshal errors:
  line 3: cannot unmarshal !!str `set -eo...` into common.Common

ERROR: Job failed: exit code 1

But on a local VM I am able to successfully generate the Ignition Config with the same command and config file. How is this possible?

Getting started doc says:

Download the latest version of fcct and the detached signature from the releases page. Verify it with gpg:
gpg --verify <fcct binary> <detached sig>

However:

$ gpg --verify fcct-x86_64-unknown-linux-gnu fcct-x86_64-unknown-linux-gnu.asc 
gpg: no valid OpenPGP data found.
gpg: the signature could not be verified.
Please remember that the signature file (.sig or .asc)
should be the first file given on the command line.

So it seems that those two files should be swapped:

$ gpg --verify fcct-x86_64-unknown-linux-gnu.asc fcct-x86_64-unknown-linux-gnu
gpg: Signature made Thu 11 Jul 2019 12:46:56 AM CEST
gpg:                using RSA key 9CEB8FE6B4F1E9E752F61C82CDDE268EBB729EC7
gpg: Can't check signature: No public key

That works better, but complains about no public key. I'm not very fluent with gpg, so I might be doing something wrong.

fcct removed the networkd section from ct. Also, the network extension is not a valid systemd unit extension.

So what's the correct way to config the network now?

I'm trying to convert FCC like below to Ignition config but getting error:

Error translating config: Error unmarshaling yaml: yaml: line 4: mapping values are not allowed in this context

My configuration file:

variant: fcos
version: 1.0.0
passwd:
  users:
    - name: drakkai
      ssh_authorized_keys:
      - ssh-rsa "<removed>"
      home_dir: /home/drakkai
      groups: 
        - wheel
      shell: /bin/bash

I tried fcct in versions 0.2.0 and 0.1.0 on windows and linux with the same error. So I thought that I wrote something wrong in FCC, but getting the same even if I use someone else configuration file. What can it be caused by?

CLCT supports setting the Afterburn provider on platforms where that needs to be explicitly configured (OpenStack and CloudStack). If we want to continue requiring that bit of configuration, FCCT should learn to generate it automatically from a platform name specified on the command line. That does mean that FCCT output becomes platform-specific, which isn't ideal.

Within the documentation for the configuration spec it calls out "variant" as being a required field. At the same time this is not present in the published documentation (though it is shown in the source code):

With simple examples like this, it would help to explain why this is specific to my use case as well as why a rendered ignition sequence[1] with the same functional content would be denied in use:

[�[0;32m  OK  �[0m] Stopped target �[0;1;39mTimers�[0m.[   17.900163] ignition[589]: failed to fetch config: unsupported config version

[1]: Rendered ignition configuration used with a different ignition based host -

{
  "ignition": {
    "config": {},
    "security": {
      "tls": {}
    },
    "timeouts": {},
    "version": "2.2.0"
  },
  "networkd": {},
  "passwd": {
    "users": [
      {
        "name": "core",
        "sshAuthorizedKeys": [
          "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDVUVfmNA8ybrjfBMJQrJS2stImcGdaiWSAdkCnVi0urbRwChnREhndrCueMyQwUeofzZuyCZAQ323QSAEussO+H5vSgCT+lM93oeN7nays5Y9l3kaj3eJtDzSeJ7e7X0Bo//BO4xevO1PqlfplwPGXJgV61O/iRYP9EtkCXbgjO40IpD3knXw6De++fqSdKe6FPdWkjQTLvR8X558UXf+C4CaP8xMLbVsH+lIWdm4yRRn/RM+qjCUTQDPMPkEZy/gsL0P0HX1Ecx0FcoXSsVcLMed3HIQFXrw+MD2gLbQ3yWvKQO+EvD47TRwlU4WcuM/Ua3cC1R/eK2LlovPih8/pS8aeSQG+BRb/UhTf6gZfDTTUyM/SuYCM9RWMT0SS9SEuyo2q9WrOOQtu6ouLJHyYZMX7jbkuzmAEJDGtvfYzSC6vvHK8rOMdJgMujnqHmV2JoUbubkivE8pcJQYcH1MzCoeNMqzzBHocJCJ/aqDPNbyselB3BeiCjgIIoI6naZBrQCFotD5mH+VC394IL36T1lfgu4gJMaPORqP4GM+DyZMcndz3LSV5zxfVJrVsamGoKAZgNlOVw+AbBf0L/+c7xfeXKT+JT/lVXdSanjhWYCcQla6oHvi+MxZO4lfGjaBXXLssxDf9+O5ikZ/5fMnffog4xJlMte12JN98idKGjw=="
        ]
      },
      {
        "name": "bharrington",
        "sshAuthorizedKeys": [
          "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDVUVfmNA8ybrjfBMJQrJS2stImcGdaiWSAdkCnVi0urbRwChnREhndrCueMyQwUeofzZuyCZAQ323QSAEussO+H5vSgCT+lM93oeN7nays5Y9l3kaj3eJtDzSeJ7e7X0Bo//BO4xevO1PqlfplwPGXJgV61O/iRYP9EtkCXbgjO40IpD3knXw6De++fqSdKe6FPdWkjQTLvR8X558UXf+C4CaP8xMLbVsH+lIWdm4yRRn/RM+qjCUTQDPMPkEZy/gsL0P0HX1Ecx0FcoXSsVcLMed3HIQFXrw+MD2gLbQ3yWvKQO+EvD47TRwlU4WcuM/Ua3cC1R/eK2LlovPih8/pS8aeSQG+BRb/UhTf6gZfDTTUyM/SuYCM9RWMT0SS9SEuyo2q9WrOOQtu6ouLJHyYZMX7jbkuzmAEJDGtvfYzSC6vvHK8rOMdJgMujnqHmV2JoUbubkivE8pcJQYcH1MzCoeNMqzzBHocJCJ/aqDPNbyselB3BeiCjgIIoI6naZBrQCFotD5mH+VC394IL36T1lfgu4gJMaPORqP4GM+DyZMcndz3LSV5zxfVJrVsamGoKAZgNlOVw+AbBf0L/+c7xfeXKT+JT/lVXdSanjhWYCcQla6oHvi+MxZO4lfGjaBXXLssxDf9+O5ikZ/5fMnffog4xJlMte12JN98idKGjw=="
        ]
      }
    ]
  },
  "storage": {},
  "systemd": {}
}

In https://github.com/coreos/fcct/blob/master/docs/configuration-v1_0.md - storage --> directories is it used to only create empty directories? Is there a way to recursively copy a directory say from S3 location

I don't think this requires any code changes, just vendoring an Ignition release containing coreos/ignition#965 and updating the 1.1.0-exp spec.

cc @sohankunkerkar

Hi,
I want to generate an ignition file from this FCC file:

root@ld3955:/home# more config.yml
variant: fcos
version: 1.0.0

hostname: vm191-fcos

passwd:
  users:
    - name: "core"
      groups: [ sudo, docker ]
      ssh_authorized_keys:
        - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAA [...]

network:
  version: 1
  config:
    - type: physical
      name: eth0
      subnets:
        - type: static
          address: 192.168.100.191/24
          gateway: 192.168.100.11
    - type: nameserver
      address:
        - 8.8.8.8

However fcct reports that key "hostname" and "network" are not available.

root@ld3955:/home# fcct --input config.yml
warning at $.hostname, line 4 col 1: Unused key hostname
warning at $.network, line 13 col 1: Unused key network

Question:
How can I configure a static network in ignition file?

This is required as no DHCP service is available.

THX

Add tests to ensure examples in the docs are valid.

Release checklist:

• Write release notes in NEWS. Get them reviewed and merged
  • If doing a branched release, also include a PR to merge the NEWS changes into master
• Ensure your local copy is up to date with master and your working directory is clean
• Ensure you can sign commits and any yubikeys/smartcards are plugged in
• Run ./tag_release.sh <vX.Y.z> <git commit hash>
• Push that tag to GitHub
• Run ./build_releases
• Sign the release artifacts by running

gpg --local-user 0xCDDE268EBB729EC7! --detach-sign --armor <path to artifact>

for each release artifact. Do not try to sign all of them at once by globbing. If you do, gpg will sign the combination of all the release artifacts instead of each one individually.

• Create a draft release on GitHub and upload all the release artifacts and their signatures. Copy and paste the release notes from NEWS here as well.
• Publish the release
• Update the release tag on Quay:
  • Visit the Quay tags page and wait for a versioned tag to appear
  • Click the gear next to the tag, select "Add New Tag", enter release, and confirm

The examples doc links to config spec, but that's a broken link. The correct link should be this one.

For our cgroup v2 strategy ticket, we identified that we need an easy way for a user to be able to set kernel arguments for many reasons, but also to toggle cgroups v2 on and off.

We need the ability for a user to easily configure kernel arguments and also documentation on how to do that.

fcct allows the same file path to be defined multiple times in a storage.files section, and will happily transpile an Ignition file. However, Fedora CoreOS does not accept duplicate file paths in the Ignition file in the storage.files section, and Ignition will fail if the user attempts to use such a file.

fcct should print an error and fail when attempting to use multiple duplicate files.

./fcct-x86_64-unknown-linux-gnu -strict -pretty -input coreos.yml -output ~/coreos.ign

failed to open /home/aj/coreos.ign: open /home/aj/coreos.ign: no such file or directory

and if i try creating a file before hand

./fcct-x86_64-unknown-linux-gnu -strict -pretty -input coreos.yml -output ~/coreos.ign

Failed to write config to /home/aj/coreos.ign: write /home/aj/coreos.ign: bad file descriptor

Release checklist:

• Write release notes in NEWS. Get them reviewed and merged
  • If doing a branched release, also include a PR to merge the NEWS changes into master
• Ensure your local copy is up to date with master and your working directory is clean
• Ensure you can sign commits and any yubikeys/smartcards are plugged in
• Run ./tag_release.sh <vX.Y.z> <git commit hash>
• Push that tag to GitHub
• Run ./build_releases
• Sign the release artifacts by running

gpg --local-user 0xCDDE268EBB729EC7! --detach-sign --armor <path to artifact>

for each release artifact. Do not try to sign all of them at once by globbing. If you do, gpg will sign the combination of all the release artifacts instead of each one individually.

• Create a draft release on GitHub and upload all the release artifacts and their signatures. Copy and paste the release notes from NEWS here as well.
• Publish the release
• Update the release tag on Quay:
  • Visit the Quay tags page and wait for a versioned tag to appear
  • Click the gear next to the tag, select "Add New Tag", enter release, and confirm
• Update Fedora RPM:
  • Create a PR to bump the FCCT spec file in Fedora.
  • Once that PR merges to master, merge master into the other relevant branches (e.g. f31) then push those.
  • On each of those branches (including master) run fedpkg build
  • Once the builds have finished, submit them to Bodhi, filling in:
    • fedora-coreos-config-transpiler for Packages
    • Selecting the build(s) that just completed, except for the Rawhide one (which gets submitted automatically)
    • Copy relevant part of release notes from the GitHub release
    • Leave Update name blank
    • Type, Severity and Suggestion can be left as unspecified unless it is a security release. In that case select security with the appropriate severity.
    • Stable karma and Unstable karma can be set to 2 and -1, respectively.

How can I incorporate a package install (rpm-ostree <pkg>) into config transpiler yaml file?

People,

I get:

# ./fcct/bin/amd64/fcct -pretty -strict -input config_network.fcc 
Error translating config: yaml: unmarshal errors:
  line 9: field networkd not found in type v1_0.Config

What am I missing?

Thanks,
Phil.

I noticed a weird issue with fcct where I can get an invalid ignition config. I copy and pasted some text into an FCC. The text ended up with extra whitespace on the lines. See attached file:
fcc-extra-space.yaml.txt

I then create an ignition config using:

$ fcct --version
Fedora CoreOS Config Transpiler v0.2.0
$ fcct -pretty -strict -input ./fcc-extra-space.yaml.txt -output simple.ign
warning at $.storage.files.0.mode, line 17 col 7: permissions unset, defaulting to 0644
$ ignition-validate --version
Ignition 2.1.1
$ ignition-validate ./simple.ign 
warning at $.storage.files.0.mode, line 18 col 8: permissions unset, defaulting to 0644

So it produced valid ignition config but I noticed that my systemd unit had a bunch of extra whitespace in it. So I trimmed the whitespace from the file and ran it again:

fcc-no-extra-space.yaml.txt

$ fcct -pretty -strict -input ./fcc-no-extra-space.yaml.txt -output simple.ign
warning at $.storage.files.0.mode, line 17 col 7: permissions unset, defaulting to 0644

$ ignition-validate ./simple.ign 
989
error at line 42 col 41: invalid character '\\' after top-level value

The invalid ignition config is here:

{
  "ignition": {
    "config": {
      "replace": {
        "source": null,
        "verification": {}
      }
    },
    "security": {
      "tls": {}
    },
    "timeouts": {},
    "version": "3.0.0"
  },
  "passwd": {},
  "storage": {
    "files": [
      {
        "group": {},
        "path": "/etc/zincati/config.d/90-disable-auto-updates.toml",
        "user": {},
        "contents": {
          "source": "data:,%5Bupdates%5D%0Aenabled%20%3D%20false%0A",
          "verification": {}
        }
      }
    ]
  },
  "systemd": {
    "units": [
      {
        "dropins": [
          {
            "contents": "[Service]\n# Override Execstart in main unit\nExecStart=\n# Add new Execstart with `-` prefix to ignore failure\nExecStart=-/usr/sbin/agetty --autologin core --noclear %I $TERM\nTTYVTDisallocate=no\n",
            "name": "autologin-core.conf"
          }
        ],
        "name": "[email protected]"
      }
    ]
  }
}                                      \n# Override Execstart in main unit                              \nExecStart=                                                     \n# Add new Execstart with `-` prefix to ignore failure          \nExecStart=-/usr/sbin/agetty --autologin core --noclear %I $TERM\nTTYVTDisallocate=no                                            \n",
            "name": "autologin-core.conf"
          }
        ],
        "name": "[email protected]"
      }
    ]
  }
}

You can see there is a whole additional section of random ignition config on the end of the file.

If I delete the simple.ign file before I run it then I get a clean file:

$ rm simple.ign 
$ fcct -pretty -strict -input ./fcc-no-extra-space.yaml.txt -output simple.ign
warning at $.storage.files.0.mode, line 17 col 7: permissions unset, defaulting to 0644

$ ignition-validate ./simple.ign 
warning at $.storage.files.0.mode, line 18 col 8: permissions unset, defaulting to 0644

So there is some issue where the simple.ign file already exists and it's being read/written erroneously.

NOTE: I could not reproduce this with latest master. It seems like the issue was fixed somewhere but it would be nice to know what fixed it.

On macOS 10.14.5, fcct --strict --pretty --input ignition.yaml --output ignition.json returns the error failed to open ignition.json: open ignition.json: no such file or directory

however without --output it works fine:

$ fcct --strict --pretty --input ignition.yaml
{
  "ignition": {
    "config": {
      "replace": {
        "source": null,
        "verification": {}
      }
    },
    "security": {
      "tls": {}
    },
    "timeouts": {},
    "version": "3.0.0"
  },
  "passwd": {
    "users": [
      {
        "name": "core",
        "sshAuthorizedKeys": [
          "ssh-ed25519 AAAA ..."
        ]
      }
    ]
  },
  "storage": {},
  "systemd": {}
}

Similar to the Container Linux validator there should be an host version of FCCT.

Rough proposal:

• A service that wraps FCCT and acts as an HTTP endpoint for translation. Run it in a container for easy hosting.
• Static page with some kind of text box that makes requests to that service to do the translation.
• The page should be able to read URL params (or something similar) to autofill the text box. This enables people to create links to specific configs when reporting bugs and allows the FCCT documentation to have "try it" links for the examples.

https://github.com/coreos/fcct/releases/tag/v0.1.0 comes with pre-built binaries for 4 platforms/architectures, with signatures.

I'd suggest to get rid of all those, and just release the source code.
We can then wire-in a container autobuilder (e.g. on quay.io), and point people to the container image.
Downstream distributions are still encouraged to package this (as deb/rpm/msi/whatever), and build for how many architectures they want.

Hi,

I cannot run the latest release 0.4.0 binary inside an alpine container:

jens:~> docker run --rm -it alpine:3.8 sh
/ # wget -O /usr/local/bin/fcct https://github.com/coreos/fcct/releases/download/v0.4.0/fcct-x86_64-unknown-linux-gnu
Connecting to github.com (140.82.118.4:443)
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (52.216.170.59:443)
fcct                 100% |*******************************************************************************************************************************************************************************************| 10203k  0:00:00 ETA
/ # chmod a+x /usr/local/bin/fcct 
/ # /usr/local/bin/fcct 
sh: /usr/local/bin/fcct: not found

After installing file (using apk --update-cache --upgrade add file) it turns out the 0.4.0 release is a dynamically linked binary, not a static binary.

/usr/local/bin # file fcct
fcct: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, BuildID[sha1]=d9bb68fd48bf269bb0271cfc71545251d8fe1343, for GNU/Linux 3.2.0, stripped

Yet there aren't any linker issues:

/usr/local/bin # ldd fcct 
	/lib64/ld-linux-x86-64.so.2 (0x7f40e5620000)
	libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f40e5620000)
	libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f40e5620000)

Shouldn't the 0.4.0 release also be a statically linked binary? The 0.2.0 release is, and I can run that one just fine.

While reviewing the FCOS docs PR (coreos/fedora-coreos-docs#22), I was using the fcct container to test what happens with invalid FCCT files:

$ cat ignition.fcc
variant: fcos
version: 1.0.0
passwd:
  users:
    - name: core
      groups: [ sudo, wheel, adm, systemd-journal ]
      password_hash: $6$xyz$GeyXohSve5L1Oh.lphSVoNZdCM1k9.qWujjXXSHeboOO0niFbei6Gsyj9E1UAZ/qYb8fDm73pihQn/MoWcW9G1
      ssh_authorized_keys:
        - ssh-rsa AAAAB3NzaC1yc2EA....
storage:
  files:
    - path: /var/helloworld

$ podman run -i --rm quay.io/coreos/fcct:v0.2.0 -pretty -strict < ignition.fcc > ignition.ign
$ jq < ignition.ign 
parse error: Invalid numeric literal at line 1, column 8
$ head ignition.ign 
warning at $.storage.files.0.mode, line 12 col 7: permissions unset, defaulting to 0644

{
  "ignition": {
    "config": {
      "replace": {
        "source": null,
        "verification": {}
      }
    },

If I remove that first line, the JSON appears valid.

Not sure if this is something we can improve in the container model, but noting for posterity.

To make further bug reporting and debugging easier for both parties, fcct should implement --version to print its version. Especially since the binaries need to be manually downloaded from github release page at the moment, with no external release versioning.

Release checklist:

• Write release notes in NEWS. Get them reviewed and merged
  • [ ] If doing a branched release, also include a PR to merge the NEWS changes into master
• Ensure your local copy is up to date with master and your working directory is clean
• Ensure you can sign commits and any yubikeys/smartcards are plugged in
• Run ./tag_release.sh <vX.Y.z> <git commit hash>
• Push that tag to Github
• Run ./build_releases
• Sign the release artifacts by running

gpg --local-user 0xCDDE268EBB729EC7! --detach-sign --armor <path to artifact>

for each release artifact. Do not try to sign all of them at once by globbing. If you do, gpg will sign the combination of all the release artifacts instead of each one individually.

• Create a draft release on Github and upload all the release artifacts and their signatures. Copy and paste the release notes from NEWS here as well.
• Publish the release

As a user it would be nice to have my systemd mount unit automatically generated for me without having to craft one myself. We could probably craft something that works for most use cases and allow the user to disable autogeneration and provide their own if they want to.

In my case I was hitting the nuance where I named my mount unit docker.mount where I needed to name it var-lib-docker.mount

I was adapting this example from the docs: https://github.com/coreos/fcct/blob/f9513c5b8e0bf03262bfe1fb3ab8c1ed8e5b55f1/docs/examples.md#filesystems-and-partitions and simply replaced var.mount with docker.mount.

fcct appears to output an unneccessary newline to STDOUT even when -output is being used.

Add CI to run ./test on PRs

When a config yaml file is fed to fcct tool such as:
~/bin/fcct -input test2.yaml -output test2.json -pretty

a ignition json file is created. Now, modify the YAML file, say, add an entry to storage or change group/user info in passwd. Rerun the same above command. Check the json file - it is incorrect. The tool tries to in-place modify the JSON instance rather overwrite the file. I would expect overwrite not update.

The docs for using the container require the user to know the current FCCT version, which is inconvenient.

Create a release tag, similar to the one for coreos-installer. Recommend that users use that instead, and update the release process to move the tag.

@jlebon @lucab Any objections?

Please can someone put a note in the old Repo:
https://github.com/coreos/container-linux-config-transpiler

All the old blogs and links state that people should use the Config-transpiler, and it points to the old CoreOS CT tool. This is now over a year with ZERO commits, and no ISSUE tabs. This causes people to abandon the search for CT. Please talk about the relationship at least between the two projects.

What is the correct way to use append to a file in YAML

  files:
    - path: /etc/sudoers
      overwrite: false
      append: 
        contents:
          inline: core	ALL=(ALL)	NOPASSWD: /usr/bin/docker

The above will result in an error with fcct:

Error translating config: Error unmarshaling yaml: yaml: line 9: mapping values are not allowed in this context

Release checklist:

• Write release notes in NEWS. Get them reviewed and merged
  • If doing a branched release, also include a PR to merge the NEWS changes into master
• Ensure your local copy is up to date with master and your working directory is clean
• Ensure you can sign commits and any yubikeys/smartcards are plugged in
• Run ./tag_release.sh <vX.Y.z> <git commit hash>
• Push that tag to Github
• Run ./build_releases
• Sign the release artifacts by running

gpg --local-user 0xCDDE268EBB729EC7! --detach-sign --armor <path to artifact>

for each release artifact. Do not try to sign all of them at once by globbing. If you do, gpg will sign the combination of all the release artifacts instead of each one individually.

• Create a draft release on Github and upload all the release artifacts and their signatures. Copy and paste the release notes from NEWS here as well.
• Publish the release

When using fcct in a GitLab pipeline to transpile the ignition config, you require a shell.
It would be nice if the default fcct image had a shell, or if another image would be provided with a shell.

Instead of downloading binaries from github and manually checking for updates, it would be much more convenient for Fedora users to install fcct from a standard repository. Is that planned? Thanks.

Using this https://docs.fedoraproject.org/en-US/fedora-coreos/static-ip-config/ as an example where the mode of the file is set to 0600, the resulting mode after translation is "mode": 384

It seems as if there's a conversion to base 10 during translation that results in the value 384.

-Version output
Fedora CoreOS Config Transpiler 0.4.0
Tried both linux exe in Ubuntu WSL and native windows exe.

I propose we merge in the functionality from https://github.com/ashcrow/filetranspiler into this project. Strawman:

fcct --units myunits  --directory myconf --input foo.fcct > foo.ign

--directory would do what filetranspiler does today.
--units would look like:

myunits/
  foo.service
  bar.service

fcct in the v1_1-exp version does not know about the mountOptions option:

mountOptions (list of strings): any special options to be passed to the mount command.

It would be nice if we don't put extraneous sections in our Ignition config that might cause confusion.

For example:

[dustymabe@media tmp]$ cat foo.yaml
variant: fcos
version: 1.0.0
passwd:
  users:
    - name: dusty
      ssh_authorized_keys:
        - ssh-rsa AAAAxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx [email protected]
[dustymabe@media tmp]$ 
[dustymabe@media tmp]$ cat foo.yaml | fcct --pretty
{
  "ignition": {
    "config": {
      "replace": {
        "source": null,
        "verification": {}
      }
    },
    "security": {
      "tls": {}
    },
    "timeouts": {},
    "version": "3.0.0"
  },
  "passwd": {
    "users": [
      {
        "name": "dusty",
        "sshAuthorizedKeys": [
          "ssh-rsa AAAAxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx [email protected]"
        ]
      }
    ]
  },
  "storage": {},
  "systemd": {}
}
[dustymabe@media tmp]$ rpm -q fedora-coreos-config-transpiler
fedora-coreos-config-transpiler-0.4.0-3.fc31.x86_64

In the above config I get a lot of extraneous sections that aren't required. A user came to me today and asked for help. I validated his Ignition config using ignition-validate, but then I saw the config/replace stuff at the top of his config and since I haven't used that particular part of Ignition much I thought maybe he was replacing his config with nothing (null). He mentioned he was just using fcct, which I confirmed.

So my proposal is that we remove unneeded sections from the generated Ignition config from fcct. For example the above would turn into:

[dustymabe@media tmp]$ cat minimal.json 
{
  "ignition": {
    "version": "3.0.0"
  },
  "passwd": {
    "users": [
      {
        "name": "dusty",
        "sshAuthorizedKeys": [
          "ssh-rsa AAAAxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx [email protected]"
        ]
      }
    ]
  }
}

We piled up a bunch of changes and bugfixes since the last release, so it would be good to tag a new release soon-ish: v0.2.0...master

One thing I think would be interesting is sugar like this:

provisioncontainer: quay.io/myuser/setup@sha256:shashasha

This would expand into a systemd unit which ran podman under ConditionFirstBoot=yes something like this:
podman run --rm quay.io/myuser/setup --privileged -v /:/host

And we could give some examples of using "first boot containers" like this that can do completely arbitrary things, say use an Ansible playbook pointing at the host, or use Ruby scripts - whatever.

This isn't hard to write manually but it's also not entirely obvious, and we'd be encouraging the pattern of containerization even for early boot.

$ ./fcct-x86_64-unknown-linux-gnu -version
Fedora CoreOS Config Transpiler v0.2.0
$ cat example.fcct | ./fcct-x86_64-unknown-linux-gnu > example.ign 
$ echo $?
0

Usually I would then take example.ign and feed it into an instance start, but it's actually invalid json because the warnings got put in the file:

$ cat example.ign | head -n 2
warning at $.systemd.units.1.name, line 41 col 13: unit "docker.mount" is enabled, but has no install section so enable does nothing
warning at $.systemd.units.2.name, line 49 col 13: unit "dockerdata.mount" is enabled, but has no install section so enable does nothing

Should we print those to stderr?

Spec v1.0 is stable and thus frozen, meaning we can't add features to it. It also does not expose things in the Ignition spec 3.1.0-experimental spec.

Create v1.1.0-experimental which targets Ignition spec v3.1.0-experimental. Include all the new features in that Ignition spec and allow for new ones.

The configuration spec shows all supported keys in CamelCase format, but the examples document uses underscores. For example, the examples show:

ssh_authorized_keys
home_dir
no_create_home

and the spec shows:

sshAuthorizedKeys
homeDir
noCreateHome

It seems that the examples-style naming is the correct one, because fcct fails when converting an input file with e.g. sshAuthorizedKeys in it.

This is very confusing. Can you make spec compliant with the implementation or vice versa? :-)

My .yaml configuration file is named tth.fcc and has been written using visual studio and the red hat yaml plugin. When saving the file, visual studio does not split any error. Everything is made from my home directory. I use fcct v 0.3.

% ls -al 
....
tth.yaml
% mv tth.yaml tth.fcc
% ls -al 
...
tth.fcc
%podman pull quay.io/coreos/fcct:v0.3.0
%podman run -i --rm quay.io/coreos/fcct:v0.3.0 -pretty -strict tth.fcc transpiled_config.ign
_

Nothing happens. The command outputs nothing and after a few minutes, I have to ctr+c to quit.
Now:

%podman run --rm -v /var/home/gabx/tth.fcc:/config.fcc:z quay.io/coreos/fcct:v0.3.0 -pretty -strict -input /config.fcc > transpiled_config.ign
%ls -al
....
transpiled_config.ign
tth.fcc

The command succeeded and I have a proper ignition file.
No idea why one succeed and not the other one.

Release checklist:

• Write release notes in NEWS. Get them reviewed and merged
  • If doing a branched release, also include a PR to merge the NEWS changes into master
• Ensure your local copy is up to date with master and your working directory is clean
• Ensure you can sign commits and any yubikeys/smartcards are plugged in
• Run ./tag_release.sh <vX.Y.z> <git commit hash>
• Push that tag to Github
• Run ./build_releases
• Sign the release artifacts by running

gpg --local-user 0xCDDE268EBB729EC7! --detach-sign --armor <path to artifact>

for each release artifact. Do not try to sign all of them at once by globbing. If you do, gpg will sign the combination of all the release artifacts instead of each one individually.

• Create a draft release on Github and upload all the release artifacts and their signatures. Copy and paste the release notes from NEWS here as well.
• Publish the release

I am currently in the process of porting a CoreOS Container Linux Ignition config to Fedora CoreOS 1.0 config.

One major issue I am running into is the following:

In the old setup, the yaml file contains no sensitive content and is tracked publicly, and some sensitive content is pulled in when it is transpiled, using the local option of file contents, like following:

storage:
  files:
    - path: /root/.ssh/id_rsa
      mode: 0600
      contents:
        local: keys/id_rsa_deployment

How can I achieve this with the new config format?

It seems like the local option is gone, and the source option seems not to apply, as it is only applied when the config is applied, and not during the transpilation process.