[question] Change Ignition Configuration? about fedora-coreos-proxmox

[TheAlgorythm]

Thank you for your patience and help 🙏. I've learned a lot about CoreOS and Proxmox.

I don't see the solution for repeatable provisioning of many VMs in the (original) project. But that is no problem, as I found/programmed a solution for my use-case.

[jimlee2048]

Sorry for response your issue slowly. Although I'm not the developer who write this script, also lacking knowledge about shell scripting and ignite configuration, I would like to try my best to answer your question.

What is the best process to to change the ignite configuration in order to change something like adding a podman service?

This project using butane to generate ignite config. It first generate a butane config according you parameter provided in template_deploy.conf, then using this generated butane config to get a ignite config.

The butane config is combined from these two script. So using these scripts, if you want to change the ignite config used by CoreOS VM, you may need to modify them manually. Note that you should be careful about indentation.

1. hook-fcos.sh:
   https://github.com/jimlee2002/fedora-coreos-proxmox/blob/2db993cc46245fc6f8cb1425c9e730f6c35ffdcf/hook-fcos.sh#L68-L145
   It's generate the leading part of the butane config, the output from this script structured like:

variant: fcos
version: 1.1.0
passwd:
  users:
    (...)
storage:
  files:
    (...)

2. fcos-base-tmplt.yaml:
   https://github.com/jimlee2002/fedora-coreos-proxmox/blob/master/fcos-base-tmplt.yaml
   It define the last part of the butane config. The config writing here would be directly append to the first part generated from hook-fcos.sh, which mean the butane config you finally get look like this:

variant: fcos
version: 1.1.0
passwd:
  users:
    (...)
storage:
  files:
    (...) # some of them already define in `hook-fcos.sh`
    (...) # the other part would be append from `fcos-base-tmplt.yaml`
(...) # the other part you may add in `fcos-base-tmplt.yaml`

If I have multiple different VMs, should I create a template per use-case?

I think it depends on your personal needs, like if you need some special templates to pre init some services not just using a universal template.

[TheAlgorythm]

Thank you for the detailed response.
I think my question wasn't clear.

If I want to run e.g. 10 VMs, each with different services, I have to create 10 templates, or is there a simpler solution to have a unique butane file for each VM.

[TheAlgorythm]

Maybe a different ignition file (generated separately) can be used.

https://github.com/jimlee2002/fedora-coreos-proxmox/blob/2db993cc46245fc6f8cb1425c9e730f6c35ffdcf/hook-fcos.sh#L170
sets the QEMU argument fw_cfg and if I would overwrite the path per VM everything should work. Or has this some disadvantages I overlooked?

[jimlee2048]

If I want to run e.g. 10 VMs, each with different services, I have to create 10 templates, or is there a simpler solution to have a unique butane file for each VM.

It's no need to create a template match a vm perfectly one by one, I think you may misunderstand the usage of VM Templates.

In my opinion, the VM Templates generated is used as a universal "template", which can be cloned to create VMs quickly.

So if you want to creat multi VMs all running CoreOS, you just need to use this script to creat one universal VM Template of CoreOS, then use this template to create as many VMs running CoreOS as you want. You can then get into each VM and configure different services for them separately, by pre-written scripts or just manually.

sets the QEMU argument fw_cfg and if I would overwrite the path per VM everything should work. Or has this some disadvantages I overlooked?

It seem to be a place to modify if you just want to load your pre-written ignition config, but note that override the path to your ignition config directly would make cloud-init support provided by this project not working, which is the major feature the generator scripts do.

[TheAlgorythm]

It's no need to create a template match a vm perfectly one by one

I know it would be a misuse of templates, so it was just an idea for a workaround.

You can then get into each VM and configure different services for them separately, by pre-written scripts or just manually.

But I thought, that the ignition file is immutable from the perspective of inside the VM. Furthermore if I understand it correctly, everything is lost when a reboot happens as it isn't part of the ignition file.

[TheAlgorythm]

I've changed the hook-fcos.sh to integrate a custom VM specific configuration. Maybe this is something for a PR.
The current state is probably not perfect and more work has to be done, which I am willing to do.
Or do I misunderstand the project and the effort is not needed?

[jimlee2048]

But I thought, that the ignition file is immutable from the perspective of inside the VM. Furthermore if I understand it correctly, everything is lost when a reboot happens as it isn't part of the ignition file.

No. Ignition file won't fully define and control CoreOS. It just a "provisioning tool" same as Cloud-Init, take effect only the first time boot of VM.
Furthermore, what this project mainly do is act as a converter between Iginition and Cloud-Init, allow you pre-init CoreOS throught Proxmox Cloud-Init interface.

You may want to check these documents below:

https://coreos.github.io/ignition/rationale/#ignition-runs-only-on-the-first-boot

https://wiki.geco-it.net/public:pve_fcos#operation_diagram

[jimlee2048]

I've changed the hook-fcos.sh to integrate a custom VM specific configuration. Maybe this is something for a PR. The current state is probably not perfect and more work has to be done, which I am willing to do. Or do I misunderstand the project and the effort is not needed?

I'm not the original developer, and this fork repo is created mainly for self-use, trying to fix some issues when woking in recent release of CoreOS, also add some features to meet my personal requirement. So if you want to share you modification, I would rather suggest you creat your own fork repo. Sorry about that.😞