gabriella439 / terraform-nixos-ng

Terraform modules for NixOS and spiritual successor to the terraform-nixos project

terraform-nixos-ng

This repository provides two Terraform modules:

• ./nixos - A Terraform module for auto-deploying a NixOS configuration to a target host

• ./ami - A Terraform module for computing the correct NixOS AMI to use

Example usage

You can find an example Terraform project using these modules in the ./example directory.

Motivation

The main reason this repository exists is because the terraform-nixos repository is abandoned. I wanted to simplify the upstream project in two major ways:

• I wanted to simplify the deployment logic with nixos-rebuild

  … using the trick documented in this blog post

• I wanted to simplify the AMI module

  … by not requiring the list to be manually updated like this

However, since the upstream project doesn't appear to be maintained I created my own opinionated fork that is simpler than the original. The main omissions compared to the original are:

• No support for secrets management

  However, you can use this in conjunction with sops-nix if you need secrets management

• No support for GCE images (yet)

  … mainly because I don't use Google Compute Engine, but I'd accept a PR to add support following the same pattern as the ami module.

terraform-nixos-ng's Issues

Incorporating tf data into the NixOS configuration

Currently, if you want to automatically incorporate some information from the tf state into your system configuration (e.g. make an SSH key owned by tf that can connect to the machine), you have to either write your flake to hard-code some path that will be generated by tf, or use tf's local_file machinery to write a flake to local (presumably gitignored) disk and use that as your flakeref.

I can think of two possible paths to improve this:

1. Make nixos-rebuild aware of some kind of command-line configurable variant of a NixOS config at the flakeref. The simplest thing might be to just allow nixosConfigurations.foo to be a function which is called using the typical --arg/--argstr mechanism (maybe augmented with --argflakeref 😅 ).
2. Add some parameters to this tf module to allow for inline specification of a flake with tf interpolation, which under the hood manages the temporary flake file creation for you.

I like 1 more, but it's obviously out of scope for this project and I suspect there will be upstream resistance. Would you be interested in 2 or some other in-scope solution?

Consider allowing pre-built NixOS images via store paths, instead of a flake/default.nix

The interaction between Nix and Terraform is undesirable. The nix builds only happen at apply time for any sufficiently complicated setup (i.e. most production usage). Imagine this simplified setup (pseudo code):

terraform:

resource r route53 { ... }

module box nixos-deploy {
  src = ./box.nix
  inputs = {
    host = r.hostname
  }
}

box.nix:

{ host }: { ... }

How do you build that nix at plan time?
The function can't get applied until terraform understands what the hostname is, which only happens at apply time.
This leads to a rather crappy user experience where your PR passes CI, then dies at apply time.

If we instead allow the terraform to take an already-built image, we can circumvent this issue. For anyone who doesn't care they could still use the current machinery.

nix path-info output schema has changed, causing instantiate.sh to fail on certain architectures

The old output schema seemed to be an array, which matches what instantiate.sh assumes in its jq invokation. However with newer nix versions the schema seems to have changed to an object of paths instead...

e.g. using nix 2.18.1

[
  {
    "path": "/nix/foo",
    ...
  }
]

using nix 2.19.2

{
  "/nix/foo": {
    ...
  }
}

causing the jq invokation to fail.

This can be solved for some versions by changing the jq invokation to jq '{ "path": keys[] } instead, but then it will fail for the older output format.

One hack could be to try the first jq invocation, and fall back to the second one on failure. But maybe a better option would be to change to something with a stable output format.

I can make a pull request with the hacky option if needed.

consider terranix and nixos-generators

https://github.com/dzmitry-lahoda/hcl-nix-for-web3/ - see how I build vm image, and feed image to TF, generate TF json via terranix, and apply (it also has Google examle)

flake.nix in root and this https://github.com/dzmitry-lahoda/hcl-nix-for-web3/blob/master/terranix/05.nix

also
terranix/terranix#7

terranix/terranix#45

https://discuss.hashicorp.com/t/configure-both-local-development-and-production-environments-with-terraform/25061/6?u=dzmitry-lahoda

https://github.com/arcnmx/tf-nix

sorry for chunked comments, hope you will be able to wire from reference

==============

Why Terraform?

It is a well documented versatile cloud configuration tool for 5+ of clouds and 100+ of S/I/F/P-aaSs. It consumes a JSON file describing desired cloud configuration. Its internal workings are identical to what NixOps is doing.

Terraform is a cloud lingua franca well known and liked by developers, architects, SRE and ops.

Terraranix is a thin layer to generate JSON from Nix and run validation routines.

Terraform with its state and configuration patterns for cloud is 1 to 1 what Nixos modules did to local machines. Cloud is just a machine from a Terraform point of view.

Why Nix?

Nix is the biggest and fastest growing software artifact, configuration and package management ecosystem built on Blockchain/Git like foundations.

Cloud VM (nixos-generators), local Nixos(VM or bare metal), local Linux(home-manager) and local containers share the same configuration language and patterns. That allows the machine once specified by developers to be tuned to any environment it runs.

Nix ecosystem is idempotent, immutable, reproducible, deterministics, rollbackable and traceable out of the box. Which is desired property by developers, architects, SRE and ops.

How to Nix?

Nix flake to be provided which configures Google Cloud basic, Terranix and VM image. SRE, CI and developers will provide their own env arguments and TF variables to layer.

It will take several lines to run our staging environment or play with an ad hoc manual environment for testing.

regarding terraform-nixos

Hi @Gabriella439

Just wanted to point out that terraform-nixos has been moved to nix-community and has 2 additional maintainers (including myself).

Maybe we could look into (eventually) merging your work back into it? If that would imply some breaking changes we could consider a major version bump for terraform-nixos.

@thufschmitt FYI