gchq / nix-bootstrap

Easily generate reproducible infrastructure

nix-bootstrap

This project aims to simplify generating the infrastructure for common types of projects.

Features

nix-bootstrap provides development infrastructure and pre-commit hooks as well as some production build configurations.

Expand feature tables

Existing features and toolchains supported have been decided based on the needs of the internal developer community within GCHQ.

That said, if there's a toolchain/feature set you'd like that we don't support, feel free to contribute it!

Key

The following symbols are used throughout this page to indicate support status:

Symbol	Meaning
✅	Fully Supported
🟠	Partially Supported
🕓	Support Planned
❌	Not Supported

Toolchain Support

Toolchain\\Feature	Development Environment	VSCode DevContainer	Gitignore	Pre-Commit Hooks1	Reproducible Production Builds
Elm	✅ Either as a standalone app or as part of a Node project with Parcel.	✅ With the Elm extension installed.	✅	✅	❌
Golang	✅	✅ With the official Go extension installed.	✅	✅	🟠 Support is currently considered experimental.
Haskell	✅ Either as a simple project or just a place to run a repl.	✅ With the haskell.haskell extension installed.	✅	✅	✅
Java	✅ With maven, google-java-format, and optionally minishift.	✅ With the official Java, and optionally the Lombok, extensions installed.	✅	✅	🟠 Only supports Spring applications. Support is currently considered experimental.
Minimal (With no project-specific tooling)	✅	✅	✅	🟠	❌
NodeJS	✅ With latest stable node, AWS CLI, and optionally PNPm/Yarn.	✅	✅	✅	❌
Python	✅ With Python version 3.9	✅ With the official Python VSCode extension.	✅	🟠	❌
Rust	✅	✅ With the official rust-analyzer extension.	✅	✅	✅

¹ Marked as fully supported if any non-nix pre-commit hooks are added. See the Pre-Commit Hooks table below for details.

Pre-Commit Hooks

Toolchain\\Hook Type	Nix Formatting2	Formatters	Linters	Testing
Elm	✅	✅ elm-format	✅ elm-review	❌
Golang	✅	✅ go-fmt	❌	✅ go test
Haskell	✅	✅ ormolu	✅ hlint	❌
Java	✅	✅ google-java-format	❌	❌
Minimal	✅	❌	❌	❌
NodeJS	✅	✅ prettier	❌	❌
Python	✅	❌	❌	❌
Rust	✅	✅ rustfmt	✅ cargo check + clippy	❌

² alejandra is set up to format nix files.

Continuous Integration

nix-bootstrap provides configurations for GitLab CI out-of-the-box.

Scenario\\CI Job	Build Development Environment	Run Pre-Commit Hooks	Build For Production
Default	✅	❌	❌
Pre-Commit Hooks Enabled	✅	✅	❌
Pre-Commit Hooks Disabled, Production Build Configured	✅	❌	✅
Pre-Commit Hooks Enabled, Production Build Configured	✅	✅	✅

For Elm projects, it provides an additional CI job to build the Elm site.

Installation & Usage

Expand installation instructions

The simplest way to get started is to run the following script:

sh <(curl -L https://raw.githubusercontent.com/gchq/nix-bootstrap/main/scripts/run.sh)

Or to run nix-bootstrap with flakes enabled:

sh <(curl -L https://raw.githubusercontent.com/gchq/nix-bootstrap/main/scripts/run.sh) --experimental-use-flakes

Alternatively, you can always clone the repo and build nix-bootstrap using nix build. The built binary will then be available in ./result/bin/nix-bootstrap.

Contributing

We welcome contributions to the project. Detailed information on our ways of working can be found in CONTRIBUTING.md.

In brief:

• Sign the GCHQ Contributor Licence Agreement.
• Submit a pull request.

Development Environment Setup

Expand development environment setup instructions

This section is for people who want to contribute to the nix-bootstrap tool.

Environment Setup

1. Install nix by running the following command:

   sh <(curl -L https://nixos.org/nix/install) --daemon

2. Enable Nix Flakes.

3. Install direnv >=2.23.0, by first installing the direnv package for your system.

   • You can check your current version by running direnv version
   • On the latest Ubuntu, this is available using apt-get
   • If you can't install it through your OS's package manager, download a release from the GitHub releases page and put it somewhere on your $PATH.

4. Hook direnv into your shell

5. Clone the nix-bootstrap repository

6. Run direnv allow in the cloned directory

7. Run setUpHaskellLanguageServer to ensure cabal and the HLS build correctly

Building nix-bootstrap with Cabal

You'll probably want to run your builds with Cabal during development as it builds incrementally by module.

1. After a fresh clone, run setUpHaskellLanguageServer. This gets around a bug in Cabal.
2. During development, run builds with cabal build -O0. The -O0 (optimisation zero) flag speeds up the compilation process at the expense of not optimising the code for quicker run times.
3. You can run tests with cabal test --test-show-details=streaming -O0

Building nix-bootstrap with Nix

Nix should be used for production builds, as it enables several additional checks.

Run nix build. This will produce a production binary at result/bin/nix-bootstrap.

Note: ^ This command has a space in it, not a dash - this is a change since v1 of the nix CLI.

Structure of the nix-bootstrap Repo

The nix-bootstrap source modules are grouped as follows (those without .hs extensions are directories):

src/
├── Bootstrap
│   ├── Cli.hs                    - Handling of nix-bootstrap's CLI options and producing its RunConfig from them
│   ├── Data                      - Data structures, including files we bootstrap
│   │   └── Bootstrappable        - All the files we bootstrap
│   │       ├── Go                - Files we bootstrap which are specific to Go projects
│   │       └── Python            - Files we bootstrap which are specific to Python projects
│   ├── Error.hs                  - Error handling helpers
│   ├── Monad.hs                  - Exposes MonadBootstrap, a collection of common constraints required by nix-bootstrap functions
│   ├── Niv.hs                    - Management of dependencies with Niv
│   ├── Nix                       - Handling of writing and structuring Nix Code
│   │   └── Expr                  - A Nix AST and some common expressions formed with it
│   │       └── ReproducibleBuild - Expressions specific to producing reproducible builds
│   ├── State.hs                  - Management of state used for user interactions
│   ├── Terminal                  - Additional helpers for things displayed in the CLI
│   ├── Terminal.hs               - Handles user interactions through the terminal
│   └── Unix.hs                   - Provides an interface for interacting with other CLIs
├── Bootstrap.hs                  - The main nix-bootstrap entrypoint
└── Prelude.hs                    - A custom prelude

Common Scenarios

Adding support for a new toolchain

1. Add a new ProjectSuperType and corresponding ProjectType to the Bootstrap.Data.ProjectType module
2. Run cabal build -O0, handling incomplete case statements until all of the warnings are fixed

Adding a new file to the list of files to be bootstrapped

1. Add a new module for the file under Bootstrap.Data.Bootstrappable

2. In the new module:

   1. Create a datatype for the file
   2. Make that datatype an instance of Bootstrap.Data.Bootstrappable.Bootstrappable
   3. Add a function which will return Maybe a where a is your datatype, depending on whether it is necessary to bootstrap the file under the given circumstances
   4. Name the function in 2.3 using a suffix of "for" according to the conventions - see Bootstrap.Data.Bootstrappable.VSCodeSettings vsCodeSettingsFor for an example.

3. Call the function in 2.3 in the mkInitialBuildPlanMap function in the Bootstrap module

License

nix-bootstrap is released under the Apache 2.0 Licence and is covered by Crown Copyright.

nix-bootstrap's Issues

Infer project name from directory name

Other tools such as npm init infer a default project name from the name of the current directory. Since we expect users to run in an empty directory created for their project, this seems sensible for us to do the same.

elm-review hook doesn't work in CI

The hook for elm-review doesn't work in CI as elm-json isn't available in the sandboxed environment mandated by nix build .\#runChecks doesn't contain it.

Find a way around this in a pure context.

Incorrect command in Elm CI

In the build site stage of generated CI for Elm projects, pnpm --frozen-lockfile should instead be pnpm install --frozen-lockfile.

Bootstrapped environments don't work with nix-env selector VSCode extension

The arrterian.nix-env-selector VSCode extension (bootstrapped) requires a shell.nix to load the environment's tools, but none is created for flake projects.

For flake projects, a shell.nix should be bootstrapped using edolstra/flake-compat with an appropriate comment to explain its presence.