Coder Social home page Coder Social logo

determinatesystems / nix-installer Goto Github PK

View Code? Open in Web Editor NEW
1.6K 13.0 44.0 1.67 MB

Install Nix and flakes with the fast and reliable Determinate Nix Installer, with over 2 million installs.

License: GNU Lesser General Public License v2.1

Nix 5.79% Shell 3.52% Rust 90.18% Dockerfile 0.10% PowerShell 0.42%
installer nix docker linux macos podman wsl wsl2 fedora ostree

nix-installer's Introduction

The Determinate Nix Installer

Crates.io Docs.rs

A fast, friendly, and reliable tool to help you use Nix with Flakes everywhere.

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

The nix-installer has successfully completed over 2,000,000 installs in a number of environments, including Github Actions and GitLab:

Platform Multi User root only Maturity
Linux (x86_64 & aarch64) ✓ (via systemd) Stable
MacOS (x86_64 & aarch64) Stable (See note)
Valve Steam Deck (SteamOS) Stable
WSL2 (x86_64 & aarch64) ✓ (via systemd) Stable
Podman Linux Containers ✓ (via systemd) Stable
Docker Containers Stable
Linux (i686) ✓ (via systemd) Unstable

Note On MacOS only, removing users and/or groups may fail if there are no users who are logged in graphically.

Usage

Install Nix with the default planner and options:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

Or, to download a platform specific Installer binary yourself:

$ curl -sL -o nix-installer https://install.determinate.systems/nix/nix-installer-x86_64-linux
$ chmod +x nix-installer
$ ./nix-installer

nix-installer installs Nix by following a plan made by a planner. Review the available planners:

$ ./nix-installer install --help
Execute an install (possibly using an existing plan)

To pass custom options, select a planner, for example `nix-installer install linux-multi --help`

Usage: nix-installer install [OPTIONS] [PLAN]
       nix-installer install <COMMAND>

Commands:
  linux
          A planner for Linux installs
  steam-deck
          A planner suitable for the Valve Steam Deck running SteamOS
  help
          Print this message or the help of the given subcommand(s)
# ...

Planners have their own options and defaults, sharing most of them in common:

$ ./nix-installer install linux --help
A planner for Linux installs

Usage: nix-installer install linux [OPTIONS]

Options:
# ...
      --nix-build-group-name <NIX_BUILD_GROUP_NAME>
          The Nix build group name

          [env: NIX_INSTALLER_NIX_BUILD_GROUP_NAME=]
          [default: nixbld]

      --nix-build-group-id <NIX_BUILD_GROUP_ID>
          The Nix build group GID

          [env: NIX_INSTALLER_NIX_BUILD_GROUP_ID=]
          [default: 3000]
# ...

Planners can be configured via environment variable or command arguments:

$ curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | NIX_BUILD_GROUP_NAME=nixbuilder sh -s -- install linux-multi --nix-build-group-id 4000
# Or...
$ NIX_BUILD_GROUP_NAME=nixbuilder ./nix-installer install linux-multi --nix-build-group-id 4000

Upgrading Nix

You can upgrade Nix to our currently recommended version of Nix by running:

sudo -i nix upgrade-nix

Alternatively, you can uninstall and reinstall with a different version of the nix-installer.

Uninstalling

You can remove a nix-installer-installed Nix by running

/nix/nix-installer uninstall

As a Github Action

You can use the nix-installer-action Github Action like so:

on:
  pull_request:
  push:
    branches: [main]

jobs:
  lints:
    name: Build
    runs-on: ubuntu-22.04
    steps:
    - uses: actions/checkout@v3
    - name: Install Nix
      uses: DeterminateSystems/nix-installer-action@main
    - name: Run `nix build`
      run: nix build .

On GitLab

GitLab CI runners are typically Docker based and run as the root user. This means systemd is not present, so the --init none option needs to be passed to the Linux planner.

On the default GitLab.com runners, nix can be installed and used like so:

test:
  script:
  - curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux --no-confirm --init none
  - . /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh
  - nix run nixpkgs#hello
  - nix profile install nixpkgs#hello
  - hello

If you are using different runners, the above example may need to be adjusted.

Without systemd (Linux only)

Warning When --init none is used, only root or users who can elevate to root privileges can run Nix:

sudo -i nix run nixpkgs#hello

If you don't use systemd, you can still install Nix by explicitly specifying the linux plan and --init none:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux --init none

In a container

In Docker/Podman containers or WSL2 instances where an init (like systemd) is not present, pass --init none.

For containers (without an init):

Warning When --init none is used, only root or users who can elevate to root privileges can run Nix:

sudo -i nix run nixpkgs#hello
# Dockerfile
FROM ubuntu:latest
RUN apt update -y
RUN apt install curl -y
RUN curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux \
  --extra-conf "sandbox = false" \
  --init none \
  --no-confirm
ENV PATH="${PATH}:/nix/var/nix/profiles/default/bin"
RUN nix run nixpkgs#hello
docker build -t ubuntu-with-nix .
docker run --rm -ti ubuntu-with-nix
docker rmi ubuntu-with-nix
# or
podman build -t ubuntu-with-nix .
podman run --rm -ti ubuntu-with-nix
podman rmi ubuntu-with-nix

For containers with a systemd init:

# Dockerfile
FROM ubuntu:latest
RUN apt update -y
RUN apt install curl systemd -y
RUN curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux \
  --extra-conf "sandbox = false" \
  --no-start-daemon \
  --no-confirm
ENV PATH="${PATH}:/nix/var/nix/profiles/default/bin"
RUN nix run nixpkgs#hello
CMD [ "/bin/systemd" ]
podman build -t ubuntu-systemd-with-nix .
IMAGE=$(podman create ubuntu-systemd-with-nix)
CONTAINER=$(podman start $IMAGE)
podman exec -ti $CONTAINER /bin/bash
podman rm -f $CONTAINER
podman rmi $IMAGE

On some container tools, such as docker, sandbox = false can be omitted. Omitting it will negatively impact compatibility with container tools like podman.

In WSL2

We strongly recommend enabling systemd, then installing Nix as normal:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

If WSLg is enabled, you can do things like open a Linux Firefox from Windows on Powershell:

wsl nix run nixpkgs#firefox

To use some OpenGL applications, you can use nixGL (note that some applications, such as blender, may not work):

wsl nix run --impure github:guibou/nixGL nix run nixpkgs#obs-studio

If enabling systemd is not an option, pass --init none at the end of the command:

Warning When --init none is used, only root or users who can elevate to root privileges can run Nix:

sudo -i nix run nixpkgs#hello
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux --init none

Skip confirmation

If you'd like to bypass the confirmation step, you can apply the --no-confirm flag:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install --no-confirm

This is especially useful when using the installer in non-interactive scripts.

Quirks

While nix-installer tries to provide a comprehensive and unquirky experience, there are unfortunately some issues which may require manual intervention or operator choices.

Using MacOS after removing nix while nix-darwin was still installed, network requests fail

If nix was previously uninstalled without uninstalling nix-darwin first, users may experience errors similar to this:

$ nix shell nixpkgs#curl
error: unable to download 'https://cache.nixos.org/g8bqlgmpa4yg601w561qy2n576i6g0vh.narinfo': Problem with the SSL CA cert (path? access rights?) (77)

This occurs because nix-darwin provisions an org.nixos.activate-system service which remains after Nix is uninstalled. The org.nixos.activate-system service in this state interacts with the newly installed Nix and changes the SSL certificates it uses to be a broken symlink.

$ ls -lah /etc/ssl/certs
total 0
drwxr-xr-x  3 root  wheel    96B Oct 17 08:26 .
drwxr-xr-x  6 root  wheel   192B Sep 16 06:28 ..
lrwxr-xr-x  1 root  wheel    41B Oct 17 08:26 ca-certificates.crt -> /etc/static/ssl/certs/ca-certificates.crt

The problem is compounded by the matter that the nix-darwin uninstaller will not work after uninstalling Nix, since it uses Nix and requires network connectivity.

It's possible to resolve this situation by removing the org.nixos.activate-system service and the ca-certificates:

$ sudo rm /Library/LaunchDaemons/org.nixos.activate-system.plist
$ sudo launchctl bootout system/org.nixos.activate-system
$ /nix/nix-installer uninstall
$ sudo rm /etc/ssl/certs/ca-certificates.crt

Then run the nix-installer again, and it should work.

Up-to-date versions of the nix-installer will refuse to uninstall until nix-darwin is uninstalled first, helping mitigate this problem.

Building a binary

Since you'll be using nix-installer to install Nix on systems without Nix, the default build is a static binary.

Build a portable Linux binary on a system with Nix:

# to build a local copy
nix build -L ".#nix-installer-static"
# to build the remote main development branch
nix build -L "github:determinatesystems/nix-installer#nix-installer-static"
# for a specific version of the installer:
export NIX_INSTALLER_TAG="v0.6.0"
nix build -L "github:determinatesystems/nix-installer/$NIX_INSTALLER_TAG#nix-installer-static"

On Mac:

# to build a local copy
nix build -L ".#nix-installer"
# to build the remote main development branch
nix build -L "github:determinatesystems/nix-installer#nix-installer"
# for a specific version of the installer:
export NIX_INSTALLER_TAG="v0.6.0"
nix build -L "github:determinatesystems/nix-installer/$NIX_INSTALLER_TAG#nix-installer"

Then copy the result/bin/nix-installer to the machine you wish to run it on.

You can also add nix-installer to a system without Nix via cargo, there are no system dependencies to worry about:

# to build and run a local copy
RUSTFLAGS="--cfg tokio_unstable" cargo run -- --help
# to build the remote main development branch
RUSTFLAGS="--cfg tokio_unstable" cargo install --git https://github.com/DeterminateSystems/nix-installer
nix-installer --help
# for a specific version of the installer:
export NIX_INSTALLER_TAG="v0.6.0"
RUSTFLAGS="--cfg tokio_unstable" cargo install --git https://github.com/DeterminateSystems/nix-installer --tag $NIX_INSTALLER_TAG
nix-installer --help

To make this build portable, pass --target x86_64-unknown-linux-musl.

Note We currently require --cfg tokio_unstable as we utilize Tokio's process groups, which wrap stable std APIs, but are unstable due to it requiring an MSRV bump.

As a library

Warning Use as a library is still experimental. This feature is likely to be removed in the future without an advocate. If you're using this, please let us know and we can make a path to stabilization.

Add nix-installer to your dependencies:

cargo add nix-installer

If you are building a CLI, check out the cli feature flag for clap integration.

You'll also need to edit your .cargo/config.toml to use tokio_unstable as we utilize Tokio's process groups, which wrap stable std APIs, but are unstable due to it requiring an MSRV bump:

# .cargo/config.toml
[build]
rustflags=["--cfg", "tokio_unstable"]

You'll also need to set the NIX_INSTALLER_TARBALL_PATH environment variable to point to a target-appropriate Nix installation tarball, like nix-2.21.2-aarch64-darwin.tar.xz. The contents are embedded in the resulting binary instead of downloaded at installation time.

Then it's possible to review the documentation:

cargo doc --open -p nix-installer

Documentation is also available via nix build:

nix build github:DeterminateSystems/nix-installer#nix-installer.doc
firefox result-doc/nix-installer/index.html

Accessing other versions

For users who desire version pinning, the version of nix-installer to use can be specified in the curl command:

VERSION="v0.6.0"
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix/tag/${VERSION} | sh -s -- install

To discover which versions are available, or download the binaries for any release, check the Github Releases.

These releases can be downloaded and used directly:

VERSION="v0.6.0"
ARCH="aarch64-linux"
curl -sSf -L https://github.com/DeterminateSystems/nix-installer/releases/download/${VERSION}/nix-installer-${ARCH} -o nix-installer
./nix-installer install

Each installer version has an associated supported nix version -- if you pin the installer version, you'll also indirectly pin to the associated nix version.

You can also override the nix version via --nix-package-url or NIX_INSTALLER_NIX_PACKAGE_URL= but doing so is not recommended since we haven't tested that combination. Here are some example nix package URLs including nix version, OS and architecture:

Installation Differences

Differing from the upstream Nix installer scripts:

  • In nix.conf:
    • the nix-command and flakes features are enabled
    • bash-prompt-prefix is set
    • auto-optimise-store is set to true (On Linux only)
    • extra-nix-path is set to nixpkgs=flake:nixpkgs
    • max-jobs is set to auto
    • upgrade-nix-store-path-url is set to https://install.determinate.systems/nix-upgrade/stable/universal, to prevent unintentional downgrades.
  • an installation receipt (for uninstalling) is stored at /nix/receipt.json as well as a copy of the install binary at /nix/nix-installer
  • nix-channel --update is not run, ~/.nix-channels is not provisioned
  • ssl-cert-file is set in /etc/nix/nix.conf if the ssl-cert-file argument is used.

Motivations

The existing upstream scripts do a good job, however they are difficult to maintain.

Subtle differences in the shell implementations and tool used in the scripts make it difficult to make meaningful changes to the installer.

The Determinate Nix installer has numerous advantages:

  • survives macOS upgrades
  • keeping an installation receipt for easy uninstallation
  • offering users a chance to review an accurate, calculated install plan
  • having 'planners' which can create appropriate install plans for complicated targets
  • offering users with a failing install the chance to do a best-effort revert
  • improving performance by maximizing parallel operations
  • supporting a expanded test suite including 'curing' cases
  • supporting SELinux and OSTree based distributions without asking users to make compromises
  • operating as a single, static binary with external dependencies such as openssl, only calling existing system tools (like useradd) where necessary
  • As a MacOS remote build target, ensures nix is not absent from path

It has been wonderful to collaborate with other participants in the Nix Installer Working Group and members of the broader community. The working group maintains a foundation owned fork of the installer.

Diagnostics

The goal of the Determinate Nix Installer is to successfully and correctly install Nix. The curl | sh pipeline and the installer collects a little bit of diagnostic information to help us make that true.

Here is a table of the diagnostic data we collect:

Field Use
version The version of the Determinate Nix Installer.
planner The method of installing Nix (linux, macos, steam-deck)
configured_settings The names of planner settings which were changed from their default. Does not include the values.
os_name The running operating system.
os_version The version of the operating system.
triple The architecture/operating system/binary format of your system.
is_ci Whether the installer is being used in CI (e.g. GitHub Actions).
action Either Install or Uninstall.
status One of Success, Failure, Pending, or Cancelled.
attribution Optionally defined by the user, associate the diagnostics of this run to the provided value.
failure_chain A high level description of what the failure was, if any. For example: Command("diskutil") if the command diskutil list failed.

To disable diagnostic reporting, set the diagnostics URL to an empty string by passing --diagnostic-endpoint="" or setting NIX_INSTALLER_DIAGNOSTIC_ENDPOINT="".

You can read the full privacy policy for Determinate Systems, the creators of the Determinate Nix Installer, here.

nix-installer's People

Contributors

abathur avatar cole-h avatar dependabot[bot] avatar edolstra avatar flexiondotorg avatar georgyo avatar grahamc avatar hofer-julian avatar hoverbear avatar lheckemann avatar lucperkins avatar lytedev avatar mig4ng avatar mkenigs avatar nanashi-1 avatar spacekookie avatar timsutton avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nix-installer's Issues

check if running as root

harmonic should check if the current user is root (or equivalent on MacOS), otherwise it will just fail with a permission denied error.

Test "HPC contexts" or "LDAP contexts"

On machines with a lot of users due to LDAP running in a big org, it would be interesting to see if the nix installer behaves well, e.g. select the proper UIDs for the build users, etc.

Also, HPC contexts are more special case of LDAP contexts, so better keep them in mind I think.

I have seen a PR for adding sandboxed QEMU tests, I think it would be great to extend those with similar scenarios.

uninstall should remove /etc/nix if empty

after uninstalling, if /etc/nix is empty (no .bak file or anything that could have been created by the administrator), it would be nice to remove that empty directory.

How to handle Nix versions?

Right now we keep our Nix package URL as a const in the package, it may be ideal to determine a way to check what the most up to date option is during plan time.

Deleting Users on Mac is not working

Deleting Users on a Mac isn't working right now for unknown reasons:

ephemeraladmin@mac-epic-turducken ~ % dscl . -read /Users/_nixbld1
dsAttrTypeNative:_writers_passwd: _nixbld1
dsAttrTypeNative:accountPolicyData:
 <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>creationTime</key>
        <real>1667407560.622637</real>
        <key>failedLoginCount</key>
        <integer>0</integer>
        <key>failedLoginTimestamp</key>
        <integer>0</integer>
        <key>passwordLastSetTime</key>
        <real>1667409170.2462959</real>
</dict>
</plist>

dsAttrTypeNative:IsHidden: 1
dsAttrTypeNative:record_daemon_version: 8770000
AppleMetaNodeLocation: /Local/Default
GeneratedUID: 95CA4ABC-6A65-4EE0-8430-1F8BC58E9C64
NFSHomeDirectory: /var/empty
Password: ********
PrimaryGroupID: 3000
RecordName: _nixbld1
RecordType: dsRecTypeStandard:Users
UniqueID: 301
UserShell: /sbin/nologin
ephemeraladmin@mac-epic-turducken ~ % sudo dscl . -delete Users/_nixbld1
<main> delete status: eDSPermissionError
<dscl_cmd> DS Error: -14120 (eDSPermissionError)

This suggests a password is required: https://apple.stackexchange.com/questions/310308/delete-a-standard-user-from-mac-os

ephemeraladmin@mac-epic-turducken ~ % sudo passwd _nixbld1
Changing password for _nixbld1.
New password:
Retype new password:

################################### WARNING ###################################
# This tool does not update the login keychain password.                      #
# To update it, run `security set-keychain-password` as the user in question, #
# or as root providing a path to such user's login keychain.                  #
###############################################################################

ephemeraladmin@mac-epic-turducken ~ % sudo dscl . -delete Users/_nixbld1
<main> delete status: eDSPermissionError
<dscl_cmd> DS Error: -14120 (eDSPermissionError)

Other resources:

Daemonless install

Right now nix-installer has three planners, linux-multi, darwin-multi, steam-deck. These all currently require an init system, specifically systemd, but that may change at a later date. They are based on the multi-user install of the existing Nix installer scripts.

We want to support uses cases like containers, where the user may use docker.

There are also some use cases where the user does not have the ability to start services or daemon processes. We'd like to support those.

The existing Nix installer scripts also offer users a "Single User" mode, in which the created /nix store:

ana@ubuntu-base:~$ sh <(curl -L https://nixos.org/nix/install) --no-daemon
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  4052  100  4052    0     0   8576      0 --:--:-- --:--:-- --:--:--  8576
downloading Nix 2.12.0 binary tarball for x86_64-linux from 'https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz' to '/tmp/nix-binary-tarball-unpack.FASdoMsq37'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 19.5M  100 19.5M    0     0  55.9M      0 --:--:-- --:--:-- --:--:-- 56.0M
Note: a multi-user installation is possible. See https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation
performing a single-user installation of Nix...
directory /nix does not exist; creating it by running 'mkdir -m 0755 /nix && chown ana /nix' using sudo
copying Nix to /nix/store................................................
installing 'nix-2.12.0'
building '/nix/store/0dg0zjj2j6hijn193x1215yssrg7n1xs-user-environment.drv'...
unpacking channels...
modifying /home/ana/.profile...

Installation finished!  To ensure that the necessary environment
variables are set, either log in again, or type

  . /home/ana/.nix-profile/etc/profile.d/nix.sh

in your shell.
ana@ubuntu-base:~$ ls -lah /ni*
total 16K
drwxr-xr-x  4 ana  root 4.0K Dec 21 09:02 .
drwxr-xr-x 21 root root 4.0K Dec 21 09:02 ..
drwxr-xr-x 51 ana  ana  4.0K Dec 21 09:03 store
drwxr-xr-x  4 ana  ana  4.0K Dec 21 09:02 var

This doesn't allow for situations where multiple users wish to install Nix, or situations where root is actually the intended user to run nix build etc.

In fact, the existing Single User script refuses to run as root:

ana@ubuntu-base:~$ sudo su
root@ubuntu-base:/home/ana# sh <(curl -L https://nixos.org/nix/install) --no-daemon
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  4052  100  4052    0     0   9293      0 --:--:-- --:--:-- --:--:--  9293
downloading Nix 2.12.0 binary tarball for x86_64-linux from 'https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz' to '/tmp/nix-binary-tarball-unpack.hujxbv7VdU'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 19.5M  100 19.5M    0     0  54.1M      0 --:--:-- --:--:-- --:--:-- 54.1M
Note: a multi-user installation is possible. See https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation
warning: installing Nix as root is not supported by this script!
performing a single-user installation of Nix...
directory /nix does not exist; creating it by running 'mkdir -m 0755 /nix && chown root /nix' using sudo
copying Nix to /nix/store................................................
warning: the group 'nixbld' specified in 'build-users-group' does not exist
warning: the group 'nixbld' specified in 'build-users-group' does not exist
installing 'nix-2.12.0'
error: the group 'nixbld' specified in 'build-users-group' does not exist
/tmp/nix-binary-tarball-unpack.hujxbv7VdU/unpack/nix-2.12.0-x86_64-linux/install: unable to install Nix into your default profile

The current single user install method also lacks some sandboxing features that are present in the daemon-based multi-user mode, since the unprivileged user cannot do certain things. This can create situations where a user can build something in Single User Nix which fails to build in Multi-user nix due to isolation. While multi-user installs can invoke --impure to build those, it's still a frustrating experience.

We'd like to explore finding a better option here. Based discussions from the Installer Working Group and some polling of the community we determined that most people use the Single User variant for inside containers (like docker) or otherwise in build pipelines.

One idea that was proposed was to have a root daemonless variant instead, attempting to offer the user a workflow like this:

FROM ubuntu:latest
RUN apt update
RUN apt install curl --yes
RUN curl -L https://install.determinate.systems/nix | sh -s -- install linux-multi --no-confirm
RUN nix build .#hello -L /out
RUN createuser runner
USER runner
CMD /out/bin/runner

In this case, we'd create the /nix folder structure as normal, and perhaps even create build users/groups, and only root would be able to invoke nix related command that may alter the store. This mode would be able to benefit from all the normal isolation of multi-user installs and work without an init system in places like a build pipeline.

One note is that in a container we can only alter our hostname if we are --privileged, which many users won't want to do:

❯ sudo podman run --rm -ti --privileged ubuntu
root@784cf2044df2:/# hostname boop 
root@784cf2044df2:/# 
exit
❯ podman run --rm -ti ubuntu
root@f5e5e97eda12:/# hostname boop
hostname: you must be root to change the host name
root@f5e5e97eda12:/# 
exit

Too many error types

Right now each of the builtin actions have their own error types.

We don't need to do that.

We should be able to create a common ActionError covering most cases. Since actions can return Box errors, they can always create their own new error if needed.

Confirm installation is successful

I just used nix-installer and the end of the output looks like

INFO Step: Create directory /nix
INFO Step: Provision Nix
INFO Step: Configure Nix

Now, I'm curious to know if it aborted or finished correctly, a simple message such as INFO Nix is installed or something like this would be nice.

Add `cargo-semver-checks`

This tool cargo-semver-checks by @obi1kenobi can help us detect when we break semver and we should be using it once we're not 0.0.0-unreleased.

This ticket includes:

  • Fixing the initial issues which semver-checks highlights, and reviewing the codebase for possible places where we need to apply its rules
  • Adding a nix flake check step for cargo-semver-checks, perhaps through the action, though, cargo-semver-checks is already in nixpkgs so we can make it one of our normal lints
  • Adding a CI job for the created nix flake check lint

Support WSL2 (without Systemd)

When trying to run the install script on WSL2 (Ubuntu 22.04) I get the following error:

Error:
   0: Planner error
   1: Error executing action
   2: No supported init system found

Location:
   /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/convert/mod.rs:726

Flesh out docs

One of the design goals of Harmonic is to be usable as a library.

As a consequence of that, we need to be usable as a library, and that means docs.

Ubuntu 16.04 lacks `sudo --preserve-env` with args

error: builder for '/nix/store/v7khh9q4bpbism2nqkpxgs1k0ay711a3-installer-test-ubuntu-v16.04-install-default.drv' failed with exit code 1;
       last 10 log lines:
       > sudo: option '--preserve-env' doesn't allow an argument
       > usage: sudo -h | -K | -k | -V
       > usage: sudo -v [-AknS] [-g group] [-h host] [-p prompt] [-u user]
       > usage: sudo -l [-AknS] [-g group] [-h host] [-p prompt] [-U user] [-u user]
       >             [command]
       > usage: sudo [-AbEHknPS] [-r role] [-t type] [-C num] [-g group] [-h host] [-p
       >             prompt] [-u user] [VAR=value] [-i|-s] [<command>]
       > usage: sudo -e [-AknS] [-r role] [-t type] [-C num] [-g group] [-h host] [-p
       >             prompt] [-u user] file ...
       > qemu-kvm: terminating on signal 15 from pid 1 ()
       For full logs, run 'nix log /nix/store/v7khh9q4bpbism2nqkpxgs1k0ay711a3-installer-test-ubuntu-v16.04-install-default.drv'.
error: 1 dependencies of derivation '/nix/store/dkn95xgy5g9icl7pbh3qrdf5c9r2y1iw-all.drv' failed to build
⏎         

This was discovered during #138

Allow users to answer `e` at install prompt

Users may wish to get an explanation for their plan and not wish to restart the application to get there. When we prompt the user to allow an install/revert, we should support y and n as we do now, but also e which re-prints with explanation.

Recognizing Existing or Prior Nix Installs (Curing)

Some users have reported issues when trying to use this package on a system that had previously had Nix installed on it.

Currently, users will ideally get a planner error (for example, that a file or volume #217 already exists), but in some cases may get an install-time error.

We'd like to address this and make it easier.

This can break down into three major chunks of work:

  • Giving better error messages in cases where there are easily fixable problems the user can take. For example in #217 we tell the user the exact command to run.
  • Recognizing artifacts from previous installs, for example, an existing /nix/nix.conf with existing configuration which we can just modify.
    • In particular with /nix/nix.conf we would ideally have a way to 'merge' or 'upsert' the config to include features we toggle on.
    • There is an open question of what to do with these files that on uninstall, as the user might want us to remove /etc/nix.conf if they uninstall.
  • Offering some reasonable way to "cure" an existing install.

The last point is a bit wishy-washy so let's consider what that might look like:

  • A recognize subcommand which creates receipts for existing (possibly partial) installs and allows for use of uninstall later.
  • Some sort of doctor subcommand which essentially follows the recommended remedy steps codified in the first point
  • A "Cure" phase that happens between the plan and install phases, where the user is interactively prompted about running remedy steps.
  • Allowing actions to add "Cure" steps to the plan and presenting those inside of the plan summary for users.

TODO:

  • Cure APFS/fstab on mac: #246
  • Cure nix.conf: #263
  • Cure service manager configurations (i.e. systemd's nix-daemon.service/socket, macos's nix-daemon LaunchDaemon)

Support resumption of partially successful installations and repair of broken installations by making every `Action` idempotent and comparing the actual state of the system to that in the plan or receipt

Hi there! I'm a Nix newbie who is very excited about this project. So far, I love it and I'm sold.

I've got some ideas that I want to share because I think they have merit. I understand that this project is experimental, and that what I'm suggesting is quite a large change. I'd like to solicit feedback from the maintainers to see if what I'm saying has value.

I want Nix installs to be automatically resumable and repairable. This means that if something goes wrong during an installation, or something gets broken after an installation, the installer can restore the system to a good state. This is not an easy thing to do. I don't think it'll be easy to retrofit either. With that said, it is possible to start supporting automatic resumption and repair, provided certain constraints on actions are made.

Currently, Actions are designed to be reversible. This is an excellent design decision. I'd like to propose two additional constraints on actions:

  1. Actions must be idempotent (in other words, safely "replayable").
  2. Actions must be able to compare the state of the actual installation against the state they expect.

Under these constraints, the application of a plan can be safely done under the following circumstances:

  1. No Nix has been installed
  2. An installation has partially succeeded
  3. Something external broke the installation in some way.

In some sense, the receipt becomes a write-ahead-log and the installer re-applies the actions in there, unless it knows the action is unnecessary.

By way of example, the following happens today:

$ curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
Nix was installed successfully!

# rm -rf /etc/nix

$ curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

Found existing plan in `/nix/receipt.json`, with the same settings, already completed, try uninstalling and reinstalling if Nix isn't working

$ /nix/nix-installer uninstall

Proceed? ([Y]es/[n]o/[e]xplain): y
 INFO Revert: Configure Nix
Error: 
   0: Error executing action
   1: Remove file `/etc/nix/nix.conf`
   2: No such file or directory (os error 2)

The problem is that a something (me) has made the actual state of the system diverge from that specified in the receipt. Currently, the installer doesn't detect this can can't patch it up. Additionally, the uninstaller cannot uninstall what's already there.

It would be valuable if the nix-installer could idempotently re-install Nix, patching over the differences between the actual and the expected state of the installation.

Various Actions can be idempotent

Actions like CreateFile, CreateDirectory, CreateGroup, and CreateUser can be idempotent.

This would be advantageous because it would help us "keep going" over partially installed Nix's.

Caution should be taken to add a new ActionState which signals to the uninstaller that the user/folder/group etc need not be removed.

These actions probably can then be renamed to like EnsureFile and EnsureFileContents and EnsureUser.

Better tracing

Right now, unless tracing is enabled, Harmonic is mostly silent during install except during plan prompts.

When tracing is enabled, there is significant boilerplate in each action to enable messaging:

https://github.com/DeterminateSystems/harmonic/blob/673464ede7bac886f5611541eb2706037afbf17f/src/action/common/configure_nix.rs#L102-L127

We should:

  • Have a more standard way for debug/trace messaging like above to happen for actions.
    • Maybe using a const and a wrapper function? We should be careful about not showing the tracing span fields accidentally.
  • Consider and explore how to do good info level logging here.

install -> uninstall -> install fails

after installing nix and uninstalling using harmonic, it seems a systemd unit (nix-daemon.socket ?) was left in an incorrect state

this path will be fetched (0.14 MiB download, 0.47 MiB unpacked):
  /nix/store/7vj1h9s1yfkj66dh9645rk51pjpb0hkr-nss-cacert-3.80
copying path '/nix/store/7vj1h9s1yfkj66dh9645rk51pjpb0hkr-nss-cacert-3.80' from 'https://cache.nixos.org'...
building '/nix/store/3m5x22m8rnylzjg8xzkfb5vr7f984l1b-user-environment.drv'...
unpacking channels...

Created symlink /etc/systemd/system/nix-daemon.service → /nix/var/nix/profiles/default/lib/systemd/system/nix-daemon.service.
Created symlink /etc/systemd/system/nix-daemon.socket → /nix/var/nix/profiles/default/lib/systemd/system/nix-daemon.socket.
Created symlink /etc/systemd/system/sockets.target.wants/nix-daemon.socket → /nix/store/nmq5zcd93qb1yskx42rs910ff0247nn2-nix-2.11.0/lib/systemd/system/nix-daemon.socket.
Job failed. See "journalctl -xe" for details.
  2022-10-26T13:17:21.719777Z ERROR harmonic::cli::subcommand::install: 
   0: Error executing action
   1: Starting systemd unit
   2: Failed to execute command
   3: Command `"systemctl" "enable" "--now" "nix-daemon.socket"` failed status

Location:
   /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/convert/mod.rs:550

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::install::execute
      at src/cli/subcommand/install.rs:43
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
    at src/cli/subcommand/install.rs:88
    in harmonic::cli::subcommand::install::execute
    in harmonic::cli::execute

This Nix uninstall is for:
Operating System: Linux
Init system: systemd
Nix channels: nixpkgs=https://nixos.org/channels/nixpkgs-unstable

Created by planner: LinuxMultiUser
            
The following actions will be taken:
* Remove the directory tree in `/nix`

* Remove build users and group

* Remove the directory `/nix`

* Unconfigure the shell profiles

* Remove channel configuration at `/root/.nix-channels`

* Remove the nix configuration in `/etc/nix/nix.conf`

* Unconfigure Nix daemon related settings with systemd

* Unset the default Nix profile

        

Are you sure? (y/N): Cancelled!Okay, didn't do anything! Bye!

a systemctl status nix-daemon.socket shows this

$ sudo systemctl status nix-daemon.socket
○ nix-daemon.socket - Nix Daemon Socket
     Loaded: loaded (/etc/systemd/system/nix-daemon.socket; enabled; preset: enabled)
     Active: inactive (dead)
   Triggers: ● nix-daemon.service
     Listen: /nix/var/nix/daemon-socket/socket (Stream)

oct. 26 13:13:54 xubuntu systemd[1]: Listening on Nix Daemon Socket.
oct. 26 13:15:02 xubuntu systemd[1]: nix-daemon.socket: Socket unit configuration has changed while unit has been running, no open socket file descriptor left. The socket unit is >
oct. 26 13:15:03 xubuntu systemd[1]: nix-daemon.socket: Socket unit configuration has changed while unit has been running, no open socket file descriptor left. The socket unit is >
oct. 26 13:15:03 xubuntu systemd[1]: nix-daemon.socket: Socket unit configuration has changed while unit has been running, no open socket file descriptor left. The socket unit is >
oct. 26 13:15:03 xubuntu systemd[1]: nix-daemon.socket: Deactivated successfully.
oct. 26 13:15:03 xubuntu systemd[1]: Closed nix-daemon.socket.
oct. 26 13:17:21 xubuntu systemd[1]: nix-daemon.socket: Socket service nix-daemon.service already active, refusing.
oct. 26 13:17:21 xubuntu systemd[1]: Failed to listen on Nix Daemon Socket.

Remove unnecessary Github Actions

We have some github actions doing builds that can be cleaned up since they are done in buildkite now

build-x86_64-linux:
name: Build x86_64 Linux
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- uses: cachix/install-nix-action@v17
with:
extra_nix_config: |
access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
- name: Cache build store (x86_64-linux)
id: build-store-x86_64-linux
uses: actions/cache@v3
with:
path: ~/.ci-store
key: build-store-x86_64-linux-${{ hashFiles('**/Cargo.lock', '**/flake.lock') }}-v1
- name: Build `nix-installer-static`
run: nix build --store ~/.ci-store --print-build-logs .#packages.x86_64-linux.nix-installer-static
- name: Copy artifact
run: |
RESULT=$(nix eval --raw --store ~/.ci-store --print-build-logs .#packages.x86_64-linux.nix-installer-static --apply "x: \"$HOME/.ci-store\${x}\"")
cp $RESULT/bin/nix-installer nix-installer
- name: Create artifact for x86_64-linux nix-installer
uses: actions/upload-artifact@v3
with:
name: nix-installer-x86_64-linux
path: |
nix-installer
# Various feature variations or versions of the build which we expect to always work
# Since it uses the `build-store-x86_64-linux-*` cache it should be quite fast
build-variants-x86_64-linux:
name: Build x86_64 Linux (Variants)
runs-on: ubuntu-22.04
needs: build-x86_64-linux # Only run this if the normal checks work, to avoid clogging builders
steps:
- uses: actions/checkout@v3
- uses: cachix/install-nix-action@v17
with:
extra_nix_config: |
access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
- name: Cache build store (x86_64-linux)
id: build-store-x86_64-linux
uses: actions/cache@v3
with:
path: ~/.ci-store
key: build-store-x86_64-linux-${{ hashFiles('**/Cargo.lock', '**/flake.lock') }}-v1
- name: Build `nix-installer`
run: nix build --store ~/.ci-store --print-build-logs .#packages.x86_64-linux.nix-installer
- name: Test `nix develop` build without default features
run: nix develop --store ~/.ci-store --print-build-logs .# --command "cargo" build --no-default-features
- name: Test `nix develop` build all features
run: nix develop --store ~/.ci-store --print-build-logs .# --command "cargo" build --all-features

Add CI jobs to do Install -> Uninstall -> Install

Since the Github CI machines don't have Nix on them, we can just use them directly.

After the artifacts are produced, try to run harmonic install $PLAN and then do uninstall, then install again. This could be done on both x86 Linux and Darwin

`plan` and `install` should detect if nix is already installed

That would be nice to detect if nix was already installed when trying install or plan, I suppose there code is there because the uninstall process already do that.

Here are the results of install and plan after a successful installation

$ sudo ./harmonic install
Error: 
   0: Error executing action
   1: Placing channel configuration
   2: Creating file
   3: File exists `/root/.nix-channels`

Location:
   src/cli/subcommand/install.rs:77

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::install::execute
      at src/cli/subcommand/install.rs:43
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

and

$ sudo ./harmonic plan
Error: 
   0: Error executing action
   1: Placing channel configuration
   2: Creating file
   3: File exists `/root/.nix-channels`

Location:
   src/cli/subcommand/plan.rs:58

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::plan::execute with channels=nixpkgs https://nixos.org/channels/nixpkgs-unstable daemon_user_count=32 no_modify_profile=false
      at src/cli/subcommand/plan.rs:24
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

`cli` feature

With harmonic usable as a library, we should provide a way to flag off CLI related code and dependencies. We have at least 8 top level dependencies which are only required for CLI:

  • eyre
  • color_eyre
  • atty
  • clap
  • crossterm
  • tracing-subscriber
  • tracing-error
  • owo-colors

These should be made optional dependencies and related code put behind a #[cfg(feature = "cli")].

This should probably wait until #65 is merged up.

User interface: don't validate input upon pressed key

It's a matter of choice I suppose, but when asked (Y/N) by harmonic, it automatically validates upon pressing a single letter, while users may expect to type ENTER to validate their input.

I think harmonic should behave that way too.

Steam Deck installation crashes desktop session

I tried the installer on my Steam Deck after reading your blog post. Thanks for your work, Nix may be a more convenient way of installing additional software on SteamOS.

The installer worked fine, but somehow crashed my desktop session. Here's some logs, maybe you can figure out what happened:

Installer output (looks as expected):

deck@steamdeck /sys curl -L https://install.determinate.systems/nix | sh -s -- install steam-deck                                                                    
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
100 15714  100 15714    0     0   5589      0  0:00:02  0:00:02 --:--:-- 48055
info: downloading installer (https://install.determinate.systems/nix/nix-installer-x86_64-linux)
`nix-installer` needs to run as `root`, attempting to escalate now via `sudo`...
Nix install plan (v0.0.0-unreleased)

Planner: steam-deck

Planner settings:

* extra_conf: []
* modify_profile: true
* nix_package_url: "https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz"
* channels: ["nixpkgs=https://nixos.org/channels/nixpkgs-unstable"]
* nix_build_group_id: 3000
* nix_build_group_name: "nixbld"
* force: false
* daemon_user_count: 32
* nix_build_user_id_base: 3000
* nix_build_user_prefix: "nixbld"
* persistence: "/home/nix"

The following actions will be taken (`--explain` for more context):

* Create directory `/home/nix`
* Create or overwrite file `/etc/systemd/system/nix-directory.service`
* Create or overwrite file `/etc/systemd/system/nix.mount`
* Create or overwrite file `/etc/systemd/system/ensure-symlinked-units-resolve.service`
* Enable (and start) the systemd unit ensure-symlinked-units-resolve.service
* Fetch `https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz` to `/nix/temp-install-dir`
* Create build users (UID 3000-3032) and group (GID 3000)
* Create a directory tree in `/nix`
* Move the downloaded Nix into `/nix`
* Setup the default Nix profile
* Configure Nix daemon related settings with systemd
* Place the Nix configuration in `/etc/nix/nix.conf`
* Place channel configuration at `/root/.nix-channels`
* Configure the shell profiles
* Enable (and start) the systemd unit nix-daemon.socket


Proceed? (y/N): y
 INFO Step: Create directory `/home/nix`
 INFO Step: Create or overwrite file `/etc/systemd/system/nix-directory.service`
 INFO Step: Create or overwrite file `/etc/systemd/system/nix.mount`
 INFO Step: Create or overwrite file `/etc/systemd/system/ensure-symlinked-units-resolve.service`
 INFO Step: Enable (and start) the systemd unit ensure-symlinked-units-resolve.service
 INFO Step: Provision Nix
 INFO Step: Configure Nix
 INFO Step: Enable (and start) the systemd unit nix-daemon.socket

Journal: journal.txt

can't uninstall an interrupted install

I've been doing an install with harmonic but it failed due to network issues, however I can't use the uninstall process to rollback the changes.

Error: 
   0: Reading receipt
   1: No such file or directory (os error 2)

Location:
   src/cli/subcommand/uninstall.rs:42

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::uninstall::execute
      at src/cli/subcommand/uninstall.rs:32
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

flaky failure conditions on macos that might need handling

Opening an issue to collect links to some failures on macOS that are known or suspected to be flaky, since they might come up here as well. I'll try to remember to update this if I spot others.

  • NixOS/nix#7615 and NixOS/nix#5566 detail an issue with sudo sometimes failing on AD systems
  • NixOS/nix#6669 details an issue that can prevent volume mounting; my mental model was that this would be a reliable blocker related to system profiles, but at least one user recently reported the install working after a few retries (NixOS/nix#6669 (comment))

detect NixOS

That's unlikely anyone would try to use harmonic on NixOS like me, but the current result isn't good

> result/bin/harmonic -vv plan
  2022-10-26T12:33:46.914391Z TRACE harmonic::actions::meta::configure_shell_profile: Did not plan to edit `/etc/profile.d/nix.sh` as it does not exist.
    at src/actions/meta/configure_shell_profile.rs:32
    in harmonic::actions::meta::configure_shell_profile::plan
    in harmonic::actions::meta::configure_nix::plan
    in harmonic::cli::subcommand::plan::execute with channels: nixpkgs https://nixos.org/channels/nixpkgs-unstable, daemon_user_count: 32, no_modify_profile: false
    in harmonic::cli::execute

  2022-10-26T12:33:46.914521Z TRACE harmonic::actions::meta::configure_shell_profile: Did not plan to edit `/etc/zshrc` as it does not exist.
    at src/actions/meta/configure_shell_profile.rs:32
    in harmonic::actions::meta::configure_shell_profile::plan
    in harmonic::actions::meta::configure_nix::plan
    in harmonic::cli::subcommand::plan::execute with channels: nixpkgs https://nixos.org/channels/nixpkgs-unstable, daemon_user_count: 32, no_modify_profile: false
    in harmonic::cli::execute

  2022-10-26T12:33:46.914608Z TRACE harmonic::actions::meta::configure_shell_profile: Did not plan to edit `/etc/bash.bashrc` as it does not exist.
    at src/actions/meta/configure_shell_profile.rs:32
    in harmonic::actions::meta::configure_shell_profile::plan
    in harmonic::actions::meta::configure_nix::plan
    in harmonic::cli::subcommand::plan::execute with channels: nixpkgs https://nixos.org/channels/nixpkgs-unstable, daemon_user_count: 32, no_modify_profile: false
    in harmonic::cli::execute

  2022-10-26T12:33:46.914674Z TRACE harmonic::actions::meta::configure_shell_profile: Did not plan to edit `/etc/zsh/zshrc` as it does not exist.
    at src/actions/meta/configure_shell_profile.rs:32
    in harmonic::actions::meta::configure_shell_profile::plan
    in harmonic::actions::meta::configure_nix::plan
    in harmonic::cli::subcommand::plan::execute with channels: nixpkgs https://nixos.org/channels/nixpkgs-unstable, daemon_user_count: 32, no_modify_profile: false
    in harmonic::cli::execute

Error: 
   0: Error executing action
   1: Placing Nix configuration
   2: Creating file
   3: File exists `/etc/nix/nix.conf`

Location:
   src/cli/subcommand/plan.rs:58

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::plan::execute with channels=nixpkgs https://nixos.org/channels/nixpkgs-unstable daemon_user_count=32 no_modify_profile=false
      at src/cli/subcommand/plan.rs:24
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Nix Installer adds a superflous newline in `nix.conf` in GitHub Actions

I encountered an issue in the Nix Installer GitHub Action where the access-tokens config is prefaced with \n when written to nix.conf:

\naccess-tokens = *****

The basic contours of my Actions configuration:

- uses: DeterminateSystems/nix-installer@main
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    extra-conf: |
      binary-caches = https://my-special-cache.org
      trusted-substituters = https://my-special-cache.org

I debugged this by adding an Action that cats the config file, which appears to be as expected otherwise.

The offending line seems to be here.

Try to migrate `action_state` out of `Action`

Right now each Action has to define action_state and set_action_state to store it. That, combined with the ActionImplementation trait, suggests we can somehow wrap these Actions in something with a contained ActionState.

Consider future design options

The process of installing Nix involves a number of steps which:

  • Are fallible
  • Should support --dry-run
  • Sometimes OS dependent
  • Sometimes option dependent
  • Depend on one another
  • Can, at times, be done in parallel
  • Would be nice to be "rolled back" in the case of a failure
  • Would be nice to summarize for the user before being performed.

Right now, we have a Harmonic type which performs each of these steps linearly, and it only supports multi-user x86 Linux systemd installs.

States and Transitions

While the current API 'works', it's very easy to misuse it, using steps in the wrong order, or missing steps entirely.

It would be desirable to use a pattern with a Builder (implementing a builder pattern) which would produce Harmonic<State> types where Harmonic held onto those built options as well as a State which models which stage of the install was occurring.

This state machine could have states such as:

  • State::Initialized: Does fetch/unpack process into a tempdir, creates users/groups.
  • State::Initialized -> State::Provisioned: Upserts directories/files, moves tempdir's store into place
  • State::Provisioned -> State::Bootstrapped: Runs nix-env -i on nix (etc), runs nix-channel --update
  • State::Bootstrapped -> State::Ready: Starts nix-daemon, does a test build

Forming a directed acyclic graph.

In the case of platforms such as Darwin, different states can be defined and swapped out, without impacting the rest of the states.

Actions & Plans

The current nix installer does a great job communicating what it is doing in great detail. In Rust, we won't always have a 1:1 matching to bash commands a user could run.

It would be quite maintainable if we could have some sort of "plan" we generate with all the steps we'd take, so that we could print this out to the user.

Furthermore, if we did have some plan we could 'drive' that plan to completion (probably via states as described above), or roll back the plan with minimal blast radius on the system.

As it stands now each step in the process is doing rough control flow do execute fairly generic actions:

https://github.com/DeterminateSystems/harmonic/blob/9873c864efd83374131085e71051aee6ce035124/src/lib.rs#L144-L197

https://github.com/DeterminateSystems/harmonic/blob/9873c864efd83374131085e71051aee6ce035124/src/lib.rs#L478-L494

If we had some way to generate a plan, or at least keep a sequence of the actions which we take, we could later roll them back.

Missing indicator for successful install

One thing that could be improved overall is that we never tell the user they may need to reload their environment (by opening a new terminal or exec $SHELL or something). If you run it without -v, it just silently exits (which is fine, though I would like some indication that it completed successfully by default...), and if you try to nix-shell right after that, it won't work and the user is left guessing if it actually completed, or if something went wrong. I'll make a ticket about this.

Originally posted by @cole-h in #34 (review)


Basically, I think this issue is twofold:

  1. We never notify the user that the install finished successfully
  2. We never notify the user that they may need to restart their shell (either by opening a new terminal, exec $SHELL, or anything else)

Example screenshot where I fumble through this exact thing:

image

`InstallPlan` should not copy self, binary should

If the user is using the nix-installer as a rust library, we can't promise that std::env::current_exe() is the nix-installer binary.

This step should happen in the binary itself, not the library code.

`install` should detect existing `/nix/receipt.json`

Right now a user who runs an install can only uninstall, then try again from scratch. This differs from how uninstall works, where if it breaks the user can resolve the issue and run it again.

During the harmonic install $PLAN phase, we should detect an existing receipt.json, if the planner is the same as the one the user invokes, we should prompt the user if they want to resume that install, or instead uninstall completely and reinstall.

This differs from #27 since it discusses an existing Nix install, not an existing Harmonic-installed Nix.

Versioned plans

Plans should include the harmonic version used, and refuse to work on semantically incompatible plans.

As a consequence of this, we need to also advise the user which version of Harmonic to use in uninstalling, etc.

`plan()` would be more useful with context

Right now if a plan has the following steps:

CreateDirectory::plan("/nix")
CreateFile::plan("/nix/boop")

The CreateFile task cannot actually validate in plan() that /nix will be created.

It would be quite helpful to have a way to include context.

This has some interaction with #35 .

installer failed due to networking issue

I had connectivity issues during the installation process, this leaves the installer in a weird state, and I couldn't resume / uninstall it.

The following actions will be taken:
* Create the directory `/nix`

* Fetch Nix from `https://releases.nixos.org/nix/nix-2.11.0/nix-2.11.0-x86_64-linux.tar.xz`

* Create build users and group

* Create a directory tree in `/nix`

* Move the downloaded Nix into `/nix`

* Setup the default Nix profile

* Configure Nix daemon related settings with systemd

* Place the nix configuration in `/etc/nix/nix.conf`

* Place channel configuration at `/root/.nix-channels`

* Configure the shell profiles

* Start the systemd Nix service and socket

        

Are you sure? (y/N): useradd warning: nixbld7's uid 3008 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld1's uid 3002 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld3's uid 3004 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld20's uid 3021 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld24's uid 3025 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld17's uid 3018 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld21's uid 3022 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld8's uid 3009 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld25's uid 3026 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld22's uid 3023 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld12's uid 3013 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld13's uid 3014 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld5's uid 3006 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld19's uid 3020 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld30's uid 3031 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld18's uid 3019 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld31's uid 3032 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld2's uid 3003 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld16's uid 3017 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld23's uid 3024 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld6's uid 3007 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld26's uid 3027 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld15's uid 3016 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld0's uid 3001 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld9's uid 3010 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld4's uid 3005 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld28's uid 3029 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld11's uid 3012 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld14's uid 3015 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld10's uid 3011 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld27's uid 3028 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld29's uid 3030 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.

  2022-10-26T12:52:21.257363Z ERROR harmonic::cli::subcommand::install: 
   0: Error executing action
   1: Fetching Nix
   2: Request error
   3: error sending request for url (https://releases.nixos.org/nix/nix-2.11.0/nix-2.11.0-x86_64-linux.tar.xz): error trying to connect: dns error: failed to lookup address information: Try again
   4: error trying to connect: dns error: failed to lookup address information: Try again
   5: dns error: failed to lookup address information: Try again
   6: failed to lookup address information: Try again

Location:
   /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/convert/mod.rs:550

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::install::execute
      at src/cli/subcommand/install.rs:43
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
    at src/cli/subcommand/install.rs:88
    in harmonic::cli::subcommand::install::execute
    in harmonic::cli::execute

This Nix uninstall is for:
Operating System: Linux
Init system: systemd
Nix channels: nixpkgs=https://nixos.org/channels/nixpkgs-unstable

Created by planner: LinuxMultiUser
            
The following actions will be taken:
* Remove the directory tree in `/nix`

* Remove build users and group

* Remove the directory `/nix`

        

Are you sure? (y/N): Cancelled!Okay, didn't do anything! Bye!

Plan compatability tests

We need some test fixtures which our unit tests and attempt to deserialize and validate, so that we will know if we ever break or alter plan format.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.