Coder Social home page Coder Social logo

determinatesystems / nix-installer Goto Github PK

View Code? Open in Web Editor NEW
1.7K 13.0 46.0 1.61 MB

Install Nix and flakes with the fast and reliable Determinate Nix Installer, with over 2 million installs.

License: GNU Lesser General Public License v2.1

Nix 5.79% Shell 3.52% Rust 90.18% Dockerfile 0.10% PowerShell 0.42%
installer nix docker linux macos podman wsl wsl2 fedora ostree

nix-installer's Issues

Better tracing

Right now, unless tracing is enabled, Harmonic is mostly silent during install except during plan prompts.

When tracing is enabled, there is significant boilerplate in each action to enable messaging:

https://github.com/DeterminateSystems/harmonic/blob/673464ede7bac886f5611541eb2706037afbf17f/src/action/common/configure_nix.rs#L102-L127

We should:

  • Have a more standard way for debug/trace messaging like above to happen for actions.
    • Maybe using a const and a wrapper function? We should be careful about not showing the tracing span fields accidentally.
  • Consider and explore how to do good info level logging here.

installer failed due to networking issue

I had connectivity issues during the installation process, this leaves the installer in a weird state, and I couldn't resume / uninstall it.

The following actions will be taken:
* Create the directory `/nix`

* Fetch Nix from `https://releases.nixos.org/nix/nix-2.11.0/nix-2.11.0-x86_64-linux.tar.xz`

* Create build users and group

* Create a directory tree in `/nix`

* Move the downloaded Nix into `/nix`

* Setup the default Nix profile

* Configure Nix daemon related settings with systemd

* Place the nix configuration in `/etc/nix/nix.conf`

* Place channel configuration at `/root/.nix-channels`

* Configure the shell profiles

* Start the systemd Nix service and socket

        

Are you sure? (y/N): useradd warning: nixbld7's uid 3008 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld1's uid 3002 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld3's uid 3004 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld20's uid 3021 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld24's uid 3025 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld17's uid 3018 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld21's uid 3022 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld8's uid 3009 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld25's uid 3026 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld22's uid 3023 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld12's uid 3013 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld13's uid 3014 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld5's uid 3006 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld19's uid 3020 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld30's uid 3031 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld18's uid 3019 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld31's uid 3032 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld2's uid 3003 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld16's uid 3017 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld23's uid 3024 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld6's uid 3007 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld26's uid 3027 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld15's uid 3016 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld0's uid 3001 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld9's uid 3010 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld4's uid 3005 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld28's uid 3029 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld11's uid 3012 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld14's uid 3015 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld10's uid 3011 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld27's uid 3028 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.
useradd warning: nixbld29's uid 3030 outside of the SYS_UID_MIN 101 and SYS_UID_MAX 999 range.

  2022-10-26T12:52:21.257363Z ERROR harmonic::cli::subcommand::install: 
   0: Error executing action
   1: Fetching Nix
   2: Request error
   3: error sending request for url (https://releases.nixos.org/nix/nix-2.11.0/nix-2.11.0-x86_64-linux.tar.xz): error trying to connect: dns error: failed to lookup address information: Try again
   4: error trying to connect: dns error: failed to lookup address information: Try again
   5: dns error: failed to lookup address information: Try again
   6: failed to lookup address information: Try again

Location:
   /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/convert/mod.rs:550

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::install::execute
      at src/cli/subcommand/install.rs:43
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
    at src/cli/subcommand/install.rs:88
    in harmonic::cli::subcommand::install::execute
    in harmonic::cli::execute

This Nix uninstall is for:
Operating System: Linux
Init system: systemd
Nix channels: nixpkgs=https://nixos.org/channels/nixpkgs-unstable

Created by planner: LinuxMultiUser
            
The following actions will be taken:
* Remove the directory tree in `/nix`

* Remove build users and group

* Remove the directory `/nix`

        

Are you sure? (y/N): Cancelled!Okay, didn't do anything! Bye!

uninstall should remove /etc/nix if empty

after uninstalling, if /etc/nix is empty (no .bak file or anything that could have been created by the administrator), it would be nice to remove that empty directory.

Support WSL2 (without Systemd)

When trying to run the install script on WSL2 (Ubuntu 22.04) I get the following error:

Error:
   0: Planner error
   1: Error executing action
   2: No supported init system found

Location:
   /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/convert/mod.rs:726

Test "HPC contexts" or "LDAP contexts"

On machines with a lot of users due to LDAP running in a big org, it would be interesting to see if the nix installer behaves well, e.g. select the proper UIDs for the build users, etc.

Also, HPC contexts are more special case of LDAP contexts, so better keep them in mind I think.

I have seen a PR for adding sandboxed QEMU tests, I think it would be great to extend those with similar scenarios.

`plan()` would be more useful with context

Right now if a plan has the following steps:

CreateDirectory::plan("/nix")
CreateFile::plan("/nix/boop")

The CreateFile task cannot actually validate in plan() that /nix will be created.

It would be quite helpful to have a way to include context.

This has some interaction with #35 .

Versioned plans

Plans should include the harmonic version used, and refuse to work on semantically incompatible plans.

As a consequence of this, we need to also advise the user which version of Harmonic to use in uninstalling, etc.

Try to migrate `action_state` out of `Action`

Right now each Action has to define action_state and set_action_state to store it. That, combined with the ActionImplementation trait, suggests we can somehow wrap these Actions in something with a contained ActionState.

Recognizing Existing or Prior Nix Installs (Curing)

Some users have reported issues when trying to use this package on a system that had previously had Nix installed on it.

Currently, users will ideally get a planner error (for example, that a file or volume #217 already exists), but in some cases may get an install-time error.

We'd like to address this and make it easier.

This can break down into three major chunks of work:

  • Giving better error messages in cases where there are easily fixable problems the user can take. For example in #217 we tell the user the exact command to run.
  • Recognizing artifacts from previous installs, for example, an existing /nix/nix.conf with existing configuration which we can just modify.
    • In particular with /nix/nix.conf we would ideally have a way to 'merge' or 'upsert' the config to include features we toggle on.
    • There is an open question of what to do with these files that on uninstall, as the user might want us to remove /etc/nix.conf if they uninstall.
  • Offering some reasonable way to "cure" an existing install.

The last point is a bit wishy-washy so let's consider what that might look like:

  • A recognize subcommand which creates receipts for existing (possibly partial) installs and allows for use of uninstall later.
  • Some sort of doctor subcommand which essentially follows the recommended remedy steps codified in the first point
  • A "Cure" phase that happens between the plan and install phases, where the user is interactively prompted about running remedy steps.
  • Allowing actions to add "Cure" steps to the plan and presenting those inside of the plan summary for users.

TODO:

  • Cure APFS/fstab on mac: #246
  • Cure nix.conf: #263
  • Cure service manager configurations (i.e. systemd's nix-daemon.service/socket, macos's nix-daemon LaunchDaemon)

Add `cargo-semver-checks`

This tool cargo-semver-checks by @obi1kenobi can help us detect when we break semver and we should be using it once we're not 0.0.0-unreleased.

This ticket includes:

  • Fixing the initial issues which semver-checks highlights, and reviewing the codebase for possible places where we need to apply its rules
  • Adding a nix flake check step for cargo-semver-checks, perhaps through the action, though, cargo-semver-checks is already in nixpkgs so we can make it one of our normal lints
  • Adding a CI job for the created nix flake check lint

User interface: don't validate input upon pressed key

It's a matter of choice I suppose, but when asked (Y/N) by harmonic, it automatically validates upon pressing a single letter, while users may expect to type ENTER to validate their input.

I think harmonic should behave that way too.

flaky failure conditions on macos that might need handling

Opening an issue to collect links to some failures on macOS that are known or suspected to be flaky, since they might come up here as well. I'll try to remember to update this if I spot others.

  • NixOS/nix#7615 and NixOS/nix#5566 detail an issue with sudo sometimes failing on AD systems
  • NixOS/nix#6669 details an issue that can prevent volume mounting; my mental model was that this would be a reliable blocker related to system profiles, but at least one user recently reported the install working after a few retries (NixOS/nix#6669 (comment))

Daemonless install

Right now nix-installer has three planners, linux-multi, darwin-multi, steam-deck. These all currently require an init system, specifically systemd, but that may change at a later date. They are based on the multi-user install of the existing Nix installer scripts.

We want to support uses cases like containers, where the user may use docker.

There are also some use cases where the user does not have the ability to start services or daemon processes. We'd like to support those.

The existing Nix installer scripts also offer users a "Single User" mode, in which the created /nix store:

ana@ubuntu-base:~$ sh <(curl -L https://nixos.org/nix/install) --no-daemon
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  4052  100  4052    0     0   8576      0 --:--:-- --:--:-- --:--:--  8576
downloading Nix 2.12.0 binary tarball for x86_64-linux from 'https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz' to '/tmp/nix-binary-tarball-unpack.FASdoMsq37'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 19.5M  100 19.5M    0     0  55.9M      0 --:--:-- --:--:-- --:--:-- 56.0M
Note: a multi-user installation is possible. See https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation
performing a single-user installation of Nix...
directory /nix does not exist; creating it by running 'mkdir -m 0755 /nix && chown ana /nix' using sudo
copying Nix to /nix/store................................................
installing 'nix-2.12.0'
building '/nix/store/0dg0zjj2j6hijn193x1215yssrg7n1xs-user-environment.drv'...
unpacking channels...
modifying /home/ana/.profile...

Installation finished!  To ensure that the necessary environment
variables are set, either log in again, or type

  . /home/ana/.nix-profile/etc/profile.d/nix.sh

in your shell.
ana@ubuntu-base:~$ ls -lah /ni*
total 16K
drwxr-xr-x  4 ana  root 4.0K Dec 21 09:02 .
drwxr-xr-x 21 root root 4.0K Dec 21 09:02 ..
drwxr-xr-x 51 ana  ana  4.0K Dec 21 09:03 store
drwxr-xr-x  4 ana  ana  4.0K Dec 21 09:02 var

This doesn't allow for situations where multiple users wish to install Nix, or situations where root is actually the intended user to run nix build etc.

In fact, the existing Single User script refuses to run as root:

ana@ubuntu-base:~$ sudo su
root@ubuntu-base:/home/ana# sh <(curl -L https://nixos.org/nix/install) --no-daemon
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  4052  100  4052    0     0   9293      0 --:--:-- --:--:-- --:--:--  9293
downloading Nix 2.12.0 binary tarball for x86_64-linux from 'https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz' to '/tmp/nix-binary-tarball-unpack.hujxbv7VdU'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 19.5M  100 19.5M    0     0  54.1M      0 --:--:-- --:--:-- --:--:-- 54.1M
Note: a multi-user installation is possible. See https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation
warning: installing Nix as root is not supported by this script!
performing a single-user installation of Nix...
directory /nix does not exist; creating it by running 'mkdir -m 0755 /nix && chown root /nix' using sudo
copying Nix to /nix/store................................................
warning: the group 'nixbld' specified in 'build-users-group' does not exist
warning: the group 'nixbld' specified in 'build-users-group' does not exist
installing 'nix-2.12.0'
error: the group 'nixbld' specified in 'build-users-group' does not exist
/tmp/nix-binary-tarball-unpack.hujxbv7VdU/unpack/nix-2.12.0-x86_64-linux/install: unable to install Nix into your default profile

The current single user install method also lacks some sandboxing features that are present in the daemon-based multi-user mode, since the unprivileged user cannot do certain things. This can create situations where a user can build something in Single User Nix which fails to build in Multi-user nix due to isolation. While multi-user installs can invoke --impure to build those, it's still a frustrating experience.

We'd like to explore finding a better option here. Based discussions from the Installer Working Group and some polling of the community we determined that most people use the Single User variant for inside containers (like docker) or otherwise in build pipelines.

One idea that was proposed was to have a root daemonless variant instead, attempting to offer the user a workflow like this:

FROM ubuntu:latest
RUN apt update
RUN apt install curl --yes
RUN curl -L https://install.determinate.systems/nix | sh -s -- install linux-multi --no-confirm
RUN nix build .#hello -L /out
RUN createuser runner
USER runner
CMD /out/bin/runner

In this case, we'd create the /nix folder structure as normal, and perhaps even create build users/groups, and only root would be able to invoke nix related command that may alter the store. This mode would be able to benefit from all the normal isolation of multi-user installs and work without an init system in places like a build pipeline.

One note is that in a container we can only alter our hostname if we are --privileged, which many users won't want to do:

❯ sudo podman run --rm -ti --privileged ubuntu
root@784cf2044df2:/# hostname boop 
root@784cf2044df2:/# 
exit
❯ podman run --rm -ti ubuntu
root@f5e5e97eda12:/# hostname boop
hostname: you must be root to change the host name
root@f5e5e97eda12:/# 
exit

detect NixOS

That's unlikely anyone would try to use harmonic on NixOS like me, but the current result isn't good

> result/bin/harmonic -vv plan
  2022-10-26T12:33:46.914391Z TRACE harmonic::actions::meta::configure_shell_profile: Did not plan to edit `/etc/profile.d/nix.sh` as it does not exist.
    at src/actions/meta/configure_shell_profile.rs:32
    in harmonic::actions::meta::configure_shell_profile::plan
    in harmonic::actions::meta::configure_nix::plan
    in harmonic::cli::subcommand::plan::execute with channels: nixpkgs https://nixos.org/channels/nixpkgs-unstable, daemon_user_count: 32, no_modify_profile: false
    in harmonic::cli::execute

  2022-10-26T12:33:46.914521Z TRACE harmonic::actions::meta::configure_shell_profile: Did not plan to edit `/etc/zshrc` as it does not exist.
    at src/actions/meta/configure_shell_profile.rs:32
    in harmonic::actions::meta::configure_shell_profile::plan
    in harmonic::actions::meta::configure_nix::plan
    in harmonic::cli::subcommand::plan::execute with channels: nixpkgs https://nixos.org/channels/nixpkgs-unstable, daemon_user_count: 32, no_modify_profile: false
    in harmonic::cli::execute

  2022-10-26T12:33:46.914608Z TRACE harmonic::actions::meta::configure_shell_profile: Did not plan to edit `/etc/bash.bashrc` as it does not exist.
    at src/actions/meta/configure_shell_profile.rs:32
    in harmonic::actions::meta::configure_shell_profile::plan
    in harmonic::actions::meta::configure_nix::plan
    in harmonic::cli::subcommand::plan::execute with channels: nixpkgs https://nixos.org/channels/nixpkgs-unstable, daemon_user_count: 32, no_modify_profile: false
    in harmonic::cli::execute

  2022-10-26T12:33:46.914674Z TRACE harmonic::actions::meta::configure_shell_profile: Did not plan to edit `/etc/zsh/zshrc` as it does not exist.
    at src/actions/meta/configure_shell_profile.rs:32
    in harmonic::actions::meta::configure_shell_profile::plan
    in harmonic::actions::meta::configure_nix::plan
    in harmonic::cli::subcommand::plan::execute with channels: nixpkgs https://nixos.org/channels/nixpkgs-unstable, daemon_user_count: 32, no_modify_profile: false
    in harmonic::cli::execute

Error: 
   0: Error executing action
   1: Placing Nix configuration
   2: Creating file
   3: File exists `/etc/nix/nix.conf`

Location:
   src/cli/subcommand/plan.rs:58

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::plan::execute with channels=nixpkgs https://nixos.org/channels/nixpkgs-unstable daemon_user_count=32 no_modify_profile=false
      at src/cli/subcommand/plan.rs:24
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Steam Deck installation crashes desktop session

I tried the installer on my Steam Deck after reading your blog post. Thanks for your work, Nix may be a more convenient way of installing additional software on SteamOS.

The installer worked fine, but somehow crashed my desktop session. Here's some logs, maybe you can figure out what happened:

Installer output (looks as expected):

deck@steamdeck /sys curl -L https://install.determinate.systems/nix | sh -s -- install steam-deck                                                                    
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
100 15714  100 15714    0     0   5589      0  0:00:02  0:00:02 --:--:-- 48055
info: downloading installer (https://install.determinate.systems/nix/nix-installer-x86_64-linux)
`nix-installer` needs to run as `root`, attempting to escalate now via `sudo`...
Nix install plan (v0.0.0-unreleased)

Planner: steam-deck

Planner settings:

* extra_conf: []
* modify_profile: true
* nix_package_url: "https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz"
* channels: ["nixpkgs=https://nixos.org/channels/nixpkgs-unstable"]
* nix_build_group_id: 3000
* nix_build_group_name: "nixbld"
* force: false
* daemon_user_count: 32
* nix_build_user_id_base: 3000
* nix_build_user_prefix: "nixbld"
* persistence: "/home/nix"

The following actions will be taken (`--explain` for more context):

* Create directory `/home/nix`
* Create or overwrite file `/etc/systemd/system/nix-directory.service`
* Create or overwrite file `/etc/systemd/system/nix.mount`
* Create or overwrite file `/etc/systemd/system/ensure-symlinked-units-resolve.service`
* Enable (and start) the systemd unit ensure-symlinked-units-resolve.service
* Fetch `https://releases.nixos.org/nix/nix-2.12.0/nix-2.12.0-x86_64-linux.tar.xz` to `/nix/temp-install-dir`
* Create build users (UID 3000-3032) and group (GID 3000)
* Create a directory tree in `/nix`
* Move the downloaded Nix into `/nix`
* Setup the default Nix profile
* Configure Nix daemon related settings with systemd
* Place the Nix configuration in `/etc/nix/nix.conf`
* Place channel configuration at `/root/.nix-channels`
* Configure the shell profiles
* Enable (and start) the systemd unit nix-daemon.socket


Proceed? (y/N): y
 INFO Step: Create directory `/home/nix`
 INFO Step: Create or overwrite file `/etc/systemd/system/nix-directory.service`
 INFO Step: Create or overwrite file `/etc/systemd/system/nix.mount`
 INFO Step: Create or overwrite file `/etc/systemd/system/ensure-symlinked-units-resolve.service`
 INFO Step: Enable (and start) the systemd unit ensure-symlinked-units-resolve.service
 INFO Step: Provision Nix
 INFO Step: Configure Nix
 INFO Step: Enable (and start) the systemd unit nix-daemon.socket

Journal: journal.txt

Support resumption of partially successful installations and repair of broken installations by making every `Action` idempotent and comparing the actual state of the system to that in the plan or receipt

Hi there! I'm a Nix newbie who is very excited about this project. So far, I love it and I'm sold.

I've got some ideas that I want to share because I think they have merit. I understand that this project is experimental, and that what I'm suggesting is quite a large change. I'd like to solicit feedback from the maintainers to see if what I'm saying has value.

I want Nix installs to be automatically resumable and repairable. This means that if something goes wrong during an installation, or something gets broken after an installation, the installer can restore the system to a good state. This is not an easy thing to do. I don't think it'll be easy to retrofit either. With that said, it is possible to start supporting automatic resumption and repair, provided certain constraints on actions are made.

Currently, Actions are designed to be reversible. This is an excellent design decision. I'd like to propose two additional constraints on actions:

  1. Actions must be idempotent (in other words, safely "replayable").
  2. Actions must be able to compare the state of the actual installation against the state they expect.

Under these constraints, the application of a plan can be safely done under the following circumstances:

  1. No Nix has been installed
  2. An installation has partially succeeded
  3. Something external broke the installation in some way.

In some sense, the receipt becomes a write-ahead-log and the installer re-applies the actions in there, unless it knows the action is unnecessary.

By way of example, the following happens today:

$ curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
Nix was installed successfully!

# rm -rf /etc/nix

$ curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

Found existing plan in `/nix/receipt.json`, with the same settings, already completed, try uninstalling and reinstalling if Nix isn't working

$ /nix/nix-installer uninstall

Proceed? ([Y]es/[n]o/[e]xplain): y
 INFO Revert: Configure Nix
Error: 
   0: Error executing action
   1: Remove file `/etc/nix/nix.conf`
   2: No such file or directory (os error 2)

The problem is that a something (me) has made the actual state of the system diverge from that specified in the receipt. Currently, the installer doesn't detect this can can't patch it up. Additionally, the uninstaller cannot uninstall what's already there.

It would be valuable if the nix-installer could idempotently re-install Nix, patching over the differences between the actual and the expected state of the installation.

Deleting Users on Mac is not working

Deleting Users on a Mac isn't working right now for unknown reasons:

ephemeraladmin@mac-epic-turducken ~ % dscl . -read /Users/_nixbld1
dsAttrTypeNative:_writers_passwd: _nixbld1
dsAttrTypeNative:accountPolicyData:
 <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>creationTime</key>
        <real>1667407560.622637</real>
        <key>failedLoginCount</key>
        <integer>0</integer>
        <key>failedLoginTimestamp</key>
        <integer>0</integer>
        <key>passwordLastSetTime</key>
        <real>1667409170.2462959</real>
</dict>
</plist>

dsAttrTypeNative:IsHidden: 1
dsAttrTypeNative:record_daemon_version: 8770000
AppleMetaNodeLocation: /Local/Default
GeneratedUID: 95CA4ABC-6A65-4EE0-8430-1F8BC58E9C64
NFSHomeDirectory: /var/empty
Password: ********
PrimaryGroupID: 3000
RecordName: _nixbld1
RecordType: dsRecTypeStandard:Users
UniqueID: 301
UserShell: /sbin/nologin
ephemeraladmin@mac-epic-turducken ~ % sudo dscl . -delete Users/_nixbld1
<main> delete status: eDSPermissionError
<dscl_cmd> DS Error: -14120 (eDSPermissionError)

This suggests a password is required: https://apple.stackexchange.com/questions/310308/delete-a-standard-user-from-mac-os

ephemeraladmin@mac-epic-turducken ~ % sudo passwd _nixbld1
Changing password for _nixbld1.
New password:
Retype new password:

################################### WARNING ###################################
# This tool does not update the login keychain password.                      #
# To update it, run `security set-keychain-password` as the user in question, #
# or as root providing a path to such user's login keychain.                  #
###############################################################################

ephemeraladmin@mac-epic-turducken ~ % sudo dscl . -delete Users/_nixbld1
<main> delete status: eDSPermissionError
<dscl_cmd> DS Error: -14120 (eDSPermissionError)

Other resources:

How to handle Nix versions?

Right now we keep our Nix package URL as a const in the package, it may be ideal to determine a way to check what the most up to date option is during plan time.

install -> uninstall -> install fails

after installing nix and uninstalling using harmonic, it seems a systemd unit (nix-daemon.socket ?) was left in an incorrect state

this path will be fetched (0.14 MiB download, 0.47 MiB unpacked):
  /nix/store/7vj1h9s1yfkj66dh9645rk51pjpb0hkr-nss-cacert-3.80
copying path '/nix/store/7vj1h9s1yfkj66dh9645rk51pjpb0hkr-nss-cacert-3.80' from 'https://cache.nixos.org'...
building '/nix/store/3m5x22m8rnylzjg8xzkfb5vr7f984l1b-user-environment.drv'...
unpacking channels...

Created symlink /etc/systemd/system/nix-daemon.service → /nix/var/nix/profiles/default/lib/systemd/system/nix-daemon.service.
Created symlink /etc/systemd/system/nix-daemon.socket → /nix/var/nix/profiles/default/lib/systemd/system/nix-daemon.socket.
Created symlink /etc/systemd/system/sockets.target.wants/nix-daemon.socket → /nix/store/nmq5zcd93qb1yskx42rs910ff0247nn2-nix-2.11.0/lib/systemd/system/nix-daemon.socket.
Job failed. See "journalctl -xe" for details.
  2022-10-26T13:17:21.719777Z ERROR harmonic::cli::subcommand::install: 
   0: Error executing action
   1: Starting systemd unit
   2: Failed to execute command
   3: Command `"systemctl" "enable" "--now" "nix-daemon.socket"` failed status

Location:
   /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/convert/mod.rs:550

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::install::execute
      at src/cli/subcommand/install.rs:43
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
    at src/cli/subcommand/install.rs:88
    in harmonic::cli::subcommand::install::execute
    in harmonic::cli::execute

This Nix uninstall is for:
Operating System: Linux
Init system: systemd
Nix channels: nixpkgs=https://nixos.org/channels/nixpkgs-unstable

Created by planner: LinuxMultiUser
            
The following actions will be taken:
* Remove the directory tree in `/nix`

* Remove build users and group

* Remove the directory `/nix`

* Unconfigure the shell profiles

* Remove channel configuration at `/root/.nix-channels`

* Remove the nix configuration in `/etc/nix/nix.conf`

* Unconfigure Nix daemon related settings with systemd

* Unset the default Nix profile

        

Are you sure? (y/N): Cancelled!Okay, didn't do anything! Bye!

a systemctl status nix-daemon.socket shows this

$ sudo systemctl status nix-daemon.socket
○ nix-daemon.socket - Nix Daemon Socket
     Loaded: loaded (/etc/systemd/system/nix-daemon.socket; enabled; preset: enabled)
     Active: inactive (dead)
   Triggers: ● nix-daemon.service
     Listen: /nix/var/nix/daemon-socket/socket (Stream)

oct. 26 13:13:54 xubuntu systemd[1]: Listening on Nix Daemon Socket.
oct. 26 13:15:02 xubuntu systemd[1]: nix-daemon.socket: Socket unit configuration has changed while unit has been running, no open socket file descriptor left. The socket unit is >
oct. 26 13:15:03 xubuntu systemd[1]: nix-daemon.socket: Socket unit configuration has changed while unit has been running, no open socket file descriptor left. The socket unit is >
oct. 26 13:15:03 xubuntu systemd[1]: nix-daemon.socket: Socket unit configuration has changed while unit has been running, no open socket file descriptor left. The socket unit is >
oct. 26 13:15:03 xubuntu systemd[1]: nix-daemon.socket: Deactivated successfully.
oct. 26 13:15:03 xubuntu systemd[1]: Closed nix-daemon.socket.
oct. 26 13:17:21 xubuntu systemd[1]: nix-daemon.socket: Socket service nix-daemon.service already active, refusing.
oct. 26 13:17:21 xubuntu systemd[1]: Failed to listen on Nix Daemon Socket.

Add CI jobs to do Install -> Uninstall -> Install

Since the Github CI machines don't have Nix on them, we can just use them directly.

After the artifacts are produced, try to run harmonic install $PLAN and then do uninstall, then install again. This could be done on both x86 Linux and Darwin

can't uninstall an interrupted install

I've been doing an install with harmonic but it failed due to network issues, however I can't use the uninstall process to rollback the changes.

Error: 
   0: Reading receipt
   1: No such file or directory (os error 2)

Location:
   src/cli/subcommand/uninstall.rs:42

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::uninstall::execute
      at src/cli/subcommand/uninstall.rs:32
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

`install` should detect existing `/nix/receipt.json`

Right now a user who runs an install can only uninstall, then try again from scratch. This differs from how uninstall works, where if it breaks the user can resolve the issue and run it again.

During the harmonic install $PLAN phase, we should detect an existing receipt.json, if the planner is the same as the one the user invokes, we should prompt the user if they want to resume that install, or instead uninstall completely and reinstall.

This differs from #27 since it discusses an existing Nix install, not an existing Harmonic-installed Nix.

Too many error types

Right now each of the builtin actions have their own error types.

We don't need to do that.

We should be able to create a common ActionError covering most cases. Since actions can return Box errors, they can always create their own new error if needed.

Nix Installer adds a superflous newline in `nix.conf` in GitHub Actions

I encountered an issue in the Nix Installer GitHub Action where the access-tokens config is prefaced with \n when written to nix.conf:

\naccess-tokens = *****

The basic contours of my Actions configuration:

- uses: DeterminateSystems/nix-installer@main
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    extra-conf: |
      binary-caches = https://my-special-cache.org
      trusted-substituters = https://my-special-cache.org

I debugged this by adding an Action that cats the config file, which appears to be as expected otherwise.

The offending line seems to be here.

Flesh out docs

One of the design goals of Harmonic is to be usable as a library.

As a consequence of that, we need to be usable as a library, and that means docs.

check if running as root

harmonic should check if the current user is root (or equivalent on MacOS), otherwise it will just fail with a permission denied error.

Plan compatability tests

We need some test fixtures which our unit tests and attempt to deserialize and validate, so that we will know if we ever break or alter plan format.

Consider future design options

The process of installing Nix involves a number of steps which:

  • Are fallible
  • Should support --dry-run
  • Sometimes OS dependent
  • Sometimes option dependent
  • Depend on one another
  • Can, at times, be done in parallel
  • Would be nice to be "rolled back" in the case of a failure
  • Would be nice to summarize for the user before being performed.

Right now, we have a Harmonic type which performs each of these steps linearly, and it only supports multi-user x86 Linux systemd installs.

States and Transitions

While the current API 'works', it's very easy to misuse it, using steps in the wrong order, or missing steps entirely.

It would be desirable to use a pattern with a Builder (implementing a builder pattern) which would produce Harmonic<State> types where Harmonic held onto those built options as well as a State which models which stage of the install was occurring.

This state machine could have states such as:

  • State::Initialized: Does fetch/unpack process into a tempdir, creates users/groups.
  • State::Initialized -> State::Provisioned: Upserts directories/files, moves tempdir's store into place
  • State::Provisioned -> State::Bootstrapped: Runs nix-env -i on nix (etc), runs nix-channel --update
  • State::Bootstrapped -> State::Ready: Starts nix-daemon, does a test build

Forming a directed acyclic graph.

In the case of platforms such as Darwin, different states can be defined and swapped out, without impacting the rest of the states.

Actions & Plans

The current nix installer does a great job communicating what it is doing in great detail. In Rust, we won't always have a 1:1 matching to bash commands a user could run.

It would be quite maintainable if we could have some sort of "plan" we generate with all the steps we'd take, so that we could print this out to the user.

Furthermore, if we did have some plan we could 'drive' that plan to completion (probably via states as described above), or roll back the plan with minimal blast radius on the system.

As it stands now each step in the process is doing rough control flow do execute fairly generic actions:

https://github.com/DeterminateSystems/harmonic/blob/9873c864efd83374131085e71051aee6ce035124/src/lib.rs#L144-L197

https://github.com/DeterminateSystems/harmonic/blob/9873c864efd83374131085e71051aee6ce035124/src/lib.rs#L478-L494

If we had some way to generate a plan, or at least keep a sequence of the actions which we take, we could later roll them back.

Confirm installation is successful

I just used nix-installer and the end of the output looks like

INFO Step: Create directory /nix
INFO Step: Provision Nix
INFO Step: Configure Nix

Now, I'm curious to know if it aborted or finished correctly, a simple message such as INFO Nix is installed or something like this would be nice.

Missing indicator for successful install

One thing that could be improved overall is that we never tell the user they may need to reload their environment (by opening a new terminal or exec $SHELL or something). If you run it without -v, it just silently exits (which is fine, though I would like some indication that it completed successfully by default...), and if you try to nix-shell right after that, it won't work and the user is left guessing if it actually completed, or if something went wrong. I'll make a ticket about this.

Originally posted by @cole-h in #34 (review)

Basically, I think this issue is twofold:

  1. We never notify the user that the install finished successfully
  2. We never notify the user that they may need to restart their shell (either by opening a new terminal, exec $SHELL, or anything else)

Example screenshot where I fumble through this exact thing:

image

Remove unnecessary Github Actions

We have some github actions doing builds that can be cleaned up since they are done in buildkite now

nix-installer/.github/workflows/ci.yml

Lines 33 to 84 in 1849d90

build-x86_64-linux:
name: Build x86_64 Linux
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- uses: cachix/install-nix-action@v17
with:
extra_nix_config: |
access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
- name: Cache build store (x86_64-linux)
id: build-store-x86_64-linux
uses: actions/cache@v3
with:
path: ~/.ci-store
key: build-store-x86_64-linux-${{ hashFiles('**/Cargo.lock', '**/flake.lock') }}-v1
- name: Build `nix-installer-static`
run: nix build --store ~/.ci-store --print-build-logs .#packages.x86_64-linux.nix-installer-static
- name: Copy artifact
run: |
RESULT=$(nix eval --raw --store ~/.ci-store --print-build-logs .#packages.x86_64-linux.nix-installer-static --apply "x: \"$HOME/.ci-store\${x}\"")
cp $RESULT/bin/nix-installer nix-installer
- name: Create artifact for x86_64-linux nix-installer
uses: actions/upload-artifact@v3
with:
name: nix-installer-x86_64-linux
path: |
nix-installer
# Various feature variations or versions of the build which we expect to always work
# Since it uses the `build-store-x86_64-linux-*` cache it should be quite fast
build-variants-x86_64-linux:
name: Build x86_64 Linux (Variants)
runs-on: ubuntu-22.04
needs: build-x86_64-linux # Only run this if the normal checks work, to avoid clogging builders
steps:
- uses: actions/checkout@v3
- uses: cachix/install-nix-action@v17
with:
extra_nix_config: |
access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
- name: Cache build store (x86_64-linux)
id: build-store-x86_64-linux
uses: actions/cache@v3
with:
path: ~/.ci-store
key: build-store-x86_64-linux-${{ hashFiles('**/Cargo.lock', '**/flake.lock') }}-v1
- name: Build `nix-installer`
run: nix build --store ~/.ci-store --print-build-logs .#packages.x86_64-linux.nix-installer
- name: Test `nix develop` build without default features
run: nix develop --store ~/.ci-store --print-build-logs .# --command "cargo" build --no-default-features
- name: Test `nix develop` build all features
run: nix develop --store ~/.ci-store --print-build-logs .# --command "cargo" build --all-features

Various Actions can be idempotent

Actions like CreateFile, CreateDirectory, CreateGroup, and CreateUser can be idempotent.

This would be advantageous because it would help us "keep going" over partially installed Nix's.

Caution should be taken to add a new ActionState which signals to the uninstaller that the user/folder/group etc need not be removed.

These actions probably can then be renamed to like EnsureFile and EnsureFileContents and EnsureUser.

Ubuntu 16.04 lacks `sudo --preserve-env` with args 

error: builder for '/nix/store/v7khh9q4bpbism2nqkpxgs1k0ay711a3-installer-test-ubuntu-v16.04-install-default.drv' failed with exit code 1;
       last 10 log lines:
       > sudo: option '--preserve-env' doesn't allow an argument
       > usage: sudo -h | -K | -k | -V
       > usage: sudo -v [-AknS] [-g group] [-h host] [-p prompt] [-u user]
       > usage: sudo -l [-AknS] [-g group] [-h host] [-p prompt] [-U user] [-u user]
       >             [command]
       > usage: sudo [-AbEHknPS] [-r role] [-t type] [-C num] [-g group] [-h host] [-p
       >             prompt] [-u user] [VAR=value] [-i|-s] [<command>]
       > usage: sudo -e [-AknS] [-r role] [-t type] [-C num] [-g group] [-h host] [-p
       >             prompt] [-u user] file ...
       > qemu-kvm: terminating on signal 15 from pid 1 ()
       For full logs, run 'nix log /nix/store/v7khh9q4bpbism2nqkpxgs1k0ay711a3-installer-test-ubuntu-v16.04-install-default.drv'.
error: 1 dependencies of derivation '/nix/store/dkn95xgy5g9icl7pbh3qrdf5c9r2y1iw-all.drv' failed to build
⏎

This was discovered during #138

Allow users to answer `e` at install prompt

Users may wish to get an explanation for their plan and not wish to restart the application to get there. When we prompt the user to allow an install/revert, we should support y and n as we do now, but also e which re-prints with explanation.

`cli` feature

With harmonic usable as a library, we should provide a way to flag off CLI related code and dependencies. We have at least 8 top level dependencies which are only required for CLI:

  • eyre
  • color_eyre
  • atty
  • clap
  • crossterm
  • tracing-subscriber
  • tracing-error
  • owo-colors

These should be made optional dependencies and related code put behind a #[cfg(feature = "cli")].

This should probably wait until #65 is merged up.

`InstallPlan` should not copy self, binary should

If the user is using the nix-installer as a rust library, we can't promise that std::env::current_exe() is the nix-installer binary.

This step should happen in the binary itself, not the library code.

`plan` and `install` should detect if nix is already installed

That would be nice to detect if nix was already installed when trying install or plan, I suppose there code is there because the uninstall process already do that.

Here are the results of install and plan after a successful installation

$ sudo ./harmonic install
Error: 
   0: Error executing action
   1: Placing channel configuration
   2: Creating file
   3: File exists `/root/.nix-channels`

Location:
   src/cli/subcommand/install.rs:77

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::install::execute
      at src/cli/subcommand/install.rs:43
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

and

$ sudo ./harmonic plan
Error: 
   0: Error executing action
   1: Placing channel configuration
   2: Creating file
   3: File exists `/root/.nix-channels`

Location:
   src/cli/subcommand/plan.rs:58

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: harmonic::cli::subcommand::plan::execute with channels=nixpkgs https://nixos.org/channels/nixpkgs-unstable daemon_user_count=32 no_modify_profile=false
      at src/cli/subcommand/plan.rs:24
   1: harmonic::cli::execute
      at src/cli/mod.rs:29

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.