Make it easier to gather log data about archinstall

[grazzolini]

I think that, there could be an installation summary with details about partitioning, bootloader, packages installed, locale, etc. This is what whomever helping the person that used archinstall could ask for. And, there could also be a more detailed log, this one with loglevels, that could also be sent in case the summary wasn't enough.

But I understand that the summary would be a different feature than logs.

[grazzolini]

Are you also going to support loglevels?

[Torxed]

Are you also going to support loglevels?

That's the idea. It previously existed. But for some reason I decided it might not be needed or the systemd dependency was to much. My only fear is that the default log-level might be to verbose / not verbose enough for support personnel, so I'd like to hone in on that aspect. What's needed? and how much of it?

The start will probably be kept simple, and more is better at the beginning.
I could also pipe it straight out to journalctl as a unit file, as a optional dependency.

[Torxed]

I think that, there could be an installation summary with details about partitioning, bootloader, packages installed, locale, etc. This is what whomever helping the person that used archinstall could ask for. And, there could also be a more detailed log, this one with loglevels, that could also be sent in case the summary wasn't enough.

But I understand that the summary would be a different feature than logs.

This seems to summarize the need pretty well.

I'll have some different stages for output and hard logs, and possibly a clever open source way of optionally delivering automatic report uploads with visual confirmation (and read-through) of bigger log data collections. So beginners can get the log data out of whatever system they're using on a non-trivial way. Either via USB-storage detection or file upload to a trusted server of sorts.

[Torxed]

b988508 [56-log-data] is a first attempt to re-introduce log levels as well as a more joint file output file.

examples/guided.py now sets up a global logfile definition in archinstall.storage['logfile'] which archinstall's different function calls calling log() will honor where applicable. archinstall.Installer() also honors this, so any subsequent installation function-call should inherit and honor it as well. If omitted, some things won't end up in the log. For instance, luks2() calls won't log to the join log file unless the global definition has been made. Because it's outside of archinstall.Installer() which sets up a centralized log file for that installation session - if no global config was defined ahead of time.

Logs are saved in ~/.cache/archinstall and the log name format is install-session_%Y-%M-%D %H:%M:%S.{milliseconds}.log. I'll probably remove the white space between date and time. And the date format is discussable, but I live where this format is the norm so that's what I was used to. Not sure what the general mass is used to : )

[Torxed]

Branch [56-log-data] has been tested.

Everything is looking good, I'll re-run some tests tomorrow to get some more extensive edge case tests in as well.

Assuming those goes according to plan, there will be a simple bug more than good enough log file I'd say under ~/.cache/archinstall/ that users can submit/upload when asking for help. Which we can work on and put more stuff in to as need be. For instance, all the user_interactions.py functionality could be plotted in there rather easy if we want all the user-responses as well, not just the result of them (which usually contains the response data one way or another anyway).

[Torxed]

I've now moved (default value) the log configuration to storage.py for a more centralized configuration.
(The file name is perhaps a bit odd, settings.py or something would prooobably make more sense in the long run, and keep storage for storage stuff - for instance session variables etc).

The change:

• https://github.com/Torxed/archinstall/blob/40622265a214be0531482ee91ffa61ef2431ae43/archinstall/lib/storage.py#L19-L20

This configuration can be overridden by any installation here:

• https://github.com/Torxed/archinstall/blob/40622265a214be0531482ee91ffa61ef2431ae43/archinstall/lib/installer.py#L44-L47

The install log will contain everything the user sees, except the input (which is reflected in the config which is outputted), here's an example from /var/log/archinstall/install.log:

-- Guided template chosen (with below config) --
{
    "disk_encryption": true,
    "harddrive": {
        "model": "QEMU_HARDDISK",
        "path": "/dev/sda",
        "size": "8G"
    },
    "hostname": "testmachine",
    "keyboard_layout": "sv-latin1",
    "mirrors": {
        "Sweden": {
            "https://archlinux.dynamict.se/$repo/os/$arch": true,
            "https://ftp.acc.umu.se/mirror/archlinux/$repo/os/$arch": true,
            "https://ftp.lysator.liu.se/pub/archlinux/$repo/os/$arch": true,
            "https://ftp.myrveln.se/pub/linux/archlinux/$repo/os/$arch": true,
            "https://mirror.osbeck.com/archlinux/$repo/os/$arch": true
        }
    },
    "network": {
        "nic": "enp2s0"
    },
    "packages": [
        "nano"
    ],
    "profile": {
        "path": "/root/archinstall-git/profiles/awesome.py"
    },
    "root_unlocked": true,
    "users": [
        "anton"
    ]
}
Adding partition to BlockDevice(/dev/sda)
Adding partition to BlockDevice(/dev/sda)
Formatting Partition(path=/dev/sda1, fs=None, mounted=None) -> fat32
Encrypting Partition(path=/dev/sda2, fs=None, mounted=None) (This might take a while)
Formatting Partition(path=/dev/mapper/luksloop, real_device=/dev/sda2, fs=None, mounted=None) -> btrfs
Mounting Partition(path=/dev/mapper/luksloop, real_device=/dev/sda2, fs=btrfs, mounted=None) to /mnt
Mounting Partition(path=/dev/sda1, fs=fat32, mounted=None) to /mnt/boot
Waiting for automatic mirror selection has completed before using custom mirrors.
A new package mirror-list has been created: /etc/pacman.d/mirrorlist
Installing packages: ['base', 'base-devel', 'linux', 'linux-firmware', 'efibootmgr', 'nano', 'btrfs-progs']
A new package mirror-list has been created: /mnt/etc/pacman.d/mirrorlist
Adding bootloader systemd-bootctl to Partition(path=/dev/sda1, fs=fat32, mounted=/mnt/boot)
Enabling service systemd-networkd
Enabling service systemd-resolved
Installing packages: ['nano']
Installing network profile Profile(awesome.py)
Installing network profile Profile(xorg)
Installing packages: ('xorg-server xorg-xinit xf86-video-vmware',)
Installing packages: ('awesome xorg-xrandr xterm feh slock terminus-font gnu-free-fonts ttf-liberation xsel',)
Installing packages: ('chromium openssh sshfs git htop pkgfile scrot dhclient wget libu2f-host nemo gpicview-gtk3 nano',)
Installing packages: ('alacritty',)
Creating user anton
Setting password for anton
Setting password for root
Updating /mnt/etc/fstab
Installation completed without any errors. You may now reboot.

And here's what it looks like with an error thrown right near the end of an installation:

Installing network profile Profile(awesome.py)
Installing packages: ('awesome xorg-xrandr xterm feh slock terminus-font-otb gnu-free-fonts ttf-liberation xsel',)
'/usr/bin/pacstrap /mnt awesome xorg-xrandr xterm feh slock terminus-font-otb gnu-free-fonts ttf-liberation xsel' did not exit gracefully, exit code 256.
==> Creating install root at /mnt
==> Installing packages to /mnt
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
error: target not found: terminus-font-otb
==> ERROR: Failed to install packages to new root

Which indicated a faulty package list in awesome.py in this case.

Any exception thrown during the installation (either system commands failing or programming errors in the Python code) will be handled by the __exit__ function of the installer. Custom messages can be thrown to it or default errors will be passed:

• https://github.com/Torxed/archinstall/blob/7eaee2cd4817c1aaca013f8baa267ddf4503f678/archinstall/lib/installer.py#L74-L83

Also added a short info about where to find the log in case it crashes:

• https://github.com/Torxed/archinstall/blob/7eaee2cd4817c1aaca013f8baa267ddf4503f678/archinstall/lib/installer.py#L81-L82

[Torxed]

Closing this as the logging has been improved enough (I think), log file is also copied over to the install medium.
There's a printout on errors where to find it and what to do with it (post it here).

If something needs adjusting before perhaps adding this as a default package on the ISO, I'll re-open again and work on it some more :)