projectpotos / potos-iso-builder Goto Github PK
View Code? Open in Web Editor NEWPotos Installation Boot Image
License: GNU General Public License v3.0
Potos Installation Boot Image
License: GNU General Public License v3.0
I did a few experiments on slow hardware. I experienced that
prepare.yml
and playbook.yml
from there.basics
which installs ansible-pull
and the associated timer
units. When this happens just before the full hour, it is possible that that timer fires while the playbook.yml
spawned from finish.sh
is still running.While the temporary file locations are different and do not conflict, most likely one or the other apt
lock fails (on either side)...
Possible Solutions:
lock
file from ansible-pull
to finish.sh
basics
to the end of my requirements.yml
(I don't think this is actually a solution)ansible-pull
timers to a handler (that will be executed only after all roles are done). Not so nice IMHO as there is a risk that if the first ansible-pull is aborted (e.g. power failure on host), the time window for a "not activated timer" becomes quite long. This would result in an unmanaged client that needs a one time manual undisturbed run of ansible-pull to fix.see above
No response
For a clean setup on every system independent of the os (version) we need to run everything in a virtual env
No response
Ansible-vault key doesn't get correctly installed on a new machine. Probably related to #
Create image with ansible-vault key
No response
I often find myself having to go onto potos.dev to copy and paste the docker command to build the ISO. It would be very nice if this + checksum check were in a script delivered with the repository. I could implement this myself but I am unsure if you prefer the makefile or the bashscript approach or even think something else is better.
No response
Version is fixed to 22.04:
potos-iso-builder/potos-iso/create-iso
Line 46 in 419357f
Create this dynamically (or at least with parameters)
See:
potos-iso-builder/potos-iso/create-iso
Line 46 in 419357f
No response
potos-iso-builder/potos-iso/create-iso
Line 74 in 419357f
We recommend the usage of ansible-core
instead of ansible
in autoinstall, but that conflicts with bug ansible/ansible#77624 - galaxy install collections are failing - as Ubuntu 22.04.01 servers the affected ansible-core v2.12.
No response
As discussed, the blocksize should be correctly determined automatically and not need to be adjusted for each iso manually
No response
When a value is not set in the config.yml
the default is not applied but instead the value null
is used.
Just leave a value out in the config. For example leave out output.iso_filename
and the iso file will be generated into a file called null
#23 tries to solve this. It works for regular values, but not for lists as required for the package list of software to be installed.
The setup is not aborted, when a task is failing. For example when an invaild specs repo is given, there are just a bunch of ignored erros but no fail of the setup.
Setup an client with an invalid specs repo
relevant sniplets from setup.log
when defining the specs repo to be at [email protected]/fadnincx/ansible-specs-fadnincx.git
instead of [email protected]:fadnincx/ansible-specs-fadnincx.git
Cloning into '/tmp/potos_ansible'...
created virtual environment CPython3.10.6.final.0-64 in 413ms
WARNING: The directory '/run/gnome-initial-setup/.cache/pip' or its parent directory is not owned or is
not writable by the current user. The cache has been disabled. Check the permissions and owner of that d
irectory. If executing pip with sudo, you should use sudo's -H flag.
Collecting ansible-core==2.12.3
Successfully built ansible-core
Installing collected packages: resolvelib, PyYAML, pycparser, packaging, MarkupSafe, jinja2, cffi, cryptography, ansible-core
Successfully installed MarkupSafe-2.1.1 PyYAML-6.0 ansible-core-2.12.3 cffi-1.15.1 cryptography-39.0.0 jinja2-3.1.2 packaging-23.0 pycparser-2.21 resolvelib-0.5.4
[WARNING]: log file at /var/log/potos/ansible.log is not writeable and we cannot create it, aborting
# ansible-playbook [core 2.12.3]
# config file = /tmp/potos_ansible/ansible.cfg
# configured module search path = ['/run/gnome-initial-setup/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
# ansible python module location = /tmp/potos_ansible/lib/python3.10/site-packages/ansible
# ansible collection location = /etc/ansible/collections
# executable location = /tmp/potos_ansible/bin/ansible-playbook
# python version = 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]
# jinja version = 3.1.2
# libyaml = True
# Using /tmp/potos_ansible/ansible.cfg as config file
# host_list declined parsing /tmp/potos_ansible/inventory as it did not pass its verify_file() method
# script declined parsing /tmp/potos_ansible/inventory as it did not pass its verify_file() method
# auto declined parsing /tmp/potos_ansible/inventory as it did not pass its verify_file() method
# Parsed /tmp/potos_ansible/inventory inventory source with ini plugin
# Skipping callback 'default', as we already have a stdout callback.
# Skipping callback 'minimal', as we already have a stdout callback.
# Skipping callback 'oneline', as we already have a stdout callback.
#
# PLAYBOOK: prepare.yml **********************************************************
# 1 plays in prepare.yml
#
# PLAY [Potos preparation playbook] **********************************************
#
# TASK [read specs repo configuration] *******************************************
# task path: /tmp/potos_ansible/prepare.yml:11
# ok: [localhost] => {
# "ansible_facts": {
# "potos_specs": {
# "client_name": "Fadnincx Linux Client",
# "client_short_name": "flc",
# "git_ansible_vault": "True",
# "git_branch": "main",
# "git_repo": "ansible-specs-fadnincx",
# "git_ssh_key": "True",
# "git_url": "ssh://[email protected]/fadnincx/"
# }
# },
# "ansible_included_var_files": [
# "/etc/potos/specs_repo.yml"
# ],
# "changed": false
# }
#
# TASK [Get specs repo for Fadnincx Linux Client] ********************************
# task path: /tmp/potos_ansible/prepare.yml:15
# ok: [localhost] => {
# "ansible_facts": {
# "discovered_interpreter_python": "/usr/bin/python3"
# },
# "changed": false,
# "cmd": "/usr/bin/git ls-remote 'ssh:********@github.com/fadnincx//ansible-specs-fadnincx.git' -h refs/heads/main",
# "failed_when_result": false,
# "invocation": {
# "module_args": {
# "accept_hostkey": false,
# "accept_newhostkey": true,
# "archive": null,
# "archive_prefix": null,
# "bare": false,
# "clone": true,
# "depth": null,
# "dest": "/tmp/potos_ansible/specs",
# "executable": null,
# "force": false,
# "gpg_whitelist": [],
# "key_file": "/etc/potos/specs_key",
# "recursive": true,
# "reference": null,
# "refspec": null,
# "remote": "origin",
# "repo": "ssh://[email protected]/fadnincx//ansible-specs-fadnincx.git",
# "separate_git_dir": null,
# "single_branch": true,
# "ssh_opts": null,
# "track_submodules": false,
# "umask": null,
# "update": true,
# "verify_commit": false,
# "version": "main"
# }
# },
# "msg": "Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.\r\nfatal: remote error: \n fadnincx//ansible-specs-fadnincx is not a valid repository name\n Visit https://support.github.com/ for help",
# "rc": 128,
# "stderr": "Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.\r\nfatal: remote error: \n fadnincx//ansible-specs-fadnincx is not a valid repository name\n Visit https://support.github.com/ for help\n",
# "stderr_lines": [
# "Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.",
# "fatal: remote error: ",
# " fadnincx//ansible-specs-fadnincx is not a valid repository name",
# " Visit https://support.github.com/ for help"
# ],
# "stdout": "",
# "stdout_lines": []
# }
# TASK [Check if specs exists] ***************************************************
# task path: /tmp/potos_ansible/prepare.yml:26
# ok: [localhost] => {
# "changed": false,
# "invocation": {
# "module_args": {
# "checksum_algorithm": "sha1",
# "follow": false,
# "get_attributes": true,
# "get_checksum": true,
# "get_md5": false,
# "get_mime": true,
# "path": "/tmp/potos_ansible/specs"
# }
# },
# "stat": {
# "exists": false
# }
# }
#
# TASK [Error handling] **********************************************************
# task path: /tmp/potos_ansible/prepare.yml:31
# ok: [localhost] => {
# "msg": "Unable to find an potos specs repo for Fadnincx Linux Client"
# }
#
# TASK [Get templates] ***********************************************************
# task path: /tmp/potos_ansible/prepare.yml:40
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [Create parent dir for templates] *****************************************
# task path: /tmp/potos_ansible/prepare.yml:47
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [Apply templates] *********************************************************
# task path: /tmp/potos_ansible/prepare.yml:54
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [Get files to copy] *******************************************************
# task path: /tmp/potos_ansible/prepare.yml:60
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [Create parent dir for files] *********************************************
# task path: /tmp/potos_ansible/prepare.yml:67
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [Copy files] **************************************************************
# task path: /tmp/potos_ansible/prepare.yml:74
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [Get vars to copy] ********************************************************
# task path: /tmp/potos_ansible/prepare.yml:80
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [Create parent dir for vars] **********************************************
# task path: /tmp/potos_ansible/prepare.yml:87
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [Copy vars] ***************************************************************
# task path: /tmp/potos_ansible/prepare.yml:94
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [Install required collections] ********************************************
# task path: /tmp/potos_ansible/prepare.yml:100
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
# PLAY RECAP *********************************************************************
# localhost : ok=4 changed=0 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0
#
[WARNING]: log file at /var/log/potos/ansible.log is not writeable and we cannot create it, aborting
# ansible-playbook [core 2.12.3]
# config file = /tmp/potos_ansible/ansible.cfg
# configured module search path = ['/run/gnome-initial-setup/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
# ansible python module location = /tmp/potos_ansible/lib/python3.10/site-packages/ansible
# ansible collection location = /etc/ansible/collections
# executable location = /tmp/potos_ansible/bin/ansible-playbook
# python version = 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]
# jinja version = 3.1.2
# libyaml = True
# Using /tmp/potos_ansible/ansible.cfg as config file
# host_list declined parsing /tmp/potos_ansible/inventory as it did not pass its verify_file() method
# script declined parsing /tmp/potos_ansible/inventory as it did not pass its verify_file() method
# auto declined parsing /tmp/potos_ansible/inventory as it did not pass its verify_file() method
# Parsed /tmp/potos_ansible/inventory inventory source with ini plugin
# Skipping callback 'default', as we already have a stdout callback.
# Skipping callback 'minimal', as we already have a stdout callback.
# Skipping callback 'oneline', as we already have a stdout callback.
#
# PLAYBOOK: playbook.yml *********************************************************
# 1 plays in playbook.yml
#
# PLAY [Potos system configuration] **********************************************
#
# TASK [Gathering Facts] *********************************************************
# task path: /tmp/potos_ansible/playbook.yml:3
# ok: [localhost]
#
# TASK [read specs repo configuration] *******************************************
# task path: /tmp/potos_ansible/playbook.yml:15
# ok: [localhost] => {
# "ansible_facts": {
# "potos_specs": {
# "client_name": "Fadnincx Linux Client",
# "client_short_name": "flc",
# "git_ansible_vault": "True",
# "git_branch": "main",
# "git_repo": "ansible-specs-fadnincx",
# "git_ssh_key": "True",
# "git_url": "ssh://[email protected]/fadnincx/"
# }
# },
# "ansible_included_var_files": [
# "/etc/potos/specs_repo.yml"
# ],
# "changed": false
# }
#
# TASK [define client_name] ******************************************************
# task path: /tmp/potos_ansible/playbook.yml:22
# ok: [localhost] => {
# "ansible_facts": {
# "potos_plays_client_full_name": "Fadnincx Linux Client",
# "potos_plays_client_name": "flc",
# "potos_plays_client_short_name": "flc"
# },
# "changed": false
# }
#
# TASK [template requirements.yml file] ******************************************
# task path: /tmp/potos_ansible/playbook.yml:30
#
# <localhost> EXEC /bin/sh -c 'rm -f -r /tmp/.ansible/tmp/ansible-tmp-1673463593.8693933-2761-1625774097
94740/ > /dev/null 2>&1 && sleep 0'
# The full traceback is:
# Traceback (most recent call last):
# File "/tmp/potos_ansible/lib/python3.10/site-packages/ansible/plugins/action/template.py", line 97, in run
# source = self._find_needle('templates', source)
# File "/tmp/potos_ansible/lib/python3.10/site-packages/ansible/plugins/action/__init__.py", line 1364, in _find_needle
# return self._loader.path_dwim_relative_stack(path_stack, dirname, needle)
# File "/tmp/potos_ansible/lib/python3.10/site-packages/ansible/parsing/dataloader.py", line 341, in path_dwim_relative_stack
# raise AnsibleFileNotFound(file_name=source, paths=[to_native(p) for p in search])
# ansible.errors.AnsibleFileNotFound: Could not find or access 'requirements.yml.j2'
# Searched in:
# /tmp/potos_ansible/templates/requirements.yml.j2
# /tmp/potos_ansible/requirements.yml.j2
# /tmp/potos_ansible/templates/requirements.yml.j2
# /tmp/potos_ansible/requirements.yml.j2 on the Ansible Controller.
# If you are using a module and expect the file to exist on the remote, see the remote_src option
# fatal: [localhost]: FAILED! => {
# "changed": false,
# "msg": "Could not find or access 'requirements.yml.j2'\nSearched in:\n\t/tmp/potos_ansible/templates/requirements.yml.j2\n\t/tmp/potos_ansible/requirements.yml.j2\n\t/tmp/potos_ansible/templates/requirements.yml.j2\n\t/tmp/potos_ansible/requirements.yml.j2 on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"
# }
# ...ignoring
#
# TASK [load roles from requirements file] ***************************************
# task path: /tmp/potos_ansible/playbook.yml:41
[WARNING]: Unable to find 'requirements.yml' in expected paths (use -vvvvv to
see paths)
# exception during Jinja2 execution: Traceback (most recent call last):
# File "/tmp/potos_ansible/lib/python3.10/site-packages/ansible/plugins/lookup/file.py", line 83, in run
# raise AnsibleParserError()
# ansible.errors.AnsibleParserError
#
# During handling of the above exception, another exception occurred:
#
# Traceback (most recent call last):
# File "/tmp/potos_ansible/lib/python3.10/site-packages/ansible/template/__init__.py", line 1015, in _lookup
# ran = instance.run(loop_terms, variables=self._available_variables, **kwargs)
# File "/tmp/potos_ansible/lib/python3.10/site-packages/ansible/plugins/lookup/file.py", line 85, in run
# raise AnsibleError("could not locate file in lookup: %s" % term)
# ansible.errors.AnsibleError: could not locate file in lookup: requirements.yml
# fatal: [localhost]: FAILED! => {
# "msg": "An unhandled exception occurred while running the lookup plugin 'file'. Error was a <class 'ansible.errors.AnsibleError'>, original message: could not locate file in lookup: requirements.yml. could not locate file in lookup: requirements.yml"
# }
# ...ignoring
#
# TASK [remove old files] ********************************************************
# task path: /tmp/potos_ansible/playbook.yml:47
# ok: [localhost] => (item=absent) => {
# "ansible_loop_var": "item",
# "changed": false,
# "invocation": {
# "module_args": {
# "_diff_peek": null,
# "_original_basename": null,
# "access_time": null,
# "access_time_format": "%Y%m%d%H%M.%S",
# "attributes": null,
# "follow": true,
# "force": false,
# "group": "0",
# "mode": "0755",
# "modification_time": null,
# "modification_time_format": "%Y%m%d%H%M.%S",
# "owner": "0",
# "path": "roles/",
# "recurse": false,
# "selevel": null,
# "serole": null,
# "setype": null,
# "seuser": null,
# "src": null,
# "state": "absent",
# "unsafe_writes": false
# }
# },
# "item": "absent",
# "path": "roles/",
# "state": "absent"
# }
# ok: [localhost] => (item=directory) => {
# "ansible_loop_var": "item",
# "changed": false,
# "diff": {
# "after": {
# "path": "roles/",
# "state": "directory"
# },
# "before": {
# "path": "roles/",
# "state": "absent"
# }
# },
# "gid": 0,
# "group": "root",
# "invocation": {
# "module_args": {
# "_diff_peek": null,
# "_original_basename": null,
# "access_time": null,
# "access_time_format": "%Y%m%d%H%M.%S",
# "attributes": null,
# "follow": true,
# "force": false,
# "group": "0",
# "mode": "0755",
# "modification_time": null,
# "modification_time_format": "%Y%m%d%H%M.%S",
# "owner": "0",
# "path": "roles/",
# "recurse": false,
# "selevel": null,
# "serole": null,
# "setype": null,
# "seuser": null,
# "src": null,
# "state": "directory",
# "unsafe_writes": false
# }
# },
# "item": "directory",
# "mode": "0755",
# "owner": "root",
# "path": "roles/",
# "size": 4096,
# "state": "directory",
# "uid": 0
# }
#
# TASK [load roles via git] ******************************************************
# task path: /tmp/potos_ansible/playbook.yml:60
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [get roles to be loaded via ansible-galaxy] *******************************
# task path: /tmp/potos_ansible/playbook.yml:83
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [get temp file to store requirements for roles to be loaded with ansible-galaxy] ***
# task path: /tmp/potos_ansible/playbook.yml:90
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [write temp file with roles to be loaded from ansible-galaxy] *************
# task path: /tmp/potos_ansible/playbook.yml:97
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
#
# TASK [load roles from ansible-galaxy] ******************************************
# task path: /tmp/potos_ansible/playbook.yml:106
# skipping: [localhost] => {
# "changed": false,
# "skip_reason": "Conditional result was False"
# }
# TASK [run all the required roles] **********************************************
# task path: /tmp/potos_ansible/playbook.yml:115
[WARNING]: Unable to find 'requirements.yml' in expected paths (use -vvvvv to
see paths)
# exception during Jinja2 execution: Traceback (most recent call last):
# File "/tmp/potos_ansible/lib/python3.10/site-packages/ansible/plugins/lookup/file.py", line 83, in run
# raise AnsibleParserError()
# ansible.errors.AnsibleParserError
#
# During handling of the above exception, another exception occurred:
#
# Traceback (most recent call last):
# File "/tmp/potos_ansible/lib/python3.10/site-packages/ansible/template/__init__.py", line 1015, in _lookup
# ran = instance.run(loop_terms, variables=self._available_variables, **kwargs)
# File "/tmp/potos_ansible/lib/python3.10/site-packages/ansible/plugins/lookup/file.py", line 85, in run
# raise AnsibleError("could not locate file in lookup: %s" % term)
# ansible.errors.AnsibleError: could not locate file in lookup: requirements.yml
# fatal: [localhost]: FAILED! => {
# "msg": "An unhandled exception occurred while running the lookup plugin 'file'. Error was a <class 'ansible.errors.AnsibleError'>, original message: could not locate file in lookup: requirements.yml. could not locate file in lookup: requirements.yml"
# }
# ...ignoring
#
# PLAY RECAP *********************************************************************
# localhost : ok=7 changed=0 unreachable=0 failed=0 skipped=5 rescued=0 ignored=3
#
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.15.0-57-generic
Found initrd image: /boot/initrd.img-5.15.0-57-generic
Found memtest86+ image: /memtest86+.elf
Found memtest86+ image: /memtest86+.bin
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
done
# Cleaning up ... please wait
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Reading package lists...
Building dependency tree...
Reading state information...
The following package was automatically installed and is no longer required:
libsysmetrics1
Use 'sudo apt autoremove' to remove it.
The following packages will be REMOVED:
gnome-initial-setup*
0 upgraded, 0 newly installed, 1 to remove and 39 not upgraded.
After this operation, 1749 kB disk space will be freed.
Purging configuration files for gnome-initial-setup (42.0.1-1ubuntu2) ...
Removing user `gnome-initial-setup' ...
Warning: group `nogroup' has no more members.
userdel: user gnome-initial-setup is currently used by process 1337
/usr/sbin/deluser: `/sbin/userdel gnome-initial-setup' returned error code 8. Exiting.
Could not remove gnome-initial-setup user.
userdel: user gnome-initial-setup is currently used by process 1337
userdel: gnome-initial-setup mail spool (/var/mail/gnome-initial-setup) not found
I noticed the different names of this repository (potos-image
) and the provided container image (potos-iso-builder
). After some thoughts I think it could make sense to rename the repository itself also to potos-iso-builder
. This seems to better capture the intention and content of the repo as it mainly contains the glue code but not the image itself. In addition using the term iso
is probably also better to avoid confusion with image like in "picture".
What do you think? Should we rename the repository to potos-iso-builder
?
No response
Sometimes the installation just randomly crashes without any useful information. The error appears randomly and I don't know a pattern yet.
How ever, restarting the installation with the same iso most likely fixes it.
Probably when an upstream service has a short timeout:
known tasks to fail:
No response
Currently you have a massive list of all available keyboard layouts to choose from.
For most enterprises, the list of keyboard layouts used company wide is limited to just a few, so we should provide a possibility to limit the selection.
The icing on the cake would be, if one could add a button to then still get the all available list.
No response
Currently just pkg.adfinis.com is used. When this mirror is down, no one can update anything. Thus we should add more diverse mirrors
No response
The installation media contains BIOS and UEFI.
potos-installer-production.iso
does not boot.No response
many files/directories are inconsistently named between the defined client_short_name and potos
have a look the source code and you will see the issue ๐
No response
Specs SSH key not installed correctly and thus installation/playbook hangs on firstboot
Create iso with specs repo that is only accessible threw a ssh deploy key
No response
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.