Coder Social home page Coder Social logo

weldr / lorax Goto Github PK

View Code? Open in Web Editor NEW
289.0 16.0 157.0 70.62 MB

Tools for creating images, including the Anaconda boot.iso, live disk images, iso's, and filesystem images.

License: GNU General Public License v2.0

Python 95.84% Shell 2.71% TeX 0.07% Makefile 1.38%

lorax's Introduction

Lorax is a set of tools used to create bootable images.

  • lorax - creates the Anaconda boot.iso used to install Fedora
  • livemedia-creator - uses Anaconda to create bootable images

See the Weldr blog for more info about BDCS and the Lorax documentation for more information about Lorax and associated tools.

lorax's People

Contributors

adamwill avatar atodorov avatar ausil avatar bcl avatar cgwalters avatar clumens avatar damarlin avatar dashea avatar dcantrell avatar dwlehman avatar evan-goode avatar jcpunk avatar jikortus avatar jkeating avatar jkonecny12 avatar jrusz avatar jstodola avatar larskarlitski avatar m4rtink avatar marmarek avatar martinpitt avatar nullr0ute avatar poncovka avatar rvykydal avatar sharkcz avatar tagoh avatar vathpela avatar vladimirslavik avatar vpodzime avatar wgwoods avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lorax's Issues

Unable to build an image using latest code.

I don't know if this is know issue or not,

FileNotFoundError: [Errno 2] No such file or directory: '/root/results/images/install.img'

More logs


2016-03-19 18:50:18,745: populating output tree and building boot images
populating output tree and building boot images
2016-03-19 18:50:18,849: Problem rendering x86.tmpl ({'glob': <function LoraxTemplateRunner.__init__.<locals>.<lambda> at 0x7f45788ffd90>, 'inroot': '/var/tmp/lorax.cgmhvmc1/installroot', 'domacboot': True, 'lower': <function string_lower at 0x7f45788e30d0>, 'udev': <function udev_escape at 0x7f45788e3048>, 'isolabel': 'Fedora-23-x86_64', 'product': {'variant': None, 'isfinal': False, 'name': 'Fedora', 'bugurl': 'your distribution provided bug reporting tool', 'version': '23', 'release': '23'}, 'outroot': '/root/results', 'basearch': 'x86_64', 'libdir': 'lib64', 'kernels': [{'path': 'boot/vmlinuz-4.4.5-300.fc23.x86_64', 'initrd': {'path': 'boot/initramfs-4.4.5-300.fc23.x86_64.img'}, 'version': '4.4.5-300.fc23.x86_64', 'flavor': None, 'arch': 'x86_64'}], 'arch': {'basearch': 'x86_64', 'libdir': 'lib64', 'bcj': 'x86', 'buildarch': 'x86_64'}, 'runtime_img': 'images/install.img', 'exists': <function LoraxTemplateRunner.__init__.<locals>.<lambda> at 0x7f45788ff6a8>, 'workdir': '/var/tmp/lorax.cgmhvmc1', 'runtime_base': 'install.img', 'doupgrade': True}):
Problem rendering x86.tmpl ({'glob': <function LoraxTemplateRunner.__init__.<locals>.<lambda> at 0x7f45788ffd90>, 'inroot': '/var/tmp/lorax.cgmhvmc1/installroot', 'domacboot': True, 'lower': <function string_lower at 0x7f45788e30d0>, 'udev': <function udev_escape at 0x7f45788e3048>, 'isolabel': 'Fedora-23-x86_64', 'product': {'variant': None, 'isfinal': False, 'name': 'Fedora', 'bugurl': 'your distribution provided bug reporting tool', 'version': '23', 'release': '23'}, 'outroot': '/root/results', 'basearch': 'x86_64', 'libdir': 'lib64', 'kernels': [{'path': 'boot/vmlinuz-4.4.5-300.fc23.x86_64', 'initrd': {'path': 'boot/initramfs-4.4.5-300.fc23.x86_64.img'}, 'version': '4.4.5-300.fc23.x86_64', 'flavor': None, 'arch': 'x86_64'}], 'arch': {'basearch': 'x86_64', 'libdir': 'lib64', 'bcj': 'x86', 'buildarch': 'x86_64'}, 'runtime_img': 'images/install.img', 'exists': <function LoraxTemplateRunner.__init__.<locals>.<lambda> at 0x7f45788ff6a8>, 'workdir': '/var/tmp/lorax.cgmhvmc1', 'runtime_base': 'install.img', 'doupgrade': True}):
2016-03-19 18:50:19,052: 

Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/pylorax/ltmpl.py", line 55, in parse
    textbuf = template.render(**variables)
  File "/usr/lib/python3.4/site-packages/mako/template.py", line 443, in render
    return runtime._render(self, self.callable_, args, data)
  File "/usr/lib/python3.4/site-packages/mako/runtime.py", line 803, in _render
    **_kwargs_for_callable(callable_, data))
  File "/usr/lib/python3.4/site-packages/mako/runtime.py", line 835, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/usr/lib/python3.4/site-packages/mako/runtime.py", line 860, in _exec_template
    callable_(context, *args, **kwargs)
  File "/usr/share/lorax/templates.d/99-generic/x86.tmpl", line 19, in render_body
    <%
FileNotFoundError: [Errno 2] No such file or directory: '/root/results/images/install.img'



Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/pylorax/ltmpl.py", line 55, in parse
    textbuf = template.render(**variables)
  File "/usr/lib/python3.4/site-packages/mako/template.py", line 443, in render
    return runtime._render(self, self.callable_, args, data)
  File "/usr/lib/python3.4/site-packages/mako/runtime.py", line 803, in _render
    **_kwargs_for_callable(callable_, data))
  File "/usr/lib/python3.4/site-packages/mako/runtime.py", line 835, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/usr/lib/python3.4/site-packages/mako/runtime.py", line 860, in _exec_template
    callable_(context, *args, **kwargs)
  File "/usr/share/lorax/templates.d/99-generic/x86.tmpl", line 19, in render_body
    <%
FileNotFoundError: [Errno 2] No such file or directory: '/root/results/images/install.img'

Traceback (most recent call last):
  File "/home/sacpatil/github/lorax/src/sbin/lorax", line 353, in <module>
    main(sys.argv)
  File "/home/sacpatil/github/lorax/src/sbin/lorax", line 209, in main
    remove_temp=True, verify=opts.verify)
  File "/usr/lib/python3.4/site-packages/pylorax/__init__.py", line 355, in run
    treebuilder.build()
  File "/usr/lib/python3.4/site-packages/pylorax/treebuilder.py", line 301, in build
    self._runner.run(templatefile, kernels=self.kernels)
  File "/usr/lib/python3.4/site-packages/pylorax/ltmpl.py", line 218, in run
    commands = t.parse(templatefile, variables)
  File "/usr/lib/python3.4/site-packages/pylorax/ltmpl.py", line 55, in parse
    textbuf = template.render(**variables)
  File "/usr/lib/python3.4/site-packages/mako/template.py", line 443, in render
    return runtime._render(self, self.callable_, args, data)
  File "/usr/lib/python3.4/site-packages/mako/runtime.py", line 803, in _render
    **_kwargs_for_callable(callable_, data))
  File "/usr/lib/python3.4/site-packages/mako/runtime.py", line 835, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/usr/lib/python3.4/site-packages/mako/runtime.py", line 860, in _exec_template
    callable_(context, *args, **kwargs)
  File "x86_tmpl", line 46, in render_body
FileNotFoundError: [Errno 2] No such file or directory: '/root/results/images/install.img'

Trying to run

lorax -p Fedora -v 23 -r 23 \
      -s http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Everything/x86_64/os/ \
      -s http://dl.fedoraproject.org/pub/fedora/linux/updates/23/x86_64/ \
      ./results/

Is it possible to affect the boot loader config?

With livecd-tools, I was able to customize the bootloader command line to do things like:

  • set the boot timeout to 1s -- the default 60s is not usable for embedded OS uses
  • set kernel vars, e.g.: console=ttyS0,115200 console=tty0 usbcore.autosuspend=-1 biosdevname=0 net.ifnames=0 8250.nr_uarts=16
  • enable systemd debugging

So far my luck with livemedia-creator has been that pretty much anything I throw at the bootloader directive in the kickstart seems to be ignored.

If this is not possible at present, how hard would it be to develop and where would I need to start?

Test /api/v0/recipes/diff/<recipe_name>/<from_commit>/<to_commit> fails

When running tests in a Fedora 27 mock, there is one failure (after fixing some others in #291):

======================================================================
FAIL: Test /api/v0/recipes/diff/<recipe_name>/<from_commit>/<to_commit>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/martin/upstream/lorax/tests/pylorax/test_server.py", line 332, in test_13_recipes_diff
    self.assertEqual(data, {"diff": [{"new": {"Version": "0.0.1"}, "old": {"Version": "0.2.1"}}]})
AssertionError: {u'diff': [{u'new': {u'Version': u'0.2.1'}, u'old': {u'Version': u'0.0.1'}}]} != {'diff': [{'new': {'Version': '0.0.1'}, 'old': {'Version': '0.2.1'}}]}
- {u'diff': [{u'new': {u'Version': u'0.2.1'}, u'old': {u'Version': u'0.0.1'}}]}
?  -          -        -           -   ^      -        -           -   ^

+ {'diff': [{'new': {'Version': '0.0.1'}, 'old': {'Version': '0.2.1'}}]}
?                                  ^                            ^

I. e. aside from the u"" unicode prefix noise (which doesn't matter for comparison) the old and new versions are swapped. Is that something specific on my system, or does it fail for you as well?

runtime-cleanup.tmpl removes sshd-keygen, breaks sshd

In 99-generic cleanup template there's a rule to remove /usr/libexec/*, which removes /usr/libexec/openssh/sshd-keygen.

When booting a Lorax-generated Anaconda rawhide image with 'inst.sshd' on the kernel command line to start sshd, sshd is started via systemd but there are no host keys so client connections are refused.

lorax.log:2016-08-18 18:42:37,696 DEBUG pylorax.ltmpl:
  removed /var/tmp/lorax.j_ttxdrd/installtree//usr/libexec/openssh/sshd-keygen
Aug 23 23:09:23 localhost systemd[1927]: [email protected]: Failed at step EXEC spawning /usr/libexec/openssh/sshd-keygen: No such file or directory
Aug 23 23:09:23 localhost systemd[1944]: [email protected]: Failed at step EXEC spawning /usr/libexec/openssh/sshd-keygen: No such file or directory
Aug 23 23:09:23 localhost audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=sshd-keygen@ecdsa comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Aug 23 23:09:23 localhost systemd[1]: [email protected]: Unit entered failed state.
Aug 23 23:09:23 localhost systemd[1]: [email protected]: Failed with result 'exit-code'.

[repeat for [email protected] too]

$ ssh [email protected]
Connection closed by 2401:db00:172::face:0:1d:0

Running something like ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa (which is in /bin) from tmux console or a pre-script repairs it and SSH is usable again.

lorax-composer fails to start: ImportError: No module named pyanaconda.queue

Since the latest COPR update, lorax-composer now fails to start:

# lorax-composer --group wheel /recipes/
Traceback (most recent call last):
  File "/usr/sbin/lorax-composer", line 37, in <module>
    from pyanaconda.queue import QueueFactory
ImportError: No module named pyanaconda.queue

[root@ccd32dd8c710 /]# rpm -qa|grep lorax-composer
lorax-composer-19.7.7-1.el7.centos.x86_64

There is no package like python-anaconda, and a yum search anaconda didn't reveal something fitting. So this is a missing dependency. Does that exist on RHEL/CentOS 7?

Impossible to install several kernel on a live iso install

Hello,

I try to build an iso file using livemedia-creator.
I wanted to add a standard kernel and a realtime kernel from CCRMA repo.
The iso is correctly generated, but the realtime kernel doesn't appears in the boot menu.
As far as I understand, lorax install only the standard kernel and ignore the rt one.
I tried to use x86.tmpl but the iso generation failed because of the arguments of mkisofs ...

Best regards,

YC

Tell users to use url + repo if a kickstart only contains repo directives.

New user to lm-c here; long time user of livecd-tools though. I'm passing a kickstart that lcd-t was plenty happy with, but lm-c fails stating:

Only url and nfs install methods are currently supported. Please fix your kickstart file.

However, there are several directives in a kickstart that can accept such URIs and I'm given no hint as to which one I'm using against the needs of lm-c. My initial encounter with this was via Koji so it was doubly obscure. I'm now in the process of running lm-c directly from workstation so that I can hack the code to add some debug messages to learn what ks directive that condition is looking at.

Move from genisoimage to xorriso

genisoimage (from cdrkit) is old and unmaintained. These days, the modern replacement available is from xorriso, which looks like it supports all the features that exist in our variant of genisoimage.

Please switch from genisoimage to xorriso.

lorax-composer cannot write its own git tree

With latest lorax-composer creating and deleting recipes fails now:

2018-02-09 14:52:32,972: (v0_recipes_new) ggit-error: Failed to create temporary file '/recipes/git/objects/tmp_object_git2_uLs0mq': Permission denied (-1)

[root@594b4069006e /]# ls -l /recipes/
drwxr-xr-x 1 root root 82 Feb  9 14:30 git

# rpm -qa|grep lorax
lorax-19.7.7-1.el7.centos.x86_64
lorax-composer-19.7.7-1.el7.centos.x86_64

This started from a completely blank /recipes directory (which is a docker volume, and docker just created it from scratch). It seems the entire git tree is initialized from the process that runs as root, but being used by the process that runs as user weldr?

Before that I still had a "foo" recipe in that volume, and deletion has the same problem:

❱❱❱ curl --unix-socket /run/weldr/weldr/api.socket http://localhost/api/v0/recipes/list
{
  "limit": 20, 
  "offset": 0, 
  "recipes": [
    "foo"
  ], 
  "total": 1
}

{
  "error": {
    "msg": "ggit-error: Failed to create temporary file '/recipes/git/objects/tmp_object_git2_iax2WE': Permission denied (-1)"
  }, 
  "status": false
}

and lorax-composer showed a similiar EPERM message.

boot.iso should have supported bit if created with --isfinal

When lorax is run with the --isfinal option, I think lorax should add --supported-iso to its invocation of implantisomd5.

Currently when you check the iso it always claims to be unsupported:

$ checkisomd5 --md5sumonly boot.iso
boot.iso:   68f71e883070f611e0862e57eb6eb0dc
Fragment sums: e2f7bf8efd4721aa28887f97b7c6deb64807a6a4b3c6a4da6e35e381ffad
Fragment count: 20
Supported ISO: no

I'm not completely sure this would be correct, but at least Pungi expects this should happen when calling lorax.

drivers/target are necessary to support installation over ISERT/SRPT

share/templates.d/99-generic/runtime-cleanup.tmpl

90 ## clean up kernel modules
91 removekmod sound drivers/media drivers/hwmon
92 net/atm net/bluetooth net/sched net/sctp
93 net/rds net/l2tp net/decnet net/netfilter net/ipv4 net/ipv6
94 drivers/watchdog drivers/target drivers/rtc drivers/input/joystick
95 drivers/bluetooth drivers/edac drivers/staging
96 drivers/usb/serial drivers/usb/host drivers/usb/misc
97 fs/ocfs2 fs/ceph fs/nfsd fs/ubifs fs/nilfs2
98 arch/x86/kvm

As modules, ib_srpt and ib_isert, depends on target modules, so please do not remove drivers/target. We need them to support installation over RDMA targets, such as SRP and ISER.

thanks

lorax-composer API modules/projects fail with: Cannot find a valid baseurl for repo: base/$releasever/x86_64

This is with the latest version 19.7.4-1.el7.centos from the COPR. Reproducer on current CentOS 7 image:

yum install -y epel-release
curl https://copr.fedorainfracloud.org/coprs/g/weldr/lorax-composer/repo/epel-7/group_weldr-lorax-composer-epel-7.repo > /etc/yum.repos.d/lorax-composer.repo
yum install -y lorax-composer 
systemctl enable --now lorax-composer
curl http://localhost:4000/api/v0/projects/list

This works:

# curl http://localhost:4000/api/v0/recipes/list
{
  "limit": 20, 
  "offset": 0, 
  "recipes": [], 
  "total": 0
}

This doesn't:

[root@m1 ~]# curl http://localhost:4000/api/v0/projects/list
{
  "error": {
    "msg": "('There was a problem listing projects: %s', 'Cannot find a valid baseurl for repo: base/$releasever/x86_64')"
  }

# curl http://localhost:4000/api/v0/modules/list
{
  "error": {
    "msg": "There was a problem listing modules: Cannot find a valid baseurl for repo: base/$releasever/x86_64"
  }

Note that I'm not a project member and thus I cannot set labels. Can you please add "lorax-composer"?

lorax-composer.service stops responding to SIGTERM after query, times out on shutdown

This is with the latest version 19.7.4-1.el7.centos from the COPR. Reproducer on current CentOS 7 image:

yum install -y epel-release
curl https://copr.fedorainfracloud.org/coprs/g/weldr/lorax-composer/repo/epel-7/group_weldr-lorax-composer-epel-7.repo > /etc/yum.repos.d/lorax-composer.repo
yum install -y lorax-composer 
systemctl start lorax-composer
curl http://localhost:4000/api/v0/projects/list
poweroff

The unit times out on shutdown:

[  OK  ] Started Show Plymouth Power Off Screen.
[  OK  ] Stopped Availability of block devices.
[...]
[ ***  ] A stop job is running for Lorax Ima...oser API Server (1min 27s / 1min 30s)

and eventually gets killed.

Install packages failed in cross build environment

HOST:X86 ubuntu
DOCKER:arm64 centos 7
2018-01-10 18:35:11,137: template command error in runtime-postinstall.tmpl:
template command error in runtime-postinstall.tmpl:
2018-01-10 18:35:11,138: symlink /lib/systemd/system/anaconda.target etc/systemd/system/default.target
symlink /lib/systemd/system/anaconda.target etc/systemd/system/default.target
2018-01-10 18:35:11,157: OSError: [Errno 2] No such file or directory
OSError: [Errno 2] No such file or directory
Traceback (most recent call last):
File "/usr/sbin/lorax", line 337, in
main(sys.argv)
File "/usr/sbin/lorax", line 235, in main
remove_temp=True)
File "/usr/lib/python2.7/site-packages/pylorax/init.py", line 272, in run
rb.postinstall()
File "/usr/lib/python2.7/site-packages/pylorax/treebuilder.py", line 136, in postinstall
self._runner.run("runtime-postinstall.tmpl", configdir=configdir_path)
File "/usr/lib/python2.7/site-packages/pylorax/ltmpl.py", line 182, in run
self._run(commands)
File "/usr/lib/python2.7/site-packages/pylorax/ltmpl.py", line 201, in _run
f(*args)
File "/usr/lib/python2.7/site-packages/pylorax/ltmpl.py", line 359, in symlink
os.symlink(target, self._out(dest))
OSError: [Errno 2] No such file or directory
2018-01-10 18:35:11,360: glibc-2.17-196.el7.aarch64 scriptlet output:
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from dbcursor->c_close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

glibc-2.17-196.el7.aarch64 scriptlet output:
error: rpmdb: BDB0060 PANIC: fatal region error detected; run recovery
error: db5 error(-30973) from dbcursor->c_close: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

2018-01-10 18:35:11,614: RPM transaction error: Error: Cannot open file /var/tmp/lorax.ffv2RL/installtree/var/tmp/lorax.ffv2RL/yum/yum.cache/lorax-repo/packages/libstdc++-4.8.5-16.el7.aarch64.rpm: [Errno 2] No such file or directory: '/var/tmp/lorax.ffv2RL/installtree/var/tmp/lorax.ffv2RL/yum/yum.cache/lorax-repo/packages/libstdc++-4.8.5-16.el7.aarch64.rpm'
RPM transaction error: Error: Cannot open file /var/tmp/lorax.ffv2RL/installtree/var/tmp/lorax.ffv2RL/yum/yum.cache/lorax-repo/packages/libstdc++-4.8.5-16.el7.aarch64.rpm: [Errno 2] No such file or directory: '/var/tmp/lorax.ffv2RL/installtree/var/tmp/lorax.ffv2RL/yum/yum.cache/lorax-repo/packages/libstdc++-4.8.5-16.el7.aarch64.rpm'
TypeError: an integer is required
FATAL ERROR: python callback <bound method RPMTransaction.callback of <yum.rpmtrans.RPMTransaction instance at 0xfc93f8>> failed, aborting!

Update the x86 templates

If the generic-logos package is installed, rather than fedora-logos, livemedia-creator "--make-iso" fails when building "boot.iso".

Python traceback:
IOError: nothing matching /var/tmp/tmpPtUQ_u/usr/share/anaconda/boot/syslinux-splash.png in /

This is due to each package installing the "syslinux-splash.png" file in different locations:

generic-logos: /usr/share/anaconda/pixmaps/syslinux-splash.png
fedora-logos: /usr/share/anaconda/boot/syslinux-splash.png

Update to install the generic-logos file, if it exists; else, default.

diff --git a/share/live/x86.tmpl b/share/live/x86.tmpl
index f60b80d..dbd61bf 100644
--- a/share/live/x86.tmpl
+++ b/share/live/x86.tmpl
@@ -29,7 +29,11 @@ install ${SYSLINUXDIR}/libutil.c32 ${BOOTDIR}
 install ${configdir}/isolinux.cfg ${BOOTDIR}
 install ${configdir}/boot.msg ${BOOTDIR}
 install ${configdir}/grub.conf ${BOOTDIR}
-install usr/share/anaconda/boot/syslinux-splash.png ${BOOTDIR}/splash.png
+%if exists("usr/share/anaconda/pixmaps/syslinux-splash.png"):
+   install /usr/share/anaconda/pixmaps/syslinux-splash.png ${BOOTDIR}/splash.png
+%else:
+   install usr/share/anaconda/boot/syslinux-splash.png ${BOOTDIR}/splash.png
+%endif
 install boot/memtest* ${BOOTDIR}/memtest

 ## configure bootloader
diff --git a/share/x86.tmpl b/share/x86.tmpl
index 7273e28..3f1e470 100644
--- a/share/x86.tmpl
+++ b/share/x86.tmpl
@@ -29,7 +29,11 @@ install ${SYSLINUXDIR}/libutil.c32 ${BOOTDIR}
 install ${configdir}/isolinux.cfg ${BOOTDIR}
 install ${configdir}/boot.msg ${BOOTDIR}
 install ${configdir}/grub.conf ${BOOTDIR}
-install usr/share/anaconda/boot/syslinux-splash.png ${BOOTDIR}/splash.png
+%if exists("usr/share/anaconda/pixmaps/syslinux-splash.png"):
+   install /usr/share/anaconda/pixmaps/syslinux-splash.png ${BOOTDIR}/splash.png
+%else:
+   install usr/share/anaconda/boot/syslinux-splash.png ${BOOTDIR}/splash.png
+%endif
 install boot/memtest* ${BOOTDIR}/memtest

 ## configure bootloader

repo with user/pass

Hi, i have a repo with username/password, is there a way for lorax to access it ?

livemedia-creator lacks a way to add files to the iso image

When livemedia-creator exits, the result is an iso image contains squashfs, which contains ext3f, which is the rootfs.
The kickstart allows installing packages, and do post processing in the rootfs. But not in the iso.

.ks files available in /usr/share/doc/lorax/fedora-livemedia.ks contains this %post section :

%post`

--nochroot
cp $INSTALL_ROOT/usr/share/licenses/-release/ $LIVE_ROOT/

# only works on x86, x86_64
if [ "$(uname -i)" = "i386" -o "$(uname -i)" = "x86_64" ]; then
if [ ! -d $LIVE_ROOT/LiveOS ]; then mkdir -p $LIVE_ROOT/LiveOS ; fi
cp /usr/bin/livecd-iso-to-disk $LIVE_ROOT/LiveOS
fi
%end

Which does not work :

  • no licences files in the iso root
  • no livecd-iso-to-disk in /LiveOS of the iso

Instead:

  • licence files are in the mock root
  • the LiveOS folder is created in the mock root

livecd-creator mechanism based on %post using $LIVE_ROOT has no equivalent, which is, for me, a lack in the said .ks file compatibility.

Having the possibility to interact with the iso file before its packing could be great to :

  • add some files (documents, manuals, MS windows stuff)
  • add some more grub menu entries

Please don't make me do stupid dances (aka results dir is created before SELinux status is checked)

[root@adam live]# lorax -p Fedora -v 25 -r 25 -s https://kojipkgs.fedoraproject.org/compose/branched/Fedora-25-20161003.n.0/compose/Everything/x86_64/os -s https://www.happyassassin.net/temp/repo/x86_64 ./results/
2016-10-03 15:50:40,029: Added 'lorax-repo-0': https://kojipkgs.fedoraproject.org/compose/branched/Fedora-25-20161003.n.0/compose/Everything/x86_64/os
2016-10-03 15:50:40,030: Fetching metadata...
2016-10-03 15:50:45,701: Added 'lorax-repo-1': https://www.happyassassin.net/temp/repo/x86_64
2016-10-03 15:50:45,701: Fetching metadata...
2016-10-03 15:50:58,950: checking for root privileges
checking for root privileges
2016-10-03 15:50:58,950: checking the selinux mode
checking the selinux mode
2016-10-03 15:50:58,951: selinux must be disabled or in Permissive mode
selinux must be disabled or in Permissive mode
[root@adam live]# setenforce Permissive
[root@adam live]# lorax -p Fedora -v 25 -r 25 -s https://kojipkgs.fedoraproject.org/compose/branched/Fedora-25-20161003.n.0/compose/Everything/x86_64/os -s https://www.happyassassin.net/temp/repo/x86_64 ./results/
usage: lorax [-h] -p PRODUCT -v VERSION -r RELEASE [-s REPOSITORY]
             [--repo REPOSITORY] [-m REPOSITORY] [-t VARIANT] [-b URL]
             [--isfinal] [-c CONFIGFILE] [--proxy HOST] [-i PACKAGE]
             [--buildarch ARCH] [--volid VOLID] [--macboot] [--nomacboot]
             [--noupgrade] [--logfile LOGFILE] [--tmp TMP]
             [--cachedir CACHEDIR] [--workdir WORKDIR] [--force]
             [--add-template ADD_TEMPLATES]
             [--add-template-var ADD_TEMPLATE_VARS]
             [--add-arch-template ADD_ARCH_TEMPLATES]
             [--add-arch-template-var ADD_ARCH_TEMPLATE_VARS] [--noverify]
             [--sharedir SHAREDIR] [--enablerepo [repo]]
             [--disablerepo [repo]] [--rootfs-size ROOTFS_SIZE] [-V]
             OUTPUTDIR
lorax: error: output directory /home/adamw/local/live/results should not exist.

just check the SELinux status before creating the output directory, and I won't look like an idiot every damn time I run lorax...:)

livemedia-creator with "--make-pxe-live" produces error "non-existent device" for Fedora 25/26

Hello.

I have a Fedora 25 using "lorax-25.19-1.fc25.x86_64" package to build live pxe boot squashfs image.

I'm using the following command

livemedia-creator --make-pxe-live --image-name=my_fedora_img --qcow2 --tmp=/var/tmp/a --ks fedora25-my.ks --iso=Fedora-Workstation-netinst-x86_64-25-1.3.iso

The "fedora25-my.ks" is pretty basic:

url --mirrorlist=https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-$releasever&arch=$basearch
lang en_US.UTF-8
keyboard us
timezone --utc Etc/UTC
network --bootproto=dhcp --device=link --activate
rootpw root
shutdown
zerombr
clearpart --all --initlabel --disklabel=msdos
part / --size 5120 --fstype ext4
bootloader --timeout=1 --append="no_timer_check console=tty1 console=ttyS0,115200n8"

%packages
@core
dracut-live
%end

Qemu did the job and generates the image, but then I got the error:

Running... qemu-img create -f qcow2 /var/tmp/a/my_fedora_img 5370806272
Formatting '/var/tmp/a/my_fedora_img', fmt=qcow2 size=5370806272 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
Return code: 0
Running... qemu-system-x86_64 -nodefconfig -m 8192 -smp 2 --machine accel=kvm -kernel /var/tmp/a/lorax.imgutils.zxwe7haj/isolinux/vmlinuz -initrd /var/tmp/a/lmc-initrd-xhshv9qs.img -drive file=/var/tmp/a/my_fedora_img,cache=unsafe,discard=unmap,format=qcow
2 -drive file=/home/ruzickap/data2/iso/Fedora-Workstation-netinst-x86_64-25-1.3.iso,media=cdrom,readonly=on -append ks=file:/fedora25-my.ks inst.stage2=hd:LABEL=Fedora-WS-dvd-x86_64-25 inst.text inst.cmdline -nographic -display vnc=127.0.0.1:0 -device virt
io-serial-pci,id=virtio-serial0 -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.fedoraproject.anaconda.log.0 -chardev socket,id=charchannel0,host=127.0.0.1,port=59921 -object rng-random,id=virtio-rng0,filename=/de
v/random -device virtio-rng-pci,rng=virtio-rng0,id=rng0,bus=pci.0,addr=0x9
Return code: 0
Running... umount /var/tmp/a/lorax.imgutils.zxwe7haj
Return code: 0
Running... kpartx -v -a -s /var/tmp/a/my_fedora_img
Return code: 0
Running... kpartx -d -s /var/tmp/a/my_fedora_img
Return code: 0
Running... /usr/sbin/fsck.ext4 -y -f -E discard /var/tmp/a/lmc-work-klz2ladw/squashfs_root/LiveOS/rootfs.img
e2fsck 1.43.3 (04-Sep-2016)
/usr/sbin/fsck.ext4: No such file or directory while trying to open /var/tmp/a/lmc-work-klz2ladw/squashfs_root/LiveOS/rootfs.img
Possibly non-existent device?
Return code: 8

Do you have any idea why I'm getting it or what may be wrong?
(I do have the same error with Fedora 26 Beta)

Thank you

PetrR

Live ISO images changed from F23 to F24 in ways that break things

Since Fedora 24 came out, I have been asking myself "What’s goin’ on (aka why is everything awful)". Seems like I was not the only one. Fedora Live ISO images changed from F23 to F24 in subtle ways without further explanation or warning that break things, e.g., SystemImageKit and testappimage.

  • The name of the ISO suddenly changed from Fedora-Live-Workstation-x86_64-$VERSION.iso to Fedora-Workstation-Live-x86_64-$VERSION.iso

Before the change, this worked:

mount -o loop,ro /isodevice/boot/iso/Fedora-Live-Workstation-x86_64-23-10.iso /tmp/27337/iso
mount -o loop,ro /tmp/27337/iso/LiveOS/squashfs.img /tmp/27337/iso/
mount /tmp/27337/iso/LiveOS/ext3fs.img /tmp/27337/unionfs/root -o loop,ro

In F24 and subsequent versions, ext3fs.img is no longer there, but rootfs.img is.

Needs to be further investigated.

What is the reason of these changes, other than break things?

fill_sack not reloading repositories and failing on setting includes/excludes

Hi,

I am having some issues on this control path:

https://github.com/rhinstaller/lorax/blob/d93adc283fae671d697b933661be39f65c315952/src/pylorax/ltmpl.py#L648-L650

I am invoking lorax cli with a repo definition that contains an includepkgs directive (the repo definition is passed via the --repo flag). This results in the following stack trace:

2017-10-06 11:24:34,410:   run_pkg_transaction
  run_pkg_transaction
2017-10-06 11:24:34,411:   ValueError: Can't set use_includes for repo with given name.
  ValueError: Can't set use_includes for repo with given name.
Traceback (most recent call last):
  File "/usr/sbin/lorax", line 279, in <module>
    main()
  File "/usr/sbin/lorax", line 135, in main
    remove_temp=True, verify=opts.verify)
  File "/usr/lib/python3.6/site-packages/pylorax/__init__.py", line 280, in run
    rb.install()
  File "/usr/lib/python3.6/site-packages/pylorax/treebuilder.py", line 122, in install
    self._runner.run("runtime-install.tmpl")
  File "/usr/lib/python3.6/site-packages/pylorax/ltmpl.py", line 220, in run
    self._run(commands)
  File "/usr/lib/python3.6/site-packages/pylorax/ltmpl.py", line 239, in _run
    f(*args)
  File "/usr/lib/python3.6/site-packages/pylorax/ltmpl.py", line 650, in run_pkg_transaction
    self.dbo.fill_sack(load_system_repo=True, load_available_repos=False)
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 347, in fill_sack
    self._setup_excludes_includes()
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 157, in _setup_excludes_includes
    self.sack.set_use_includes(True, r.id)
ValueError: Can't set use_includes for repo with given name.

This comes from libdnf, which attempts to iterate over all the repos defined in the sack object and raises when it cannot find a match. It appears that recently dnf library switched to using a new interface exposed by libdnf (i.e. set_use_includes method of the sack object). Specifically, it happened here. Now, my understanding is that fill_sack creates a brand new sack and as a consequence all the repo definitions are lost. Not forcing the reload of all repositories via load_available_repos (in addition to the systems ones) makes that code path blow up on set_use_includes (actually further down in the libdnf C import module). I do not have a complete understanding of why it used to work before switching to the new interface. It seems that add_includes was following a control path that did not depend on having the repositories defined (ending up here).

What I can tell is that s/load_available_repos=False/load_available_repos=True/ seems to fix this issue.

As a side question, I wonder what

self.dbo.reset(repos=False) 

is supposed to do? All the actions in that function are gated by flags that are False by default.

Could you please have a look at that usage of self.dbo.fill_sack?

when printing out packages being installed print the n-v-r.a not n.a

something changed in rawhide recently that has caused lorax to start to be unable to complete.

the error is
2017-01-29 07:38:48,191: /usr/bin/perl, needed by /var/tmp/lorax.vbio_v1v/installtree/usr/bin/rxe_cfg, does not exist
/usr/bin/perl, needed by /var/tmp/lorax.vbio_v1v/installtree/usr/bin/rxe_cfg, does not exist

the package set has gone from 730 to 733 packages, however I can not see the versions that changed to find out which one of them changed and started pulling in more.

Transfer some requires from blivet to lorax

Blivet will be losing libblockdev-plugins-all and dosfstools requires. I believe that lorax will need to add a libblockdev-plugins-all requires when this blivet change goes through.

Difference Package Installation from Kickstart form Lorax template

For me it is not clear when to use kickstart and when to use lorax template files to install a package or configure a file. E.g. it is not clear which package repos the template files would pull in on installpkg command. Is it relying on anaconda's preparation part? Is /mnt/sysimage mounted when the template magic is done?

Could you shortly answer where to use kickstart and where to use the templates?

I wonder if I could use the gihub lorax version in RHEL 7.2. Which one is currently the last one supported in RHEL?

Is lm-c or lorax responsible for the funny business with /var/cache/man

I've been trying to figure out why 'man -k bla' doesn't work in my live images. I initially followed the fedora templates by calling mandb in %post but that didn't help. In debugging this I also added to %post things like find /var/cache/man -type f and found that the DB files do exist at that time, but they're gone once I actually go to run my image. I just took the same kickstart and bypassed lm-c and lorax, essentially doing a standard kickstart install and found the DB files present in that case. I grepped for var, cache, man in the lorax templates but found no smoking gun. I can keep digging further, but thought "those in the know" might be able to save me a whole lot of effort since I run lm-c via koji. I guess my next step would be to run lm-c outside of koji for another data point.

livemedia-creator 25.19 --novirt --make-pxe-live --ks fedora-livemedia.ks not respecting %packages -@libreoffice or -libreoffice*

I dont want libreoffice. livemedia-creator using the default fedora-livemedia.ks is not respecting the -@LibreOffice or -libreoffice* in %packages.

I dont want to sort out and list all the packages to include in %packages, I just want to remove libreoffice.

Why does it insist on installing? Ive grepped the logs and cant find any clues. Ive looked in the other packages and cant find it as a dependency.

Is there --exclude support in lorax?:

* Thu Jan 12 2012 Martin Gracik <[email protected]> 17.2-1
  - lorax: Add option to exclude packages (bcl)

What about a way to ad an exclude section to the dnf.conf, yum.conf or the .repo's from within the kickstart using %pre --nochroot??

[dnf]: no attribute 'basearch'

FYI

I was trying to run lorax after installing using make install, and got below error.


 File "/usr/lib/python3.4/site-packages/pylorax/__init__.py", line 65, in __init__
    self.basearch = dnf.rpm.basearch(buildarch)
AttributeError: 'module' object has no attribute 'basearch'

dnf version 1.1.6 was not having dnf.rpm.basearch(), it was moved from dnf.arch.basearch() to dnf.rpm.basearch() since commit 19766a601550a.

You need to update dnf to version 1.1.7

Random losetup errors while building inside mock

I am trying to build a Fedora 26 live image inside mock. For this I am currently using "--old-chroot" option for mock as otherwise /dev/loop does not even exist.

(The -squashfs_args "-noD -noI -noF -noX -no-duplicates" was me trying to speed up mksquashfs, not that it worked.)

2017-08-10 16:15:46,260: Configuring installed system
2017-08-10 16:15:46,312: .
2017-08-10 16:15:46,313: Creating users
2017-08-10 16:15:46,327: .
2017-08-10 16:15:46,327: Configuring addons
2017-08-10 16:15:59,737: .
2017-08-10 16:15:59,738: Generating initramfs
2017-08-10 16:16:01,522: .
2017-08-10 16:16:01,522: Running post-installation scripts
2017-08-10 16:16:01,523: .
2017-08-10 16:16:01,523: Storing configuration files and kickstarts
2017-08-10 16:16:01,524: .
2017-08-10 16:16:34,460: Shutting down log processing
2017-08-10 16:16:35,316: Disk Image install successful
2017-08-10 16:16:35,316: working dir is /var/tmp/lmc-work-_eugkhmt
2017-08-10 16:31:10,101: Rebuilding initrds
2017-08-10 16:31:10,101: dracut args = ['--xz', '--add', 'livenet dmsquash-live convertfs pollcdrom qemu qemu-net', '--omit', 'plymouth', '--no-hostonly', '--debug', '--no-early-microcode']
2017-08-10 16:31:10,103: rebuilding boot/initramfs-4.11.11-300.fc26.x86_64.img
2017-08-10 16:31:52,932: Building boot.iso
2017-08-10 16:31:52,981: running x86.tmpl
2017-08-10 16:32:00,012: command output:
Traceback (most recent call last):
  File "/usr/sbin/mkefiboot", line 167, in <module>
    main()
  File "/usr/sbin/mkefiboot", line 160, in main
    mkefiboot(opt.bootdir, opt.outfile, opt.label)
  File "/usr/sbin/mkefiboot", line 33, in mkefiboot
    mkdosimg(None, outfile, label=label, graft={'EFI/BOOT':bootdir})
  File "/usr/lib/python3.6/site-packages/pylorax/imgutils.py", line 437, in mkdosimg
    mkfsargs=["-n", label], graft=graft)
  File "/usr/lib/python3.6/site-packages/pylorax/imgutils.py", line 417, in mkfsimage
    with LoopDev(outfile, size) as loopdev:
  File "/usr/lib/python3.6/site-packages/pylorax/imgutils.py", line 297, in __enter__
    self.loopdev = loop_attach(self.filename)
  File "/usr/lib/python3.6/site-packages/pylorax/imgutils.py", line 154, in loop_attach
    dev = runcmd_output(["losetup", "--find", "--show", outfile])
  File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 347, in runcmd_output
    return execWithCapture(cmd[0], cmd[1:], **kwargs)
  File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 249, in execWithCapture
    reset_handlers=reset_handlers, reset_lang=reset_lang)[1]
  File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 201, in _run_program
    raise subprocess.CalledProcessError(proc.returncode, argv, output)
subprocess.CalledProcessError: Command '['losetup', '--find', '--show', '/var/tmp/lmc-work-_eugkhmt/images/efiboot.img']' returned non-zero exit status 1.

2017-08-10 16:32:00,012: command returned failure (1)
2017-08-10 16:32:00,013: template command error in x86.tmpl:
2017-08-10 16:32:00,013:   runcmd mkefiboot --label=ANACONDA /var/tmp/lmc-work-_eugkhmt/EFI/BOOT /var/tmp/lmc-work-_eugkhmt/images/efiboot.img
2017-08-10 16:32:00,015:   subprocess.CalledProcessError: Command '['mkefiboot', '--label=ANACONDA', '/var/tmp/lmc-work-_eugkhmt/EFI/BOOT', '/var/tmp/lmc-work-_eugkhmt/images/efiboot.img']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/usr/sbin/livemedia-creator", line 1476, in <module>
    main()
  File "/usr/sbin/livemedia-creator", line 1413, in main
    result_dir = make_livecd(opts, mount_dir, work_dir)
  File "/usr/sbin/livemedia-creator", line 637, in make_livecd
    tb.build()
  File "/usr/lib/python3.6/site-packages/pylorax/treebuilder.py", line 311, in build
    self._runner.run(templatefile, kernels=self.kernels)
  File "/usr/lib/python3.6/site-packages/pylorax/ltmpl.py", line 220, in run
    self._run(commands)
  File "/usr/lib/python3.6/site-packages/pylorax/ltmpl.py", line 239, in _run
    f(*args)
  File "/usr/lib/python3.6/site-packages/pylorax/ltmpl.py", line 515, in runcmd
    stdout = runcmd_output(cmd)
  File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 347, in runcmd_output
    return execWithCapture(cmd[0], cmd[1:], **kwargs)
  File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 249, in execWithCapture
    reset_handlers=reset_handlers, reset_lang=reset_lang)[1]
  File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 201, in _run_program
    raise subprocess.CalledProcessError(proc.returncode, argv, output)
subprocess.CalledProcessError: Command '['mkefiboot', '--label=ANACONDA', '/var/tmp/lmc-work-_eugkhmt/EFI/BOOT', '/var/tmp/lmc-work-_eugkhmt/images/efiboot.img']' returned non-zero exit status 1.
Finish: chroot ['livemedia-creator --squashfs_args "-noD -noI -noF -noX -no-duplicates" --ks flat-hwtest-live.ks --no-virt --resultdir /var/lmc --project Fedora-HWTest --make-iso --kernel-args="crashkernel=256M" --volid Fedora-HWTest-26 --iso-only --iso-name Fedora-HWTest-26-x86_64.iso --releasever 26 --title Fedora-HWTest-live --macboot']
ERROR: Command failed: 
 # /bin/sh -c livemedia-creator --squashfs_args "-noD -noI -noF -noX -no-duplicates" --ks flat-hwtest-live.ks --no-virt --resultdir /var/lmc --project Fedora-HWTest --make-iso --kernel-args="crashkernel=256M" --volid Fedora-HWTest-26 --iso-only --iso-name Fedora-HWTest-26-x86_64.iso --releasever 26 --title Fedora-HWTest-live --macboot

Investigate rpm-ostree for ISO

When generating ISO-with-content-embedded, as we're doing for Atomic Host and I am doing for workstationostree, currently there's lots of duplicate files between the ISO and the data.

This isn't easily fixable with ISO + traditional RPM, but with ostree, if we had an ostree commit for the ISO data, then all we'd need to do is set the content repo to use it as a parent, and everything should be transparently dedup'd.

This would be most of a win for the desktop case since the content already has a desktop. For the AH case I'd expect only a small space increase over the anaconda base.

produce new medadata files

in order to capture all teh data about a compose I would like to have some json files created with metadata about the lorax runs that can be consumed by productmd https://github.com/release-engineering/productmd with details easily consumable of things like what is in the anaconda runtime we can look at improving how and when we make new anaconda runtimes and make them only when things change. the same is also true of livemedia-creator output and other deliverables that we can make with lorax. with the info provided we could then suck it into and include with the entire compose metadata

x86.tmpl fails when running mkefiboot (exit status 1)

# livemedia-creator --ks tools/serveurlibre-flat.ks --no-virt --make-iso --iso-only --iso-name ServeurLibre.iso --volid ServeurLibre --title ServeurLibre --releasever 25 --proxy=http://localhost:3128
[...]
2017-05-22 20:12:03,268: Building boot.iso
2017-05-22 20:12:03,394: running x86.tmpl
2017-05-22 20:12:08,103: template command error in x86.tmpl:
2017-05-22 20:12:08,103:   runcmd mkefiboot --label=ANACONDA /var/tmp/lmc-work-55byjqxr/EFI/BOOT /var/tmp/lmc-work-55byjqxr/images/efiboot.img
2017-05-22 20:12:08,106:   subprocess.CalledProcessError: Command '['mkefiboot', '--label=ANACONDA', '/var/tmp/lmc-work-55byjqxr/EFI/BOOT', '/var/tmp/lmc-work-55byjqxr/images/efiboot.img']' returned non-zero exit status 1
Traceback (most recent call last):
  File "/usr/sbin/livemedia-creator", line 1476, in <module>
    main()
  File "/usr/sbin/livemedia-creator", line 1413, in main
    result_dir = make_livecd(opts, mount_dir, work_dir)
  File "/usr/sbin/livemedia-creator", line 637, in make_livecd
    tb.build()
  File "/usr/lib/python3.5/site-packages/pylorax/treebuilder.py", line 308, in build
    self._runner.run(templatefile, kernels=self.kernels)
  File "/usr/lib/python3.5/site-packages/pylorax/ltmpl.py", line 220, in run
    self._run(commands)
  File "/usr/lib/python3.5/site-packages/pylorax/ltmpl.py", line 239, in _run
    f(*args)
  File "/usr/lib/python3.5/site-packages/pylorax/ltmpl.py", line 515, in runcmd
    stdout = runcmd_output(cmd)
  File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 347, in runcmd_output
    return execWithCapture(cmd[0], cmd[1:], **kwargs)
  File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 249, in execWithCapture
    reset_handlers=reset_handlers, reset_lang=reset_lang)[1]
  File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 201, in _run_program
    raise subprocess.CalledProcessError(proc.returncode, argv, output)
subprocess.CalledProcessError: Command '['mkefiboot', '--label=ANACONDA', '/var/tmp/lmc-work-55byjqxr/EFI/BOOT', '/var/tmp/lmc-work-55byjqxr/images/efiboot.img']' returned non-zero exit status 1

I executed the command by myself and it worked just fine:

# mkefiboot --label=ANACONDA /var/tmp/lmc-work-55byjqxr/EFI/BOOT /var/tmp/lmc-work-55byjqxr/images/efiboot.img ; echo $?
0

I tried to cleanup the mock environment and to run the procedure multiple times, but the problem persists. I am using F25 x86_64.

/depsolve error: There was a problem depsolving []: [u'Success - empty transaction']

This happens when running welder-web against current lorax-composer 19.7.5:

  • Start with no recipes; open JS console to see error messages
  • Create a new recipe; I called it "test"
  • Several errors are thrown:
    Error fetching recipe: TypeError: null is not iterable!
    Error in fetchModalRecipeContents promise: undefined
    uncaught at fetchRecipeContents TypeError: Cannot read property 'id' of undefined

AFAICS this happens during calling

❱❱❱ curl http://localhost:4000/api/v0/recipes/depsolve/test
{
  "errors": [
    {
      "msg": "There was a problem depsolving []: [u'Success - empty transaction']", 
      "recipe": "test"
    }
  ], 
  "recipes": [
    {
      "dependencies": [], 
      "modules": [], 
      "recipe": {
        "description": "foo", 
        "modules": null, 
        "name": "test", 
        "packages": null, 
        "version": "0.0.1"
      }
    }
  ]
}

In the systemd journal I see the same error:

lorax-composer[1206]: 2017-12-12 03:17:30,521: (v0_recipes_depsolve) There was a problem depsolving []: [u'Success - empty transaction']

Feature Request - Create ISO from Chroot folder

It would be easier for me in some cases to do quick edits in a chroot folder and then to build and iso from that current folder. If I could just point to a folder instead of doing a KickStart file etc.. and create a bootable ISO or image. Thanks

Improvements for enabling lorax to build chroots different from OS you run from

@dcantrell and I just had a long discussion about how to improve this and came up with some relatively simple improvements that I wanted to get written down somewhere before it falls out of my head. At some point I may send the relevant PRs... but I'm not committing to it yet ;)

Problem:

  • If you use an older OS to build newer installer media (i.e. run RHEL 7 lorax to build a rawhide installer) you will run into templates that don't match the chroot and it will fail
  • If you grab the newer postinst template from github so that the install rules match the new packages, then all of the templates will fail later when it tries to install the "config_files" (sshd config, pam config, etc.)
  • If you munge the templates for the version of lorax you have, you'll build an installer environment that will not work (lacking the right config files for the version of rawhide used to build the chroot)

Solution:

  1. The artifacts in config_files are what anaconda needs for it's environment and thus should be stored in the anaconda repo and thus subsequently in the anaconda package
  2. Update lorax to have an option to overwrite a template (rather than add a template). Ala. --postinst-template=... or possibly --template-dir

Once that's done you should in theory be able to cross build easily with any version of lorax and the latest templates from github.

I'm stumped on failure during mkmacboot()

I'm attempting to build a live image via our Koji and have gotten past most issues, but the following (from livemedia-out.log) has me stumped:

2016-10-14 10:06:14,545 DEBUG pylorax.ltmpl: running command: ('mkefiboot', '--label=ANACONDA', '--apple', '--icon=/var/tmp/lorax.imgutils.iyo8x2zq/usr/share/pixmaps/bootloader/fedora.icns', '--diskname=/var/tmp/lorax.imgutils.iyo8x2zq/usr/share/pixmaps/bootloader/fedora-media.vol', '--product=mdct-aos-flash glenah', '/var/tmp/lmc-work-uq14bv7m/EFI/BOOT', '/var/tmp/lmc-work-uq14bv7m/images/macboot.img')
2016-10-14 10:06:15,046 DEBUG pylorax.ltmpl: command output:
Traceback (most recent call last):
  File "/usr/sbin/mkefiboot", line 167, in <module>
    main()
  File "/usr/sbin/mkefiboot", line 158, in main
    opt.diskname)
  File "/usr/sbin/mkefiboot", line 45, in mkmacboot
    mkhfsimg(None, outfile, label=label, graft=graft, size=size)
  File "/usr/lib/python3.5/site-packages/pylorax/imgutils.py", line 451, in mkhfsimg
    mkfsargs=["-v", label], graft=graft)
  File "/usr/lib/python3.5/site-packages/pylorax/imgutils.py", line 424, in mkfsimage
    with Mount(loopdev, mountargs) as mnt:
  File "/usr/lib/python3.5/site-packages/pylorax/imgutils.py", line 315, in __enter__
    self.mnt = mount(self.dev, self.opts, self.mnt)
  File "/usr/lib/python3.5/site-packages/pylorax/imgutils.py", line 198, in mount
    runcmd(cmd)
  File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 339, in runcmd
    return execWithRedirect(cmd[0], cmd[1:], **kwargs)
  File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 226, in execWithRedirect
    env_add=env_add, reset_handlers=reset_handlers, reset_lang=reset_lang)[0]
  File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 199, in _run_program
    raise subprocess.CalledProcessError(proc.returncode, argv, output)
subprocess.CalledProcessError: Command '['mount', '/dev/loop1', '/tmp/lorax.imgutils.kyxgf0lo']' returned non-zero exit status 32

2016-10-14 10:06:15,046 DEBUG pylorax.ltmpl: command returned failure (1)
2016-10-14 10:06:15,046 ERROR pylorax.ltmpl: template command error in x86.tmpl:
2016-10-14 10:06:15,047 ERROR pylorax.ltmpl:   runcmd mkefiboot --label=ANACONDA --apple --icon=/var/tmp/lorax.imgutils.iyo8x2zq/usr/share/pixmaps/bootloader/fedora.icns --diskname=/var/tmp/lorax.imgutils.iyo8x2zq/usr/share/pixmaps/bootloader/fedora-media.vol --product=mdct-aos-flash glenah /var/tmp/lmc-work-uq14bv7m/EFI/BOOT /var/tmp/lmc-work-uq14bv7m/images/macboot.img
2016-10-14 10:06:15,052 ERROR pylorax.ltmpl:   subprocess.CalledProcessError: Command '['mkefiboot', '--label=ANACONDA', '--apple', '--icon=/var/tmp/lorax.imgutils.iyo8x2zq/usr/share/pixmaps/bootloader/fedora.icns', '--diskname=/var/tmp/lorax.imgutils.iyo8x2zq/usr/share/pixmaps/bootloader/fedora-media.vol', '--product=mdct-aos-flash glenah', '/var/tmp/lmc-work-uq14bv7m/EFI/BOOT', '/var/tmp/lmc-work-uq14bv7m/images/macboot.img']' returned non-zero exit status 1
2016-10-14 10:06:15,052 DEBUG pylorax.ltmpl:   Traceback (most recent call last):
2016-10-14 10:06:15,053 DEBUG pylorax.ltmpl:     File "/usr/lib/python3.5/site-packages/pylorax/ltmpl.py", line 515, in runcmd
2016-10-14 10:06:15,053 DEBUG pylorax.ltmpl:       stdout = runcmd_output(cmd)
2016-10-14 10:06:15,053 DEBUG pylorax.ltmpl:     File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 345, in runcmd_output
2016-10-14 10:06:15,053 DEBUG pylorax.ltmpl:       return execWithCapture(cmd[0], cmd[1:], **kwargs)
2016-10-14 10:06:15,054 DEBUG pylorax.ltmpl:     File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 247, in execWithCapture
2016-10-14 10:06:15,054 DEBUG pylorax.ltmpl:       reset_handlers=reset_handlers, reset_lang=reset_lang)[1]
2016-10-14 10:06:15,054 DEBUG pylorax.ltmpl:     File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 199, in _run_program
2016-10-14 10:06:15,054 DEBUG pylorax.ltmpl:       raise subprocess.CalledProcessError(proc.returncode, argv, output)
2016-10-14 10:06:15,054 DEBUG pylorax.ltmpl:   subprocess.CalledProcessError: Command '['mkefiboot', '--label=ANACONDA', '--apple', '--icon=/var/tmp/lorax.imgutils.iyo8x2zq/usr/share/pixmaps/bootloader/fedora.icns', '--diskname=/var/tmp/lorax.imgutils.iyo8x2zq/usr/share/pixmaps/bootloader/fedora-media.vol', '--product=mdct-aos-flash glenah', '/var/tmp/lmc-work-uq14bv7m/EFI/BOOT', '/var/tmp/lmc-work-uq14bv7m/images/macboot.img']' returned non-zero exit status 1

Intuition tells me that mkefiboot fails because the mount in mkmacboot failed, but I've been unable to uncover why the mount fails. Regardless, I've failed to grok why this happens and what I need to do about it. I don't even need the macboot support as all my targets use BIOS boot, but it appears that Koji's LiveMediaTask class forces this if arch is x86_64.

I have been comparing notes heavily with a flattened fedora-live-xfce.ks but can't seem to find where I've went astray. Any hints would be greatly appreciated!

boot.iso has no metadata inside

If you go to https://getfedora.org and download a netinst image (which is just renamed boot.iso), there is pretty much no way other than parsing a filename to figure out what it is (for example what architecture the image is for).

Lorax generates .discinfo and .treeinfo files together with the iso, but does not include them in the image. These files are only available in the lorax output dir.

Do you think it would make sense to put them inside the boot.iso itself?

SQL Injection..?

Same as in F25 I'm trying to generate installed packages list for anaconda-env for rhel7-branch.

I hacked runtime-postinstall.tmpl in a way:

runcmd chroot ${root} /bin/rpm -qa --qf '%{NAME},%{NVRA}\n' --pipe "sort | tee /root/lorax-packages-formatted.log"

but I get empty log /root/lorax-packages-formatted.log and this msgs in lorax log:

2017-02-04 00:07:22,268 DEBUG pylorax.ltmpl: template line 68: runcmd chroot /var/tmp/lorax.fdP_n3/installtree /bin/rpm -qa --qf %NAME,%NVRA\n --pipe sort | tee /root/lorax-packages-formatted.log
2017-02-04 00:07:22,268 DEBUG pylorax.ltmpl: running command: ('chroot', '/var/tmp/lorax.fdP_n3/installtree', '/bin/rpm', '-qa', '--qf', '%NAME,%NVRA\\n', '--pipe', 'sort | tee /root/lorax-packages-formatted.log')
2017-02-04 00:07:22,587 DEBUG pylorax.ltmpl: command finished successfully

see - no command output. And in program.log:

Running... chroot /var/tmp/lorax.fdP_n3/installtree /bin/rpm -qa --qf %NAME,%NVRA\n --pipe sort | tee /root/lorax-packages-formatted.log
error: incorrect format: invalid field width
error: incorrect format: invalid field width
error: incorrect format: invalid field width
...

And with this format all fine:

runcmd chroot ${root} /bin/rpm -qa --qf '%{NAME} %{NVRA}\n' --pipe "sort | tee /root/lorax-packages.log"

in console (manually) both works fine. Other delimeters (; -) works fine too. Looks like runcmd could handle , in args correctly...

mkefiboot macmunge IndexError: list index out of range

On RHEL 7, with following packages:
livecd-tools-20.4-1.6.el7.x86_64
lorax-19.6.78-1.el7.x86_64
python-2.7.5-16.el7.x86_64

It fails while execute livecd-creator command, and the error info is as following:

Traceback (most recent call last):
  File "/sbin/mkefiboot", line 166, in <module>
    opt.diskname)
  File "/sbin/mkefiboot", line 46, in mkmacboot
    macmunge(outfile, product)
  File "/sbin/mkefiboot", line 73, in macmunge
    shim = glob.glob(os.path.join(mnt, 'EFI/BOOT/BOOT*.EFI'))[0]
IndexError: list index out of range

After copy the macboot.img and mount it by loop device, and write a test code, I found out that the 'glob.glob(os.path.join(mnt, 'EFI/BOOT/BOOT*.EFI'))' returns a empty list. So it will crash on where to get the first element of the list. The data 'BOOT*.EFI' in macboot.img is renamed to lower suffix, such as BOOTX64.efi. we need to adjust the code to adopt to this change, for example:

            shimList = glob.glob(os.path.join(mnt, 'EFI/BOOT/BOOT*.EFI'))
            if len(shimList):
                pass
            else:
                shimList = glob.glob(os.path.join(mnt, 'EFI/BOOT/BOOT*.efi'))
            shim = shimList[0]

Fedora 25 no '-e' option in mkisofs

lorax-templates-generic-25.19-1.fc25.x86_64
lorax-25.19-1.fc25.x86_64
mkisofs-3.02-a07.2.fc25.x86_64

Results in the following:

command finished successfully
template line 51: runcmd mkisofs -o /root/results/images/boot.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -boot-load-size 4 -boot-info-table -no-emul-boot -eltorito-alt-boot -e images/efiboot.img -no-emul-boot -eltorito-alt-boot -e images/macboot.img -no-emul-boot -R -J -V Fedora-Generic-25-x86_64 -T -graft-points images/install.img=/root/results/images/install.img isolinux=/root/results/isolinux images/pxeboot=/root/results/images/pxeboot EFI/BOOT=/root/results/EFI/BOOT images/efiboot.img=/root/results/images/efiboot.img images/macboot.img=/root/results/images/macboot.img
running command: ('mkisofs', '-o', '/root/results/images/boot.iso', '-b', 'isolinux/isolinux.bin', '-c', 'isolinux/boot.cat', '-boot-load-size', '4', '-boot-info-table', '-no-emul-boot', '-eltorito-alt-boot', '-e', 'images/efiboot.img', '-no-emul-boot', '-eltorito-alt-boot', '-e', 'images/macboot.img', '-no-emul-boot', '-R', '-J', '-V', 'Fedora-Generic-25-x86_64', '-T', '-graft-points', 'images/install.img=/root/results/images/install.img', 'isolinux=/root/results/isolinux', 'images/pxeboot=/root/results/images/pxeboot', 'EFI/BOOT=/root/results/EFI/BOOT', 'images/efiboot.img=/root/results/images/efiboot.img', 'images/macboot.img=/root/results/images/macboot.img')
command output:
Bad Option '-e' (error -1 BADFLAG).
Usage: mkisofs [options] [-find] file... [find expression]

Use mkisofs -help
to get a list all of valid options.

Use mkisofs -find -help
to get a list of all valid -find options.

Most important Options:
	-posix-H		Follow sylinks encountered on command line
	-posix-L		Follow all symlinks
	-posix-P		Do not follow symlinks (default)
	-o FILE, -output FILE	Set output file name
	-R, -rock		Generate Rock Ridge directory information
	-r, -rational-rock	Generate rationalized Rock Ridge directory info
	-J, -joliet		Generate Joliet directory information
	-print-size		Print estimated filesystem size and exit
	-UDF			Generate UDF file system
	-dvd-audio		Generate DVD-Audio compliant UDF file system
	-dvd-video		Generate DVD-Video compliant UDF file system
	-dvd-hybrid		Generate a hybrid (DVD-Audio/DVD-Video) compliant UDF file system
	-iso-level LEVEL	Set ISO9660 level (1..3) or 4 for ISO9660 v 2
	-V ID, -volid ID	Set Volume ID
	-graft-points		Allow to use graft points for filenames
	-M FILE, -prev-session FILE	Set path to previous session to merge

command returned failure (255)
template command error in x86.tmpl:
  runcmd mkisofs -o /root/results/images/boot.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -boot-load-size 4 -boot-info-table -no-emul-boot -eltorito-alt-boot -e images/efiboot.img -no-emul-boot -eltorito-alt-boot -e images/macboot.img -no-emul-boot -R -J -V Fedora-Generic-25-x86_64 -T -graft-points images/install.img=/root/results/images/install.img isolinux=/root/results/isolinux images/pxeboot=/root/results/images/pxeboot EFI/BOOT=/root/results/EFI/BOOT images/efiboot.img=/root/results/images/efiboot.img images/macboot.img=/root/results/images/macboot.img
  subprocess.CalledProcessError: Command '['mkisofs', '-o', '/root/results/images/boot.iso', '-b', 'isolinux/isolinux.bin', '-c', 'isolinux/boot.cat', '-boot-load-size', '4', '-boot-info-table', '-no-emul-boot', '-eltorito-alt-boot', '-e', 'images/efiboot.img', '-no-emul-boot', '-eltorito-alt-boot', '-e', 'images/macboot.img', '-no-emul-boot', '-R', '-J', '-V', 'Fedora-Generic-25-x86_64', '-T', '-graft-points', 'images/install.img=/root/results/images/install.img', 'isolinux=/root/results/isolinux', 'images/pxeboot=/root/results/images/pxeboot', 'EFI/BOOT=/root/results/EFI/BOOT', 'images/efiboot.img=/root/results/images/efiboot.img', 'images/macboot.img=/root/results/images/macboot.img']' returned non-zero exit status 255
  Traceback (most recent call last):
    File "/usr/lib/python3.5/site-packages/pylorax/ltmpl.py", line 515, in runcmd
      stdout = runcmd_output(cmd)
    File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 347, in runcmd_output
      return execWithCapture(cmd[0], cmd[1:], **kwargs)
    File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 249, in execWithCapture
      reset_handlers=reset_handlers, reset_lang=reset_lang)[1]
    File "/usr/lib/python3.5/site-packages/pylorax/executils.py", line 201, in _run_program
      raise subprocess.CalledProcessError(proc.returncode, argv, output)
  subprocess.CalledProcessError: Command '['mkisofs', '-o', '/root/results/images/boot.iso', '-b', 'isolinux/isolinux.bin', '-c', 'isolinux/boot.cat', '-boot-load-size', '4', '-boot-info-table', '-no-emul-boot', '-eltorito-alt-boot', '-e', 'images/efiboot.img', '-no-emul-boot', '-eltorito-alt-boot', '-e', 'images/macboot.img', '-no-emul-boot', '-R', '-J', '-V', 'Fedora-Generic-25-x86_64', '-T', '-graft-points', 'images/install.img=/root/results/images/install.img', 'isolinux=/root/results/isolinux', 'images/pxeboot=/root/results/images/pxeboot', 'EFI/BOOT=/root/results/EFI/BOOT', 'images/efiboot.img=/root/results/images/efiboot.img', 'images/macboot.img=/root/results/images/macboot.img']' returned non-zero exit status 255

After running this:

lorax -p Fedora-Generic -v 25 -r 25 -s https://mirror.umd.edu/fedora/linux/releases/25/Everything/x86_64/os/ -s https://mirror.umd.edu/fedora/linux/updates/25/x86_64/ ./results

Ensure all content is tracked by metadata

Problem Description

I'll start by saying I wasn't sure whether to file this issue against Pungi or Lorax (or some other tool I don't yet know about). If there is nothing to be done in Lorax to fix this problem, I would greatly appreciate getting pointed in the right direction(s) 😄

I work on a project called Pulp. One thing Pulp tries to do is mirror yum repositories and installation trees via HTTP. For yum repositories this is fairly straightforward using repodata. However, we've noticed that there are several images and pieces of content that aren't in the treeinfo file.

It would be great if Lorax ensured every piece of content that makes it into installation trees is described by metadata (presumably in the treeinfo file).

Examples

Here are a few examples I've noticed.

Fedora

Fedora 23's Server variant released repo has an EFI/ and isolinux/ directory. None of the files in either of those directories are present in the treeinfo metadata or any other metadata as far as I can tell, so Pulp isn't able to discover them short of guessing filenames.

Fedora 24's Server variant beta repo looks to be missing both the directories Fedora 23 does, but it's also missing the images/efiboot.img and images/macboot.img.

CentOS

In addition to the files missing in Fedora, CentOS also has GPG keys, the GPL, a EULA, and a text file with the build tag in its tree. I'm not certain when this gets added, so maybe there's nothing to be done in Lorax for this.

RHEL

RHEL is similiar to CentOS, but it also contains release notes. Again, I don't know that Lorax is responsible for these files.

Template improvements

Problem: We have changes we want to make to templates, but we don't want to fully fork them and then loose upstream changes.

Proposed solutions:

  1. templates.d/ is already built for multiple directories. It would be great if they stacked. In order. So 1-site-early/runtime-postinstall.tmpl happened before 99-generic/runtime-postinstall.tmpl which happened before 100-site-late/runtime-postinstall.tmpl. This would give great access to leverage the existing templates while also extending them.
  2. If that's not desirable and "one overrides the other" is really important, the default templates could have an include syntax that included other files, if they were present, at specific points providing "hook points" for people to hook into them.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.