ios-xr / gisobuild Goto Github PK

The below behaviour explained in README is not working

To override any input in the yaml config file, please use the corresponding cli option and parameter.
./src/gisobuild.py --yamlfile <input_yaml_cfg> --label <new_label>
The above command will override the label specified in yaml file with new option provided via cli option --label.

GISOs above 1.8GB in size are not supported as per CSCvw23760 and the Limitations section in the Customize Installation using Golden ISO chapter of the System Setup and Software Installation Guide for NCS 5500 (and similarly for ASR 9000). Systems can upgrade successfully using GISOs bigger than 1.8GB but then later run into problems the first time there is a hardware failure and replacement hardware needs to PXE boot to reimage/synchronise its software packages with the rest of the system.

This is an enhancement request so gisobuild refuses to build a GISO bigger than 1.8GB unless a force option is used or generates a warning that needs to be acknowledged for the GiSO build to complete.

Similarly when building GISOs for asr9k platform, warn for GISOs bigger than 1.599GB that they are not supported on RSP880-LT-SE/TR as per System Setup and Software Installation Guide for Cisco ASR 9000 Series Routers.

If pkglist = [ 'all' ], the GISO built done without any SMUs in tar files.

Hello,

Do you have any script to create GISO for new cisco 8000 series routers ? like IOS XR 7.3.2.
Thank you.

Regards,
Abdul Samad

Anyone can provide RPM 5.0 packet? I can't find a proper one and many errors occurred after I upgrade it(seems I downloaded a wrong one)

Tool broken since version 0.35.

Environment:

$ cat /etc/issue
Ubuntu 20.04.3 LTS \n \l

Version 0.34 works:

$ ./gisobuild-0.34.py -v
gisobuild-0.34.py (version 0.34)
$ sudo ./gisobuild-0.34.py -l v0 -i ncs5500-mini-x-7.1.2.iso -r ncs5500-sysadmin-7.1.2.CSCvx16766

System requirements check [PASS]
Golden ISO build process starting...

Platform: ncs5500 Version: 7.1.2

Scanning repository [/usr/local/giso/giso-ncs5500-712/ncs5500-sysadmin-7.1.2.CSCvx16766]...

Building RPM Database...

Total 4 RPM(s) present in the repository path provided in CLI
[ 1] ncs5500-sysadmin-system-7.1.2.3-r712.CSCvx16766.x86_64.rpm
[ 2] ncs5500-sysadmin-ncs5500-7.1.2.3-r712.CSCvx16766.arm.rpm
[ 3] ncs5500-sysadmin-ncs5500-7.1.2.3-r712.CSCvx16766.x86_64.rpm
[ 4] ncs5500-sysadmin-system-7.1.2.3-r712.CSCvx16766.arm.rpm

Following SYSADMIN x86_64 rpm(s) will be used for building Golden ISO:

	(+) ncs5500-sysadmin-system-7.1.2.3-r712.CSCvx16766.x86_64.rpm
	(+) ncs5500-sysadmin-ncs5500-7.1.2.3-r712.CSCvx16766.x86_64.rpm

Following SYSADMIN arm rpm(s) will be used for building Golden ISO:

	(+) ncs5500-sysadmin-ncs5500-7.1.2.3-r712.CSCvx16766.arm.rpm
	(+) ncs5500-sysadmin-system-7.1.2.3-r712.CSCvx16766.arm.rpm

	...RPM signature check [PASS]

	...RPM compatibility check [PASS]

Building Golden ISO...
Summary .....

SYSADMIN rpms:
	ncs5500-sysadmin-system-7.1.2.3-r712.CSCvx16766.x86_64.rpm
	ncs5500-sysadmin-ncs5500-7.1.2.3-r712.CSCvx16766.x86_64.rpm
	ncs5500-sysadmin-ncs5500-7.1.2.3-r712.CSCvx16766.arm.rpm
	ncs5500-sysadmin-system-7.1.2.3-r712.CSCvx16766.arm.rpm

	...Golden ISO creation SUCCESS.

Golden ISO Image Location: /usr/local/giso/giso-ncs5500-712/ncs5500-golden-x-7.1.2-v0.iso

Version 0.35 broken:

$ gisobuild.py -v
gisobuild.py (version 0.35)
$ gisobuild.py -l v0 -i ncs5500-mini-x-7.1.2.iso -r ncs5500-sysadmin-7.1.2.CSCvx16766

System requirements check [PASS]
Golden ISO build process starting...

 Exception: <class 'RuntimeError'> Error CMD=IFS='[] ' read -a a <<< $(isoinfo -i ncs5500-mini-x-7.1.2.iso -R -l | grep " boot.catalog") && dd bs=2048 skip=${a[8]} if=ncs5500-mini-x-7.1.2.iso | head -${a[4]}c > /usr/local/giso/giso-ncs5500-712/tmpq7k5uzsq/boot.catalog returned --->/bin/sh: 1: Syntax error: redirection unexpected

Detailed logs:

2021-09-09 08:18:13::  ##############START#####################
2021-09-09 08:18:13::  Tool version is 0.35
2021-09-09 08:18:13::  Info: XR Congifuration file not specified.
2021-09-09 08:18:13::  Info: Golden ISO will not have XR configuration file
2021-09-09 08:18:13::  Info: User script is not specified.
2021-09-09 08:18:13::  Argument passed to /usr/local/bin/gisobuild.py : Namespace(bundle_iso=['ncs5500-mini-x-7.1.2.iso'], fullISO=False, gisoExtend=False, gisoInfo=False, gisoLabel=['v0'], migTar=False, pkglist=None, rpmRepo=[['ncs5500-sysadmin-7.1.2.CSCvx16766']], script=None, skipDepCheck=False, x86_only=False, xrConfig=None, ztp_ini=None)
2021-09-09 08:18:13::
Performing System requirements check...
2021-09-09 08:18:13::
System requirements check [PASS]
2021-09-09 08:18:13::  Golden ISO build process starting...
2021-09-09 08:18:13::  Cleaning Iso
2021-09-09 08:18:13::  Exiting with exception
2021-09-09 08:18:13::  TB:
Traceback (most recent call last):
 File "/usr/local/bin/gisobuild.py", line 3702, in <module>
   main(args)
 File "/usr/local/bin/gisobuild.py", line 3367, in main
   giso.set_giso_info(argv.bundle_iso[0])
 File "/usr/local/bin/gisobuild.py", line 2016, in set_giso_info
   self.bundle_iso.set_iso_info(iso_path)
 File "/usr/local/bin/gisobuild.py", line 1699, in set_iso_info
   readiso(self.iso_path, self.iso_mount_path)
 File "/usr/local/bin/gisobuild.py", line 3665, in readiso
   run_cmd(cmd)
 File "/usr/local/bin/gisobuild.py", line 94, in run_cmd
   raise RuntimeError("Error CMD=%s returned --->%s" % (cmd, out))
RuntimeError: Error CMD=IFS='[] ' read -a a <<< $(isoinfo -i ncs5500-mini-x-7.1.2.iso -R -l | grep " boot.catalog") && dd bs=2048 skip=${a[8]} if=ncs5500-mini-x-7.1.2.iso | head -${a[4]}c > /usr/local/giso/giso-ncs5500-712/tmpq7k5uzsq/boot.catalog returned --->/bin/sh: 1: Syntax error: redirection unexpected

Hi guys,

Is there any possibility to run the script for creating GISO with normal user rights on Ubuntu without using "Sudo"?

Need some help with the GISO creation.

Have earlier successfully created GISO on WSL. But when i have installed a new virtual debian machine and trying to run the script i get a runtime error. Isn't the script checking in the wrong tmp-directory for the isis rpm?

sadada@server1:~$ sudo python3 /home/sadada/gisobuild/src/gisobuild.py --iso 5500_762_software/ncs5500-mini-x-7.6.2.iso --repo /home/sadada/5500_762_software/ --pkglist ncs5500-isis-2.1.0.0-r762.x86_64.rpm --label v1
System requirements check [PASS]

Platform: ncs5500 Version: 7.6.2

Scanning repository [/home/sadada/5500_762_software]...

Building RPM Database...
Traceback (most recent call last):
File "/home/sadada/gisobuild/src/gisobuild.py", line 532, in
main()
File "/home/sadada/gisobuild/src/gisobuild.py", line 528, in main
execute_build(cli_args)
File "/home/sadada/gisobuild/src/exrmod/isotools_exr.py", line 40, in execute_build
system_build_main (cli_args, infile)
File "/home/sadada/gisobuild/src/exrmod/gisobuild_exr.py", line 193, in main
rpm_db.populate_rpmdb(fs_root, argv.rpmRepo, pkglist,
File "/home/sadada/gisobuild/src/exrmod/gisobuild_exr_engine.py", line 748, in populate_rpmdb
rpm.populate_mdata(fs_root, os.path.basename(file_name),
File "/home/sadada/gisobuild/src/exrmod/gisobuild_exr_engine.py", line 321, in populate_mdata
group_info = run_cmd("chroot "+fs_root+" rpm -qp --qf '%{GROUP}' "+rpm)
File "/home/sadada/gisobuild/src/exrmod/gisobuild_exr_engine.py", line 114, in run_cmd
raise RuntimeError("Error CMD=%s returned --->%s" % (cmd, out))
RuntimeError: Error CMD=chroot /home/sadada/output_gisobuild/tmp4xywesm2 rpm -qp --qf '%{GROUP}' ncs5500-isis-2.1.0.0-r762.x86_64.rpm returned --->

sadada@server1:~/output_gisobuild$ ls
logs tmp9_y6q13q

root@server1:/home/sadada/output_gisobuild/tmp9_y6q13q# ls
ncs5500-isis-2.1.0.0-r762.x86_64.rpm

sadada@server1:~/output_gisobuild/logs$ more gisobuild.log-2023-01-13:08:19:04.198732
2023-01-13 08:19:04:: Performing System requirements check...
2023-01-13 08:19:04:: Available space 25.50180435180664 GB
2023-01-13 08:19:04:: Required space 17.534122467041016 GB
2023-01-13 08:19:04:: System requirements check [PASS]
2023-01-13 08:19:52:: Golden ISO RPM_PATH: giso/
2023-01-13 08:19:52:: Skipping the top level iso wrapper
2023-01-13 08:22:03:: ISO ncs5500-mini-x extract path /home/sadada/output_gisobuild/tmp106otls0
2023-01-13 08:22:03:: Iso top initrd path /home/sadada/output_gisobuild/tmp106otls0
2023-01-13 08:22:16:: Intermal System_image.iso /home/sadada/5500_762_software/ncs5500-mini-x-7.6.2.iso
2023-01-13 08:22:16:: Cleaning Iso
2023-01-13 08:22:16:: iso extract path /home/sadada/output_gisobuild/tmp106otls0
2023-01-13 08:23:02::
Found Bundle ISO: /home/sadada/5500_762_software/ncs5500-mini-x-7.6.2.iso
2023-01-13 08:23:02::
Platform: ncs5500 Version: 7.6.2
2023-01-13 08:25:55:: ISO ncs5500-mini-x extract path /home/sadada/output_gisobuild/tmp4xywesm2
2023-01-13 08:25:55::
Scanning repository [/home/sadada/5500_762_software]...

2023-01-13 08:25:55::
File list After Unification [['/home/sadada/5500_762_software/ncs5500-isis-2.1.0.0-r762.x86_64.rpm']]

2023-01-13 08:25:55:: Building RPM Database...
2023-01-13 08:25:56:: Cleaning Iso
2023-01-13 08:25:56:: iso extract path /home/sadada/output_gisobuild/tmp4xywesm2

Hi,
I am trying to build a GISO for NCS 540 platform , ang getting this error "Gisobuild failed: Missing environment requirements packaging"
Just wondering what is the issue and possible solution for that?
I used the same script for ASR9k platform and successfully built the GISO image.

When put tar files under repo and pkglist specify list of tar file name, if the it has sysadmin SMU tar file, the GISO build for eXR failed. I think validate_and_return_list() in gisobuild_exr_engine.py require similar logic for "if pkg.endswith('.tar'):" as "elif pkg.endswith('.rpm'):" to treat sysadmin SMUs.

The below is error message. I've put some debug lines so the line numbers in below error are not aligned with original source code.

Traceback (most recent call last):
  File "/opt/giso/./gisobuild/src/gisobuild.py", line 532, in <module>
    main()
  File "/opt/giso/./gisobuild/src/gisobuild.py", line 528, in main
    execute_build(cli_args)
  File "/opt/giso/gisobuild/src/exrmod/isotools_exr.py", line 40, in execute_build
    system_build_main (cli_args, infile)
  File "/opt/giso/gisobuild/src/exrmod/gisobuild_exr.py", line 193, in main
    rpm_db.populate_rpmdb(fs_root, argv.rpmRepo, pkglist,
  File "/opt/giso/gisobuild/src/exrmod/gisobuild_exr_engine.py", line 703, in populate_rpmdb
    self.tmp_smu_repo_path, repo_files = Rpmdb.validate_and_return_list(platform, repo_paths, pkglist)
  File "/opt/giso/gisobuild/src/exrmod/gisobuild_exr_engine.py", line 551, in validate_and_return_list
    result = run_cmd(cmd)
  File "/opt/giso/gisobuild/src/exrmod/gisobuild_exr_engine.py", line 114, in run_cmd
    raise RuntimeError("Error CMD=%s returned --->%s" % (cmd, out))
RuntimeError: Error CMD=rpm -qpR /opt/giso/output/R08_ISO_01/tmppo67pzdn/ncs5500-sysadmin-hostos-7.2.1.1-r721.CSCvv89835.host.arm.rpm | grep -e ncs5500 | grep  ' = ' returned --->warning: /opt/giso/output/R08_ISO_01/tmppo67pzdn/ncs5500-sysadmin-hostos-7.2.1.1-r721.CSCvv89835.host.arm.rpm: Header V4 RSA/SHA256 Signature, key ID 5746bd08: NOKEY

Hi, we have tried to use it with XR 6.6.25 but it wont load in the configuration then.

We get it to work with XR 6.5.3 perfectly

Is there some features that are unsupported in 6.6.25 that is making it break?

On line 160, in function create_migration_tar. You reference a command "iso-read" which is not found in the dependencies. After some research, it seems that this tool is deprecated. Please replace this with mkisofs.

cXR has a file size limitation of maximum 2 GB size for a single file. If you include lots of SMUs into the GISO or migration TAR then you end up with an error message when you try to copy that file to the harddisk of the ASR9k.

Please either stop the GISO build process with an error if the generated file exceeds that limiation or at least drop a warning at the end to avoid unnecessary service requests.

Thanks!

('\n', 'Exception:', <type 'exceptions.RuntimeError'>, RuntimeError("Error CMD=chroot /mnt/hgfs/aaa/tmpfMc211 rpm -qp --qf '%{NAME};%{VERSION};%{RELEASE};%{ARCH};%{PACKAGETYPE};%{PACKAGEPRESENCE};%{PIPD};%{CISCOHW};%{CARDTYPE};%{BUILDTIME};%{GROUP};%{VMTYPE};%{SUPPCARDS};%{PREFIXES};%{XRRELEASE};' asr9k-bng-x64-1.0.0.0-r702.x86_64.rpm returned --->chroot: failed to run command \xe2\x80\x98rpm\xe2\x80\x99: No such file or directory\n",))
Detail logs: /mnt/hgfs/aaa/Giso_build.log-2020-07-27:17:17:50.748757

Anyone knows how to solve this issue?

https://github.com/ios-xr/gisobuild/blob/master/gisobuild.py#L167

Uses logger.Info()

Should be logger.info()

Hey!

It seems the USB image is not built when the giso is for the NCS platform. We have to use https://github.com/venkateswarannagarajan/usbboot. Would it be possible to integrate it in gisobuild instead?

Thanks!

Hi,
I've built an iso using a repository that had around 50 RPMs in it. While the iso was built without any warning or error, the image installed on the router has significantly less RPMs than what it should have.

As an example: ncs5500-dpa-3.0.0.8-r7721.CSCwi35880.x86_64.rpm was inside of the repository but I cannot see that installed, when on the router I issue the "show install active summary" on the router I see:

Active Packages: 17
ncs5500-xr-7.7.21 version=7.7.21 [Boot image]
libpci3-3.6.4.p1-r0.r7721.CSCwe49269.xr
ncs5500-mpls-2.1.0.0-r7721
ncs5500-mpls-te-rsvp-3.1.0.0-r7721
ncs5500-ospf-2.0.0.0-r7721
ncs5500-mgbl-3.0.0.0-r7721
ncs5500-mcast-3.0.0.0-r7721
ncs5500-k9sec-3.1.0.0-r7721
ncs5500-infra-5.1.0.6-r7721.CSCwi09344
ncs5500-parser-2.0.0.1-r7721.CSCwf53989
ncs5500-iosxr-fwding-4.0.0.8-r7721.CSCwh79153
ncs5500-ospf-2.0.0.1-r7721.CSCwi08034
ncs5500-bgp-2.0.0.4-r7721.CSCwe14754
ncs5500-dpa-3.0.0.9-r7721.CSCwf98671
ncs5500-dpa-fwding-4.0.0.7-r7721.CSCwf57277
ncs5500-mpls-te-rsvp-3.1.0.1-r7721.CSCwe78378
ncs5500-os-support-4.0.0.2-r7721.CSCwi26163

and in admin:

ncs5500-sysadmin-7.7.21 version=7.7.21 [Boot image]
libpci3-3.6.4.p1-r0.r7721.CSCwe49269.admin
ncs5500-sysadmin-ncs5500-7.7.21.2-r7721.CSCwe59739
libpci3-3.6.4.p1-r0.r7721.CSCwe49269.host

Is the giso script skipping all RPMs not listed or, they're being merged into others? Could you please advise?

Thank you
rpms_packaged_in_giso.txt
gisobuild.log-2024-02-29%3A16%3A55%3A41.log

Running the following helper script for LNT isols.py
located at src/lntmod

The command referenced in this guide fails on python3.6

https://xrdocs.io/8000/tutorials/8000-software-xr7-giso/#giso-verification

root@NA----DEV-Machine:~/gisobuild/src/lntmod# ./isols.py /root/gisobuild/nsight_builds/builds/N540L_752_v4/giso/ncs540l-golden-x86_64-7.5.2-N540L_752_v4.iso --optional-packages
Traceback (most recent call last):
File "./isols.py", line 28, in
sys.path.append(str(pathlib.Path(file).parents[1]))
File "/usr/lib/python3.6/pathlib.py", line 594, in getitem
raise IndexError(idx)
IndexError: 1

Changing the script to the following works

#!/usr/bin/env python3
# ----------------------------------------------------------------

""" Wrapper script to extract ISO information.

Copyright (c) 2022 Cisco and/or its affiliates.
This software is licensed to you under the terms of the Cisco Sample
Code License, Version 1.1 (the "License"). You may obtain a copy of the
License at

        https://developer.cisco.com/docs/licenses

All use of the material herein must be in accordance with the terms of
the License. All rights not expressly granted by the License are
reserved. Unless required by applicable law or agreed to separately in
writing, software distributed under the License is distributed on an "AS
IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
or implied.

"""

__all__ = ()

from pathlib import Path
import sys

sys.path.append(str(Path(__file__).resolve().parents[1]))
from lnt import tools


if __name__ == "__main__":
    tools.isols(sys.argv[1:])

Hi
I try to create the GISO on a Ubuntu 22.04.
System requirements check passed, but creation fails.
Any help is appreciated
thanks in advance for your help

root@tzhpemac-ubu20:/home/tzhpemac/GISO# sudo ./gisobuild/src/gisobuild.py --iso /home/tzhpemac/GISO/RPM/ASR9K-x64-iosxr-7.4.2/asr9k-mini-x64-7.4.2.iso --repo /home/tzhpemac/GISO/RPM/7.4.2_SMU/ --clean
System requirements check [PASS]
Traceback (most recent call last):
File "/home/tzhpemac/GISO/./gisobuild/src/gisobuild.py", line 513, in
main()
File "/home/tzhpemac/GISO/./gisobuild/src/gisobuild.py", line 509, in main
execute_build(cli_args)
File "/home/tzhpemac/GISO/gisobuild/src/exrmod/isotools_exr.py", line 40, in execute_build
system_build_main (cli_args, infile)
File "/home/tzhpemac/GISO/gisobuild/src/exrmod/gisobuild_exr.py", line 101, in main
giso.set_giso_info(argv.bundle_iso)
File "/home/tzhpemac/GISO/gisobuild/src/exrmod/gisobuild_exr_engine.py", line 2056, in set_giso_info
if not args.optimize:
AttributeError: 'Namespace' object has no attribute 'optimize'

root@tzhpemac-ubu20:/home/tzhpemac/GISO# sudo ./gisobuild/src/gisobuild.py --version
gisobuild.py (version 1.0)

root@tzhpemac-ubu20:/home/tzhpemac/GISO# more /home/tzhpemac/GISO/gisobuild/src/output_gisobuild/logs/gisobuild.log-2022-06-21:11:03:19.032310
2022-06-21 11:03:19:: Performing System requirements check...
2022-06-21 11:03:19:: Available space 30.58446502685547 GB
2022-06-21 11:03:19:: Required space 14.372596740722656 GB
2022-06-21 11:03:19:: System requirements check [PASS]
2022-06-21 11:03:42:: Cleaning Iso

When running the script inside rootless Podman container ( $UID = 0) causes the script to hang.
There is a workaround to use Podman run parameter '--userns=keep-id' :

2021-02-10 12:56:47:: ISO /home/git/gisobuild/tmpbz8bv034/iso/host.iso vm_type HOST searchkey HOST.ISO
2021-02-10 12:56:51:: Cleaning Iso
2021-02-10 12:56:51:: iso extract path /home/git/gisobuild/tmpphdrdw8f
2021-02-10 12:56:51:: Cleaning Iso
2021-02-10 12:56:51:: iso extract path /home/git/gisobuild/tmpbz8bv034
2021-02-10 12:56:52:: Exiting with exception
2021-02-10 12:56:52:: TB:
Traceback (most recent call last):
File "/home/git/gisobuild/./gisobuild.py", line 3235, in
main(args)
File "/home/git/gisobuild/./gisobuild.py", line 3039, in main
supp_arch = giso.get_supp_arch(vm_type)
File "/home/git/gisobuild/./gisobuild.py", line 1859, in get_supp_arch
fs_root = iso.get_iso_extract_path()
File "/home/git/gisobuild/./gisobuild.py", line 1425, in get_iso_extract_path
run_cmd("zcat -f %s%s | cpio -id" % (self.iso_mount_path,
File "/home/git/gisobuild/./gisobuild.py", line 76, in run_cmd
raise RuntimeError("Error CMD=%s returned --->%s" % (cmd, out))
RuntimeError: Error CMD=zcat -f /home/git/gisobuild/tmplrxzonj8/boot/initrd.img | cpio -id returned --->cpio: var: C
annot change ownership to uid 434115, gid 25: Invalid argument
cpio: var/backups: Cannot change ownership to uid 434115, gid 25: Invalid argument
cpio: var/lib: Cannot change ownership to uid 434115, gid 25: Invalid argument
.
.
.

Hi team ,

I very aprecciated your development its a very good tool .

I have been build the Golden ISO for eXR 7.7.2 and when I run the script :

--clean: ./src/gisobuild.py --yamlfile /app/gisobuild/repo/< NAME OF MY YAML FILE >

The app show me the following log :

Minimum 15 GB of free disk space is required for building Golden ISO.
Error: System requirements check [FAIL]

My HD in VM have a 17GB of memory and avaliable 9.G . The questions are :

Why does the app need so much available space to build the goldeniso ?
Is it possible to modify the available space validation to a lower value? xr isos and rpm packages are generally no larger than 5 GB.

I very aprecciated your time , thanks in advance.

the name of the output file is different than in the documentation:

output:
asr9k-goldenk9-x64-6.5.3-v1-migrate_to_eXR

Cisco documentation:
asr9k-goldenk9-x64-migrate_to_eXR.tar-6.5.3.v1

gisobuild_exr_engine.py

1. Typo : ciso_rpm_files should be cisco_rpm_files

2. ciso_rpm_files keep growing with duplicated items. The below lines do not need to execute multiple times, should be only once around line 527.

                                    for repo_path in repo_paths:
                                        cisco_rpm=("%s/%s*.rpm" %(repo_path, platform))
                                        ciso_rpm_files += glob.glob(cisco_rpm)