cumulusnetworks / due Goto Github PK

Hi, I'm trying to create onie-debian-10 image using DUE but building failed due to can't finding efitools from this link:
https://git.kernel.org/pub/scm/linux/kernel/git/jejb/efitools.git/snapshot/efitools-1.9.2.tar.gz
Please change this mirror link to a valid one in the post-install-config.sh.template file.

Hi,
I have a basic insight of Docker and DUE as build environment for ONIE.
I want to know their pros and cons in order to make a decision.
-When and Why would you use DUE when you can achieve it on a simple docker?
-Does DUE require root access?
Thank you in advance.

What happened:

$ debcheckout foo
E: You must put some 'deb-src' URIs in your sources.list
unknown package 'foo'
$ 

I see two options to make debcheckout possible:
1: Copy apt sources list from host
2: Add the deb-src in sources.list

Option 1 has the advantage that you copy / clone what the developer already has configured on the host. It supports local configuration.

configure: error: missing required tool: unzip
make/crosstool-ng.make:92: recipe for target '/home/sonic/onie/build/crosstool-ng/stamp/crosstool-ng-configure' failed
make: *** [/home/sonic/onie/build/crosstool-ng/stamp/crosstool-ng-configure] Error 1
make: *** Waiting for unfinished jobs....
2023-11-08 09:32:06 URL:http://mirror.opencompute.org/onie/crosstool-NG/isl-0.20.tar.xz [1539064/1539064] -> "/home/sonic/onie/build/download/isl-0.20.tar.xz" [1]
Got isl-0.20.tar.xz

isl-0.20.tar.xz: OK
==== Getting upstream mpc-1.1.0.tar.gz ====
Downloading mpc-1.1.0.tar.gz from: http://mirror.opencompute.org/onie/crosstool-NG/mpc-1.1.0.tar.gz
2023-11-08 09:32:08 URL:http://mirror.opencompute.org/onie/crosstool-NG/mpc-1.1.0.tar.gz [701263/701263] -> "/home/sonic/onie/build/download/mpc-1.1.0.tar.gz" [1]
Got mpc-1.1.0.tar.gz

mpc-1.1.0.tar.gz: OK
==== Getting upstream mpfr-4.1.0.tar.xz ====
Downloading mpfr-4.1.0.tar.xz from: http://mirror.opencompute.org/onie/crosstool-NG/mpfr-4.1.0.tar.xz
2023-11-08 09:32:15 URL:http://mirror.opencompute.org/onie/crosstool-NG/mpfr-4.1.0.tar.xz [1525476/1525476] -> "/home/sonic/onie/build/download/mpfr-4.1.0.tar.xz" [1]
Got mpfr-4.1.0.tar.xz

mpfr-4.1.0.tar.xz: OK
==== Getting upstream strace-4.26.tar.xz ====
Downloading strace-4.26.tar.xz from: http://mirror.opencompute.org/onie/crosstool-NG/strace-4.26.tar.xz
2023-11-08 09:32:22 URL:http://mirror.opencompute.org/onie/crosstool-NG/strace-4.26.tar.xz [1444216/1444216] -> "/home/sonic/onie/build/download/strace-4.26.tar.xz" [1]
Got strace-4.26.tar.xz

strace-4.26.tar.xz: OK
==== Getting upstream zlib-1.2.11.tar.xz ====
Using cached /home/sonic/onie/build/download/zlib-1.2.11.tar.xz
zlib-1.2.11.tar.xz: OK
sonic@due-onie-build-debian-9:~/onie/build-config$

Hi folks,
I'm attempting to build an ONIE image using DUE - I don't have an installed debian system so I decided to go with a VM running Debian 10 and then running docker that way. However when I execute the due create command I get the following:

sri@debian:~/src/DUE$ ./due --create --from debian:9 --description "ONIE Build Debian 9" --name onie-build-debian-9 --prompt ONIE-9 --tag onie --use-template onie
==== Setting --use-template path to /home/sri/src/DUE/templates/onie ====

|
| Creating work directory /home/sri/src/DUE/due-build-merge
|__________________________________________________________________________

|
| Copying /home/sri/src/DUE/templates/common-templates to /home/sri/src/DUE/due-build-merge/onie-build-debian-9-template-merge
|__________________________________________________________________________

==== Contents of: /home/sri/src/DUE/due-build-merge/onie-build-debian-9-template-merge ====
total 36
-rw-r--r-- 1 sri sri 554 Jan 21 10:29 Dockerfile.config
-rw-r--r-- 1 sri sri 1904 Jan 21 10:29 Dockerfile.template
-rw-r--r-- 1 sri sri 3453 Jan 21 10:29 duebuild-readme.md
drwxr-xr-x 4 sri sri 4096 Jan 21 10:29 filesystem
-rw-r--r-- 1 sri sri 5267 Jan 21 10:29 install-config-common-lib.template
-rwxr-xr-x 1 sri sri 589 Jan 21 10:29 post-install-config.sh.template
-rwxr-xr-x 1 sri sri 2479 Jan 21 10:29 pre-install-config.sh.template
-rw-r--r-- 1 sri sri 1215 Jan 21 10:29 README.md

|
| Merging in [ onie-build-debian-9 ] specific files from /home/sri/src/DUE/templates/onie to /home/sri/src/DUE/due-build-merge/onie-build-debian-9-template-merge
|__________________________________________________________________________

==== Contents of: /home/sri/src/DUE/due-build-merge/onie-build-debian-9-template-merge ====
total 48
-rw-r--r-- 1 sri sri 656 Jan 21 10:29 Dockerfile.config
-rw-r--r-- 1 sri sri 1904 Jan 21 10:29 Dockerfile.template
-rw-r--r-- 1 sri sri 3453 Jan 21 10:29 duebuild-readme.md
drwxr-xr-x 4 sri sri 4096 Jan 21 10:29 filesystem
-rw-r--r-- 1 sri sri 5267 Jan 21 10:29 install-config-common-lib.template
-rwxr-xr-x 1 sri sri 7882 Jan 21 10:29 post-install-config.sh.template
-rwxr-xr-x 1 sri sri 2479 Jan 21 10:29 pre-install-config.sh.template
-rw-r--r-- 1 sri sri 8722 Jan 21 10:29 README.md

|
| Creating directory to be used for container build at: [ onie-build-debian-9 ]
|__________________________________________________________________________

==== Copying non-template files from [ /home/sri/src/DUE/due-build-merge/onie-build-debian-9-template-merge ] to [ onie-build-debian-9 ] ====
sending incremental file list
created directory /home/sri/src/DUE/due-build-merge/onie-build-debian-9
./
Dockerfile.config
README.md
duebuild-readme.md
filesystem/
filesystem/etc/
filesystem/etc/apt/
filesystem/etc/apt/sources.list.d/
filesystem/etc/apt/sources.list.d/stretch-backports.list
filesystem/usr/
filesystem/usr/local/
filesystem/usr/local/bin/
filesystem/usr/local/bin/container-create-user.sh
filesystem/usr/local/bin/duebuild

sent 42,201 bytes received 245 bytes 84,892.00 bytes/sec
total size is 41,476 speedup is 0.98
==== Contents of: /home/sri/src/DUE/due-build-merge/onie-build-debian-9 ====
total 24
-rw-r--r-- 1 sri sri 656 Jan 21 10:29 Dockerfile.config
-rw-r--r-- 1 sri sri 3453 Jan 21 10:29 duebuild-readme.md
drwxr-xr-x 4 sri sri 4096 Jan 21 10:29 filesystem
-rw-r--r-- 1 sri sri 8722 Jan 21 10:29 README.md

ERROR at libdue line 1331 : Cannot map [ i686 ] to a Docker contianer type. Exiting.

I don't know if this is because I'm not running Debian on bare metal or what. I'm running this under GNOME Boxes - which is a libvirt based VM technology running on Fedora Silverblue 33.

Please add wget to the default installed programs.

FWIW it feels strange to pick an environment from a menu.

Or expressed that feeling somewhat stronger

selecting a environment from a menu breaks command line workflow

Please make it possible to select environment as parameter of run

Good afternoon. Some time ago I tried to compile an image kvm_x86_64. With varying success, thanks to YouTube videos and issues addressed in this project. Everything turned out great.
But when I tried to build an image for the switch accton_as4610_54, I ran into a problem and I don’t know where to dig further.
When trying to build an image, it gives the following line.
user@due-onie-build-debian-9:~/onie/build-config$ make -j4 MACHINEROOT=../machine/accton MACHINE=accton_as4610_54 all
make: *** No rule to make target 'conf/crosstool/gcc-4.9.2/uClibc-ng-1.0.38/crosstool.armv7a.config', needed by '/home/ychernov/onie/build/x-tools/armv7a-g4.9.2-lnx3.2.69-uClibc-ng-1.0.38/build/.config'. Stop.

I will be grateful for your help.

Seeing

run dpkg-buildpackage -us -uc with 4 jobs (unless a different -jX has been passed)

made me creating this issue :-)

I don't know where the 4 comes from, so I felt the need to tell about nproc.

Tool nproc is default available and returns the number of processors a system has.
Typical use case is

make -j $(nproc)

So I'm suggestion

run dpkg-buildpackage -us -uc with nproc jobs (unless a different -jX has been passed)

And do feel free to close this issue.

Looking at the debian package, I see

$ grep binfmt-support -R
libdue:RECOMMENDED_PACKAGES=" binfmt-support qemu qemu-user-static "
debian/control:Recommends: binfmt-support, qemu, qemu-user-static
docs/Troubleshooting.md:`sudo apt update ; sudo apt install docker.io git rsync binfmt-support qemu qemu-user-static`  
docs/Troubleshooting.md:`binfmt-support` not being configured to handle ARM binaries. Ideally, qemu should register
docs/Troubleshooting.md:the architectures it can run with binfmt-support, so that when non-native code is encountered,
docs/Troubleshooting.md:`sudo apt update ; sudo apt install qemu qemu-user-static binfmt-support`
docs/Troubleshooting.md:which should have configured binfmt-support to have the entries. I had to do this on one system, 
docs/Troubleshooting.md:Restart binfmt-support  
docs/Troubleshooting.md:`sudo systemctl restart binfmt-support.service` 

systemd provides a builtin, cross-distro facility named systemd-binfmt to
register binary formats.

You can read more about it at
https://www.freedesktop.org/software/systemd/man/systemd-binfmt.html
https://www.freedesktop.org/software/systemd/man/binfmt.d.html
https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html

qemu in sid and bookworm ships binfmt.d configs, which means, installing binfmt-support explicitly is no longer necessary.
The docs should be updated to reflect that.

Hi...
im really new to this github and code thing, very sorry for my bad explanation and terrible english.

im trying to create debian 8 with DUE,
but alot of trouble I face.
from pre-install, and post install whole download package was error.

I think thats because the key is expired or invalid,
I have the change the code (pre install, commonlib, post install) to apply apt-key adv --recv-keys from debian8 repo,
and add another --force-yes to every apt-get install command.

maybe you can help another ppl to patch the code if they want to use debian 8.

the error is like this...

W: GPG error: http://deb.debian.org jessie-updates InRelease: The following signatures were invalid: KEYEXPIRED 1668891673
W: GPG error: http://deb.debian.org stretch-backports InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 648ACFD622F3D138 NO_PUBKEY 0E98404D386FA1D9
W: GPG error: http://deb.debian.org jessie Release: The following signatures were invalid: KEYEXPIRED 1668891673

been fighting with building onie for 7 days now, i've followed the tutorial videos very closely and believe this to be an issue with either DUE or ONIE src code itself. the error im getting is "Make[2] *** no rule to make target '/home/awalls/src/onie/encryption/machines/kvm_x86_64/keys/ONIE/efi-keys/ONIE-shim-key-cert.pem' ---

I want to know exactly what you are running DUE in. im using the latest everything i've been following your notes very closely. nothing seems to get around signing.keys.x509.
Please help!

Run the due command described at https://github.com/CumulusNetworks/DUE/tree/master/templates/onie

due --create --from debian:8 --description "ONIE Build Debian 8" --name onie-build-debian-8 --prompt ONIE-8 --tag onie-8 --use-template onie

After some time, the build fails when resources can not be downloaded:

wget https://github.com/github/hub/releases/download/v2.12.8/hub-linux-amd64-2.12.8.tgz
converted 'https://github.com/github/hub/releases/download/v2.12.8/hub-linux-amd64-2.12.8.tgz' (ANSI_X3.4-1968) -> 'https://github.com/github/hub/releases/download/v2.12.8/hub-linux-amd64-2.12.8.tgz' (UTF-8)
--2022-03-09 19:33:04-- https://github.com/github/hub/releases/download/v2.12.8/hub-linux-amd64-2.12.8.tgz
Resolving github.com (github.com)... 192.30.255.113
Connecting to github.com (github.com)|192.30.255.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/401025/87045880-e589-11e9-9915-6000d9cdaa81?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220309%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220309T193304Z&X-Amz-Expires=300&X-Amz-Signature=5aabf0b437b0a6c1622f75f73db5ba2048eeb154ebfeb4cbdc74b984da386782&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=401025&response-content-disposition=attachment%3B%20filename%3Dhub-linux-amd64-2.12.8.tgz&response-content-type=application%2Foctet-stream [following]
converted 'https://objects.githubusercontent.com/github-production-release-asset-2e65be/401025/87045880-e589-11e9-9915-6000d9cdaa81?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220309%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220309T193304Z&X-Amz-Expires=300&X-Amz-Signature=5aabf0b437b0a6c1622f75f73db5ba2048eeb154ebfeb4cbdc74b984da386782&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=401025&response-content-disposition=attachment%3B%20filename%3Dhub-linux-amd64-2.12.8.tgz&response-content-type=application%2Foctet-stream' (ANSI_X3.4-1968) -> 'https://objects.githubusercontent.com/github-production-release-asset-2e65be/401025/87045880-e589-11e9-9915-6000d9cdaa81?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A/20220309/us-east-1/s3/aws4_request&X-Amz-Date=20220309T193304Z&X-Amz-Expires=300&X-Amz-Signature=5aabf0b437b0a6c1622f75f73db5ba2048eeb154ebfeb4cbdc74b984da386782&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=401025&response-content-disposition=attachment; filename=hub-linux-amd64-2.12.8.tgz&response-content-type=application/octet-stream' (UTF-8)
--2022-03-09 19:33:04-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/401025/87045880-e589-11e9-9915-6000d9cdaa81?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A/20220309/us-east-1/s3/aws4_request&X-Amz-Date=20220309T193304Z&X-Amz-Expires=300&X-Amz-Signature=5aabf0b437b0a6c1622f75f73db5ba2048eeb154ebfeb4cbdc74b984da386782&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=401025&response-content-disposition=attachment;%20filename=hub-linux-amd64-2.12.8.tgz&response-content-type=application/octet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.110.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Username/Password Authentication Failed.

For what it is worth: There is no counter part of --create

It might be as simple as some docker prune command, but please document it.