bash 4.4 support about dracut

[tpgxyz]

@danimo @haraldh thanks for not merging my PR (#119), and rather than creating new one...

[Ratler]

@haraldh The problem is the modules without a signature in this case. The ones I've looked at have multiple '00' in the end of the file. Adding head -c 27 won't do much good.

00030350  de fd 02 00 00 00 00 00  d5 00 00 00 00 00 00 00  |................|
00030360  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
00030370  00 00 00 00 00 00 00 00                           |........|

Maybe using something like modinfo "$f" | grep '^sig_key:' to find the signed modules?

EDIT: Actually when looking at it again, why not just skip assigning the data to the variable SIG and do: tail -c 28 "$f" | grep -q '~Module signature appended~' || { printf ... } it's the variable assignment that is the issue here right?.

[tpgxyz]

running dracut -f --debug on already existing initrams

//usr/lib/dracut/dracut-init.sh@1050(find_kernel_modules_by_path): continue
//usr/lib/dracut/dracut-init.sh@1049(find_kernel_modules_by_path): read a rest
//usr/lib/dracut/dracut-init.sh@1050(find_kernel_modules_by_path): [[ kernel/drivers/block/floppy.ko.xz = /drivers/gpu/drm/ ]]
//usr/lib/dracut/dracut-init.sh@1050(find_kernel_modules_by_path): [[ kernel/drivers/block/floppy.ko.xz = updates/* ]]
//usr/lib/dracut/dracut-init.sh@1050(find_kernel_modules_by_path): continue
//usr/lib/dracut/dracut-init.sh@1049(find_kernel_modules_by_path): read a rest
//usr/lib/dracut/dracut-init.sh@1050(find_kernel_modules_by_path): [[ kernel/drivers/block/pmem.ko.xz = /drivers/gpu/drm/ ]]
//usr/lib/dracut/dracut-init.sh@1050(find_kernel_modules_by_path): [[ kernel/drivers/block/pmem.ko.xz = updates/* ]]
//usr/lib/dracut/dracut-init.sh@1050(find_kernel_modules_by_path): continue
//usr/lib/dracut/dracut-init.sh@1049(find_kernel_modules_by_path): read a rest
/usr/lib/dracut/modules.d/50drm/module-setup.sh: line 29: warning: command substitution: ignored null byte in input
//usr/lib/dracut/dracut-init.sh@1050(find_kernel_modules_by_path): [[ kernel/drivers/block/loop.ko.xz = /drivers/gpu/drm/ ]]
/usr/lib/dracut/modules.d/50drm/module-setup.sh: line 29: warning: command substitution: ignored null byte in input
/usr/lib/dracut/modules.d/50drm/module-setup.sh: line 29: warning: command substitution: ignored null byte in input
/usr/lib/dracut/modules.d/50drm/module-setup.sh: line 29: warning: command substitution: ignored null byte in input
//usr/lib/dracut/dracut-init.sh@1050(find_kernel_modules_by_path): [[ kernel/drivers/block/loop.ko.xz = updates/* ]]
/usr/lib/dracut/modules.d/50drm/module-setup.sh: line 29: warning: command substitution: ignored null byte in input
/usr/lib/dracut/modules.d/50drm/module-setup.sh: line 29: warning: command substitution: ignored null byte in input
/usr/lib/dracut/modules.d/50drm/module-setup.sh: line 29: warning: command substitution: ignored null byte in input
/usr/lib/dracut/modules.d/50drm/module-setup.sh: line 29: warning: command substitution: ignored null byte in input
//usr/lib/dracut/dracut-init.sh@1050(find_kernel_modules_by_path): continue
/usr/lib/dracut/modules.d/50drm/module-setup.sh: line 29: warning: command substitution: ignored null byte in input

[tpgxyz]

talk on #bash maybe this will help

[19:57] Is it this? $(xz -dc <$_f)
[19:57] <_TPG> greycat: yes
[19:58] Well, you are decompressing a *.ko.xz file (and *.ko implies it's a BINARY COMPILED OBJECT FILE) and trying to load it into a bash string.
[19:59] Bash cannot handle binary data in that way. It will either lose all the NULs, or simply truncate at the first one.
[19:59] <_TPG> greycat: well that was working fine before bash 4.4
[19:59] _TPG: you mean "I was not made aware of the HORRIBLE GAPING HOLES until..."
[20:01] If you are SURE you want to proceed with this approach at all (recognizing now that you are doing something tremendously silly) and you want to suppress the warnings, you can either redirect whatever FD bash is writing the warnings to, or you can strip the NUL bytes in the $() yourself.
[20:03] by the way, should be <"$_f"
[20:03] <_TPG> greycat: thanks for help

[tpgxyz]

Ping @haraldh can you please merge it ?

[alesak]

Can somebody do something with this issue please?

[tpgxyz]

@alesak What do you want to do about this issue ?

[haraldh]

in master the kernel module handling is now done with libkmod.

[danimo]

Updated proposed workaround for 044 at danimo@3c823a4

[haraldh]

can you test master HEAD please, if this is still an issue?

[haraldh]

I changed the whole insmod/firmware code. So again... is there still an issue with bash 4.4 and master HEAD?

[bobwya]

So I finally got fed-up waiting for a fix to reach the main tree...
Why am I having to apply tpgxyz 's working PR manually as a user patch to dracut?
This issue has been going on for months...

[danimo]

@bobwya Because these patches are not applicable to the main tree, which handles the functions in question in pure C. If you can positively confirm that there are still issues, please respond to @haraldh 's question.

[bobwya]

@danimo @haraldh
So I've test a live, vanilla build of dracut - using commit cc81672 ...
With an install of Gentoo, kernel 4.8.8, on my laptop. The resulting initramfs boots fine and there are no errors, console spam during the build process (as with 044).
So looks good to me, thanks! Would be nice to have a release of the master (as I don't really want to use a live build of dracut permanently!)

[Ratler]

There seems to be a similar bash 4.4 issue in dracut.sh using the latest commit from master when strip of kernel modules is enabled.

The problem is in the function that strips the modules unless they are signed (https://github.com/dracutdevs/dracut/blob/master/dracut.sh#L1596), see below:

/usr/bin/dracut@1596(main): SIG=$'\001'
/usr/bin/dracut@1597(main): [[  == \~\M\o\d\u\l\e\ \s\i\g\n\a\t\u\r\e\ \a\p\p\e\n\d\e\d\~ ]]
/usr/bin/dracut@1597(main): printf '%s\000' /var/tmp/dracut.4YEmfq/initramfs/lib/modules/4.8.11/kernel/fs/fuse/fuse.ko
/usr/bin/dracut@1595(main): read -r -d '' f
//usr/bin/dracut@1596(main): tail -c 28 /var/tmp/dracut.4YEmfq/initramfs/lib/modules/4.8.11/kernel/fs/fuse/cuse.ko
/usr/bin/dracut: line 1596: warning: command substitution: ignored null byte in input
... 

[haraldh]

Hmm.. our kernel modules have '0a' as the last byte.

000371c0  00 00 00 00 00 02 d3 7e  4d 6f 64 75 6c 65 20 73  |.......~Module s|
000371d0  69 67 6e 61 74 75 72 65  20 61 70 70 65 6e 64 65  |ignature appende|
000371e0  64 7e 0a                                          |d~.|

guess I will have to:
SIG=$(tail -c 28 "$f" | head -c 27)

any better solution?

[tpgxyz]

Closing it as it works :)