gozora / ebiso Goto Github PK
View Code? Open in Web Editor NEWUEFI bootable ISO image creator
License: GNU General Public License v3.0
UEFI bootable ISO image creator
License: GNU General Public License v3.0
Hello @gozora
I was recently informed via a SUSE customer issue report
that ebiso silently corrupts files greater or equal 2GiB.
As far as I understand it the "silently" is the actual issue in ebiso because
the actual root cause is a limit of the ISO 9660 file system according to
https://en.wikipedia.org/wiki/ISO_9660#The_2/4_GiB_file_size_limit
How it can happen that ebiso has to deal with files greater or equal 2GiB:
In ReaR it is possible to include the backup in the ISO
with OUTPUT=ISO
and BACKUP_URL=iso://
, cf.
https://github.com/rear/rear/blob/master/doc/rear.8.adoc
When the backup archive file (e.g. backup.tar.gz) is greater or equal 2GiB
ebiso has to deal with files greater or equal 2GiB.
Because of the 2GiB file size limit in the ISO 9660 file system
the backup archive file gets corrupted in the ISO.
The issue is that ebiso does not detect that problem
so for the user it appears af if everything went OK during "rear mkbackup"
until he finds out later during "rear recover" that his backup got corrupted.
So it would be good if ebiso could detect files greater or equal 2GiB
and show an error message in such cases and finish with non-zero exit code.
If ebiso could not detect (with reasonable effort) files greater or equal 2GiB
we could perhaps do something in ReaR to detect such a problem?
Perhaps in general it would be best to have something in ReaR
when making the ISO (regardless what ISO-making tool is used)
that verifies there is no file greater or equal 2GiB?
Also the ReaR recovery system initrd could become greater or equal 2GiB
(e.g. because of accidentally too much in COPY_AS_IS).
Reproducer script ebiso-test.sh
:
#!/bin/bash
#set -x
mkdir -p ebiso-test
cd ebiso-test
echo "2 GiB = 2 * 1024 * 1024 * 1024 = 2147483648"
mkdir -p below2g exact2g above2g
test -s below2g/testfile || dd if=/dev/urandom of=below2g/testfile bs=1M count=2147483647 iflag=count_bytes status=progress
test -s exact2g/testfile || { cp below2g/testfile exact2g/testfile ; echo -n 'x' >>exact2g/testfile ; }
test -s above2g/testfile || { cp below2g/testfile above2g/testfile ; echo -n 'yy' >>above2g/testfile ; }
for subdir in below2g exact2g above2g ; do
pushd $subdir
echo "in $subdir"
ls -l testfile
md5sum testfile
echo 'dummy' >dummy.efi
ebiso -e dummy.efi -o test.iso .
mkdir -p /tmp/ebiso-test
mount test.iso /tmp/ebiso-test
ls -l /tmp/ebiso-test/testfile
md5sum /tmp/ebiso-test/testfile
umount /tmp/ebiso-test
popd
done
Result of the reproducer script:
# ./ebiso-test.sh
2 GiB = 2 * 1024 * 1024 * 1024 = 2147483648
2096103424 bytes (2.1 GB, 2.0 GiB) copied, 37 s, 56.6 MB/s
2047+1 records in
2047+1 records out
2147483647 bytes (2.1 GB, 2.0 GiB) copied, 37.8585 s, 56.7 MB/s
~/ebiso-test/below2g ~/ebiso-test
in below2g
-rw-r--r-- 1 root root 2147483647 Nov 20 09:53 testfile
e964b39bce5e7a3aafb2656b1e357f69 testfile
mount: /tmp/ebiso-test: WARNING: device write-protected, mounted read-only.
-r-xr-xr-x 1 root root 2147483647 Nov 20 09:53 /tmp/ebiso-test/testfile
e964b39bce5e7a3aafb2656b1e357f69 /tmp/ebiso-test/testfile
~/ebiso-test
~/ebiso-test/exact2g ~/ebiso-test
in exact2g
-rw-r--r-- 1 root root 2147483648 Nov 20 09:54 testfile
9ae5c212a37bd45cf5a6b0a8b0c0d246 testfile
mount: /tmp/ebiso-test: WARNING: device write-protected, mounted read-only.
-r-xr-xr-x 1 root root 2147483647 Nov 20 09:54 /tmp/ebiso-test/testfile
e964b39bce5e7a3aafb2656b1e357f69 /tmp/ebiso-test/testfile
~/ebiso-test
~/ebiso-test/above2g ~/ebiso-test
in above2g
-rw-r--r-- 1 root root 2147483649 Nov 20 09:54 testfile
fde433ea7108fbe31fe68fb39bba13f1 testfile
mount: /tmp/ebiso-test: WARNING: device write-protected, mounted read-only.
-r-xr-xr-x 1 root root 2147483648 Nov 20 09:54 /tmp/ebiso-test/testfile
789f0fb931c30af13be3f68bedf0ebba /tmp/ebiso-test/testfile
~/ebiso-test
So in my case it seems to work up to a file size of 2 GiB - 1 (2147483647 bytes).
But problems start with a file size of 2 GiB (2147483648 bytes) because
then the file in the ISO is listed with only 2147483647 bytes and different md5sum
and same for a a file size of 2 GiB + 1 (2147483649 bytes) where
the file in the ISO is listed with only 2147483648 bytes and different md5sum.
This issue was created for reference to ReaR issue rear/rear#1208.
When building ebiso for openSUSE, following issues are observed by automatic checks of build log:
I: Statement is overflowing a buffer
E: ebiso bufferoverflow lib/filename.c:44:13
E: ebiso bufferoverflow lib/iso9660.c:594:16
See also openSUSE bug 974360
ReaR/GRUB had trouble to automatically boot with 8.3 file format, so it was disabled until RRIP will be implemented (v0.1.1).
ISO is now not fully ISO9660 compatible, but shout work anyhow...
please try to create rpm builds (you can use OpenSuse Build Service for this). Would be easier to download binary rpms to use immediately on SLES architectures.
Thanks for EFI Boot ISO maker.... glad we have something working now on SLES/UEFI systems.
Is there any chance your could create a package for CentOS 7?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.