error initializing file system: unsupported compression type: ZSTD about dwarfs

[zhuyifei1999]

I had a quick look at this (because reddit) and I couldn't figure out how the ebuild should specify these libraries to make libdwarfs_compression become static libraries (building everything static seems to be highly discouraged and I'm not sure how to toggle each library individually).

Then I took a look at the actual compilation commands on my side. This is that of dwarfsextract:

x86_64-pc-linux-gnu-g++ -march=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=12 -fuse-linker-plugin -falign-functions=32 -pipe -ggdb -frecord-gcc-switches -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -I/usr/include -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -march=native -O3 -fgraphite-identity -floop-nest-optimize -fdevirtualize-at-ltrans -fipa-pta -fno-semantic-interposition -flto=12 -fuse-linker-plugin -falign-functions=32 -pipe -ggdb -frecord-gcc-switches CMakeFiles/dwarfsextract.dir/src/dwarfsextract.cpp.o -o dwarfsextract  -Wl,-rpath,/var/tmp/portage/sys-fs/dwarfs-0.7.4/work/dwarfs-0.7.4:/var/tmp/portage/sys-fs/dwarfs-0.7.4/work/dwarfs-0.7.4/folly:  libdwarfsextract_main.so  -Wl,--push-state,--whole-archive  libdwarfs_compression.so  -Wl,--pop-state  libdwarfs_tool.so  libdwarfs.so  libmetadata_thrift.so  libthrift_light.so  libfsst.so  /usr/lib64/libboost_chrono.so.1.83.0  /usr/lib64/libboost_iostreams.so.1.83.0  /usr/lib64/libbrotlidec.so  /usr/lib64/libbrotlienc.so  /usr/lib64/libarchive.so  /usr/lib64/libzstd.so  /usr/lib64/libxxhash.so  /usr/lib64/libjemalloc.so  folly/libfolly.so.0.58.0-dev  /usr/lib64/libboost_program_options.so.1.83.0  /usr/lib64/libfmt.so.9.1.0  /usr/lib64/libboost_context.so.1.83.0  /usr/lib64/libboost_filesystem.so.1.83.0  /usr/lib64/libboost_atomic.so.1.83.0  /usr/lib64/libboost_regex.so.1.83.0  /usr/lib64/libboost_system.so.1.83.0  /usr/lib64/libboost_thread.so.1.83.0  /usr/lib64/libdouble-conversion.so  /usr/lib64/libgflags.so.2.2.2  /usr/lib64/libglog.so  /usr/lib64/libevent.so  /usr/lib64/libz.so  /usr/lib64/libssl.so  /usr/lib64/libcrypto.so  /usr/lib64/libbz2.so  /usr/lib64/liblz4.so  /usr/lib64/liblzma.so  /usr/lib64/libsnappy.so  /usr/lib64/libiberty.a  /usr/lib64/libaio.so  /usr/lib64/liburing.so  /usr/lib64/libsodium.so  -ldl  /usr/lib64/libunwind.so

Despite having -Wl,--push-state,--whole-archive libdwarfs_compression.so -Wl,--pop-state, libdwarfs_compression.so does not appear in ldd dwarfsextract.

This is a case of -Wl,--as-needed doing its thing. -Wl,--as-needed is part of my system CFLAGS, so it deletes the DT_NEEDED entry for libdwarfs_compression.so since it thinks it's not necessary. Removing that flag, or changing -Wl,--push-state,--whole-archive to -Wl,--push-state,--whole-archive,--no-as-needed restores the SO in the ldd output.

[mhx]

Thanks for the additional information, @zhuyifei1999!

Indeed, --as-needed is a culprit here. The following patch fixes the ebuild for me:

--- dwarfs-0.7.2-r1.ebuild.orig	2024-01-05 01:19:54.486584449 +0100
+++ dwarfs-0.7.2-r1.ebuild	2024-01-05 01:13:57.399919972 +0100
@@ -80,6 +80,7 @@
 
 src_configure(){
 	append-cxxflags "-I/usr/include"
+	append-ldflags $(no-as-needed)
 
 	mycmakeargs=(
 		-DUSE_JEMALLOC=$(usex jemalloc ON OFF)
@@ -111,6 +112,7 @@
 	cmake_src_install
 	dolib.so libdwarfs.so libdwarfs_main.so libdwarfs_tool.so libdwarfs_compression.so libthrift_light.so libmetadata_thrift.so || die "Install failed"
 	dolib.so folly/libfolly.so folly/libfolly.so.0.58.0-dev libmkdwarfs_main.so libdwarfsbench_main.so libdwarfsck_main.so libdwarfsextract_main.so || die "Install failed"
+	dolib.so folly/folly/experimental/exception_tracer/libfolly_exception_tracer_base.so folly/folly/experimental/exception_tracer/libfolly_exception_tracer_base.so.0.58.0-dev || die "Install failed"
 }
 
 pkg_postinst(){

Pinging @RarogCmex, as it seems they're the author of the ebuild.

[RarogCmex]

@mhx
That's the weirdest, that I have pure 64bit system and ebuild works! And I have thought shared libs were intended. Furthermore there is no need in experimental system.
!!! dolib: folly/folly/experimental/exception_tracer/libfolly_exception_tracer_base.so does not exist
So as @zhuyifei1999 mentioned --as-needed brokes everything.

I will update ebuild with append-ldflags $(no-as-needed)
To be honest, I have no more performed active gentoo maintainership since I became undergraduate student of Business Informatics in financial university under the government of the Russian Federation. That long text states that I am a bit ovewhelmed, so I may stop maintaining ebuild completely.

[RarogCmex]

@zhuyifei1999 Also there are sys-fs/dwarfs-bin in the GURU tree, which installs prebuilt version and therefore it is a lot more reliable. I will update it too.

[zhuyifei1999]

To be honest, I have no more performed active gentoo maintainership

@RarogCmex if you don't mind I can co-maintain these packages in GURU. I maintain a few packages of my own there (I am also a GURU contributer)

[RarogCmex]

@zhuyifei1999 I would be happy. Just add your maintainership in metadata.xml.

[Turkceqwertyx]

i am closing this issue because it's fixed. thank you guys!

[mhx]

Hi, thanks for your report.

It unfortunately looks like nobody has really tested the dwarfs ebuild properly and it is badly broken. Looking at the ebuild itself, it seems to be building the internal libraries, in particular libdwarfs_compression, as shared libraries, which is not supported. This causes the individual tools, including dwarfsextract, to not find any compression implementations.

Please file a bug report for the dwarfs ebuild.

In the meantime, you can download a binary release or a universal binary.