Misc. build issues about mold

[petr-tik]

I have a workaround to use nixpkgs.tbb and comment out the make -C oneTBB line in the submodules target.

Start the shell like this

nix-shell -p cmake clang_10 zlib gcc10Stdenv tbb

Apply this change to the makefile, since tbb is provided by nix

--- a/Makefile
+++ b/Makefile
@@ -27,7 +27,7 @@ mold-wrapper.so: mold-wrapper.c Makefile
 $(OBJS): mold.h elf.h Makefile

 submodules:
-       $(MAKE) -C oneTBB
+#      $(MAKE) -C oneTBB

Now make submodules works fine

[nix-shell:~/Coding/mold]$ make submodules
mkdir -p mimalloc/out/release
(cd mimalloc/out/release; cmake ../..)
-- No build type selected, default to: Release
-- Override standard malloc (MI_OVERRIDE=ON)
--
-- Library base name: mimalloc
-- Build type       : release
-- Compiler         : /nix/store/azayfhqyg9mjyv6rv9bcypi6ws8aqfmy-gcc-wrapper-9.3.0/bin/gcc
-- Install directory: /var/empty/local/lib/mimalloc-1.6
-- Build targets    : shared;static;object;tests
--
-- Configuring done
-- Generating done
-- Build files have been written to: /home/petr_tik/Coding/mold/mimalloc/out/release
make -C mimalloc/out/release
make[1]: Entering directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
make[2]: Entering directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
make[3]: Entering directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
make[3]: Leaving directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
[ 42%] Built target mimalloc-static
make[3]: Entering directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
make[3]: Leaving directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
[ 48%] Built target mimalloc-test-api
make[3]: Entering directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
make[3]: Leaving directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
[ 90%] Built target mimalloc
make[3]: Entering directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
make[3]: Leaving directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
[ 96%] Built target mimalloc-test-stress
make[3]: Entering directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
make[3]: Leaving directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
[100%] Built target mimalloc-obj
make[2]: Leaving directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
make[1]: Leaving directory '/home/petr_tik/Coding/mold/mimalloc/out/release'
make -C xxHash
make[1]: Entering directory '/home/petr_tik/Coding/mold/xxHash'
make[1]: Nothing to be done for 'default'.
make[1]: Leaving directory '/home/petr_tik/Coding/mold/xxHash'

but make fails to find span.

[nix-shell:~/Coding/mold]$ make
clang++  -g -IoneTBB/include -IxxHash -pthread -std=c++20 -Wno-deprecated-volatile -Wno-switch -O2 -DGIT_HASH=\"3e6fe6b928eb5cee91a7a37ca0e6e4d9f498eaa6\"  -c -o main.o main.cc
In file included from main.cc:1:
./mold.h:16:10: fatal error: 'span' file not found
#include <span>
         ^~~~~~
1 error generated.
make: *** [<builtin>: main.o] Error 1

despite the fact it's using clang++10

[nix-shell:~/Coding/mold]$ clang++ --version
clang version 10.0.1
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /nix/store/w8a9x07rai37srm9xyq827bspdvvb4xq-clang-10.0.1/bin

[petr-tik]

Thanks for the makefile improvement. Can confirm that I have successfully built mold in a docker container, here are my steps for others to use.

Start a Dockerfile like this

FROM ubuntu:20.04

ENV DEBIAN_FRONTEND noninteractive
RUN DEBIAN_FRONTEND=noninteractive apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt install -y build-essential libstdc++-10-dev clang-10 libssl-dev zlib1g-dev cmake bsdmainutils git
# the makefile requires clang++
RUN ln -s /usr/bin/clang++-10 /usr/bin/clang++

RUN git clone --recursive https://github.com/rui314/mold.git

# tests require clang
RUN ln -s /usr/bin/clang-10 /usr/bin/clang

WORKDIR /mold

put this in a Dockerfile and run docker build . in the same dir (can be a temp dir).

get the image hash and start a container instance with it docker run -it IMAGE_HASH, which will be printed at the end of the docker build output

now run this in your docker shell

root@a456c389d366:/mold# make submodules
root@a456c389d366:/mold# make STATIC=1

After you have built mold, open another shell on your host (keep the original container running) and cp files out of it with docker cp but replace a456c389d366 with the hash of your running container (not image ID)

docker cp a456c389d366:/mold/mold ~/.local/bin/
docker cp a456c389d366:/mold/mold-wrapper.so ~/.local/bin/

verify you now have mold on your host by running

$ which mold
/home/petr_tik/.local/bin/mold

Linking rust is still not working

still am struggling to point cargo/rustc to use it even after following your advice on cargo.toml from here
#26 (comment)

[rui314]

@petr-tik

Recently I made another change to make it easy to use mold instead of /usr/bin/ld. Please see https://github.com/rui314/mold/blob/main/README.md#how-to-use. Essentially, if you invoke mold -run cargo, mold intercepts all invocations of /usr/bin/ld to redirect it to mold. So you want to make sure that cargo uses GNU ld instead of gold or lld if you take this route.

[deliciouslytyped]

The solution to the first issue is sed -i "s|[email protected]:|https://github.com/|" .gitmodules.
The google doesn't yield much info about the lifetime-dse issue other than maybe being a clang 9 thing but I didn't look harder.

[deliciouslytyped]

Ran into this somewhere:

clang++  -g -IoneTBB/include -IxxHash -pthread -std=c++20 -Wno-deprecated-volatile -Wno-switch -O2  -c -o main.o main.cc
warning: unknown warning option '-Wno-deprecated-volatile' [-Wunknown-warning-option]
error: invalid value 'c++20' in '-std=c++20'
note: use 'c++98' or 'c++03' for 'ISO C++ 1998 with amendments' standard
note: use 'gnu++98' or 'gnu++03' for 'ISO C++ 1998 with amendments and GNU extensions' standard
note: use 'c++11' for 'ISO C++ 2011 with amendments' standard
note: use 'gnu++11' for 'ISO C++ 2011 with amendments and GNU extensions' standard
note: use 'c++14' for 'ISO C++ 2014 with amendments' standard
note: use 'gnu++14' for 'ISO C++ 2014 with amendments and GNU extensions' standard
note: use 'c++17' for 'ISO C++ 2017 with amendments' standard
note: use 'gnu++17' for 'ISO C++ 2017 with amendments and GNU extensions' standard
note: use 'c++2a' for 'Working draft for ISO C++ 2020' standard
note: use 'gnu++2a' for 'Working draft for ISO C++ 2020 with GNU extensions' standard

https://clang.llvm.org/cxx_status.html#cxx20 ??

[rui314]

Thank you for your report. I fixed the gitsubmodule issue by changing the URL.

I haven't seen the oneTBB issue before. Maybe it cannot be compiled with clang? When I run make submodules, oneTBB uses g++ as a compiler.

As to the last issue, please make sure that you are using clang 10. clang++ might be a different version of clang, and it's possible that clang 10 is installed as clang-10 and clang++-10.

[deliciouslytyped]

Ok I'll try.
Isn't it unsafe to link C++ libraries from different compilers due to possible ABI incompatibilities?

[Alcaro]

GCC and Clang try to be ABI-compatible with each other.

But their C++ stdlib implementations do (to my knowledge) not, so better make sure everything is built with the same one (libc++ or libstdc++).

[InfRandomness]

I run into the same issues with oneTBB on Fedora
using :
clang version 11.0.0 (Fedora 11.0.0-2.fc33)
(clang and clang++ print the same information)

[rui314]

Does your clang++ print out this error message?

error: invalid value 'c++20' in '-std=c++20'

My clang is version 10, so yours is newer.

[InfRandomness]

It does not look like I am getting that message from clang++,

I am getting this though :

clang-11: error: unknown argument: '-flifetime-dse=1'clang-11:
clang-11clang-11clang-11: error: : : error: errorunknown argument: '-flifetime-dse=1': unknown argument: '-flifetime-dse=1'unknown argument: '-flifetime-dse=1'error:
unknown argument: '-flifetime-dse=1'

clang-11: error: unknown argument: '-flifetime-dse=1'

make[1]: *** [../../build/common_rules.inc:80: dynamic_link.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [../../build/common_rules.inc:80: itt_notify.o] Error 1
make[1]: *** [../../build/common_rules.inc:80: concurrent_queue.o] Error 1
make[1]: *** [../../build/common_rules.inc:80: concurrent_vector.o] Error 1
make[1]: *** [../../build/common_rules.inc:80: concurrent_hash_map.o] Error 1
clang-11make[1]: *** [../../build/common_rules.inc:80: cache_aligned_allocator.o] Error 1
clang-11: : error: error: unknown argument: '-flifetime-dse=1'
unknown argument: '-flifetime-dse=1'
make[1]: *** [../../build/common_rules.inc:80: queuing_mutex.o] Error 1
make[1]: *** [../../build/common_rules.inc:80: pipeline.o] Error 1
make[1]: Leaving directory '/home/infrandomness/src/mold/oneTBB/build/linux_intel64_gcc_cc10.2.1_libc2.32_kernel5.11.8_release'
make: *** [Makefile:29: tbb] Error 2

[petr-tik]

getting the same error with a nix-shell started like this

nix-shell -p cmake clang_10 zlib gcc10Stdenv

then start with building submodules

[nix-shell:~/Coding/mold]$ make submodules
make -C oneTBB
make[1]: Entering directory '/home/petr_tik/Coding/mold/oneTBB'
Created the ./build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release directory
make -C "./build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release"  -r -f ../../build/Makefile.tbb cfg=release
make[2]: Entering directory '/home/petr_tik/Coding/mold/oneTBB/build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release'
../../build/Makefile.tbb:28: CONFIG: cfg=release arch=intel64 compiler=gcc target=linux runtime=cc9.3.0_libc2.31_kernel5.4.0
clang++ -o concurrent_hash_map.o -c -MMD -O2 -g -DDO_ITT_NOTIFY -DUSE_PTHREAD -pthread -m64 -mrtm  -fPIC -flifetime-dse=1 -D__TBB_BUILD=1 -Wall -Wextra -Wno-parentheses -Wno-sized-deallocation  -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1   -I../../src -I../../src/rml/include -I../../include ../../src/tbb/concurrent_hash_map.cpp
clang-10: error: unknown argument: '-flifetime-dse=1'
make[2]: *** [../../build/common_rules.inc:80: concurrent_hash_map.o] Error 1
make[2]: Leaving directory '/home/petr_tik/Coding/mold/oneTBB/build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release'
make[1]: *** [Makefile:29: tbb] Error 2
make[1]: Leaving directory '/home/petr_tik/Coding/mold/oneTBB'
make: *** [Makefile:30: submodules] Error 2

[rui314]

It looks like clang++ is used to build oneTBB, while in my environment gcc is automatically selected. Did you guys install gcc and g++?

[InfRandomness]

It looks like clang++ is used to build oneTBB, while in my environment gcc is automatically selected. Did you guys install gcc and g++?

Yes, but my CC, CXX, LD are respectively set to clang, clang++ and mold

[rui314]

@petr-tik

If you were using Ubuntu, I believe you can fix it by installing libstdc++-10-dev using apt-get. Don't know how to do that in your env, though.

[petr-tik]

It looks like clang++ is used to build oneTBB, while in my environment gcc is automatically selected. Did you guys install gcc and g++?

I am trying to build in nix-shell independently (but inspired yb @InfRandomness) and i am getting a different eror (with a temporary workaround not to build oneTBB from source).

[petr-tik]

@petr-tik

If you were using Ubuntu, I believe you can fix it by installing libstdc++-10-dev using apt-get. Don't know how to do that in your env, though.

I was actually trying to define a shell.nix file that can help me and others reproducibly set up their dependencies and build environment. I am running nix-shell on ubuntu, but my ubuntu is old and my apt is pretty fubared, so am trapped atm.

Thanks for the help

[InfRandomness]

I have checked and I have libstdc++-devel (the equivalent of debian and ubuntu's libstdc++-10-dev) but the error still happens

[rui314]

Let's break it down.

• Can you build oneTBB not as a submodule but as an independent project?
• Can you build a C++ source file containing just #include <span> with clang++?

[InfRandomness]

Let's break it down.

* Can you build oneTBB not as a submodule but as an independent project?

No, the same error happens when I try this

* Can you build a C++ source file containing just `#include <span>` with clang++?

Yes, I can

[rui314]

@InfRandomness
OK, that's the problem you want to fix. Maybe you can just try using g++ instead of clang++ to build oneTBB?

[InfRandomness]

Yes, it is what I did after posting my first comment in this issue

[petr-tik]

Let's break it down.

* Can you build oneTBB not as a submodule but as an independent project?

I cd'ed into thte oneTBB directory in the mold dir and ran make successfully. I don't see how it would be different if I cloned the oneTBB repo to a non-submodule directory?

make clean
clean done

[nix-shell:~/Coding/mold/oneTBB]$ make
Created the ./build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release directory
make -C "./build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release"  -r -f ../../build/Makefile.tbb cfg=release
...
...
...
g++ -fPIC -o libtbbmalloc_proxy.so.2 proxy.o tbb_function_replacement.o  -ldl -lrt libtbbmalloc.so -shared -Wl,-soname=libtbbmalloc_proxy.so.2 -pthread -m64  -Wl,--version-script,tbbmallocproxy.def
make[1]: Leaving directory '/home/petr_tik/Coding/mold/oneTBB/build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release'

* Can you build a C++ source file containing just `#include <span>` with clang++?

No. Get a long lsit of errors - won't post them there, but will try to get that working.

[InfRandomness]

Let's break it down.

* Can you build oneTBB not as a submodule but as an independent project?

I cd'ed into thte oneTBB directory in the mold dir and ran make successfully. I don't see how it would be different if I cloned the oneTBB repo to a non-submodule directory?

make clean
clean done

[nix-shell:~/Coding/mold/oneTBB]$ make
Created the ./build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release directory
make -C "./build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release"  -r -f ../../build/Makefile.tbb cfg=release
...
...
...
g++ -fPIC -o libtbbmalloc_proxy.so.2 proxy.o tbb_function_replacement.o  -ldl -lrt libtbbmalloc.so -shared -Wl,-soname=libtbbmalloc_proxy.so.2 -pthread -m64  -Wl,--version-script,tbbmallocproxy.def
make[1]: Leaving directory '/home/petr_tik/Coding/mold/oneTBB/build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release'

* Can you build a C++ source file containing just `#include <span>` with clang++?

No. Get a long lsit of errors - won't post them there, but will try to get that working.

I don't think it would, apart if a specific older version of oneTBB has been specified in the submodule (and there is a new commit that contains a fix for the problem we are encountering)

[petr-tik]

@InfRandomness any luck with that? Would be pretty good to define a shell.nix file to build mold

[InfRandomness]

I do not use nixOS but it would be beneficial, indeed

[petr-tik]

I do not use nixOS but it would be beneficial, indeed

I mistook you for @deliciouslytyped, who originally opened the issue with the nix-shell build error. I only use nix on ubuntu and think it's a great way to prepare a dev environment and package the final artefact for release, especially when it relies on specific versions of dependencies.

[rui314]

I made a few changes to Makefile so that you can now build a statically-linked mold executable by running make STATIC=1 (after re-running make submodules if you already have a repository). So, you can now build mold on Ubuntu 20 (on a real machine or a Docker environment) and copy its executable file to any Linux-based system. I'm sure this isn't what you guys are trying to fix, but I hope this helps.

[InfRandomness]

You could use docker volumes in order to act directly on your host's filesystem, this is what I would recommend instead of copying the files from the container to the host

[petr-tik]

@petr-tik

Recently I made another change to make it easy to use mold instead of /usr/bin/ld. Please see https://github.com/rui314/mold/blob/main/README.md#how-to-use. Essentially, if you invoke mold -run cargo, mold intercepts all invocations of /usr/bin/ld to redirect it to mold. So you want to make sure that cargo uses GNU ld instead of gold or lld if you take this route.

Thanks - that's exactly what I have been trying to do to no avail. I will take it to the original cargo/rust issue, if that's ok?

[rui314]

Sure, that's ok.

[rui314]

I made further improvements to Makefile to make it easy to build mold. I also added a shell script to automate building mold in a Docker environment. Please see https://github.com/rui314/mold#how-to-build.

Given that, I think we no longer have build issues, so closing this bug.

[InfRandomness]

After pulling the changes, it looks like the tbb/ folder is missing

[rui314]

Intel TBB is now a dependency, so you need to install Intel TBB to your system. This can be done by running sudo apt-get install libtbb-dev on Ubuntu 20.

If you can't install Intel TBB, please run ./build-static.sh script after installing Docker. This script builds mold as a statically-linked executable.

[InfRandomness]

oh, alright, thanks