rui314 / mold Goto Github PK

View Code? Open in Web Editor NEW

13.6K 13.6K 449.0 22.1 MB

Mold: A Modern Linker 🦠

License: MIT License

C++ 79.19% Shell 18.88% C 0.29% CMake 1.63%

mold's People

Contributors

Stargazers

Watchers

Forkers

crackercat doytsujin 16yuki0702 hexhexd yr-wang-hit mewbak terminus hotelzululima raphaelzulliger zeta1999 linecode taroyuyu deadalnix laeeth saeta dakerfp crazypython seeekr beyonddream-productions axiomabsolute seemethere matzke1 yetanotheropensource mbrukman wildmichael sarvex tesuji kyulee-com justanotherdot imaginary-person mu-l googol-lab zabrane andreybokhanko muryliang knweiss vittorioromeo zamazan4ik dendisuhubdy ezhangle michaeljclark bassosimone blockspacer gjimye expenses 3a2l sirspudd mfkiwl romanz nehaljwani yota9 johejo logarithmus piegamesde indianboy42 ali-robot alehacksp syrusakbary mqudsi fuyuton cruisercoder wormon luxufan rbartlensky calinou icodein twon zjukk unshorn keruspe sigmasd lacc97 corelibcore haunt98 passw br337 mrcodechef miguelraz cyberflamego killvxk watch-later amutu frederik-baetens bugorz maekawatoshiki flowervine sylvestre dankamongmen sualehasif sunxfancy motis-project iiicp tonalidadehidrica phial3 hj3938 litaotju nexa loyal1213 exonorid eikeschumann971

mold's Issues

Error when running mold on Manjaro: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found

Hello!

I'm trying to use mold on Manjaro and receiving these errors:

$ cd ~/projects
$ git clone --recursive https://github.com/rui314/mold.git
$ cd mold
$ make submodules
$ make

$ build-webkit --gtk --debug --cmakeargs="-DUSE_LD_LLD=OFF -DUSE_LD_GOLD=OFF"
Building flatpak based environment
+  /home/twilco/projects/mold/mold -run cmake --build /app/webkit/WebKitBuild/Debug --config Debug -- 
/home/twilco/projects/mold/mold: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /home/twilco/projects/mold/mold)
/home/twilco/projects/mold/mold: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /home/twilco/projects/mold/oneTBB/build/linux_intel64_gcc_cc10.2.0_libc2.33_kernel5.4.105_release/libtbb.so.2)

It looks like mold expects these .sos to be in the directories Ubuntu / apt-get puts them in (e.g. see the file listing for libc6-dev, depended on by Ubuntu's build-essential package). Neither /usr/lib/x86_64-linux-gnu nor /lib/x86_64-linux-gnu exists on my system, so I'm guessing Arch-based repos don't follow this pattern.

I do have glibc installed, just not in the place mold expects:

$ /lib/libc.so.6
GNU C Library (GNU libc) release release version 2.33.
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 10.2.0.
libc ABIs: UNIQUE IFUNC ABSOLUTE
For bug reporting instructions, please see:
<https://bugs.archlinux.org/>.

I tried a symlink:

$ sudo mkdir /usr/lib/x86_64-linux-gnu
$ sudo ln -s /lib/libc.so.6 /usr/lib/x86_64-linux-gnu/libc.so.6

But get the exact same errors, so there must be more to it.

Have you tested mold on Arch-based distros yet?

Thanks for making this project -- excited to continue experimenting with it!

gcc-11.1.0 - template-id not allowed for destructor

Wanting to try out mold but am using GCC-11.
This seemed relevant.
https://stackoverflow.com/questions/63513984/can-class-template-constructors-have-a-redundant-template-parameter-list-in-c2?rq=1
Change in standard the GCC honors but clang does not?

g++ -g -IoneTBB/include -IxxHash -pthread -std=c++20 -Wno-deprecated-volatile -Wno-switch -DGIT_HASH="4d3cc43d9f3c73377ec690745f8d1b4bd48cdb28" -O2 -c -o object_file.o object_file.cc
object_file.cc:66:22: error: template-id not allowed for destructor
66 | MemoryMappedFile::~MemoryMappedFile() {
| ^
(Arrow is supposed to be pointed at the ~)

Misc. build issues

Failure to clone xxHash repo

$ nix-shell -p cmake git clang_10 xclip --run 'pth="$(mktemp -d)"; echo "$pth" | xclip -i -selection clipboard -f; pushd "$pth"; echo Working in: "$pth"; git clone --recursive https://github.com/[67/521]
old.git; cd mold; make submodules; make'                                                                                
/run/user/1000/tmp.HGvrS5rbhx                       
/run/user/1000/tmp.HGvrS5rbhx /tmp/tmp.tMWaLqN3Se                                                                                                                                                                                             
Working in: /run/user/1000/tmp.HGvrS5rbhx
Cloning into 'mold'...                                                                                                                                                                                                                        
remote: Enumerating objects: 591, done.
remote: Counting objects: 100% (591/591), done.
remote: Compressing objects: 100% (333/333), done.                                                                                                                                                                                            
remote: Total 7737 (delta 417), reused 423 (delta 258), pack-reused 7146
Receiving objects: 100% (7737/7737), 1.32 MiB | 6.54 MiB/s, done.                                                                                                                                                                             
Resolving deltas: 100% (5655/5655), done.                                                                              
Submodule 'mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'mimalloc'
Submodule 'oneTBB' (https://github.com/oneapi-src/oneTBB.git) registered for path 'oneTBB'
Submodule 'xxHash' ([email protected]:Cyan4973/xxHash.git) registered for path 'xxHash'                                   
Cloning into '/run/user/1000/tmp.HGvrS5rbhx/mold/mimalloc'...
remote: Enumerating objects: 69, done.                                                                                                                                                                                                        
remote: Counting objects: 100% (69/69), done.        
remote: Compressing objects: 100% (50/50), done.                                                                       
remote: Total 8395 (delta 41), reused 37 (delta 19), pack-reused 8326        
Receiving objects: 100% (8395/8395), 3.07 MiB | 6.59 MiB/s, done.
Resolving deltas: 100% (6460/6460), done.      
Cloning into '/run/user/1000/tmp.HGvrS5rbhx/mold/oneTBB'... 
remote: Enumerating objects: 95, done.                                                                                 
remote: Counting objects: 100% (95/95), done.                                                                          
remote: Compressing objects: 100% (89/89), done.        
remote: Total 18772 (delta 9), reused 28 (delta 3), pack-reused 18677        
Receiving objects: 100% (18772/18772), 42.18 MiB | 5.93 MiB/s, done.
Resolving deltas: 100% (13966/13966), done.                                                                                                                                                                                                   
Cloning into '/run/user/1000/tmp.HGvrS5rbhx/mold/xxHash'...                                                                                                                                                                                   
[email protected]: Permission denied (publickey).                                                                                                                                                                                                
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:Cyan4973/xxHash.git' into submodule path '/run/user/1000/tmp.HGvrS5rbhx/mold/xxHash' failed
Failed to clone 'xxHash'. Retry scheduled
Cloning into '/run/user/1000/tmp.HGvrS5rbhx/mold/xxHash'... 
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:Cyan4973/xxHash.git' into submodule path '/run/user/1000/tmp.HGvrS5rbhx/mold/xxHash' failed
Failed to clone 'xxHash' a second time, aborting
make -C oneTBB
make[1]: Entering directory '/run/user/1000/tmp.HGvrS5rbhx/mold/oneTBB'
make[1]: *** No targets specified and no makefile found.  Stop.
make[1]: Leaving directory '/run/user/1000/tmp.HGvrS5rbhx/mold/oneTBB'
make: *** [Makefile:24: submodules] Error 2
clang++  -g -IoneTBB/include -IxxHash -pthread -std=c++20 -Wno-deprecated-volatile -Wno-switch -O2  -c -o main.o main.cc
In file included from main.cc:1:
./mold.h:8:10: fatal error: 'xxHash/xxhash.h' file not found
#include "xxHash/xxhash.h"
         ^~~~~~~~~~~~~~~~~
1 error generated.
make: *** [<builtin>: main.o] Error 1

(For what it's worth, I think subtrees are better than submodules)

Failure to build oneTBB

$ nix-shell -p cmake git clang_10 xclip --run 'pth="$(mktemp -d)"; echo "$pth" | xclip -i -selection clipboard -f; pushd "$pth"; echo Working in: "$pth"; git clone https://github.com/rui314/mold.git; cd mold; sed -i "s|[email protected]:|https://github.com/|" .gitmodules; git submodule update --init; make submodules; make'
/run/user/1000/tmp.tnOI8qWbwK
/run/user/1000/tmp.tnOI8qWbwK /tmp/tmp.tMWaLqN3Se
Working in: /run/user/1000/tmp.tnOI8qWbwK
Cloning into 'mold'...
remote: Enumerating objects: 591, done.
remote: Counting objects: 100% (591/591), done.
remote: Compressing objects: 100% (333/333), done.
remote: Total 7737 (delta 417), reused 423 (delta 258), pack-reused 7146
Receiving objects: 100% (7737/7737), 1.32 MiB | 3.55 MiB/s, done.
Resolving deltas: 100% (5655/5655), done.
Submodule 'mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'mimalloc'
Submodule 'oneTBB' (https://github.com/oneapi-src/oneTBB.git) registered for path 'oneTBB'
Submodule 'xxHash' (https://github.com/Cyan4973/xxHash.git) registered for path 'xxHash'
Cloning into '/run/user/1000/tmp.tnOI8qWbwK/mold/mimalloc'...
Cloning into '/run/user/1000/tmp.tnOI8qWbwK/mold/oneTBB'...
Cloning into '/run/user/1000/tmp.tnOI8qWbwK/mold/xxHash'...
Submodule path 'mimalloc': checked out '4cc8bff90d9e081298ca2c1a94024c7ad4a9e478'
Submodule path 'oneTBB': checked out 'eca91f16d7490a8abfdee652dadf457ec820cc37'
Submodule path 'xxHash': checked out 'ea5393ddb997999aace2be4e101a4efa8e8f3cb3'
make -C oneTBB
make[1]: Entering directory '/run/user/1000/tmp.tnOI8qWbwK/mold/oneTBB'
Created the ./build/linux_intel64_gcc_cc10.2.0_libc2.32_kernel5.4.104_release directory
make -C "./build/linux_intel64_gcc_cc10.2.0_libc2.32_kernel5.4.104_release"  -r -f ../../build/Makefile.tbb cfg=release
make[2]: Entering directory '/run/user/1000/tmp.tnOI8qWbwK/mold/oneTBB/build/linux_intel64_gcc_cc10.2.0_libc2.32_kernel5.4.104_release'
../../build/Makefile.tbb:28: CONFIG: cfg=release arch=intel64 compiler=gcc target=linux runtime=cc10.2.0_libc2.32_kernel5.4.104
clang++ -o concurrent_hash_map.o -c -MMD -O2 -g -DDO_ITT_NOTIFY -DUSE_PTHREAD -pthread -m64 -mrtm  -fPIC -flifetime-dse=1 -D__TBB_BUILD=1 -Wall -Wextra -Wno-parentheses -Wno-sized-deallocation  -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1   -I../../src -I../../src/rml/include -I../../include ../../src/tbb/concurrent_hash_map.cpp
clang-10: error: unknown argument: '-flifetime-dse=1'

[...]

Some tests fail on my machine & on Void Linux CI

The failing tests (on Void Linux) (WIP):

copyrel-protected.sh
expected:
cannot make copy relocation for protected symbol
found:
nothing
copyrel-relro.sh
expected:
[21] .dynbss.rel.ro
found:
[20] .dynbss.rel.ro
Log: https://gist.github.com/Logarithmus/502ffb024f5cc8205894c2eb0d18e640#file-copyrel-relro-log
dt_init.sh
expected:
(INIT) 0x201010
found:
(INIT) 0x20100c
Log: https://gist.github.com/Logarithmus/502ffb024f5cc8205894c2eb0d18e640#file-dt_init-log
dynamic.sh
hello-dynamic.sh
mold: cannot open /usr/lib/x86_64-linux-gnu/crt1.o
i386-*.sh
fatal error: gnu/stubs-32.h: No such file or directory
this should be fixed by installing glibc-devel-32bit package

TODO:

ifunc-dso.sh
image-base.sh
incompatible-libs.sh
init-array.sh
library-path-static.sh
note-property.sh
run.sh
shared.sh

I'll create a PR later and try to gradually fix all the tests.

Support compressed sections

Hi, I tried this out on a larger project but it ends up in an endless loop and printing a lot of string is not null terminated errors.

mold/input_sections.cc

Lines 565 to 576 in 6630781

    
           while (!data.empty()) { 
        
             size_t end = find_null(data, entsize); 
        
             if (end == std::string_view::npos) 
        
               Error() << *this << ": string is not null terminated"; 
        
             std::string_view substr = data.substr(0, end + entsize); 
        
             data = data.substr(end + entsize); 
        
             SectionFragment *frag = parent.insert(substr, isec->shdr.sh_addralign); 
        
             fragments.push_back(frag); 
        
             frag_offsets.push_back(substr.data() - begin); 
        
           }

I modified that loop to just add the whole data to fragments and break, which makes it link successfully in 2–4 s. lld takes 4 s, both are really impressive!

It still fails to run afterwards (a new breaks somewhere in the llvm SmallVector), but that’s another issue.

Support for Aarch64 and more architectures?

Hi @rui314 , first great work on the linker! The numbers are impressive, and it's much more lightweight than other alternatives (namely LLVM's lld).

We are researching embedding a linker in Wasmer as part of our dynamic library engine.

I was wondering what is the timeline for supporting more architectures, namely Aarch64 (but also curious about other architectures)

Mold causes Rust to lose backtraces in certain cases.

Minimal repro:
https://github.com/itsfarseen/repro-sqlx-rocket-anyhow-no-backtrace

A notable thing is, if I comment out use sqlx; import, it works fine. If I don't import sqlx, I think Rust compiler is smart enough to skip compiling it.
I don't have to use anything inside sqlx, just importing it causes backtraces to be lost.

Cargo.toml

[target.x86_64-unknown-linux-gnu]
linker = "/usr/bin/clang"
# rustflags = ["-Clink-arg=-fuse-ld=lld"]
rustflags = ["-Clink-arg=-fuse-ld=/usr/local/bin/mold"]

Rust version: rustc 1.53.0-nightly (ca075d268 2021-04-28)

Optimize string merging

String merging seems to be the biggest bottleneck when linking a large binary with debug info enabled.

For example, when we create a clang executable with debug info, .debug_str sections from all input object files contain ~70M string fragments in total. After merging, the total number of string constants is ~2M. In split_section() in object_files.cc, we split section contents into smaller pieces and insert the pieces into a concurrent hash table. It looks like the hash insertion takes about a half of total execution time. Even with that cost, mold is still faster than lld on machine, but it's slower than 2x of lld.

I have no concrete idea as to how to improve it. But there are a few things we can do to experiment:

Use other concurrent hash table, e.g. concurrent cuckoo hashmap
Design a new data structure and implement it. Since our usage pattern is special, we don't really need a full-fledged hashmap implementation. We never delete elements, for example.

Possibility of a sane linker script language?

The actual reason I stumbled across mold was, somehow, searching for information about linker scripts.
(Specifically, I was searching in the context of there seemingly not existing anything for generating a "default" linker script, for, e.g., whatever the default configuration of a linker is Edit: https://stackoverflow.com/questions/28688484/actual-default-linker-script-and-settings-gcc-uses).

I currently have no real understanding of linker scripts, so sorry if this goes in the wrong direction.

I should read https://ftp.gnu.org/old-gnu/Manuals/ld-2.9.1/html_chapter/ld_3.html.

There seems to be very little material on dealing with linker scripts and I imagine part of it is that it's hard to deal with linkers and linker scripts? Maybe it doesn't need to be that way. Old *nix tools seem to have a pathology of hard to use domain specific languages? - such as xkeyboard config files which are also powerful but massively underdocumented?

I saw you mentioned in your readme that you don't intend to implement ld scripts, nevertheless there is a linker_script.cc?

In any case for my actual proposal; a new linker implementation with the potential to - pardon my language - disrupt the linker space, seems like a good opportunity to try to introduce some better scripting?

I naively see a few possibilities here

do something like embed Lua
a) write your own DSL semantics and b) use a simple frontend like S-Expressions

I hope this is not incompatible with your speed demon dreams?

(1) would provide a full featured language out of the box, meaning you don't have to implement any general purpose semantics yourself. With all the advantages that come with it, and without the issues of home-rolling another language. (?)

For (2):
(a) I say from personal experience that Nix/NixOS for example has quite a lot of success using a purely functional langugage for declaratively describing builds - another variant of "generating files". The approach used uses the DSL to generate a low level configuration which is then executed by the build evaluation semantics (for example, the Nixpkgs library has functionality for generating certain bash scripts, which then are run by the builder sandboxes). It may be possible to "precompile/preevaluate" scripts to help with possible speed issues. (I don't know how things would benchmark.)

(b) SMT-LIB appears to use the SExpr approach for example: http://smtlib.cs.uiowa.edu/papers/smt-lib-reference-v2.6-r2017-07-18.pdf

The choice of the S-expression syntax and the design of the concrete syntax
was mostly driven by the goal of simplifying parsing, as opposed to facilitating
human readability.
[...]
Preferring  ease  of  parsing  over  human  readability  is  reasonable  in  this
 context  not  only  because  SMT-LIB benchmarks are meant to be read by
solvers but also because they are produced in the first place by automated
tools like verification condition generators or translators from other formats.

The human issue mentioned could later be solved by using some manner of frontend, if deemed necessary, but it's "perfectly" writable by hand.
This would also(, as a significant point?, ) allow other tools to interface better with the linker if flexibility is necessary.

Look into splice syscall for faster I/O

While reading through the README.md, I found that you were using cat to concatenate input object files.

If you are just interested in copying the files into another file without modification, there are ways to speed up this process on Linux with the splice syscall. This is what I used for fcat (short for "fast-cat"). It resulted in a 3x speedup in my tests. There is also a write-up on my blog on that topic if you're interested.

For macOS there is copyfile, which seems to support COPYFILE_CLONE which will attempt to use the APFS clone capabilities for faster copies. (See e.g. mono/mono#9038.)
There is a similar API for windows.
I wouldn't be surprised if the C++ I/O copy function already uses these optimizations under the hood already, but at least for Linux splice could yield some improvements.

mold built with makepkg always fails assertion

I'm trying to create an AUR package for mold. You can see what I've come up with here: https://github.com/wezm/aur/tree/master/mold

I'm using a modified Makefile which includes some of the changes @Logarithmus made for their Void package. With this Makefile and my PKGBUILD mold builds fine but the resulting binary always fails an assertion. If I build mold without using makepkg, just using the Makefile supplied in the tarball the resulting binary works fine. The same issue happens if I don't link against the system mimalloc and let mold build its vendored copy.

As this is a packaging issue I'll understand if you don't have time to took into this issue. Nonetheless I figured the error might mean more to you.

If I try to build this small C program:

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
  printf("Linked with mold\n");
  return EXIT_SUCCESS;
}

as follows: mold -run clang -o mold-test test.c I get this error:

/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/11.1.0/../../../../include/c++/11.1.0/span:276: std::span::reference std::span<ElfSym<X86_64>, 18446744073709551615>::operator[](std::span::size_type) const [_Type = ElfSym<X86_64>, _Extent = 18446744073709551615]: Assertion '__idx < size()' failed.
clang-12: error: unable to execute command: Aborted (core dumped)
clang-12: error: linker command failed due to signal (use -v to see invocation)

The compiler used to build mold (and the test binary above) is:

clang version 12.0.0
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

My system is an AMD Ryzen 9 3900X running Arch Linux.

If you have any ideas I'd love to hear them.

Settings up some basic CI for guaranteeing build-pass

Hi, with build issue in #21, I would like to suggest to create a basic CI for Linux (maybe macOS also)
to guarantee the build pass. I can help with setting up CI for github actions services if you'd like.

Project cannot be built with both recent gcc and clang

I cannot build this project with both gcc and clang on latest commit as this time: 6630781.

I'm happy to answer more questions about my build environment if that helps.

Expected Behavior

Build shall pass.

Current Behavior

Build fails.

With clang:

`make -j32`

archive_file.cc:39:17: error: no matching constructor for initialization of 'std::string' (aka 'basic_string<char>')
    std::string name(start, strstr(start, "/\n"));
                ^    ~~~~~~~~~~~~~~~~~~~~~~~~~~~
archive_file.cc:70:12: error: no viable overloaded '='
      name = {start, strstr(start, "/\n")};
      ~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

With gcc by changing CC and CXX in makefile.

`make -j32`

subprocess.cc: In function ‘std::function<void()> fork_child()’:
subprocess.cc:33:34: error: taking address of temporary array
   33 |     if (read(pipefd[0], (char[1]){}, 1) == 1)
      |                                  ^~
subprocess.cc: In lambda function:
subprocess.cc:48:44: error: taking address of temporary array
   48 |   return [=]() { write(pipefd[1], (char []){1}, 1); };
      |                                            ^~~
subprocess.cc: In function ‘bool resume_daemon(char**, i64*)’:
subprocess.cc:153:31: error: taking address of temporary array
  153 |   i64 r = read(conn, (char[1]){}, 1);
      |                               ^~
subprocess.cc: In lambda function:
subprocess.cc:218:47: error: taking address of temporary array
  218 |   *on_complete = [=]() { write(conn, (char []){1}, 1); };
      |                                               ^~~
icf.cc:69:19: warning: ‘template<class Key> class tbb::tbb_hash’ is deprecated: tbb::tbb_hash is deprecated, use std::hash [-Wdeprecated-declarations]
   69 | template<> struct tbb_hash<Digest> {
      |                   ^~~~~~~~
In file included from oneTBB/include/tbb/concurrent_hash_map.h:35,
                 from mold.h:21,
                 from icf.cc:53:
oneTBB/include/tbb/internal/_tbb_hash_compare_impl.h:86:74: note: declared here
   86 | class __TBB_DEPRECATED_MSG("tbb::tbb_hash is deprecated, use std::hash") tbb_hash
      |                                                                          ^~~~~~~~
At global scope:
cc1plus: note: unrecognized command-line option ‘-Wno-deprecated-volatile’ may have been intended to silence earlier diagnostics

Context (Environment)

gcc --version: gcc (GCC) 10.2.0
clang version 11.0.0 x86_64-unknown-linux-gnu
OS: Linux 4.19.0-0.bpo.9-amd64 #1 SMP Debian 4.19.118-2+deb10u1~bpo9+1 (2020-06-09) x86_64 GNU/Linux
glibc:

ICF digest calculation fails when asserts are disabled

When the -icf flag is passed, mold will attempt to calculate digests (hashes) for each section and eliminate sections with identical hashes.

I have been playing around with mold and moved the build to use CMake, which defines NDEBUG when building a release configuration, which ultimately disables assert. The icf.sh test case was failing for me, on closer inspection using the --print-icf-sections flag, I found that all sections were being marked as identical and removed.

The problem is that SHA256_Final was being called from inside an assert, in icf.cc:118

assert(SHA256_Final(buf, &sha) == 1);

Changing this to call the function outside the assert and then checking the return result fixes the problem.

Assertion failure building llvm with mold as linker

Environment:

Arch Linux with gcc 10.2.0, mold 33c7719

Building llvm from git, version 27dfcd978edc94b4b46218b12a3dfbcdc47ae4c8

llvm-project$ mkdir build && cd build
llvm-project/build$ cmake -G 'Ninja' -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX=/opt/llvm '-DLLVM_ENABLE_PROJECTS=clang;libcxx;libunwind;lldb;compiler-rt;lld;libcxxabi'  ../llvm
llvm-project/build$ mold -run ninja -j16
<...>
[340/6394] Linking CXX executable bin/llvm-tblgen
FAILED: bin/llvm-tblgen
: && /usr/bin/c++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -O2 -g -DNDEBUG -Wl,-rpath-link,/home/jade/dev/llvm-project/build/./lib  -Wl,-O3 -Wl,--gc-sections utils/TableGen/CMakeFiles/llvm-tblgen.dir/AsmMatcherEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/AsmWriterEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/AsmWriterInst.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/Attributes.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CallingConvEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeBeadsGen.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeEmitterGen.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenDAGPatterns.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenHwModes.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenInstruction.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenMapTable.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenRegisters.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenSchedule.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenTarget.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DAGISelEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DAGISelMatcherEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DAGISelMatcherGen.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DAGISelMatcherOpt.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DAGISelMatcher.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DFAEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DFAPacketizerEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DirectiveEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DisassemblerEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/ExegesisEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/FastISelEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/FixedLenDecoderEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/GICombinerEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/GlobalISelEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/InfoByHwMode.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/InstrInfoEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/InstrDocsEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/IntrinsicEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/OptEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/OptParserEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/OptRSTEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/PredicateExpander.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/PseudoLoweringEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/RISCVCompressInstEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/RegisterBankEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/RegisterInfoEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/SDNodeProperties.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/SearchableTableEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/SubtargetEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/SubtargetFeatureInfo.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/TableGen.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/Types.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/X86DisassemblerTables.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/X86EVEX2VEXTablesEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/X86FoldTablesEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/X86ModRMFilters.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/X86RecognizableInstr.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/WebAssemblyDisassemblerEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CTagsEmitter.cpp.o -o bin/llvm-tblgen  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMSupport.a  lib/libLLVMTableGen.a  -lpthread  lib/libLLVMTableGenGlobalISel.a  lib/libLLVMTableGen.a  lib/libLLVMSupport.a  -lrt  -ldl  -lpthread  -lm  /usr/lib/libz.so  -ltinfo  lib/libLLVMDemangle.a && :
mold: ./mold.h:1466: u64 Symbol<X86_64>::get_addr(Context<E> &) const [E = X86_64]: Assertion `frag->is_alive' failed.
collect2: fatal error: ld terminated with signal 6 [Aborted], core dumped
compilation terminated.
[355/6394] Building CXX object lib/MC/MCParser/CMakeFiles/LLVMMCParser.dir/MasmParser.cpp.o
ninja: build stopped: subcommand failed.

Comment on implementation details

Hi @rui314,

I've just stumbled upon this project coming from some reddit comment, and skimmed through the readme. The following excerpt from the Details section

At least on Linux, it looks like the filesystem's performance to allocate new blocks to a new file is the limiting factor when creating a new large file and filling its contents using mmap. If you already have a large file in the buffer cache, writing to it is much faster than creating a new fresh file and writing to it. [...]

reminded me of something I had read before at https://stackoverflow.com/a/34579110/3451198:

Secondly, never ever extend an output file during a write, you force an extent allocation and possibly a metadata flush. Instead fallocate the file's maximum extent to some suitably large value, and keep an internal atomic counter of the end of file. That should reduce the problem to just extent allocation which for ext4 is batched and lazy - more importantly you won't be forcing a metadata flush.

Assuming that actually applies to this project and you didn't consider it already, maybe fallocate(2)ing in large chunks would be an alternate option given that common filesystems (ext4, btrfs) support it.

Cross-compiling mold itself for Windows fails due to missing header

I'm trying to cross-compile for Windows, using Docker and wclang:
https://github.com/tpoechtrager/wclang

FROM ubuntu:20.04

ENV DEBIAN_FRONTEND noninteractive
RUN DEBIAN_FRONTEND=noninteractive apt update && apt install -y \
  build-essential \
  libstdc++-10-dev \
  clang \
  mingw-w64 \
  libssl-dev \
  zlib1g-dev \
  cmake \
  bsdmainutils \
  git

RUN git clone --recursive https://github.com/rui314/mold.git

WORKDIR /mold

RUN git clone https://github.com/tpoechtrager/wclang && \
  cd wclang  && \
  cmake -DCMAKE_INSTALL_PREFIX=_prefix_ . && \
  make && make install && \
  cd ../ && \
  sed -i 's/C=clang/C=/mold/wclang/_prefix_/bin/w64-clang/g' Makefile && \
  sed -i 's/CXX=clang++/C=/mold/wclang/_prefix_/bin/w64-clang++/g' Makefile && \
  make submodules && make

root@01e8e468bd02:/mold# make STATIC=1
/mold/wclang/_prefix_/bin/w64-clang++  -g -IoneTBB/include -IxxHash -pthread -std=c++20 -Wno-deprecated-volatile -Wno-switch -march=native -DGIT_HASH=\"e6cf989d442867114fdf8ddc656944b10430272f\" -O2  -c -o main.o main.cc
In file included from main.cc:1:
./mold.h:19:10: fatal error: 'span' file not found
#include <span>
         ^~~~~~
1 error generated.
make: *** [<builtin>: main.o] Error 1

root@01e8e468bd02:/mold# clang --version
clang version 10.0.0-4ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Am I doing anything wrong? Or what's the best/easiest way to compile this for Windows? 🤔
(Running it with regular make or mingw32-make gives me syntax errors, in Windows)

rayga@DESKTOP-88A51E5 MINGW64 ~/Projects/tmp/mold (main)
$ mingw32-make.exe submodules
C:/msys64/mingw64/bin/mingw32-make.exe -C oneTBB
mingw32-make[1]: Entering directory 'C:/Users/rayga/Projects/tmp/mold/oneTBB'
Created the .\\build\\windows_unknown_cl_unknown_release directory
C:/msys64/mingw64/bin/mingw32-make.exe -C ".\\build\\windows_unknown_cl_unknown_release"  -r -f ../../build/Makefile.tbb cfg=release
mingw32-make[2]: Entering directory 'C:/Users/rayga/Projects/tmp/mold/oneTBB/build/windows_unknown_cl_unknown_release'
../../build/Makefile.tbb:28: CONFIG: cfg=release arch=unknown compiler=cl target=windows runtime=unknown
cl /nologo /Foconcurrent_hash_map.obj /c /MD /O2 /Zi /EHsc /GR /Zc:forScope /Zc:wchar_t /D__TBB_LIB_NAME=tbb.lib /DDO_ITT_NOTIFY /GS /DUSE_WINTHREAD /D_CRT_SECURE_NO_DEPRECATE /D_WIN32_WINNT=0x0502    /D__TBB_BUILD=1 /W4   /DTBB_SUPPRESS_DEPRECATED_MESSAGES=1   /I../../src /I../../src/rml/include /I../../include ../../src/tbb/concurrent_hash_map.cpp
process_begin: CreateProcess(NULL, cl /nologo /Foconcurrent_hash_map.obj /c /MD /O2 /Zi /EHsc /GR /Zc:forScope /Zc:wchar_t /D__TBB_LIB_NAME=tbb.lib /DDO_ITT_NOTIFY /GS /DUSE_WINTHREAD /D_CRT_SECURE_NO_DEPRECATE /D_WIN32_WINNT=0x0502 /D__TBB_BUILD=1 /W4 /DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 /I../../src /I../../src/rml/include /I../../include ../../src/tbb/concurrent_hash_map.cpp, ...) failed.
make (e=2): The system cannot find the file specified.
mingw32-make[2]: *** [../../build/common_rules.inc:80: concurrent_hash_map.obj] Error 2
mingw32-make[2]: Leaving directory 'C:/Users/rayga/Projects/tmp/mold/oneTBB/build/windows_unknown_cl_unknown_release'
mingw32-make[1]: *** [makefile:29: tbb] Error 2
mingw32-make[1]: Leaving directory 'C:/Users/rayga/Projects/tmp/mold/oneTBB'
mingw32-make: *** [makefile:63: submodules] Error 2

Nixification

As soon as I read that mold uses Gentoo for testing because it's source-based, I instantly thought about Nix being a perfect fit for it. Achieving this however turned out to be a lot harder than I initially thought, because the bootstrapping code is rather complex compared to the simple overrides one usually does.

Nevertheless, here's a file that builds any selected packages (list at the bottom) and all their transitive dependencies using mold as links (if they used bfd before). (Edit: updated file here)

let
  # Pin some fairly new nixpkgs
  sources = builtins.fetchTarball {
    name = "nixpkgs-unstable-2021-07-06";
    url = "https://github.com/nixos/nixpkgs/archive/291b3ff5af268eb7a656bb11c73f2fe535ea2170.tar.gz";
    sha256 = "1z2l7q4cmiaqb99cd8yfisdr1n6xbwcczr9020ss47y2z1cn1x7x";
  };

  # For bootstrapping
  pkgs0 = import sources {
    overlays = [
      (pkgs: super: {
        # TODO replace with proper packaging once https://github.com/NixOS/nixpkgs/pull/128889 is merged
        mold = pkgs.stdenv.mkDerivation {
          pname = "mold";
          version = "0.9.1";
          src = pkgs.fetchgit {
            url = "https://github.com/rui314/mold";
            rev = "v0.9.1";
            sha256 = "sha256-yIkW6OCXhlHZ1jC8/yMAdJbSgY9K40POT2zWv6wYr5E=";
          };
          nativeBuildInputs = with pkgs; [ clang_12 cmake lld_12 tbb xxHash zlib openssl git ];
          dontUseCmakeConfigure = "true";
          buildPhase = "make -j $NIX_BUILD_CORES";
          installPhase = "mkdir -p $out $out/bin $out/share/man/man1 && PREFIX=$out make install";
        };

        binutils_mold = pkgs.wrapBintoolsWith {
          bintools = pkgs.binutils-unwrapped.overrideAttrs (old: {
            postInstall = ''
              rm $out/bin/ld.gold
              rm $out/bin/ld.bfd
              ln -sf ${pkgs.mold}/bin/mold $out/bin/ld.bfd
            '';
          });
        };
        
        stdenv_mold = super.overrideCC super.stdenv (super.wrapCCWith rec {
          cc = super.gcc-unwrapped;
          bintools = pkgs.binutils_mold;
        });
      })
    ];
  };

  # Actual nixpkgs with patched linker in all packages
  pkgs = import sources {
    overlays = [
      (pkgs: super: {
        stdenv = pkgs0.stdenv_mold;
        mold = pkgs0.mold;
      })
    ];
  };
in with pkgs;
  # Packages we want to build
  linkFarmFromDrvs "packages-with-mold" [
    mold
    binutils
    stdenv
    opencv
    jack2
    chromium
  ]

Let me know what you think.

CC @nitsky

[SETUP] issue making it default linker

running ./mold -run make while in the mold directory gives:
make: Nothing to be done for 'all'.

so i moved the mold executable to a separate directory and run ../manma/mold -run make
it gave: ERROR: ld.so: object '/home/superman/packed/manma/mold-wrapper.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. make: Nothing to be done for 'all'.

even having moved the shared object file to appropriate folder and rebuilt mold with make as a sub process, it doesn't seem as though it worked, resulting binaries dont seem to have used mold

system: pop!_os

Add preloading support to CMake and other build commands

mold supports object file preloading feature. In order to use it, someone has to run a linker command with --preload flag a few seconds prior to the actual linker invocation. And I think "someone" should be a build system, such as make or ninja executing CMake-generated ninja or makefiles.

As a starter, I think we should add the following feature to CMake:

Detect if a linker is mold
run a linker command with --preload if it is mold before invoking compiler instances to generate object files

SIMD

Full disclosure, I'm completely unfamiliar with the kind of work you're doing and especially the problem you're trying to tackle. I've made the assumption that there is code where SIMD is applicable, but I have not assumed that it will improve performance. That out of the way, I hope my ignorance isn't more than mildly offensive 😅

Would it be possible to utilize SIMD to improve upon performance further? I notice that aside from everything that's being done on the concurrency side of things, there's a lot of IO work being done. I'm not sure if SIMD would even help there or if it would be bottle-necked by the drive speed. Or even if what you're doing is worth the overhead of SIMD, which would instead mean that SIMD may actually hurt performance.

Support preloading without changing command line arguments

According to the README:

if you run mold with -preload flag along with other command line flags, it becomes a daemon and halts after parsing input files. Then, if you invoke mold with the same command line options (except -preload flag), it tells the daemon to reload only updated files and proceed.

I am using mold with my rust project to improve incremental link times, and the rust compiler does not provide a way to change the linker flags depending on whether this is the first build or a subsequent build. Because of this, I am unable to take advantage of preloading.

Would you consider changing this behavior so that you do not have to omit the -preload flag on the second run? This means the behavior with the -preload flag would be:

if you run mold with -preload flag along with other command line flags, it becomes a daemon and halts after parsing input files. Then, if you invoke mold with the same command line options (including -preload flag), it tells the daemon to reload only updated files and proceed.

Go's tests fail when linked with mold

There's one test in the Go's testsuite that fails only when linked with mold. The same test fails with LLVM lld too. I don't know if this is a bug fo mold/lld or Go, but we need to fix it anyhow.

It is reported to Go as golang/go#46560.

Unable to use mold with a (pseudo) cross compiler

~/r/mold ❯❯❯ conda create -n gccdev gxx_linux-64 -c conda-forge
~/r/mold ❯❯❯ conda activate gccdev

(gccdev) ~/r/mold ❯❯❯ $CC --version 
x86_64-conda-linux-gnu-cc (crosstool-NG 1.24.0.133_b0863d8_dirty) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

(gccdev) ~/r/mold ❯❯❯ /tmp/ld --version 
mold 0.1.1 (782b7f9c55427f0ffea44543e6de44bc77f9e47e; compatible with GNU ld and GNU gold)

(gccdev) ~/r/mold ❯❯❯ echo 'int main() {return 0;}' | $CC -xc - -B/tmp
mold: library not found: gcc
collect2: error: ld returned 1 exit status

Any plans for PE/COFF support?

Hi,

Is there any plans to support linking on windows? I'll would be interested in helping as we right now mantain our own patches for objcopy and lld to make it a little faster. I think on DWARF there's no need to produce one global symbol table like there is on PDB, So I'm guessing mold for windows will look quite different than it looks now, was there any thought put on it already?

Things like ghashes for coff:
http://blog.llvm.org/2018/01/improving-link-time-on-windows-with.html

building on debian 10 buster

I'm using debian 10 which has no gcc-10 or clang-10 package
so I'm using clang and libc++ from llvm apt repo
and added -stdlib=libc++
but i'm unable to build as some headers are missing like "cstdint" "atomic"

Many tests fail with musl libc

These tests fail with musl libc (used in Alpine Linux & is optional in Void Linux).
I think some of them can't run successfully due to musl lacking some features of glibc e. g ifunc. So it would be nice to detect system libc somewhere in test/Makefile and conditionally skip those tests. Though some tests can be fixed I guess.

Note: Void Linux package build environment uses grep compiled without pcre support. Normal grep can't be installed due to name conflicts (you can't have 2 /usr/bin/grep binaries). As a workaround, I'm replacing grep -P with rg -P in all tests via sed (rg is ripgrep, a faster grep alternative). This is only Void specific, so never mind.

ctors-dtors.sh

+ set -e
++ dirname test/ctors-dtors.sh.fail
+ cd test
++ basename -s .sh test/ctors-dtors.sh.fail
+ echo -n 'Testing ctors-dtors.sh.fail ... '
Testing ctors-dtors.sh.fail ... ++ pwd
++ basename -s .sh test/ctors-dtors.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/ctors-dtors.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/ctors-dtors.sh.fail
+ cat
+ clang -c -o /builddir/mold-0.9.3-rc2/test/tmp/ctors-dtors.sh.fail/a.o -x assembler -
+ cat
+ clang -c -o /builddir/mold-0.9.3-rc2/test/tmp/ctors-dtors.sh.fail/b.o -x assembler -
+ cat
+ clang -c -o /builddir/mold-0.9.3-rc2/test/tmp/ctors-dtors.sh.fail/c.o -xc -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/ctors-dtors.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/ctors-dtors.sh.fail/a.o /builddir/mold-0.9.3-rc2/test/tmp/ctors-dtors.sh.fail/b.o /builddir/mold-0.9.3-rc2/test/tmp/ctors-dtors.sh.fail/c.o
+ /builddir/mold-0.9.3-rc2/test/tmp/ctors-dtors.sh.fail/exe
+ grep -q '^013289baefdc6754$'

dt_init.sh

+ set -e
++ dirname test/dt_init.sh.fail
+ cd test
++ basename -s .sh test/dt_init.sh.fail
+ echo -n 'Testing dt_init.sh.fail ... '
Testing dt_init.sh.fail ... ++ pwd
++ basename -s .sh test/dt_init.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/dt_init.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/dt_init.sh.fail
+ cat
+ clang -c -o /builddir/mold-0.9.3-rc2/test/tmp/dt_init.sh.fail/a.o -x assembler -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/dt_init.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/dt_init.sh.fail/a.o
+ readelf -a /builddir/mold-0.9.3-rc2/test/tmp/dt_init.sh.fail/exe
+ rg -PqU '(?s)\(INIT\)\s+0x([0-9a-f]+)\b.*\1\s+0 FUNC    GLOBAL HIDDEN\s+\d+ _init\b' /builddir/mold-0.9.3-rc2/test/tmp/dt_init.sh.fail/log

dynamic.sh

+ set -e
++ dirname test/dynamic.sh.fail
+ cd test
++ basename -s .sh test/dynamic.sh.fail
+ echo -n 'Testing dynamic.sh.fail ... '
Testing dynamic.sh.fail ... ++ pwd
++ basename -s .sh test/dynamic.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/dynamic.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/dynamic.sh.fail
+ echo '.globl main; main:'
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/dynamic.sh.fail/a.o -c -x assembler -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/dynamic.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/dynamic.sh.fail/a.o
+ readelf --dynamic /builddir/mold-0.9.3-rc2/test/tmp/dynamic.sh.fail/exe
+ fgrep -q 'Shared library: [libc.so.6]' /builddir/mold-0.9.3-rc2/test/tmp/dynamic.sh.fail/log

exception.sh

+ set -e
++ dirname test/exception.sh.fail
+ cd test
++ basename -s .sh test/exception.sh.fail
+ echo -n 'Testing exception.sh.fail ... '
Testing exception.sh.fail ... ++ pwd
++ basename -s .sh test/exception.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/exception.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/exception.sh.fail
+ cat
++ pwd
+ clang++ -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/exception.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/exception.sh.fail/a.cc -static
+ /builddir/mold-0.9.3-rc2/test/tmp/exception.sh.fail/exe
test/exception.sh.fail: line 20:   487 Segmentation fault      $t/exe

hello-dynamic.sh

+ set -e
++ dirname test/hello-dynamic.sh.fail
+ cd test
++ basename -s .sh test/hello-dynamic.sh.fail
+ echo -n 'Testing hello-dynamic.sh.fail ... '
Testing hello-dynamic.sh.fail ... ++ pwd
++ basename -s .sh test/hello-dynamic.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/hello-dynamic.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/hello-dynamic.sh.fail
+ cat
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/hello-dynamic.sh.fail/a.o -c -x assembler -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/hello-dynamic.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/hello-dynamic.sh.fail/a.o
+ /builddir/mold-0.9.3-rc2/test/tmp/hello-dynamic.sh.fail/exe
+ grep -q 'Hello world'

hello-static.sh

+ set -e
++ dirname test/hello-static.sh.fail
+ cd test
++ basename -s .sh test/hello-static.sh.fail
+ echo -n 'Testing hello-static.sh.fail ... '
Testing hello-static.sh.fail ... ++ pwd
++ basename -s .sh test/hello-static.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/hello-static.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/hello-static.sh.fail
+ cat
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/hello-static.sh.fail/a.o -c -x assembler -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/hello-static.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/hello-static.sh.fail/a.o -static
+ clang -fuse-ld=gold -o /builddir/mold-0.9.3-rc2/test/tmp/hello-static.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/hello-static.sh.fail/a.o -static
+ /builddir/mold-0.9.3-rc2/test/tmp/hello-static.sh.fail/exe
+ grep -q 'Hello world'

ifunc-dso.sh

+ set -e
++ dirname test/ifunc-dso.sh.fail
+ cd test
++ basename -s .sh test/ifunc-dso.sh.fail
+ echo -n 'Testing ifunc-dso.sh.fail ... '
Testing ifunc-dso.sh.fail ... ++ pwd
++ basename -s .sh test/ifunc-dso.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/ifunc-dso.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dso.sh.fail
+ cat
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dso.sh.fail/a.o -c -x assembler -
+ cat
+ cc -shared -fPIC -o /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dso.sh.fail/b.so -x assembler -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dso.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dso.sh.fail/a.o /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dso.sh.fail/b.so
+ /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dso.sh.fail/exe
+ grep -q 'Hello world'
Error relocating /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dso.sh.fail/exe: foobar: symbol not found

ifunc-dynamic.sh

+ set -e
++ dirname test/ifunc-dynamic.sh.fail
+ cd test
++ basename -s .sh test/ifunc-dynamic.sh.fail
+ echo -n 'Testing ifunc-dynamic.sh.fail ... '
Testing ifunc-dynamic.sh.fail ... ++ pwd
++ basename -s .sh test/ifunc-dynamic.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/ifunc-dynamic.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dynamic.sh.fail
+ cat
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dynamic.sh.fail/a.o -c -x assembler -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dynamic.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dynamic.sh.fail/a.o
+ /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dynamic.sh.fail/exe
+ grep -q 'Hello world'
Error relocating /builddir/mold-0.9.3-rc2/test/tmp/ifunc-dynamic.sh.fail/exe: unsupported relocation type 37

ifunc-static.sh

+ set -e
++ dirname test/ifunc-static.sh.fail
+ cd test
++ basename -s .sh test/ifunc-static.sh.fail
+ echo -n 'Testing ifunc-static.sh.fail ... '
Testing ifunc-static.sh.fail ... ++ pwd
++ basename -s .sh test/ifunc-static.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/ifunc-static.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/ifunc-static.sh.fail
+ cat
+ clang -o /builddir/mold-0.9.3-rc2/test/tmp/ifunc-static.sh.fail/a.o -c -xc -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/ifunc-static.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/ifunc-static.sh.fail/a.o -static
+ /builddir/mold-0.9.3-rc2/test/tmp/ifunc-static.sh.fail/exe
+ grep -q 'Hello world'

image-base.sh

+ set -e
++ dirname test/image-base.sh.fail
+ cd test
++ basename -s .sh test/image-base.sh.fail
+ echo -n 'Testing image-base.sh.fail ... '
Testing image-base.sh.fail ... ++ pwd
++ basename -s .sh test/image-base.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/image-base.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/image-base.sh.fail
+ cat
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/image-base.sh.fail/a.o -c -xc -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/image-base.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/image-base.sh.fail/a.o -Wl,--image-base=0x8000000
+ /builddir/mold-0.9.3-rc2/test/tmp/image-base.sh.fail/exe
+ grep -q 'Hello world'
+ readelf -W --sections /builddir/mold-0.9.3-rc2/test/tmp/image-base.sh.fail/exe
+ rg -Pq '.interp\s+PROGBITS\s+0000000008000...\b'

incompatible-libs.sh

+ set -e
++ dirname test/incompatible-libs.sh.fail
+ cd test
++ basename -s .sh test/incompatible-libs.sh.fail
+ echo -n 'Testing incompatible-libs.sh.fail ... '
Testing incompatible-libs.sh.fail ... ++ pwd
++ basename -s .sh test/incompatible-libs.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/incompatible-libs.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/incompatible-libs.sh.fail
+ cat
+ cc -m32 -c -o /builddir/mold-0.9.3-rc2/test/tmp/incompatible-libs.sh.fail/a.o -xc -
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/incompatible-libs.sh.fail/lib32
+ ar crs /builddir/mold-0.9.3-rc2/test/tmp/incompatible-libs.sh.fail/lib32/libfoo.a /builddir/mold-0.9.3-rc2/test/tmp/incompatible-libs.sh.fail/a.o
+ clang -m32 -shared -o /builddir/mold-0.9.3-rc2/test/tmp/incompatible-libs.sh.fail/lib32/libfoo.so /builddir/mold-0.9.3-rc2/test/tmp/incompatible-libs.sh.fail/a.o
/usr/bin/x86_64-unknown-linux-musl-ld: cannot find crtbeginS.o: No such file or directory
/usr/bin/x86_64-unknown-linux-musl-ld: cannot find -lgcc
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /usr/bin/../lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /usr/bin/../lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /usr/lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /usr/lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: cannot find libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /usr/bin/../lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /usr/lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: skipping incompatible /usr/lib/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/x86_64-unknown-linux-musl-ld: cannot find -lgcc
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)

library-path-dynamic.sh

+ set -e
++ dirname test/library-path-dynamic.sh.fail
+ cd test
++ basename -s .sh test/library-path-dynamic.sh.fail
+ echo -n 'Testing library-path-dynamic.sh.fail ... '
Testing library-path-dynamic.sh.fail ... ++ pwd
++ basename -s .sh test/library-path-dynamic.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/library-path-dynamic.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/library-path-dynamic.sh.fail
+ cat
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/library-path-dynamic.sh.fail/a.o -c -x assembler -
+ clang -o /builddir/mold-0.9.3-rc2/test/tmp/library-path-dynamic.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/library-path-dynamic.sh.fail/a.o
+ /builddir/mold-0.9.3-rc2/test/tmp/library-path-dynamic.sh.fail/exe
+ grep -q 'Hello world'

linker_script.sh

+ set -e
++ dirname test/linker-script.sh.fail
+ cd test
++ basename -s .sh test/linker-script.sh.fail
+ echo -n 'Testing linker-script.sh.fail ... '
Testing linker-script.sh.fail ... ++ pwd
++ basename -s .sh test/linker-script.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/linker-script.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/linker-script.sh.fail
+ cat
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/linker-script.sh.fail/a.o -c -x assembler -
+ cat
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/linker-script.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/linker-script.sh.fail/script
+ + grep /builddir/mold-0.9.3-rc2/test/tmp/linker-script.sh.fail/exe-q 
'Hello world'

mergeable-strings.sh

+ set -e
++ dirname test/mergeable-strings.sh.fail
+ cd test
++ basename -s .sh test/mergeable-strings.sh.fail
+ echo -n 'Testing mergeable-strings.sh.fail ... '
Testing mergeable-strings.sh.fail ... ++ pwd
++ basename -s .sh test/mergeable-strings.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/mergeable-strings.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/mergeable-strings.sh.fail
+ cat
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/mergeable-strings.sh.fail/a.o -c -x assembler -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -static -o /builddir/mold-0.9.3-rc2/test/tmp/mergeable-strings.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/mergeable-strings.sh.fail/a.o
+ /builddir/mold-0.9.3-rc2/test/tmp/mergeable-strings.sh.fail/exe
+ grep -q 'Hello world'

nocopyreloc.sh

+ set -e
++ dirname test/nocopyreloc.sh.fail
+ cd test
++ basename -s .sh test/nocopyreloc.sh.fail
+ echo -n 'Testing nocopyreloc.sh.fail ... '
Testing nocopyreloc.sh.fail ... ++ pwd
++ basename -s .sh test/nocopyreloc.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/nocopyreloc.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/nocopyreloc.sh.fail
+ cat
+ cc -shared -o /builddir/mold-0.9.3-rc2/test/tmp/nocopyreloc.sh.fail/a.so -xc -
+ cat
+ cc -fno-PIC -c -o /builddir/mold-0.9.3-rc2/test/tmp/nocopyreloc.sh.fail/b.o -xc -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -no-pie -o /builddir/mold-0.9.3-rc2/test/tmp/nocopyreloc.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/nocopyreloc.sh.fail/a.so /builddir/mold-0.9.3-rc2/test/tmp/nocopyreloc.sh.fail/b.o
+ /builddir/mold-0.9.3-rc2/test/tmp/nocopyreloc.sh.fail/exe
+ grep -q '3 5'

plt.sh

+ set -e
++ dirname test/plt.sh.fail
+ cd test
++ basename -s .sh test/plt.sh.fail
+ echo -n 'Testing plt.sh.fail ... '
Testing plt.sh.fail ... ++ pwd
++ basename -s .sh test/plt.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/plt.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/plt.sh.fail
+ cat
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/plt.sh.fail/a.o -c -x assembler -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/plt.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/plt.sh.fail/a.o
+ readelf --sections /builddir/mold-0.9.3-rc2/test/tmp/plt.sh.fail/exe
+ fgrep -q .got
+ readelf --sections /builddir/mold-0.9.3-rc2/test/tmp/plt.sh.fail/exe
+ fgrep -q .got.plt
+ /builddir/mold-0.9.3-rc2/test/tmp/plt.sh.fail/exe
+ grep -q 'Hello world'

preload.sh

+ set -e
++ dirname test/preload.sh.fail
+ cd test
++ basename -s .sh test/preload.sh.fail
+ echo -n 'Testing preload.sh.fail ... '
Testing preload.sh.fail ... ++ pwd
++ basename -s .sh test/preload.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/preload.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/preload.sh.fail
+ cat
+ cc -o /builddir/mold-0.9.3-rc2/test/tmp/preload.sh.fail/a.o -c -x assembler -
+ rm -f /builddir/mold-0.9.3-rc2/test/tmp/preload.sh.fail/exe
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/preload.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/preload.sh.fail/a.o -Wl,-preload
+ test -e /builddir/mold-0.9.3-rc2/test/tmp/preload.sh.fail/exe
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/preload.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/preload.sh.fail/a.o
+ /builddir/mold-0.9.3-rc2/test/tmp/preload.sh.fail/exe
+ grep -q 'Hello world'

reloc.sh

+ set -e
++ dirname test/reloc.sh.fail
+ cd test
++ basename -s .sh test/reloc.sh.fail
+ echo -n 'Testing reloc.sh.fail ... '
Testing reloc.sh.fail ... ++ pwd
++ basename -s .sh test/reloc.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail
+ cat
+ cc -fPIC -c -o /builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail/a.o -x assembler -
+ cat
+ cc -fPIC -c -o /builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail/b.o -xc -
+ cc -shared -o /builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail/c.so /builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail/a.o /builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail/b.o
+ cat
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail/c.so /builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail/d.s -no-pie
+ /builddir/mold-0.9.3-rc2/test/tmp/reloc.sh.fail/exe
+ grep -q 42

shared.sh

+ set -e
++ dirname test/shared.sh.fail
+ cd test
++ basename -s .sh test/shared.sh.fail
+ echo -n 'Testing shared.sh.fail ... '
Testing shared.sh.fail ... ++ pwd
++ basename -s .sh test/shared.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail
+ cat
+ clang -fPIC -c -o /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/a.o -x assembler -
++ pwd
+ clang -shared -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/b.so /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/a.o
+ readelf --dyn-syms /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/b.so
+ grep -q '0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND fn2' /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/log
+ rg -Pq '0 NOTYPE  GLOBAL DEFAULT   \d+ fn1' /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/log
+ cat
+ clang -fPIC -c -o /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/c.o -xc -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/c.o /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/b.so
+ /builddir/mold-0.9.3-rc2/test/tmp/shared.sh.fail/exe
+ grep -q hello

synthetic-symbols.sh

+ set -e
++ dirname test/synthetic-symbols.sh.fail
+ cd test
++ basename -s .sh test/synthetic-symbols.sh.fail
+ echo -n 'Testing synthetic-symbols.sh.fail ... '
Testing synthetic-symbols.sh.fail ... ++ pwd
++ basename -s .sh test/synthetic-symbols.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/synthetic-symbols.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/synthetic-symbols.sh.fail
+ cat
+ clang -c -o /builddir/mold-0.9.3-rc2/test/tmp/synthetic-symbols.sh.fail/a.o -x assembler -
+ cat
+ clang -c -o /builddir/mold-0.9.3-rc2/test/tmp/synthetic-symbols.sh.fail/b.o -xc -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -Wl,--image-base=0x40000 -o /builddir/mold-0.9.3-rc2/test/tmp/synthetic-symbols.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/synthetic-symbols.sh.fail/a.o /builddir/mold-0.9.3-rc2/test/tmp/synthetic-symbols.sh.fail/b.o
+ /builddir/mold-0.9.3-rc2/test/tmp/synthetic-symbols.sh.fail/exe
+ grep -q '^__ehdr_start=0x40000$' /builddir/mold-0.9.3-rc2/test/tmp/synthetic-symbols.sh.fail/log

versioned-undef.sh

+ set -e
++ dirname test/versioned-undef.sh.fail
+ cd test
++ basename -s .sh test/versioned-undef.sh.fail
+ echo -n 'Testing versioned-undef.sh.fail ... '
Testing versioned-undef.sh.fail ... ++ pwd
++ basename -s .sh test/versioned-undef.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail
+ cat
+ cc -fPIC -c -o /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/a.o -xc -
+ echo 'VER1 { local: *; }; VER2 { local: *; }; VER3 { local: *; };'
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -shared -o /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/c.so /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/a.o -Wl,--version-script=/builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/b.ver
+ cat
+ cc -c -o /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/d.o -x assembler -
+ cat
+ cc -c -o /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/e.o -xc -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -o /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/d.o /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/e.o /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/c.so
+ /builddir/mold-0.9.3-rc2/test/tmp/versioned-undef.sh.fail/exe
+ grep -q '^1 2$'

z_text.sh

+ set -e
++ dirname test/z_text.sh.fail
+ cd test
++ basename -s .sh test/z_text.sh.fail
+ echo -n 'Testing z_text.sh.fail ... '
Testing z_text.sh.fail ... ++ pwd
++ basename -s .sh test/z_text.sh.fail
+ t=/builddir/mold-0.9.3-rc2/test/tmp/z_text.sh.fail
+ mkdir -p /builddir/mold-0.9.3-rc2/test/tmp/z_text.sh.fail
+ cat
+ clang -c -o /builddir/mold-0.9.3-rc2/test/tmp/z_text.sh.fail/a.o -x assembler -
+ cat
+ clang -c -o /builddir/mold-0.9.3-rc2/test/tmp/z_text.sh.fail/b.o -xc -
++ pwd
+ clang -fuse-ld=/builddir/mold-0.9.3-rc2/test/../mold -pie -o /builddir/mold-0.9.3-rc2/test/tmp/z_text.sh.fail/exe /builddir/mold-0.9.3-rc2/test/tmp/z_text.sh.fail/a.o /builddir/mold-0.9.3-rc2/test/tmp/z_text.sh.fail/b.o
+ /builddir/mold-0.9.3-rc2/test/tmp/z_text.sh.fail/exe
+ grep -q 3

LLVM IR bitcode, or a temporary workaround

Mold is a very fast and useful linker, to me an exciting prospect; however, it isn't quite ready to replace LLVM's LLD yet in many cases, and one reason for that is that it does not support LLVM IR bitcode.

Of course, implementing full support for IR bitcode might not even be within Mold's scope. It might as well just invoke another LLVM tool to turn it into an object file in a language it can understand. Alternatively, and this is my original proposal, if it encounters an unknown file type, it will fallback to ld.lld for just that invocation (although using ld.lld to turn it into, say, a static library file, would probably be preferrable still!)

Rust program can't get command line args when linked with mold

Hello, this project is cool and I try to use it with Rust. But met a problem.

Code

fn main() {
    let me = std::env::args().next();
    println!("{:?}", me);
}

Run it with cargo run

Expected result

Some(PATH_OF_EXECUTABLE)

Actually

None

Cargo config

[target.x86_64-unknown-linux-gnu]
linker = "/usr/bin/clang"
rustflags = [
  "-C", "link-arg=-fuse-ld=/usr/bin/mold"
]

Document of rust std::env::args: https://doc.rust-lang.org/std/env/fn.args.html

Alternatives to Jemalloc

I'd like to ask if you've considered using TCMalloc and or mimalloc over Jemalloc, since you're focus on this project is speed.

The former two are generally faster than Jemalloc when threads are created and destroyed [1], with mimalloc being the shining star in all benchmarks except Redis [2].

[1] : https://stackoverflow.com/questions/7852731/c-memory-allocation-mechanism-performance-comparison-tcmalloc-vs-jemalloc#7853060
[2] : https://github.com/microsoft/mimalloc#on-24-core-amd-epyc

Consider linking tbb statically

At the moment, mold is compatible with tbb v2019 & v2020 but incompatible with tbb v2021. Similar to mimalloc, can it also be added as a sub-tree and linked in statically?

This is how I build the static libs currently (for v2020.3):

cd /path/to/oneTBB
make extra_inc=big_iron.inc tbb_build_prefix=$PWD/install

And then use the following while building mold:

make EXTRA_CPPFLAGS="-I/path/to/oneTBB/include" EXTRA_LDFLAGS="-L/path/to/oneTBB/build/install_release" ...

When mold moves to tbb v2021, one can follow oneapi-src/oneTBB#297

empty output sections need __start_ symbols too

D uses __start___minfo, __stop___minfo to mark module info section. Sometimes that section can be empty. (I don't fully understand when?) But even empty sections need __start and __stop segments. Something like this?

diff --git a/main.cc b/main.cc
index b8d2173..40d98ed 100644
--- a/main.cc
+++ b/main.cc
@@ -1259,8 +1259,7 @@ int main(int argc, char **argv) {
 
   // Add sections to the section lists
   for (OutputSection *osec : OutputSection::instances)
-    if (osec->shdr.sh_size)
-      out::chunks.push_back(osec);
+    out::chunks.push_back(osec);

Seems to work for me.

Support -r option

It looks like the last missing major feature to link most user-land program is -r or --relocatable. That option lets a linker to emit an object file (as opposed to an executable or a shared library file) by combining input object files into one.

Delegating functionality to objcopy

https://github.com/rui314/mold/blame/830a9985a887e83ca5435cd69d3bfd9189f18960/README.md#L158 suggests most things should be doable with objcopy. From what I recall / have seen, objcopy doesn't deal with rewriting dynamic symbols at all.

Once again I am ignorant about internals and feasibility, but the main reason I went down the linker script rabbit hole is that I couldn't (and still can't) figure out how to hide a dynamic symbol. In my case I wanted to hide an undefined import that shouldn't have even existed. (largely as a debugging measure, given that this suggested problems in the upstream compilation.

Handle --no-add-needed

My build of mold:

(custom.env) [user@machine user.1624664285 ]$ /tmp/mold --version
mold 0.1.1 (bd1b044afced19f7ac0b447388770494d1a62437; compatible with GNU ld and GNU gold)

My build of clang:

(custom.env) [user@machine user.1624664285 ]$ echo 'int main() {return 0;}' | clang -xc - -v
TRC clang version 8.0.1 (tags/RELEASE_801/final) (based on LLVM 8.0.1)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /spare/local/user/.conda/envs/custom.env/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.2
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.5
Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.5
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
 "/path/to/some/prefix/bin/clang-8" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /path/to/some/prefix/lib/clang/8.0.1 -internal-isystem /usr/local/include -internal-isystem /path/to/some/prefix/lib/clang/8.0.1/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /spare/scratch/user.1624664285 -ferror-limit 19 -fmessage-length 238 -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/--2789a0.o -x c - -faddrsig
clang -cc1 version 8.0.1 based upon LLVM 8.0.1 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /path/to/some/prefix/lib/clang/8.0.1/include
 /usr/include
End of search list.
 "/spare/local/user/.conda/envs/custom.env/bin/ld.lld" --hash-style=gnu --no-add-needed --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtbegin.o -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../.. -L/path/to/some/prefix/bin/../lib -L/lib -L/usr/lib /tmp/--2789a0.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtend.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crtn.o

My build of clang with mold:

(custom.env) [user@machine user.1624664285 ]$ echo 'int main() {return 0;}' | clang -xc - -fuse-ld=/tmp/mold  -v
TRC clang version 8.0.1 (tags/RELEASE_801/final) (based on LLVM 8.0.1)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /spare/local/user/.conda/envs/custom.env/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.2
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.5
Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.5
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
 "/path/to/some/prefix/bin/clang-8" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /path/to/some/prefix/lib/clang/8.0.1 -internal-isystem /usr/local/include -internal-isystem /path/to/some/prefix/lib/clang/8.0.1/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /spare/scratch/user.1624664285 -ferror-limit 19 -fmessage-length 238 -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/--0bee18.o -x c - -faddrsig
clang -cc1 version 8.0.1 based upon LLVM 8.0.1 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /path/to/some/prefix/lib/clang/8.0.1/include
 /usr/include
End of search list.
 "/tmp/mold" --hash-style=gnu --no-add-needed --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtbegin.o -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../.. -L/path/to/some/prefix/bin/../lib -L/lib -L/usr/lib /tmp/--0bee18.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtend.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crtn.o
mold: mold: unknown command line option: --no-add-needed
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)

build-static.sh fails

On 50b7497 I get this

+ set -e                                                                                                                                                                                                                                      
+ '[' -f mold ']'
+ cat                                                                                                                                                                                                                                         
+ docker build -t mold-build-ubuntu20 -                                                                                                                                                                                                       
Sending build context to Docker daemon  2.048kB                                                                                                                                                                                               
Step 1/2 : FROM ubuntu:20.04                                                                                                                                                                                                                  
 ---> 9873176a8ff5
Step 2/2 : RUN apt-get update &&   TZ=Europe/London apt-get install -y tzdata &&   apt-get install -y build-essential git clang lld cmake     libstdc++-10-dev libxxhash-dev zlib1g-dev libssl-dev &&   rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> e46d3bf61cc1
Successfully built e46d3bf61cc1
Successfully tagged mold-build-ubuntu20:latest                                                                                                                                                                                                
++ pwd                                                                                                                                                                                                                                        
++ id -u                                                                                                                                                                                                                                      
++ id -g                                                                                                                                                                                                                                      
++ nproc                                                                                                                                                                                                                                      
+ docker run -it --rm -v /home/pacak/github/mold:/mold -u 1000:1000 mold-build-ubuntu20 make -C /mold -j12 'EXTRA_LDFLAGS=-fuse-ld=lld -static'                                                                                               
make: Entering directory '/mold'                                                                                                                                                                                                              
mkdir -p oneTBB/out
(cd oneTBB/out; cmake -DBUILD_SHARED_LIBS=OFF -DTBB_TEST=OFF -DCMAKE_CXX_FLAGS=-D__TBB_DYNAMIC_LOAD_ENABLED=0 ..)                                                                                                                             
CMake Warning at CMakeLists.txt:109 (message):                                                                                                                                                                                                
  You are building oneTBB as a static library.  This is highly discouraged
  and such configuration is not supported.  Consider building a dynamic
  library to avoid unforeseen issues.


Usage: cmake -DTBB_ROOT=<tbb_root> -DTBB_OS=Linux|Windows|Darwin [-DSAVE_TO=<path>] -P tbb_config_generator.cmake
CMake Error at cmake/tbb_config_generator.cmake:21 (message):
  Required parameter TBB_ROOT is not defined
Call Stack (most recent call first):
  CMakeLists.txt:177 (include)


-- Configuring incomplete, errors occurred!
See also "/mold/oneTBB/out/CMakeFiles/CMakeOutput.log".
See also "/mold/oneTBB/out/CMakeFiles/CMakeError.log".
make: *** [Makefile:81: oneTBB/out/libs/libtbb.a] Error 1
make: Leaving directory '/mold'
elakelaiset% make clean                                                                                                                                                                                                          ~/github/mold
rm -rf *.o *~ mold mold-wrapper.so test/tmp mimalloc/out oneTBB/out ld                                                                                                                                                                        
elakelaiset% ./build-static.sh                                                                                                                                                                                                   ~/github/mold
+ set -e                                                                                                                                                                                                                                      
+ '[' -f mold ']'                                                                                                                                                                                                                             
+ cat                                                                                                                                                                                                                                         
+ docker build -t mold-build-ubuntu20 -                                                                                                                                                                                                       
Sending build context to Docker daemon  2.048kB                                                                                                                                                                                               
Step 1/2 : FROM ubuntu:20.04                                                                                                                                                                                                                  
 ---> 9873176a8ff5                                                                                                                                                                                                                            
Step 2/2 : RUN apt-get update &&   TZ=Europe/London apt-get install -y tzdata &&   apt-get install -y build-essential git clang lld cmake     libstdc++-10-dev libxxhash-dev zlib1g-dev libssl-dev &&   rm -rf /var/lib/apt/lists/*           
 ---> Using cache                                                                                                                                                                                                                             
 ---> e46d3bf61cc1
Successfully built e46d3bf61cc1                                                                                                                                                                                                               
Successfully tagged mold-build-ubuntu20:latest                                                                                                                                                                                                
++ pwd                                                                                                                                                                                                                                        
++ id -u                                                                                                                                                                                                                                      
++ id -g                                                                                                                                                                                                                                      
++ nproc                                                                                                                                                                                                                                      
+ docker run -it --rm -v /home/pacak/github/mold:/mold -u 1000:1000 mold-build-ubuntu20 make -C /mold -j12 'EXTRA_LDFLAGS=-fuse-ld=lld -static'                                                                                               
make: Entering directory '/mold'                                                                                                                                                                                                              
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o main.o main.cc                                                               
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o object_file.o object_file.cc                                                 
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o input_sections.o input_sections.cc                                           
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o output_chunks.o output_chunks.cc                                             
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o mapfile.o mapfile.cc                                                         
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o perf.o perf.cc                                                               
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o linker_script.o linker_script.cc                                             
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o archive_file.o archive_file.cc                                               
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o output_file.o output_file.cc                                                 
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o subprocess.o subprocess.cc                                                   
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o gc_sections.o gc_sections.cc                                                 
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o icf.o icf.cc                                                                 
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o symbols.o symbols.cc                                                         
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o cmdline.o cmdline.cc                                                         
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o filepath.o filepath.cc                                                       
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o passes.o passes.cc                                                           
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o tar.o tar.cc                                                                 
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o compress.o compress.cc                                                       
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o memory_mapped_file.o memory_mapped_file.cc                                   
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o relocatable.o relocatable.cc                                                 
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o arch_x86_64.o arch_x86_64.cc                                                 
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o arch_i386.o arch_i386.cc                                                     
clang++  -g -Imimalloc/include -pthread -std=c++20 -DMOLD_VERSION=\"0.9.1\" -DGIT_HASH=\"50b7497e19accedb0750d806b208a6c273a5c7c8\"  -O2 -IoneTBB/include  -c -o arch_aarch64.o arch_aarch64.cc                                               
mkdir -p mimalloc/out/release                                                                                                                                                                                                                 
(cd mimalloc/out/release; CFLAGS=-DMI_USE_ENVIRON=0 cmake ../..)                                                                                                                                                                              
-- The C compiler identification is GNU 9.3.0                                                                                                                                                                                                 
-- The CXX compiler identification is GNU 9.3.0                                                                                                                                                                                               
-- Check for working C compiler: /usr/bin/cc                                                                                                                                                                                                  
-- Check for working C compiler: /usr/bin/cc -- works                                                                                                                                                                                         
-- Detecting C compiler ABI info                                                                                                                                                                                                              
-- Detecting C compiler ABI info - done                                                                                                                                                                                                       
-- Detecting C compile features                                                                                                                                                                                                               
-- Detecting C compile features - done                                                                                                                                                                                                        
-- Check for working CXX compiler: /usr/bin/c++                                                                                                                                                                                               
-- Check for working CXX compiler: /usr/bin/c++ -- works                                                                                                                                                                                      
-- Detecting CXX compiler ABI info                                                                                                                                                                                                            
-- Detecting CXX compiler ABI info - done                                                                                                                                                                                                     
-- Detecting CXX compile features                                                                                                                                                                                                             
-- Detecting CXX compile features - done                                                                                                                                                                                                      
-- No build type selected, default to: Release                                                                                                                                                                                                
-- Override standard malloc (MI_OVERRIDE=ON)                                                                                                                                                                                                  
--                                                                                                                                                                                                                                            
-- Library base name: mimalloc                                                                                                                                                                                                                
-- Build type       : release                                                                                                                                                                                                                 
-- Compiler         : /usr/bin/cc                                                                                                                                                                                                             
-- Install directory: /usr/local/lib/mimalloc-1.7                                                                                                                                                                                             
-- Build targets    : shared;static;object;tests                                                                                                                                                                                              
--                                                                                                                                                                                                                                            
-- Configuring done                                                                                                                                                                                                                           
-- Generating done                                                                                                                                                                                                                            
-- Build files have been written to: /mold/mimalloc/out/release                                                                                                                                                                               
make -C mimalloc/out/release mimalloc-static                                                                                                                                                                                                  
make[1]: Entering directory '/mold/mimalloc/out/release'                                                                                                                                                                                      
make[2]: Entering directory '/mold/mimalloc/out/release'                                                                                                                                                                                      
make[3]: Entering directory '/mold/mimalloc/out/release'                                                                                                                                                                                      
make[4]: Entering directory '/mold/mimalloc/out/release'                                                                                                                                                                                      
Scanning dependencies of target mimalloc-static                                                                                                                                                                                               
make[4]: Leaving directory '/mold/mimalloc/out/release'                                                                                                                                                                                       
make[4]: Entering directory '/mold/mimalloc/out/release'                                                                                                                                                                                      
[  6%] Building C object CMakeFiles/mimalloc-static.dir/src/stats.c.o                                                                                                                                                                         
mkdir -p oneTBB/out                                                                                                                                                                                                                           
(cd oneTBB/out; cmake -DBUILD_SHARED_LIBS=OFF -DTBB_TEST=OFF -DCMAKE_CXX_FLAGS=-D__TBB_DYNAMIC_LOAD_ENABLED=0 ..)                                                                                                                             
-- The CXX compiler identification is GNU 9.3.0                                                                                                                                                                                               
-- Check for working CXX compiler: /usr/bin/c++                                                                                                                                                                                               
[ 13%] Building C object CMakeFiles/mimalloc-static.dir/src/random.c.o                                                                                                                                                                        
-- Check for working CXX compiler: /usr/bin/c++ -- works                                                                                                                                                                                      
-- Detecting CXX compiler ABI info                                                                                                                                                                                                            
-- Detecting CXX compiler ABI info - done                                                                                                                                                                                                     
-- Detecting CXX compile features                                                                                                                                                                                                             
-- Detecting CXX compile features - done                                                                                                                                                                                                      
CMake Warning at CMakeLists.txt:109 (message):                                                                                                                                                                                                
  You are building oneTBB as a static library.  This is highly discouraged                                                                                                                                                                    
  and such configuration is not supported.  Consider building a dynamic                                                                                                                                                                       
  library to avoid unforeseen issues.                                                                                                                                                                                                         
                                                                                                                                                                                                                                              
                                                                                                                                                                                                                                              
-- CMAKE_BUILD_TYPE is not specified. Using default: RelWithDebInfo                                                                                                                                                                           
-- Looking for C++ include pthread.h                                                                                                                                                                                                          
[ 20%] Building C object CMakeFiles/mimalloc-static.dir/src/os.c.o                                                                                                                                                                            
clang -fPIC -shared -o mold-wrapper.so mold-wrapper.c -ldl                                                                                                                                                                                    
-- Looking for C++ include pthread.h - found                                                                                                                                                                                                  
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD                                                                                                                                                                                                    
[ 26%] Building C object CMakeFiles/mimalloc-static.dir/src/bitmap.c.o                                                                                                                                                                        
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed                                                                                                                                                                                           
-- Check if compiler accepts -pthread                                                                                                                                                                                                         
[ 33%] Building C object CMakeFiles/mimalloc-static.dir/src/arena.c.o                                                                                                                                                                         
[ 40%] Building C object CMakeFiles/mimalloc-static.dir/src/segment.c.o                                                                                                                                                                       
[ 46%] Building C object CMakeFiles/mimalloc-static.dir/src/region.c.o
-- Check if compiler accepts -pthread - yes                                                                                                                                                                                                   
-- Found Threads: TRUE                                                                                                                                                                                                                        
Usage: cmake -DTBB_ROOT=<tbb_root> -DTBB_OS=Linux|Windows|Darwin [-DSAVE_TO=<path>] -P tbb_config_generator.cmake                                                                                                                             
CMake Error at cmake/tbb_config_generator.cmake:21 (message):                                                                                                                                                                                 
  Required parameter TBB_ROOT is not defined                                                                                                                                                                                                  
Call Stack (most recent call first):                                                                                                                                                                                                          
  CMakeLists.txt:177 (include)                                                                                                                                                                                                                
                                                                                                                                                                                                                                              
                                                                                                                                                                                                                                              
-- Configuring incomplete, errors occurred!                                                                                                                                                                                                   
See also "/mold/oneTBB/out/CMakeFiles/CMakeOutput.log".                                                                                                                                                                                       
See also "/mold/oneTBB/out/CMakeFiles/CMakeError.log".                                                                                                                                                                                        
make: *** [Makefile:81: oneTBB/out/libs/libtbb.a] Error 1                                                                                                                                                                                     
make: *** Waiting for unfinished jobs....                                                                                                                                                                                                     
[ 53%] Building C object CMakeFiles/mimalloc-static.dir/src/page.c.o                                                                                                                                                                          
[ 60%] Building C object CMakeFiles/mimalloc-static.dir/src/alloc.c.o                                                                                                                                                                         
[ 66%] Building C object CMakeFiles/mimalloc-static.dir/src/alloc-aligned.c.o                                                                                                                                                                 
[ 73%] Building C object CMakeFiles/mimalloc-static.dir/src/alloc-posix.c.o                                                                                                                                                                   
[ 80%] Building C object CMakeFiles/mimalloc-static.dir/src/heap.c.o                                                                                                                                                                          
[ 86%] Building C object CMakeFiles/mimalloc-static.dir/src/options.c.o                                                                                                                                                                       
[ 93%] Building C object CMakeFiles/mimalloc-static.dir/src/init.c.o                                                                                                                                                                          
[100%] Linking C static library libmimalloc.a                                                                                                                                                                                                 
make[4]: Leaving directory '/mold/mimalloc/out/release'                                                                                                                                                                                       
[100%] Built target mimalloc-static                                                                                                                                                                                                           
make[3]: Leaving directory '/mold/mimalloc/out/release'                                                                                                                                                                                       
make[2]: Leaving directory '/mold/mimalloc/out/release'                                                                                                                                                                                       
make[1]: Leaving directory '/mold/mimalloc/out/release'                                                                                                                                                                                       
                                                                                                                                                                                                                                              
make: Leaving directory '/mold'

Documentation plans?

This is related to issue #2,

Do you have any plans for documentation? (perhaps once the project is less in flux?)
An Issue I have often is that linker man pages are unclear or lacking. (once again, the fault may just be with me.)

It would be nice to have a well architected linker with good documentation. :)

Support nix by not calling `git clone` during build

Hi, I would like to build mold with the nix package manager, but nix does not support network access while building, so the git clone of mimalloc in the Makefile fails.

I suggest moving the build of mimalloc out of the Makefile, like you did for TBB, to give the user control over how mimalloc is built and how to link to it. By default, the Makefile can link to mimalloc dynamically with -lmimalloc. The build-static.sh script can then clone mimalloc, build it, and pass in the appropriate EXTRA_CPPFLAGS and EXTRA_LDFLAGS like it does for TBB.

I don't know why you chose to always statically link mimalloc, but I assume it is either because mimalloc is not packaged in ubuntu or because static linking may have better performance.

If you approve of this approach, I can submit a PR.

Linking rust code results in `library not found: gcc`

Hello! I'm trying to use mold to link some rust code.

I tried the following in my ~/.cargo/config:

[target.x86_64-unknown-linux-gnu]
rustflags = [
    "-C", "linker-flavor=ld",
]
linker = "mold"

This runs the following mold command (censored a bit, hope that isn't a problem):

= note: "mold" "--eh-frame-hdr" "-L" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "<my dir>/<my project>/target/debug/deps/<my project>-53526ca339f51915.<my project>.49v4udum-cgu.0.rcgu.o" "<my dir>/<my project>/target/debug/deps/<my project>-53526ca339f51915.<my project>.49v4udum-cgu.1.rcgu.o" "<my dir>/<my project>/target/debug/deps/<my project>-53526ca339f51915.<my project>.49v4udum-cgu.2.rcgu.o" "<my dir>/<my project>/target/debug/deps/<my project>-53526ca339f51915.<my project>.49v4udum-cgu.3.rcgu.o" "<my dir>/<my project>/target/debug/deps/<my project>-53526ca339f51915.<my project>.49v4udum-cgu.4.rcgu.o" "<my dir>/<my project>/target/debug/deps/<my project>-53526ca339f51915.<my project>.49v4udum-cgu.5.rcgu.o" "<my dir>/<my project>/target/debug/deps/<my project>-53526ca339f51915.<my project>.49v4udum-cgu.6.rcgu.o" "-o" "<my dir>/<my project>/target/debug/deps/<my project>-53526ca339f51915" "<my dir>/<my project>/target/debug/deps/<my project>-53526ca339f51915.bup2hku2a0i3608.rcgu.o" "--gc-sections" "-pie" "-zrelro" "-znow" "-L" "<my dir>/<my project>/target/debug/deps" "-L" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--start-group" "-Bstatic" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-0a9489cf400f65e4.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-ff5dc44c66f8c479.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-3317c66a83501f9c.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-5d8dec11fc25537d.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-3af2a9328550e2a6.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-3092e2ecef0f49f1.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-29b776c021389465.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-e8873bd287db0d28.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-1171b49d77e47426.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-c29894d22dc88b51.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-796a7750df3d8218.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-ff7772d803d3e0de.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-dae3eac9cfa44200.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-0fa02f580e987af5.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-7f7254233be843ed.rlib" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-1395c6db3d116086.rlib" "--end-group" "<my dir>/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-b4bd87926720b651.rlib" "-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc"

This fails with:

  = note: mold: library not found: gcc_s

We need to link with /usr/lib/libgcc_s.so, so that can be done with:

[target.x86_64-unknown-linux-gnu]
rustflags = [
    "-C", "linker-flavor=ld",
    "-C", "link-arg=-L/usr/lib",
]
linker = "mold"

And then the linking fails with:

  = note: mold: library not found: gcc

Please cut a new release to simplify packaging

My #62, #64 PRs has been accepted, but they didn't end up in any release.
Most distros have a policy about using the most recent release, and not master/main.
So please make a new release 0.9.2.

Building tagged release tarball fails when attempting to build mimalloc

I tried building the 0.9 release tarball on Arch Linux following the build steps in the README:

Download https://github.com/rui314/mold/archive/refs/tags/v0.9.tar.gz and extract
cd mold-0.9
make

Make fails when trying to build mimalloc. It seems the cmake ../.. step of

(cd mimalloc/out/release; CFLAGS=-DMI_USE_ENVIRON=0 cmake ../..)

from the Makefile is trying to find mimalloc/out/release/libmimalloc.a before its been built (which is the very next line in the Makefile). Errors:

mkdir -p mimalloc/out/release
clang -fPIC -shared -o mold-wrapper.so mold-wrapper.c -ldl
(cd mimalloc/out/release; CFLAGS=-DMI_USE_ENVIRON=0 cmake ../..)
-- The C compiler identification is GNU 11.1.0
-- The CXX compiler identification is GNU 11.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc - broken
CMake Error at /usr/share/cmake-3.20/Modules/CMakeTestCCompiler.cmake:66 (message):
  The C compiler

    "/usr/bin/cc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /home/wmoore/aur/mold-git/src/mold-0.9/mimalloc/out/release/CMakeFiles/CMakeTmp

    Run Build Command(s):/usr/bin/make -f Makefile cmTC_66e43/fast && make[1]: Entering directory '/home/wmoore/aur/mold-git/src/mold-0.9/mimalloc/out/release/CMakeFiles/CMakeTmp'
    /usr/bin/make  -f CMakeFiles/cmTC_66e43.dir/build.make CMakeFiles/cmTC_66e43.dir/build
    make[2]: Entering directory '/home/wmoore/aur/mold-git/src/mold-0.9/mimalloc/out/release/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_66e43.dir/testCCompiler.c.o
    /usr/bin/cc   -DMI_USE_ENVIRON=0  -o CMakeFiles/cmTC_66e43.dir/testCCompiler.c.o -c /home/wmoore/aur/mold-git/src/mold-0.9/mimalloc/out/release/CMakeFiles/CMakeTmp/testCCompiler.c
    Linking C executable cmTC_66e43
    /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_66e43.dir/link.txt --verbose=1
    /usr/bin/cc -DMI_USE_ENVIRON=0  -Wl,-whole-archive mimalloc/out/release/libmimalloc.a -Wl,-no-whole-archive  -rdynamic CMakeFiles/cmTC_66e43.dir/testCCompiler.c.o -o cmTC_66e43
    /usr/bin/ld: cannot find mimalloc/out/release/libmimalloc.a: No such file or directory
    collect2: error: ld returned 1 exit status
    make[2]: *** [CMakeFiles/cmTC_66e43.dir/build.make:99: cmTC_66e43] Error 1
    make[2]: Leaving directory '/home/wmoore/aur/mold-git/src/mold-0.9/mimalloc/out/release/CMakeFiles/CMakeTmp'
    make[1]: *** [Makefile:127: cmTC_66e43/fast] Error 2
    make[1]: Leaving directory '/home/wmoore/aur/mold-git/src/mold-0.9/mimalloc/out/release/CMakeFiles/CMakeTmp'





  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)


-- Configuring incomplete, errors occurred!
See also "/home/wmoore/aur/mold-git/src/mold-0.9/mimalloc/out/release/CMakeFiles/CMakeOutput.log".
See also "/home/wmoore/aur/mold-git/src/mold-0.9/mimalloc/out/release/CMakeFiles/CMakeError.log".
make: *** [Makefile:64: mimalloc/out/release/libmimalloc.a] Error 1
make: *** Waiting for unfinished jobs....

Tag a release

Please consider adding a git tag per release (and also adding a license).

It makes mold easier to package for Linux distros, Homebrew and other systems.

Thanks!

Version number in Makefile needs a bump to 0.9

Debugging facilities for the linking process

(Possibly related to #4)

Linking issues seem generally to be hard to debug?

I don't have a lot in terms of concrete proposals here but for starters, getting more information out of a linker, about what it's doing, would be a start? Ideally perhaps even machine readable information which could be consumed by analysis frontends.
There seems to be a lot of space for improvement here?

If I have a linker problem, I can't seem to do anything right now other than poke around in the object files with objdump and nm.

Using mold with Rust causes std::length_error

First of all, thanks for your work on mold; it looks promising!

When using mold with Cargo and Rust, it crashes with a std::length_error:

Output log

error: linking with `/usr/local/bin/mold` failed: signal: 6
  |
  = note: "/usr/local/bin/mold" "--eh-frame-hdr" "-L" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/caelum/dev/testbed/target/debug/deps/compress-0d51133de7c305fa.19rihm3aa63nqzaw.rcgu.o" "/home/caelum/dev/testbed/target/debug/deps/compress-0d51133de7c305fa.1hmk8zzpkw7zpvgp.rcgu.o" "/home/caelum/dev/testbed/target/debug/deps/compress-0d51133de7c305fa.1mlkfdqyrhjxbcu2.rcgu.o" "/home/caelum/dev/testbed/target/debug/deps/compress-0d51133de7c305fa.29eq5g97bv1rdv32.rcgu.o" "/home/caelum/dev/testbed/target/debug/deps/compress-0d51133de7c305fa.380fen02zzk4uzxm.rcgu.o" "/home/caelum/dev/testbed/target/debug/deps/compress-0d51133de7c305fa.dhul26qa21e8n3v.rcgu.o" "/home/caelum/dev/testbed/target/debug/deps/compress-0d51133de7c305fa.udyfmmt9fy88lgk.rcgu.o" "-o" "/home/caelum/dev/testbed/target/debug/deps/compress-0d51133de7c305fa" "/home/caelum/dev/testbed/target/debug/deps/compress-0d51133de7c305fa.3tf5i5yky9ydfot9.rcgu.o" "--gc-sections" "-pie" "-zrelro" "-znow" "-L" "/home/caelum/dev/testbed/target/debug/deps" "-L" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--start-group" "-Bstatic" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-e3026a7ea720d3a3.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-dea899c54966188d.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-dda4c0b69607e93b.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-4b7dae8949ac132c.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-978e97832b309706.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-073b1b693304b876.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-c07f996a53ee6558.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-0ae8ed6a282247d0.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-72f6aee6e444f535.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-14b94bdd9a47d665.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-97d562419076c156.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-022f1a0e7cd794ec.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-6c8051b8141a3b3d.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-3aeb407930ebd519.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-6ab1ee6dbc17ad08.rlib" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-166dae07beec0398.rlib" "--end-group" "/home/caelum/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-761b290f47712921.rlib" "-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-fuse-ld=lld"
  = note: terminate called after throwing an instance of 'std::length_error'
            what():  basic_string::_M_create

The steps to reproduce are:

Create a new, empty Cargo project (cargo new <directory>)
Set the linker to mold by adding this to .cargo/config.toml:

[target.x86_64-unknown-linux-gnu]
rustflags = [
    "-C", "linker-flavor=ld",
]
linker = "/path/to/mold"

Build with cargo build

Resources / Literature review?

i'm not well versed in linkers, and I'm having a hard time learning more about them. Pardon me if I commit some faux pas in the next few issues.

I'm familiar with the Linkers and loaders book by John R. Levine, but that's all the domain specific material I know.
This book is 20 years old at this point, surely there have been developments in linkers since then?
From what little I've looked at linker manual pages, they've been unclear at best. (but perhaps the fault lies in me.)

Do you have any (other) recommended reading material?
Are you perhaps able to release the bibliography for your thesis early, if you have anything meaningful there? - or do you already have everything you need in your head and citations are just going to be a formality, if you even need any?

Do you have any other related advice, or is the only way to really figure this stuff out currently, to write your own linker, or at least get neck deep in the code?

It's always nice to have high quality notes about design.

TODO:

Do some more of my own literature search, maybe trying to find things that cite the L&L book.

rust-analyzer fails to expand proc-macro when mold linker is used

I'm using stable rustc, most recent rust-analyzer and static build of mold as of a week ago or so.

config.toml:

[build]
rustflags = "-C target-feature=+fma,+avx2"

[target.x86_64-unknown-linux-gnu]
linker = "/usr/bin/clang"
rustflags = ["-Clink-arg=-fuse-ld=/path/to/mold"]

When deriving via proc macro is used (here serde, but any other proc macro fails

use serde::Serialize;

#[derive(Debug, Serialize)]
pub enum Distance {
    Long,
    Short,
}

rust analyzer fails to expand the macro with an error like this:
(this error is most likely about some different proc macro)

proc-macro /path/target/debug/deps/libtracing_attributes-7c1265016b85e087.so failed to find the given version. Reason: Invalid ELF section header offset/size/alignment                                                       
Failed to find proc macros. Error: ExpansionError(                                                                                                                                                                                            
    "Cannot create expander for /path/target/debug/deps/libtracing_attributes-7c1265016b85e087.so: Custom { kind: InvalidData, error: Error(\"Invalid ELF section header offset/size/alignment\") }",
)

You can invoke rust analyzer like this

rust-analyzer analysis-stats .

The same code works as expected when compiled in a usual way

$ cat a.cpp
#include <filesystem>

int main() {
  auto cwd = std::filesystem::current_path();
  return static_cast<int>(cwd.string().size());
}

My clang version:

$ clang++ --version
TRC clang version 8.0.1 (tags/RELEASE_801/final) (based on LLVM 8.0.1)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /path/to/prefix/bin

My mold version:

$ /tmp/m/bin/mold --version
mold 0.9.2 (6695c49379d6d053ea37ba716bf94acb5c4767aa; compatible with GNU ld and GNU gold)

It doesn't compile 😭

$ clang++ a.cpp -std=c++17 -fuse-ld=/tmp/m/bin/mold -stdlib=libc++ -lc++fs
mold: undefined symbol: /tmp/a-d457e4.o: std::__1::__fs::filesystem::__current_path(std::__1::error_code*)
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)

Perhaps it was using the wrong libc++fs? Try full path? 😭

$ clang++ a.cpp -std=c++17 -fuse-ld=/tmp/m/bin/mold -stdlib=libc++ /path/to/prefix/lib/libc++fs.a
mold: undefined symbol: /tmp/a-d76e7d.o: std::__1::__fs::filesystem::__current_path(std::__1::error_code*)
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)

Let's just use the object files directly! 😭

$ ar x /path/to/prefix/lib/libc++fs.a

$ clang++ a.cpp -std=c++17 -fuse-ld=/tmp/m/bin/mold directory_iterator.cpp.o int128_builtins.cpp.o operations.cpp.o
mold: directory_iterator.cpp.o: looks like this is an LLVM bitcode, but mold does not support LTO
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)

Perhaps this has something to do with the fact that I build LLVM libraries with -DLLVM_ENABLE_LTO=ON and there is no shared library for c++fs.

As an addendum: Even gold and bfd error on this one, but lld is happy 😃

	while (!data.empty()) {
	size_t end = find_null(data, entsize);
	if (end == std::string_view::npos)
	Error() << *this << ": string is not null terminated";

	std::string_view substr = data.substr(0, end + entsize);
	data = data.substr(end + entsize);

	SectionFragment *frag = parent.insert(substr, isec->shdr.sh_addralign);
	fragments.push_back(frag);
	frag_offsets.push_back(substr.data() - begin);
	}