opencilk / cheetah

Source for the OpenCilk runtime system, based on Cheetah.

Building a standalone copy of the OpenCilk runtime

These instructions assume that you are building the OpenCilk runtime system using the OpenCilk compiler.

Using Makefiles

1. If necessary, update the COMPILER_BASE variable in config.mk to point to the directory containing the OpenCilk compiler binaries, e.g., /path/to/opencilk-project/build/bin/. When it executes clang and other OpenCilk compiler binaries, the Makefile prepends this path to those binaries.
2. Run make.

To clean the build, run make clean.

Using CMake

1. Make a build directory at the top level and enter it:

$ mkdir build
$ cd build

2. Configure CMake. Make sure to specify CMAKE_C_COMPILER and LLVM_CMAKE_DIR to point to the corresponding build or installation of the OpenCilk compiler binaries. In addition, set CMAKE_BUILD_TYPE to specify the build type, such as, Debug, for an unoptimized build with all assertions enabled; Release, for an fully optimized build with assertions disabled; or RelWithDebInfo, to enable some optimizations and assertions. (The default build type is Debug.)

Example configuration:

$ cmake -DCMAKE_C_COMPILER=/path/to/opencilk-project/build/bin/clang -DCMAKE_BUILD_TYPE=Release -DLLVM_CMAKE_DIR=/path/to/opencilk-project/build ../

3. Build the runtime:

$ cmake --build . -- -j<number of build threads>

To clean the build, run cmake --build . --target clean from the build directory.

Linking against a standalone build of the OpenCilk runtime

The OpenCilk compiler accepts the flag --opencilk-resource-dir=/path/to/cheetah to specify where to find all relevant OpenCilk runtime files, including the runtime library, the bitcode ABI file, and associated header files. This resource directory should have include/ and lib/<target triple> as subdirectories. For example, if you built the standalone OpenCilk runtime using CMake, then pass the flag --opencilk-resource-dir=/path/to/cheetah/build to the OpenCilk compiler to link against that standalone build, e.g.,

/path/to/opencilk-project/build/bin/clang -o fib fib.c -fopencilk -O3 --opencilk-resource-dir=/path/to/cheetah/build

cheetah's Issues

Worker local storage

We have a user request for a function to get the worker number, usable as an array index, but it's really a request for worker local storage.

get_workerwarn_msg breaks the linker

I ran into the following error message:

/usr/bin/ld.gold: error: hidden symbol 'get_workerwarn_msg' is not defined locally

After removing lines https://github.com/OpenCilk/cheetah/blob/release/runtime/cilk2c.c#L56-L61, the error went away and I was able to compile without issues.

Build fail on cmake

I am trying to build the cheetah runtime and on the cmake --build . step I get the following log:

CMakeFiles/cheetah_shared.dir/closure.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.h:8: multiple definition of `cilkrts_zero'
CMakeFiles/cheetah_shared.dir/cilk2c.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.h:8: first defined here
CMakeFiles/cheetah_shared.dir/fiber.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/init.h:12: multiple definition of `cilkg_nproc'
CMakeFiles/cheetah_shared.dir/cilk2c.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.c:17: first defined here
CMakeFiles/cheetah_shared.dir/global.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/init.h:12: multiple definition of `cilkg_nproc'
CMakeFiles/cheetah_shared.dir/cilk2c.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.c:17: first defined here
CMakeFiles/cheetah_shared.dir/init.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/init.h:12: multiple definition of `cilkg_nproc'
CMakeFiles/cheetah_shared.dir/cilk2c.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.c:17: first defined here
CMakeFiles/cheetah_shared.dir/invoke-main.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/init.h:12: multiple definition of `cilkg_nproc'
CMakeFiles/cheetah_shared.dir/cilk2c.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.c:17: first defined here
CMakeFiles/cheetah_shared.dir/invoke-main.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.h:8: multiple definition of `cilkrts_zero'
CMakeFiles/cheetah_shared.dir/cilk2c.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.h:8: first defined here
CMakeFiles/cheetah_shared.dir/personality.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.h:8: multiple definition of `cilkrts_zero'
CMakeFiles/cheetah_shared.dir/cilk2c.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.h:8: first defined here
CMakeFiles/cheetah_shared.dir/reducer_impl.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/init.h:12: multiple definition of `cilkg_nproc'
CMakeFiles/cheetah_shared.dir/cilk2c.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.c:17: first defined here
CMakeFiles/cheetah_shared.dir/ZERO.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.h:8: multiple definition of `cilkrts_zero'
CMakeFiles/cheetah_shared.dir/cilk2c.c.o:/home/giannis/cilk/opencilk-project/cheetah/runtime/cilk2c.h:8: first defined here
collect2: error: ld returned 1 exit status
gmake[2]: *** [runtime/CMakeFiles/cheetah_shared.dir/build.make:354: lib64/libopencilk.so.1.0] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:323: runtime/CMakeFiles/cheetah_shared.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....

Any ideas on how to fix this?

`dummy(alloca(ZERO))` in hand-compiled tests

Something I've noticed happens in all hand-compiled tests is the following line, right before the definition of a __cilkrts_stack_frame:

dummy(alloca(ZERO));

Why is this required? I'm guessing it's maintaining some invariant of the stack pointer, but I'm not sure why exactly. I'm trying to understand it as I work on my own hand-compiled code for the loop frame code.

Race condition in parallel build

Using make, the first parallel build of OpenCilk often fails because of a missing dependency:

ld: error: cannot open /usr/obj/Cilk/17/lib/clang/17/lib/freebsd/libclang_rt.asan_static-x86_64.a: No such file or directory
*** [/usr/obj/Cilk/17/lib/clang/17/lib/x86_64-unknown-freebsd13.2/libopencilk-asan-personality-c.so.1.0] Error code 1

This might happen more than once if all builds are parallel. Running make -k builds libclang_rt.asan_static and the next build will succeed. A serial build, make -j1, will also succeed.

My CmakeCache.txt says the runtime components appear in the order compiler-rt;cheetah;cilktools.

Cilk: stack mmap fails on WSL Linux (due to unsupported mmap operation using MAP_GROWSDOWN flag)

Minor issue, but just wanted to report this in case anyone else runs into similar problems.
I am unable to initialize the runtime when building/running using WSL (WIndows Subsystem for Linux).
Functional workaround seems to be to remove MAP_GROWSDOWN from the flags to mmap. I did not attempt to do any performance testing or any significant correctness tests however.

Details:

Basic programs (fib) fail with errno code 95, which is an unsupported operation, at the following line that calls mmap.

cheetah/runtime/fiber.c

Line 210 in 832aeeb

MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK | MAP_GROWSDOWN, -1, 0);

The MAP_GROWSDOWN flag seems to be unsupported on my test machine.
Building OpenCilk using WSL2 with Linux version Ubuntu 20.04.3 LTS. (Windows 10 build, 19043.1466)

I couldn't find any obvious documentation to indicate whether this is expected behavior for WSL Linux or not, so it might be a gap in the WSL implementation? TBD.