projectmitosisos / mitosis-core

An OS kernel module for fast **remote** fork using advanced datacenter networking (RDMA).

MITOSIS: An OS primitive of fast remote fork

Mitosis is a kernel module that provides a new system primitive of fast remote fork based on RDMA.

[toc]

Getting Started Instructions

Prerequisite

1. Software

• OS: Ubuntu16.04 (throughly tested, in general is irrelevant to the OS)
• Linux kernel: 4.15.0-46-generic (porting needed to fit other OSes)
• MLNX_OFED driver: 4.9-3.1.5.0 (throughly, use our modified driver in case to support DCT)
• Rustc: 1.60.0-nightly (71226d717 2022-02-04)
• Clang-9

2. Hardware

• A machine with Mellanox RDMA-capable IB NIC (later than or equal to ConnectX-4).
  • In principle there is no difficult in supporting RoCE, but we have lack such NIC for testing. We welcome testing and porting on other RNICs.
• X86-64 servers

Please refer to the document here for how to configure software environments.

---

Compile the mitosis

Assumptions: we have finished installing the software dependencies described in the Prerequisite.

make km ## building the kernel module
file mitosis-kms/fork.ko
# mitosis-kms/fork.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), BuildID[sha1]=xxx, not stripped

Mitosis has different configurations, including:

- Prefetch: Read ahead some pages with RDMA
- Page cache: Cache some pages locally instead of read through RDMA
- COW: Use Copy-On-Write instead of directly copying page content
- Eager resume: Read all the pages during the startup
- Profile: Print performance profile during the execution
- Use rc: Use RC to get remote pages instead of using DCT

These configurations are specified in the mitosis-kms/Kbuild file with Rust features. Without further explanation, we will use the default configuration "COW+Prefetch". If you want to use other configurations, you can copy the Kbuild file before the compilation.

ls mitosis-kms/Kbuild* # will show the available Kbuild configurations
cp mitosis-kms/Kbuild-mitosis-prefetch mitosis-kms/Kbuild

Example

We have provided a simple demo on how to use the kernel module to remote fork a process.

1. Choose two machines, one as the parent machine and one as the child machine. Get the gid (RDMA address) of the parent machine.

show_gids
# DEV     PORT    INDEX   GID                                     IPv4            VER     DEV
# ---     ----    -----   ---                                     ------------    ---     ---
# mlx5_0  1       0       fe80:0000:0000:0000:ec0d:9a03:00ca:2f4c                 v1
# mlx5_1  1       0       fe80:0000:0000:0000:ec0d:9a03:0078:6376                 v1
# n_gids_found=2

Mitosis uses the first RDMA nic by default, so we will use the gid fe80:0000:0000:0000:ec0d:9a03:00ca:2f4c here.

2. Prepare the demo C++ programs on both machines.

cd exp
cmake .
make connector `## the utility program to connect mitosis kernel-space rpc, source code: exp/common/connector.cc` \
     simple_parent `## a simple parent program, which prints a number every second, source code: exp/common/simple_parent.cc` \
     simple_child `## a simple child program, which is to fork the parent program from a remote machine, source code: exp/common/simple_child.cc`

// an excerpt of exp/common/simple_parent.cc
int
main(int argc, char *argv[]) {
    // omitted
    int sd = sopen(); // open the mitosis device in /dev
    int cnt = 0;
    assert(sd != 0);

    sleep(1);
    printf("time %d\n", cnt++);
    fork_prepare(sd, FLAGS_handler_id); // call the fork_prepare with id here
    // the child program will resume the execution from this point

    while (1) {
        printf("time %d\n", cnt++);
        sleep(1);
    }
}

// an excerpt of exp/common/simple_child.cc
int
main(int argc, char *argv[]) {
    // omitted
    int sd = sopen();
    assert(sd != 0);
    // the child program call the fork_resume to resume the execution from the parent program with `handler_id` on machine `mac_id`
    fork_resume_remote(sd, FLAGS_mac_id, FLAGS_handler_id);
    assert(false); // we should never reach this point
    return 0;
}

3. Compile and insert the kernel module on both machines.

# at each machine, run the following command: 
make km && make insmod
file /dev/mitosis-syscalls
# /dev/mitosis-syscalls: setuid, setgid, sticky, character special (238/0)

4. Run the connector on the child machine to let the child machine to connect to the parent machine.

cd exp
./connector -gid="fe80:0000:0000:0000:ec0d:9a03:00ca:2f4c" -mac_id=0 -nic_id=0 # this gid is marked as the nic 0 on the machine 0 on the child machine, and we will send connect request to it

5. Run the parent program on the parent machine.

cd exp
./simple_parent -pin=false -handler_id=73 # the parent program will print an increasing counter from 0 repeatedly
# the remote fork identification for the parent program is 73 and we choose to leave the program in the foreground and do not pin it in the kernel

6. Run the client program on the client machine

cd exp
./simple_child -mac_id=0 -handler_id=73 # the child will start printing the counter from 1 as if it has forked the parent program on machine 0 (val01) with id 73 from the point before it starts print the counter 1

7. Use Ctrl+C to kill the parent and child and use make rmmod to uninstall the kernel module.

Animated Example of Remote Fork

The example below is to illustrate how to fork the parent process at machine val01 to machine val02.

Note: We will try to remove the kernel module before we insert it, so as to avoid double insertion of the module. This will cause error like "Module is not loaded" when the module is not inserted before. This error is OK.

Testing and Benchmarking

We have provided unit tests, stress tests, and benchmarks for mitosis. Please refer to the documents here.

Roadmap

MITOSIS is still under development and code refactory, which current codebase has the following limitations:

1. We don't support multi-threading program.
2. We only support child communicating with the server via RDMA DCT.
3. We don't support fallback handler,which is still under refactor.
4. We assume the swap is disabled on the parent machine.

Detailed roadmap:

• Supporting languages with GC and multi-threading
• Fallback handler to support unmapped page
• Add Reliable connection, RPC and TCP as an alternative to RDMA DCT-based network communications
• Other unfinished features/code refinement
  • RPC disconnection and elegant error handling in session creation
  • Doorbell optimization in prefetcher module

Contribution

Want to contribute to mitosis? We have some unfinished designs and implementations. Please refer to the documents here.

Related Projects

• KRCORE is a rust RDMA library for user-space and kernel-space applications.

License

This project is licensed under the MIT license.

Credits

Some code is insipired by (or directly borrowed) from

• nix
• tokio

We have to borrow some code because they don't support kernel space.

mitosis-core's Issues

Some Rust dependency issues prevent it from compiling

Hi, really excellent work! The research we were doing wanted to perform a performance comparison with mitosis, but we found that it had some Rust dependency issues that prevented it from compiling.

My environment:
Ubuntu 18.04
4.15.0-46-generic
MLNX_OFED_LINUX-4.9-3.1.5.0

After I completed all the instructions in mitosis-core/docs/setup.md. When I use rustc of nightly-2022-02-04-x86_64-unknown-linux-gnu, I get the following results:

root@liu-9:~/mitosis-core# make clean & make km
[1] 88763
rm -rf mitosis-kms/Cargo.lock
cd mitosis-kms ; python build.py fork
rm -rf mitosis-kms/target

• [fork]
• [RUNNING] ['make', '-C', '/root/mitosis-core/mitosis-kms', 'TEST_NAME=fork', 'TEST_PATH=fork']
  make[1]: Entering directory '/root/mitosis-core/mitosis-kms'
  cp -f /usr/src/ofa_kernel/default/Module*.symvers /root/mitosis-core/mitosis-kms/Module.symvers
  make -C /lib/modules/4.15.0-46-generic/build M=/root/mitosis-core/mitosis-kms CC=clang-9 CONFIG_CC_IS_CLANG=y
  make[2]: Entering directory '/usr/src/linux-headers-4.15.0-46-generic'
  cd /root/mitosis-core/mitosis-kms/fork; CARGO_TARGET_DIR=../target cargo build -Z build-std=core,alloc --target=x86_64-unknown-none-linuxkernel --features "mitosis krdma-test cow use_rc" --no-default-features
  warning: /root/mitosis-core/mitosis/Cargo.toml: dependency (x86_64) specified without providing a local path, Git repository, or version to use. This will be considered an error in future versions
  Updating crates.io index
  ...
  Downloaded 13 crates (2.0 MB) in 2.27s
  error: package regex-syntax v0.8.2 cannot be built because it requires rustc 1.65 or newer, while the currently active rustc version is 1.60.0-nightly
  /root/mitosis-core/mitosis-kms/Kbuild:11: recipe for target '/root/mitosis-core/mitosis-kms/target/x86_64-unknown-none-linuxkernel/debug/libfork.a' failed
  make[3]: *** [/root/mitosis-core/mitosis-kms/target/x86_64-unknown-none-linuxkernel/debug/libfork.a] Error 101
  Makefile:1551: recipe for target 'module/root/mitosis-core/mitosis-kms' failed
  make[2]: *** [module/root/mitosis-core/mitosis-kms] Error 2
  make[2]: Leaving directory '/usr/src/linux-headers-4.15.0-46-generic'
  Makefile:38: recipe for target 'all' failed
  make[1]: *** [all] Error 2
  make[1]: Leaving directory '/root/mitosis-core/mitosis-kms'
  Traceback (most recent call last):
  File "build.py", line 36, in
  main(sys.argv)
  File "build.py", line 31, in main
  "TEST_PATH={}".format(path),
  File "build.py", line 15, in run
  subprocess.check_call(list(args), cwd=cwd, env=environ)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
  raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['make', '-C', '/root/mitosis-core/mitosis-kms', 'TEST_NAME=fork', 'TEST_PATH=fork']' returned non-zero exit status 2.
  makefile:12: recipe for target 'km' failed
  make: *** [km] Error 1
  [1]+ Done make clean

This error is easy to understand, so I switched to nightly-2022-11-03-x86_64-unknown-linux-gnu, which corresponds to rustc 1.67.

(
I also found that rustc 1.68 and later versions cannot be used. For example, when I use nightly-2023-01-07-x86_64-unknown-linux-gnu, corresponding to rustc 1.68, an error will be reported:

root@liu-9:~/mitosis-core# make clean & make km
[1] 88923
rm -rf mitosis-kms/Cargo.lock
cd mitosis-kms ; python build.py fork
rm -rf mitosis-kms/target

• [fork]
• [RUNNING] ['make', '-C', '/root/mitosis-core/mitosis-kms', 'TEST_NAME=fork', 'TEST_PATH=fork']
  make[1]: Entering directory '/root/mitosis-core/mitosis-kms'
  cp -f /usr/src/ofa_kernel/default/Module*.symvers /root/mitosis-core/mitosis-kms/Module.symvers
  make -C /lib/modules/4.15.0-46-generic/build M=/root/mitosis-core/mitosis-kms CC=clang-9 CONFIG_CC_IS_CLANG=y
  make[2]: Entering directory '/usr/src/linux-headers-4.15.0-46-generic'
  cd /root/mitosis-core/mitosis-kms/fork; CARGO_TARGET_DIR=../target cargo build -Z build-std=core,alloc --target=x86_64-unknown-none-linuxkernel --features "mitosis krdma-test cow use_rc" --no-default-features
  warning: /root/mitosis-core/mitosis/Cargo.toml: dependency (x86_64) specified without providing a local path, Git repository, or version to use. This will be considered an error in future versions
  error: failed to run rustc to learn about target-specific information

Caused by:
process didn't exit successfully: rustc - --crate-name ___ --print=file-names --target x86_64-unknown-none-linuxkernel --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg (exit status: 1)
--- stderr
error: Error loading target specification: Could not find specification for target "x86_64-unknown-none-linuxkernel". Run rustc --print target-list for a list of built-in targets

I confirmed in rustc's update log that the target "x86_64-unknown-none-linuxkernel" was indeed removed after 1.68.
)

When I used nightly-2022-11-03-x86_64-unknown-linux-gnu (corresponding to rustc 1.67), I got the following running results:

root@liu-9:~/mitosis-core# make clean & make km
[1] 89474
rm -rf mitosis-kms/Cargo.lock
rm -rf mitosis-kms/target
cd mitosis-kms ; python build.py fork

• [fork]
• [RUNNING] ['make', '-C', '/root/mitosis-core/mitosis-kms', 'TEST_NAME=fork', 'TEST_PATH=fork']
  make[1]: Entering directory '/root/mitosis-core/mitosis-kms'
  cp -f /usr/src/ofa_kernel/default/Module*.symvers /root/mitosis-core/mitosis-kms/Module.symvers
  make -C /lib/modules/4.15.0-46-generic/build M=/root/mitosis-core/mitosis-kms CC=clang-9 CONFIG_CC_IS_CLANG=y
  make[2]: Entering directory '/usr/src/linux-headers-4.15.0-46-generic'
  cd /root/mitosis-core/mitosis-kms/fork; CARGO_TARGET_DIR=../target cargo build -Z build-std=core,alloc --target=x86_64-unknown-none-linuxkernel --features "mitosis krdma-test cow use_rc" --no-default-features
  warning: /root/mitosis-core/mitosis/Cargo.toml: dependency (x86_64) specified without providing a local path, Git repository, or version to use. This will be considered an error in future versions
  Updating crates.io index
  ...
  Compiling rustversion v1.0.14
  error: incorrect value ... for codegen option split-debuginfo - one of supported split-debuginfo modes (off, packed, or unpacked) was expected

error: could not compile proc-macro2 due to previous error
warning: build failed, waiting for other jobs to finish...
error: could not compile compiler_builtins due to previous error
error: could not compile unicode-ident due to previous error
error: could not compile libc due to previous error
error: could not compile glob due to previous error
error: could not compile version_check due to previous error
error: could not compile regex-syntax due to previous error
error: could not compile termcolor due to previous error
error: could not compile quick-error due to previous error
error: could not compile unicode-width due to previous error
error: could not compile memchr due to previous error
error: could not compile vec_map due to previous error
error: could not compile spin due to previous error
error: could not compile strsim due to previous error
error: could not compile crossbeam-utils due to previous error
error: could not compile futures-core due to previous error
error: could not compile syn due to previous error
error: could not compile bindgen due to previous error
error: could not compile ansi_term due to previous error
error: could not compile rustversion due to previous error
error: could not compile futures-util due to previous error
error: could not compile unicode-xid due to previous error
error: could not compile failure_derive due to previous error
error: could not compile paste due to previous error
error: could not compile futures-channel due to previous error
error: could not compile rustc-hash due to previous error
error: could not compile shlex due to previous error
error: could not compile bitflags due to previous error
error: could not compile lazycell due to previous error
error: could not compile cfg-if due to previous error
error: could not compile peeking_take_while due to previous error
error: could not compile autocfg due to previous error
error: could not compile futures-task due to previous error
error: could not compile log due to previous error
/root/mitosis-core/mitosis-kms/Kbuild:11: recipe for target '/root/mitosis-core/mitosis-kms/target/x86_64-unknown-none-linuxkernel/debug/libfork.a' failed
make[3]: *** [/root/mitosis-core/mitosis-kms/target/x86_64-unknown-none-linuxkernel/debug/libfork.a] Error 101
Makefile:1551: recipe for target 'module/root/mitosis-core/mitosis-kms' failed
make[2]: *** [module/root/mitosis-core/mitosis-kms] Error 2
make[2]: Leaving directory '/usr/src/linux-headers-4.15.0-46-generic'
Makefile:38: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/root/mitosis-core/mitosis-kms'
Traceback (most recent call last):
File "build.py", line 36, in
main(sys.argv)
File "build.py", line 31, in main
"TEST_PATH={}".format(path),
File "build.py", line 15, in run
subprocess.check_call(list(args), cwd=cwd, env=environ)
File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-C', '/root/mitosis-core/mitosis-kms', 'TEST_NAME=fork', 'TEST_PATH=fork']' returned non-zero exit status 2.
makefile:12: recipe for target 'km' failed
make: *** [km] Error 1
[1]+ Done make clean

This error is also easy to solve. I modified split-debuginfo = 'off' in mitosis-core/mitosis-kms/fork/Cargo.toml. After that I ran it again using make clean & make km and a lot of errors appeared. I modified the mitosis-core/mitosis-kms/Kbuild

$(src)/target/$(TARGET)/debug/lib%.a: cargo_will_determine_dependencies
cd $(src)/$(TEST_PATH); CARGO_TARGET_DIR=../target $(CARGO) build -Z build-std=core,alloc --target=$(TARGET) --features "mitosis krdma-test cow use_rc " --no-default-features

for:

$(src)/target/$(TARGET)/debug/lib%.a: cargo_will_determine_dependencies
cd $(src)/$(TEST_PATH); CARGO_TARGET_DIR=../target $(CARGO) build **--jobs=1 --verbose** -Z build-std=core,alloc --target=$(TARGET) --features "mitosis krdma-test cow use_rc" --no-default-features **> error.log 2>&1**

I got the following error file, I put it in the attachment.
error.log

The location where the error occurred is:
Compiling crossbeam-utils v0.8.17
Running rustc --crate-name build_script_build --edition=2018 /root/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-utils-0.8.17/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C split-debuginfo=off --cfg 'feature="std"' -C metadata=6dab265580e1aaa6 -C extra-filename=-6dab265580e1aaa6 --out-dir /root/mitosis-core/mitosis-kms/fork/../target/debug/build/crossbeam-utils-6dab265580e1aaa6 -L dependency=/root/mitosis-core/mitosis-kms/fork/../target/debug/deps --cap-lints allow
Running /root/mitosis-core/mitosis-kms/fork/../target/debug/build/crossbeam-utils-6dab265580e1aaa6/build-script-build
Running rustc --crate-name syn --edition=2018 /root/.cargo/registry/src/github.com-1ecc6299db9ec823/syn-1.0.109/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C split-debuginfo=off --cfg 'feature="clone-impls"' --cfg 'feature="default"' --cfg 'feature="derive"' --cfg 'feature="extra-traits"' --cfg 'feature="full"' --cfg 'feature="parsing"' --cfg 'feature="printing"' --cfg 'feature="proc-macro"' --cfg 'feature="quote"' --cfg 'feature="visit"' -C metadata=0ef28ab6ec5ed8d0 -C extra-filename=-0ef28ab6ec5ed8d0 --out-dir /root/mitosis-core/mitosis-kms/fork/../target/debug/deps -L dependency=/root/mitosis-core/mitosis-kms/fork/../target/debug/deps --extern proc_macro2=/root/mitosis-core/mitosis-kms/fork/../target/debug/deps/libproc_macro2-07e60ae63583f8d5.rmeta --extern quote=/root/mitosis-core/mitosis-kms/fork/../target/debug/deps/libquote-c11b7c3a87515dfa.rmeta --extern unicode_ident=/root/mitosis-core/mitosis-kms/fork/../target/debug/deps/libunicode_ident-6b7a6c518270e692.rmeta --cap-lints allow
Running rustc --crate-name crossbeam_utils --edition=2018 /root/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-utils-0.8.17/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=2 -C embed-bitcode=no -C codegen-units=16 -C debug-assertions=on --cfg 'feature="std"' -C metadata=534cd6346c3d89d2 -C extra-filename=-534cd6346c3d89d2 --out-dir /root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps --target x86_64-unknown-none-linuxkernel -L dependency=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps -L dependency=/root/mitosis-core/mitosis-kms/fork/../target/debug/deps --extern 'noprelude:alloc=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/liballoc-dd3bb74c47f5e9c4.rmeta' --extern cfg_if=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libcfg_if-d6ae4eaa3a8b28e8.rmeta --extern 'noprelude:compiler_builtins=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libcompiler_builtins-ec3a9d9aa6f74f81.rmeta' --extern 'noprelude:core=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libcore-3c3b1f4ea09e76fb.rmeta' -Z unstable-options --cap-lints allow
error[E0463]: can't find crate for std
|
= note: the x86_64-unknown-none-linuxkernel target may not support the standard library
= note: std is required by crossbeam_utils because it does not declare #![no_std]
= help: consider building the standard library from source with cargo build -Zbuild-std

error[E0463]: can't find crate for std
--> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-utils-0.8.17/src/lib.rs:90:24
|
90 | pub(crate) use std::sync::{Arc, Condvar, Mutex};
| ^^^ can't find crate
|
= note: the x86_64-unknown-none-linuxkernel target may not support the standard library
= help: consider building the standard library from source with cargo build -Zbuild-std

error[E0463]: can't find crate for std
--> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-utils-0.8.17/src/sync/once_lock.rs:7:5
|
7 | use std::sync::Once;
| ^^^ can't find crate
|
= note: the x86_64-unknown-none-linuxkernel target may not support the standard library
= help: consider building the standard library from source with cargo build -Zbuild-std

error[E0463]: can't find crate for std
--> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-utils-0.8.17/src/sync/parker.rs:3:5
|
3 | use std::fmt;
| ^^^ can't find crate
|
= note: the x86_64-unknown-none-linuxkernel target may not support the standard library
= help: consider building the standard library from source with cargo build -Zbuild-std

error[E0463]: can't find crate for std
--> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-utils-0.8.17/src/sync/parker.rs:4:5
|
4 | use std::marker::PhantomData;
| ^^^ can't find crate
|
= note: the x86_64-unknown-none-linuxkernel target may not support the standard library
= help: consider building the standard library from source with cargo build -Zbuild-std

error[E0463]: can't find crate for std
--> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-utils-0.8.17/src/sync/parker.rs:5:5
|
5 | use std::time::{Duration, Instant};
| ^^^ can't find crate
|
= note: the x86_64-unknown-none-linuxkernel target may not support the standard library
= help: consider building the standard library from source with cargo build -Zbuild-std

error[E0463]: can't find crate for std
--> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/crossbeam-utils-0.8.17/src/sync/wait_group.rs:2:5
|
2 | use std::fmt;
| ^^^ can't find crate
|
= note: the x86_64-unknown-none-linuxkernel target may not support the standard library
= help: consider building the standard library from source with cargo build -Zbuild-std

I use it every time before compiling

cd mitosis-core/mitosis-kms/fork
rm ~/.cargo/registry/cache/github.com-1ecc6299db9ec823/* && cargo clean --target-dir ../target && rm Cargo.lock

To eliminate the impact of crate caching.

Why can't find crate for std? How should I modify it to make the compilation pass correctly? We have three students here who all hope to continue research based on mitosis. This is greatly appreciated!

(
I manually modified the versions of many dependencies in xx.toml. Finally, I got the following results when I ran make km:

Compiling KRdmaKit v0.1.0 (/root/mitosis-core/deps/krcore/KRdmaKit)
Running rustc --crate-name KRdmaKit --edition=2018 /root/mitosis-core/deps/krcore/KRdmaKit/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=2 -C embed-bitcode=no -C codegen-units=16 -C debug-assertions=on --cfg 'feature="dct"' --cfg 'feature="kernel"' -C metadata=f2feb77b7d6ad1e0 -C extra-filename=-f2feb77b7d6ad1e0 --out-dir /root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps --target x86_64-unknown-none-linuxkernel -L dependency=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps -L dependency=/root/mitosis-core/mitosis-kms/fork/../target/debug/deps --extern 'noprelude:alloc=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/liballoc-9ced603fa71d9d6d.rmeta' --extern 'noprelude:compiler_builtins=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libcompiler_builtins-2b74463a1123f367.rmeta' --extern 'noprelude:core=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libcore-1de6f02c45ec0ce8.rmeta' --extern delegate=/root/mitosis-core/mitosis-kms/fork/../target/debug/deps/libdelegate-bd79b7824996c53c.so --extern futures_micro=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libfutures_micro-4c353ae8bb7e3319.rmeta --extern hashbrown=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libhashbrown-0b35d2ed94ba52fd.rmeta --extern lazy_static=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/liblazy_static-e8a8d5269ce9c5ce.rmeta --extern libc=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/liblibc-7af2676e6ddf03bd.rmeta --extern no_std_net=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libno_std_net-f9234d0514f7da73.rmeta --extern nostd_async=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libnostd_async-7189a9152b19d05e.rmeta --extern pin_utils=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libpin_utils-19a10d0df7f4b3df.rmeta --extern rdma_shim=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/librdma_shim-5a9350531b71068e.rmeta --extern thiserror_no_std=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libthiserror_no_std-00f5ab0082f0562d.rmeta -Z unstable-options -L native=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/build/rust-kernel-rdma-base-ab7aa8ed4829c369/out -L native=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/build/linux-kernel-module-6abcdab002bb9fde/out -L native=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/build/rust-kernel-linux-util-cba08ab43d214209/out
error[E0433]: failed to resolve: use of undeclared crate or module std
--> /root/mitosis-core/deps/krcore/KRdmaKit/src/memory_region.rs:13:5
|
13 | use std::ptr::null_mut;
| ^^^ use of undeclared crate or module std

warning: unused import: libc::*
--> /root/mitosis-core/deps/krcore/KRdmaKit/src/memory_region.rs:11:5
|
11 | use libc::*;
| ^^^^^^^
|
= note: #[warn(unused_imports)] on by default

error[E0560]: struct rdma_shim::bindings::ib_qp_init_attr has no field named sq_sig_all
--> /root/mitosis-core/deps/krcore/KRdmaKit/src/queue_pairs/builder.rs:383:13
|
383 | sq_sig_all : 0,
| ^^^^^^^^^^ rdma_shim::bindings::ib_qp_init_attr does not have this field
|
= note: available fields are: event_handler, qp_context, send_cq, recv_cq, srq ... and 8 others

Some errors have detailed explanations: E0433, E0560.
For more information about an error, try rustc --explain E0433.
warning: KRdmaKit (lib) generated 1 warning
error: could not compile KRdmaKit due to 2 previous errors; 1 warning emitted

Caused by:
process didn't exit successfully: rustc --crate-name KRdmaKit --edition=2018 /root/mitosis-core/deps/krcore/KRdmaKit/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=2 -C embed-bitcode=no -C codegen-units=16 -C debug-assertions=on --cfg 'feature="dct"' --cfg 'feature="kernel"' -C metadata=f2feb77b7d6ad1e0 -C extra-filename=-f2feb77b7d6ad1e0 --out-dir /root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps --target x86_64-unknown-none-linuxkernel -L dependency=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps -L dependency=/root/mitosis-core/mitosis-kms/fork/../target/debug/deps --extern 'noprelude:alloc=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/liballoc-9ced603fa71d9d6d.rmeta' --extern 'noprelude:compiler_builtins=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libcompiler_builtins-2b74463a1123f367.rmeta' --extern 'noprelude:core=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libcore-1de6f02c45ec0ce8.rmeta' --extern delegate=/root/mitosis-core/mitosis-kms/fork/../target/debug/deps/libdelegate-bd79b7824996c53c.so --extern futures_micro=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libfutures_micro-4c353ae8bb7e3319.rmeta --extern hashbrown=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libhashbrown-0b35d2ed94ba52fd.rmeta --extern lazy_static=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/liblazy_static-e8a8d5269ce9c5ce.rmeta --extern libc=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/liblibc-7af2676e6ddf03bd.rmeta --extern no_std_net=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libno_std_net-f9234d0514f7da73.rmeta --extern nostd_async=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libnostd_async-7189a9152b19d05e.rmeta --extern pin_utils=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libpin_utils-19a10d0df7f4b3df.rmeta --extern rdma_shim=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/librdma_shim-5a9350531b71068e.rmeta --extern thiserror_no_std=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/deps/libthiserror_no_std-00f5ab0082f0562d.rmeta -Z unstable-options -L native=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/build/rust-kernel-rdma-base-ab7aa8ed4829c369/out -L native=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/build/linux-kernel-module-6abcdab002bb9fde/out -L native=/root/mitosis-core/mitosis-kms/fork/../target/x86_64-unknown-none-linuxkernel/debug/build/rust-kernel-linux-util-cba08ab43d214209/out (exit status: 1)
/root/mitosis-core/mitosis-kms/Kbuild:14: recipe for target '/root/mitosis-core/mitosis-kms/target/x86_64-unknown-none-linuxkernel/debug/libfork.a' failed
make[3]: *** [/root/mitosis-core/mitosis-kms/target/x86_64-unknown-none-linuxkernel/debug/libfork.a] Error 101
Makefile:1551: recipe for target 'module/root/mitosis-core/mitosis-kms' failed
make[2]: *** [module/root/mitosis-core/mitosis-kms] Error 2
make[2]: Leaving directory '/usr/src/linux-headers-4.15.0-46-generic'
Makefile:38: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/root/mitosis-core/mitosis-kms'
Traceback (most recent call last):
File "build.py", line 36, in
main(sys.argv)
File "build.py", line 31, in main
"TEST_PATH={}".format(path),
File "build.py", line 15, in run
subprocess.check_call(list(args), cwd=cwd, env=environ)
File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-C', '/root/mitosis-core/mitosis-kms', 'TEST_NAME=fork', 'TEST_PATH=fork']' returned non-zero exit status 2
makefile:12: recipe for target 'km' failed
make: *** [km] Error 1

What should we do next, thank you very much!

)

Need Cargo.lock file

Even though we used nightly-2022-02-04-x86_64-unknown-linux-gnu as the toolchain, cargo still failed due to version incompatibility when building third-party dependencies(crates.io).

some of errors like:
error[E0658]: cfg(target_has_atomic) is experimental and subject to change
--> /home/xxx/.cargo/registry/src/mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd/log-0.4.19/src/lib.rs:350:7
|
350 | #[cfg(target_has_atomic = "ptr")]
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: see issue #32976 rust-lang/rust#32976 for more information
= help: add #![feature(cfg_target_has_atomic)] to the crate attributes to enable

Please remove cargo.lock file from .gitignore.
We need this file to reproduce your work.
Extremely grateful！