bupt-os / rros

RROS is a dual-kernel OS for satellites or other scenarios that need both real-time and general-purpose abilities. RROS = RTOS (Rust) + Linux (C).

---

RROS is a dual-kernel OS, consisting of a real-time kernel (in Rust) and a general-purpose kernel (Linux). RROS is compatible with almost all native Linux programs and offers real-time performance superior to RT-Linux. It is also being experimented with as the host OS for in-orbit satellites (Tiansuan Project).

Here is an architecture diagram and a demo video of RROS.

News

• [2024.07.13] Shan Yuxuan shared research on RROS at the 2024 Rust Meetup Beijing. (photos)
• [2024.07.10] 🔥🔥 “An Empirical Study of Rust-for-Linux: The Success, Dissatisfaction, and Compromise” is presented USENIX ATC 2024 Best Paper Award!!!(photos)
• [2024.06.15] Li Hongyu gave an oral talk at the 26th ACM SIGOPS ChinaSys. (photos)
• [2024.03.30] RROS is presented in Open Source Operating System Annual Technical Conference 2024. (photos)
• [2023.12.09] 🔥🔥 RROS is successfully launched into space! (photos).
• [2023.11.30] RROS is presented at Xenomai Workshop 2023 (photos).
• [2023.11.28] 🔥 RROS is open-sourced!

Why RROS

RROS is primarily intended for satellites (onboard computers, payloads, etc). The key incentive is the trend that nowadays satellites serve both traditional satellite-borne real-time tasks (e.g., communication and positioning) and general-purpose tasks that need mature, complicated software support (e.g., data compression and machine learning). That catalyzes the dual-kernel architecture of RROS. Taking a step further, the real-time kernel of RROS is fully implemented in Rust for better safety and robustness. However, RROS can be used in scenarios like automatic cars, IoTs, industrial control, etc.

The advantages of RROS are:

• Hard real-time: RROS offers superior real-time performance compared to RT-Linux. RROS is designed with an efficient task scheduler that can quickly respond to external events, reducing task switching and processing delays.
• Compatibility: RROS is compatible with almost every Linux program, allowing seamless migration of complex Linux applications such as TensorFlow and Kubernetes. You can also easily modify your general Linux programs into a more real-time counterpart.
• Easy to use: RROS facilitates easy programming and debugging of real-time programs. RROS uses the libevl interface to call real-time APIs for user programs, allowing you to use tools like gdb, kgdb, and QEMU.
• Robustness: The real-time kernel of RROS is carefully written in Rust, making it safer and more robust, especially for memory and concurrency issues.

Quick start

Kick off with RROS easily: boot, run, test, and develop with RROS.

Document

See the document including:

• Quick Start
• Setup the Environment
• Choose a file System
• Deploy RROS on the Raspberry Pi
• Debug
• Kernel Tools

Communication & Contribution

Contact us at Zulip Forum or with email buptrros AT gmail.com.

Contributions are also very welcomed! Check it out.

Roadmap

See here for our future roadmap.

Who are we

We are a research group at BUPT.

Release

The RROS relies on both the dovetail and the Rust for Linux(RFL), neither of which currently provides patches. Integrating one into the other at a high frequency is challenging. As a result, RROS is currently tied to Linux kernel version 5.13, built on top of linux-dovetail-v5.13, and readily compatible with RFL patch v1. Fortunately, RFL is swiftly making its way into the mainline Linux kernel. We plan to release new versions once most of the RFL APIs we depend on are available in the linux-dovetail mainline. At that point, we will further consider Long-Term Support (LTS) versions.

Acknowledgements

RROS has benefitted from the following projects/resources.

• Evl/xenomai (linux-evl). We learned from evl core how to implement a dual kernel and use dovetail for interrupt virtualization and libevl for user library. Thanks, @Philippe for his genius work and patient explanation in the riot!
• Rust-for-Linux: We use RFL to write RROS in Linux. We ask a lot of questions on RFL Zulip and constructing safety abstractions. Kudos to @ojeda, @Wedson, @Alex, @boqun, @Gary, @Björn in the RFL community for patiently helping us. We hope to bring more safety abstraction back to the RFL in return!
• Muduo: His detailed blog gives us insights for xenomai/evl project.
• All prospective contributors to RROS in the future!

Citation

@misc{li2023rros,
    title = {RROS: A Dual-kernel Real-time Operating System in Rust},
    url = {https://github.com/BUPT-OS/RROS},
    author = {Hongyu Li and Jiangtao Hu and Qichen Qiu and Yuxuan Shan and Bochen Wang and Jiajun Du and Yexuan Yang and Xinge Wang and Shangguang Wang and Mengwei Xu},
    month = {December},
    year = {2023}
}

License

The source code of RROS is under the License of GPL-2.0.

rros's Issues

mmap file_operation in control factory may exist some errors

• mmap file_operation calls remap_pfn_range with a parameter PAGE_SHARED. Is there a bug here regarding constant values?
• mmap file_operation calls remap_pfn_range with a parameter ptn, which is returned by function named rust_helper_pa. Is there an error regarding the return value type?

Failed to build "Hello World" application

Hello, I tried to build demo application as described on this page https://bupt-os.github.io/website/docs/introduction/quick-start/ and get the following error:

root@891d48fefc59:~/hello_world# LIBEVL=/root/libevl/ make
aarch64-linux-gnu-gcc -o demo demo.c -I. -I/root/libevl//include /root/libevl//lib/libevl-0.a -lrt -static
In file included from demo.c:8:
/root/libevl//include/evl/proxy.h:14:10: fatal error: uapi/evl/proxy.h: No such file or directory
   14 | #include <uapi/evl/proxy.h>
      |          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:20: demo] Error 1

It looks like that this file is missing in docker image:

root@891d48fefc59:~/hello_world# find / -name "proxy.h"
/root/libevl/include/eshi/evl/proxy.h
/root/libevl/include/evl/proxy.h

Any ideas how to fix?

lacking of trace subsystem

Currently, RROS lacks a trace subsystem to trace the function calls and profile the system.

Add `monitor factory` in RROS

The RROS project lacks the capability for inter-thread synchronization, including: mutexs, semaphores, and condition variables. A monitor factory is needed to implement this functionality. For real-time operating systems, it is necessary to ensure that the kernel does not experience deadlock while implementing synchronization mechanisms, and to implement priority inheritance and the ceiling protocol.

Lack of SMP support in the kernel.

Currently in the RROS kernel can only support running on a single CPU, all tasks are executed concurrently on a single CPU, and does not provide the function of executing multiple tasks in parallel on multiple CPUs.

However, in complex real-time application scenarios, multiple tasks are often triggered at the same time. Moreover, current computer hardware often has multiple CPU cores to provide better processing performance, such as the common SMP (Symmetric Multi-processing) architecture. If there is no corresponding SMP support in the kernel, there is no way to improve performance even with multiple CPUs. Therefore, it is necessary to provide SMP support for RROS.

In order to implement SMP support in RROS, kernel modifications are required, involving changes to the scheduling subsystem, clock subsystem, interrupt subsystem, and so on. Eventually, it will be possible to support parallel execution of multiple tasks in RROS.

rros_get/put_thread_rq is lacking of the lock/unlock part

This will break when in the smp config with migaration function.

miss unsafe documentation

// rust/kernel/kernelh.rs
    arg1: *const c_char,
    arg2: *const c_char,
) -> *mut c_char {
    unsafe { bindings::kasprintf(gfp, fmt, arg1, arg2) }

There should be an unsafe comment above the unsafe block.

Remove all warnings generated during compilation.

Compiling the code on the current mainline will generate a lot of warnings, part of which is caused by the wrapper of the RFL layer not writing comments, and part of which is caused by the irregularity of the code. Therefore, these warnings need to be removed to standardize the code. This work roughly includes the following parts:

1. Add documentation for the RFL part code.
2. Modify some wrapper's format in RFL layer.
3. Remove redundant comments in the RROS code.
4. Standardize code and eliminate some irregular codes.

Add `rros_defconfig`

I consider that RROS needs a defconfig to compile it and run it easily.

enable github bot && gitmaya

• enable a github bot to @ reviewers
• enable gitmaya to send the github related messages to the lark

Need a `new` method in `Timespec64`

The purpose of struct Timespec64's existence is to avoid using bindings. But now it doesn't have a new method, we have to import bindings::timespec64 when we want to init a Timespec64. That makes me upset😢

Failed to build kernel

I configured build environment in Docker using Dockerfile from manual and proceed with Quick start steps (https://bupt-os.github.io/website/docs/introduction/quick-start/). On Compile Kernel Step (4) I got the following error:

  RUSTC L rust/build_error.o
  EXPORTS rust/exports_alloc_generated.h
  RUSTC L rust/kernel.o
error[E0425]: cannot find function `skb_alloc_oob_head` in module `bindings`
  --> rust/kernel/skbuff.rs:11:24
   |
11 |     unsafe { bindings::skb_alloc_oob_head(gfp_mask) }
   |                        ^^^^^^^^^^^^^^^^^^ not found in `bindings`

error[E0425]: cannot find function `__netdev_alloc_oob_skb` in module `bindings`
      --> rust/kernel/skbuff.rs:35:24
       |
35     |       unsafe { bindings::__netdev_alloc_oob_skb(dev, len, gfp_mask) }
       |                          ^^^^^^^^^^^^^^^^^^^^^^ help: a function with a similar name exists: `__netdev_alloc_skb`
       | 
      ::: /root/RROS/rust/bindings_generated.rs:132602:5
       |
132602 | /     pub fn __netdev_alloc_skb(
132603 | |         dev: *mut net_device,
132604 | |         length: c_types::c_uint,
132605 | |         gfp_mask: gfp_t,
132606 | |     ) -> *mut sk_buff;
       | |______________________- similarly named function `__netdev_alloc_skb` defined here

error: aborting due to 2 previous errors

kindly ask to help to resolve it

Add pre-check `make LLVM=1 rustfmtcheck` in ci

Add this check will make sure the code in our system is formatted.

Spinlock bug in poll.rs

RROS/kernel/rros/poll.rs

Lines 379 to 382 in 8368c16

	flags = head.watchpoints.irq_lock_noguard();
	list_for_each_entry!(
	poco,
	head.watchpoints.lock().deref(),

Try to acquire spinlock when spinlock is already locked.

There is a bug in poll.rs

Hi All!
Last year I contributed the poll factory code, and I made tests poll-close.c and poll-nested.c passed. But I find something goes wrong when the poll-close test runs in a loop shell. This shell just runs poll-close test 10 times and the kernel will throw an invalid address request.
This error occurs in the function rros_drop_watchpoints which is written at 565 lines in poll.rs.
Following is the error message:

[   79.822868] Unable to handle kernel paging request at virtual address dead0000000000d0
[   79.826177] Mem abort info:
[   79.826321]   ESR = 0x96000004
[   79.826538]   EC = 0x25: DABT (current EL), IL = 32 bits
[   79.826749]   SET = 0, FnV = 0
[   79.826858]   EA = 0, S1PTW = 0
[   79.826975] Data abort info:
[   79.827065]   ISV = 0, ISS = 0x00000004
[   79.827292]   CM = 0, WnR = 0
[   79.827458] [dead0000000000d0] address between user and kernel address ranges
[   79.833567] Internal error: Oops: 96000004 [#1] SMP
[   79.833621] Modules linked in:
[   79.833639] CPU: 0 PID: 618 Comm: new_thread_lhy0 Not tainted 5.13.19-v8+ #2
[   79.833646] Hardware name: linux,dummy-virt (DT)
[   79.833652] IRQ stage: Linux
[   79.833658] pstate: 90000005 (NzcV daif -PAN -UAO -TCO BTYPE=--)
[   79.833665] pc : _RNvMs_NtCshGpAVYOtgW1_5alloc7raw_vecINtB4_6RawVecNtNtCshGpAVYOtgW1_4init4poll17RrosPollConnectorE3ptrBP_+0x8/0x18
[   79.833673] lr : _RNvMs_NtCshGpAVYOtgW1_5alloc3vecINtB4_3VecNtNtCshGpAVYOtgW1_4init4poll17RrosPollConnectorE10as_mut_ptrBI_+0xc/0x14
[   79.833681] sp : ffffffc011bf3c00
[   79.833686] x29: ffffffc011bf3c00 x28: ffffff8006525318 x27: ffffff8006615b98
[   79.833824] x26: 0000000000000000 x25: ffffffc010ccdb80 x24: ffffffc010ccdb68
[   79.833846] x23: ffffffc010ccdb50 x22: dead0000000000c8 x21: 0000000000000038
[   79.833868] x20: dead000000000100 x19: dead0000000000d0 x18: ffffffc010cca880
[   79.833890] x17: 0000000000000000 x16: 00000000000000b4 x15: 00000000000000de
[   79.833911] x14: 0000000000000040 x13: 0000000000000000 x12: ffffff80091ca998
[   79.833933] x11: 0000000000000000 x10: ffffffc01136b000 x9 : 0000000000000000
[   79.833954] x8 : ffffffffffffffc8 x7 : 6f6e203a736f7272 x6 : ffffffc0115b52b0
[   79.833975] x5 : 000000008020001f x4 : 0000000000000080 x3 : 0000000000000001
[   79.833997] x2 : ffffff8005239e00 x1 : ffffff80052d3a80 x0 : dead0000000000d0
[   79.834018] Call trace:
[   79.834025]  _RNvMs_NtCshGpAVYOtgW1_5alloc7raw_vecINtB4_6RawVecNtNtCshGpAVYOtgW1_4init4poll17RrosPollConnectorE3ptrBP_+0x8/0x18
[   79.834033]  _RNvMs_NtCshGpAVYOtgW1_5alloc3vecINtB4_3VecNtNtCshGpAVYOtgW1_4init4poll17RrosPollConnectorE10as_mut_ptrBI_+0xc/0x14
[   79.834042]  _RNvXs9_NtCshGpAVYOtgW1_5alloc3vecINtB5_3VecNtNtCshGpAVYOtgW1_4init4poll17RrosPollConnectorENtNtNtCshGpAVYOtgW1_4core3ops5deref8DerefMut9deref_mutBJ_+0x14/0x28
[   79.834053]  _RNvMs_NtCshGpAVYOtgW1_5alloc3vecINtB4_3VecNtNtCshGpAVYOtgW1_4init4poll17RrosPollConnectorE12as_mut_sliceBI_+0xc/0x14
[   79.834061]  _RNvNtCshGpAVYOtgW1_4init4poll21rros_drop_watchpoints+0xa4/0x1e8
[   79.834068]  _RNvNtCshGpAVYOtgW1_4init4file16drop_watchpoints+0x28/0x34
[   79.834075]  rust_uninstall_inband_fd+0x58/0x74
[   79.834081]  uninstall_inband_fd+0x1c/0x40
[   79.834086]  pick_file+0xf0/0x138
[   79.834092]  close_fd+0x30/0x54
[   79.834097]  __arm64_sys_close+0x20/0x50
[   79.834103]  invoke_syscall+0x4c/0x124
[   79.834109]  el0_svc_common+0xbc/0x130
[   79.834114]  do_el0_svc+0x30/0x8c
[   79.834120]  el0_svc+0x2c/0x8c
[   79.834125]  el0_sync_handler+0x84/0xe4
[   79.834133]  el0_sync+0x198/0x1c0
[   79.834139] Code: a8c17bfd d65f03c0 a9bf7bfd 910003fd (f9400000) 
[   79.834146] ---[ end trace c17059f3ccb71b43 ]---

Add `poll factory` in RROS

RROS don't have poll factory. Need a poll factory to support poll events.

replace element.ref with rc/arc