dvhart / librtpi Goto Github PK

To complete the core purpose of this library, we need to replace the wrapper of the pthread condvar implementation with a patched version of the condvar implementation from before v2.25 of glibc.

The patches to add Priority Inheritance support to the condvar is tracked in the glibc bug 11588 [1]

The implementation should be adapted from this implementation, but can be simplified to only support the defined usages (see README.md).

1. https://sourceware.org/bugzilla/show_bug.cgi?id=11588

With the current futex syscall implementation, tst-condpi2.c is hanging:

$ sudo ./tst-condpi2
low_tf: locked
high_tf: locked
med_tf: spinning
high_tf: done waiting
med_tf: done spinning
[it doesn't progress past this point]

The current librtpi API creates a strong association between a user mutex and a conditional variable by passing a pointer to the mutex to pi_cond_init() along with the pointer to the conditional variable to be initialized. The pointer to the associated mutex is stored in the private data for the conditional variable.

This is done because in order to guarantee proper RT behaviour we have to use the FUTEX_WAIT_REQUEUE_PI and FUTEX_CMP_REQUEUE_PI futex ops for waiting and signalling respectively and the user mutex is a parameter for both calls. Storing a pointer to the user mutex at init time prevents the practice of signalling the condition prior to the association of the mutex.

The problem is the current librtpi API breaks down for process shared conditional variables as illustrated by glibc tests: tst-cond4.c, tst-cond6.c, tst-cond12.c, and tst-cond13.c.

For process-shared use cases the user mutex and the condvar are allocated in shared memory. This shared memory is (usually) mapped at different addresses in the different processes that interact with the condvar. As a result the pointer to the mutex stored by the pi_cond_init() call points to a valid address only for the process that calls init. All other processes that interact with the process-shared condvar will cause an invalid memory access when accessing the stored mutex pointer.

A good example of this problem is glibc test tst-cond12.c. In this test a parent process creates a memory mapping for a process shared conditional variable and mutex that is later re-mapped by the child process at a different address and the old mapping is unmapped via munmap(). Because the conditional variable was initialized before the fork() the stored mutex pointer now points to an unmapped address in the child process and it results in a SIGSEGV crash when the mutex is
accessed inside the pi_cond.c code.

A proposed solution would be to change the librtpi API to pass the user mutex for the condvar operations that need it and do away with storing a pointer to the user mutex inside the condvar at init.

The new API would look like this:

int pi_cond_init(pi_cond_t *cond, uint32_t flags)
int pi_cond_destroy(pi_cond_t *cond)
int pi_cond_wait(pi_cond_t *cond, pi_mutex_t *mutex)
int pi_cond_timedwait(pi_cond_t *cond, pi_mutex_t *mutex, const struct timespec *restrict abstime)
int pi_cond_signal(pi_cond_t *cond, pi_mutex_t *mutex)
int pi_cond_broadcast(pi_cond_t *cond, pi_mutex_t *mutex)

I can follow up with patches if we're in agreement this is the correct path forward.

I find that an include guard like “RPTI_H” can be too short for the safe reuse of your header file (when it belongs to an application programming interface).

The test cases from the original mailing list do not reproduce the hang case.

The library should include test programs that verify correct functioning.

I would like to point out that an identifier like “_RTPI_H” does eventually not fit to the expected naming convention of the C language standard.
Would you like to adjust your selection for unique names?

tst-condpi2-orig is the original testcase which demonstrated failure on the previous glibc condvar implementation. We need to add the build steps to Makefile.am. (we'll track if this properly reproduces the original bug in a separate issue).

Hi everyone,

we are currently considering the use of librtpi in a part of our software stack and were wondering what the current status of the project is. Are there any issues left to solve? As the upstream glibc bug report is still open, I assume that glibc itself still has not fixed the issue or is planning to address it in the foreseeable future.

In the 2020 ELC talk, @gratian mentioned that there could be an "official" release when remaining issues are merged. Is this still something you consider doing? With an official release, it might also be easier to create packages of librtpi for various distributions as they can refer to a specific version instead of a commit hash. Do you have any plans for packaging the library yourself?

In any case, thanks for building this library already!

Replace the test.c testcase in this repository with a port of the test case attached to glibc bug 11588 [1][2]. Note that the testcase itself is Copyright IBM and released under the GPLv2 (not the LGPL). This doesn't impact the license of the library itself, just the testcase.

The testcase will need to be ported to use the librtpi interface.

1. https://sourceware.org/bugzilla/show_bug.cgi?id=11588
2. https://sourceware.org/bugzilla/attachment.cgi?id=4785