atollk / copper

Port of Golang channels to C++

---

---

Copper is a C++ library of a powerful queue object for communication between threads. It is based on Go's channels and follows the quote:

Don't communicate by sharing memory; share memory by communicating.

See docs/motivation.adoc for a high-level description of what Copper is capable of and how it stands out from previous C / C++ implementations of queues and Go-like channels.

Copper has...

• only a single header to include.
• no deadlocks; no race conditions; no undefined behavior; no polling.
• support for multiple producers and multiple consumers.
• an API based on that of std to avoid style clashes.

---

Quick Example

#include <future>
#include <iostream>
#include "copper.h"

copper::buffered_channel<int> channel_1;
copper::buffered_channel<int> channel_2;

void producer_1() {
    // Push the numbers 0 to 9 into channel_1.
    for (auto i = 0; i < 10; ++i) {
        (void) channel_1.push(i);
    }
    channel_1.close();
}

void producer_2() {
    // Push the numbers 0 to 9 into channel_2.
    for (auto i = 0; i < 10; ++i) {
        (void) channel_2.push(i);
    }
    channel_2.close();
}

void consumer() {
    // Until both channel_1 and channel_2 are closed, get the next message from either and print it.
    while (copper::select(
        channel_1 >> [](int x) { std::cout << "Message from producer 1: " << x << std::endl; },
        channel_2 >> [](int x) { std::cout << "Message from producer 2: " << x << std::endl; }
    ) == copper::channel_op_status::success);
}

int main() {
    const auto p1 = std::async(producer_1);
    const auto p2 = std::async(producer_2);
    const auto c = std::async(consumer);
    return 0;
}

---

More Information

API Reference

See docs/reference.adoc for a detailed reference of the public code interface.

Efficiency & Benchmarks

See docs/techdetails.adoc for technical information about efficiency and some benchmarks.

---

Dependency managers

FetchContent

include(FetchContent)
FetchContent_Declare(
  copper
  GIT_REPOSITORY https://github.com/atollk/copper
  GIT_TAG        v1.1
)
FetchContent_MakeAvailable(copper)
target_link_libraries(MyProject Threads::Threads copper)

CPM

CPMAddPackage(
  GITHUB_REPOSITORY atollk/copper
  VERSION 1.1
)
target_link_libraries(MyProject Threads::Threads copper)

copper's Issues

Add coverage CI

Right now, measurement of test coverage is manually started and uploaded to Codecov. This would better be done as a Github Action.

Add dynamic analysis to CI

Add some dynamic analysis tools to the CI to make sure the unit tests don't cause any UB (or other problems).
For example:

• UBSan
• ASan
• ThreadSanitizer
• Helgrind

ms abi warnings, unused variable warnings

with this clang:

Ubuntu clang version 11.0.0-2
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

I get warnings like these:

./copper.h:876:12: warning: class 'waiting_op_group_base' was previously declared as a struct; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags]
friend class waiting_op_group_base;
^
./copper.h:274:8: note: previous use is here
struct waiting_op_group_base {
^
./copper.h:876:12: note: did you mean struct here?
friend class waiting_op_group_base;
^~~~~
struct
./copper.h:980:12: warning: class 'waiting_op_group_base' was previously declared as a struct; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags]
friend class waiting_op_group_base;
^
./copper.h:274:8: note: previous use is here
struct waiting_op_group_base {
^
./copper.h:980:12: note: did you mean struct here?
friend class waiting_op_group_base;
^~~~~
struct
./copper.h:1649:10: warning: unused variable 'l' [-Wunused-variable]
auto l = {(f(Is, std::get(t)), 0)...};

Improve test coverage

While 100% is impossible due to how some functions are just defined to satisfy the compiler but are never meant to be called, the current 82% can be improved upon.

Add "CONTRIBUTING" file

Value Selects

The current select keeps the channel locked for the entire time until the statement finishes; most importantly, the callable is run while the channel is locked. This can cause huge performance losses that are not obvious to see.

An alternative select statement could be introduced that pops into a variable / pushes from a variable and takes a callable for "post-call" execution.

Go:

func fibonacci(c, quit chan int) {
	x, y := 0, 1
	for {
		select {
		case c <- x:
			x, y = y, x+y
		case <-quit:
			fmt.Println("quit")
			return
		}
	}
}

-->

C++:

void fibonacci(buffered_channel<int>& chan, unbuffered_channel<void> quit) {
    auto x = 0;
    auto y = 1;
    while (select(
        chan << x, [&x, &y] {x, y = y, x+y;},
        quit >> 0, [] {std::cout << "quit" << std::endl;}
    ));
}

remove_cvref unavailable with c++17

On Windows with Microsoft's clang:

clang version 11.0.0
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\Llvm\x64\bin

I get this:

./copper.h(1400,55): error: no member named 'remove_cvref_t' in namespace 'std'
COPPER_STATIC_ASSERT((std::is_same_v<std::remove_cvref_t, _detail::voidval_t>));
~~~~~^
./copper.h(49,82): note: expanded from macro 'COPPER_STATIC_ASSERT'
COPPER_GET_MACRO2(VA_ARGS, COPPER_STATIC_ASSERT2, COPPER_STATIC_ASSERT1)(VA_ARGS)
^~~~~~~~~~~
./copper.h(58,79): note: expanded from macro 'COPPER_STATIC_ASSERT1'
#define COPPER_STATIC_ASSERT1(condition) ::copper::_detail::static_assert_ifc()
^~~~~~~~~
./copper.h(1400,70): error: 'Rhs' does not refer to a value
COPPER_STATIC_ASSERT((std::is_same_v<std::remove_cvref_t, _detail::voidval_t>));
^
./copper.h(1394,20): note: declared here
template <typename Rhs, bool is_buffered, template <typename...> typename... Args>
^
etc

close cannot close all pops...

copper::chan channel;

void ch1() {
printf("ch1 start\n");
for (const auto& item : channel) {
printf("%s\n", item.c_str());
}
printf("ch1 end\n");
}

void ch2(copper::chan& channel) {
printf("ch2 start\n");

channel.push("2");

printf("ch2 end\n");

}

void aa() {
for (size_t i = 0; i < 5; i++)
{
new std::thread(& {
ch1();
});

}
//sleep(1000 * 2);
std::thread* my_thread2 = new std::thread([&]() {
	ch2(channel);
	channel.close();
	}); 

}

Mingw builds fail

When building copper_tests in CI with Mingw64, the linking fails for both GCC and Clang but with different errors and only when building in Debug mode.

see e.g. https://github.com/atollk/copper/actions/runs/794387279

Clang:

[100%] Linking CXX executable copper_tests
D:/a/_temp/msys/msys64/mingw64/bin/ld: DWARF error: could not find variable specification at offset 141e
D:/a/_temp/msys/msys64/mingw64/bin/ld: DWARF error: could not find variable specification at offset 7ce5
D:/a/_temp/msys/msys64/mingw64/bin/ld: DWARF error: could not find variable specification at offset 7d2f
CMakeFiles/copper_tests.dir/tests/main.cpp.obj:main.cpp:(.debug_info+0x16): relocation truncated to fit: IMAGE_REL_AMD64_SECREL against `.debug_line'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

GCC:

[100%] Linking CXX executable copper_tests
D:/a/_temp/msys/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: CMakeFiles/copper_tests.dir/tests/main.cpp.obj:main.cpp:(.text+0xa2e): undefined reference to `typeinfo for Catch::TestFailureException'
D:/a/_temp/msys/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: CMakeFiles/copper_tests.dir/tests/main.cpp.obj: in function `Catch::handleExceptionMatchExpr(Catch::AssertionHandler&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Catch::StringRef const&)':
D:/a/copper/copper/build/_deps/catch2-src/single_include/catch2/catch.hpp:8275: undefined reference to `Catch::Matchers::StdString::EqualsMatcher::~EqualsMatcher()'
D:/a/_temp/msys/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: D:/a/copper/copper/build/_deps/catch2-src/single_include/catch2/catch.hpp:8275: undefined reference to `Catch::Matchers::StdString::EqualsMatcher::~EqualsMatcher()'
D:/a/_temp/msys/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: CMakeFiles/copper_tests.dir/tests/main.cpp.obj: in function `Catch::handleExceptionMatchExpr(Catch::AssertionHandler&, Catch::Matchers::Impl::MatcherBase<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > const&, Catch::StringRef const&)':

...

max/min macro trouble

Building a project that includes copper.h on Windows with Microsoft's clang:

clang version 11.0.0
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\Llvm\x64\bin

I get this:

./copper.h(206,80): error: too few arguments provided to function-like macro invocation
explicit channel_buffer(size_t max_size = std::numeric_limits<size_t>::max()) : _max_size(max_size) {}
^
C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared\minwindef.h(193,9): note: macro 'max' defined here
#define max(a,b) (((a) > (b)) ? (a) : (b))

tsan lock-order-inversion reports

Describe the bug

With two threads, each running a copper::select loop and messaging each other, tsan gives lock-order-inversion reports.

To Reproduce

test-copper.cpp:

#include <thread>
#include "copper.h"


void b_run(copper::buffered_channel<bool>* a_to_b,
           copper::buffered_channel<bool>* b_to_a) {

    while (copper::channel_op_status::success == copper::select(
               (*a_to_b) >> [&](bool v) {
                   std::cout << "ok b" << std::endl;
                   (void)b_to_a->push(true);
               }));
}


int main(int argc, char* argv[]) {

    copper::buffered_channel<bool> a_to_b;
    copper::buffered_channel<bool> b_to_a;

    new std::thread(&b_run, &a_to_b, &b_to_a);

    (void)a_to_b.push(true);

    while (copper::channel_op_status::success == copper::select(
               b_to_a >> [&](bool v) {
                   std::cout << "ok a" << std::endl;
                   (void)a_to_b.push(true);
               }));

    return 0;
}

gnu-make Makefile:

CLANGFLAGS=-Wall -pedantic -Wno-gnu-zero-variadic-macro-arguments
CXXFLAGS=-std=c++20 -g -pedantic -O2
OBJ_EXT=o
EXE_SUFFIX=
LINKER=clang++ -o $@
SAN=-fsanitize=thread

%.${OBJ_EXT} : %.cpp
	${CXX} ${SAN} ${CLANGFLAGS} ${CXXFLAGS} -c -o $@ $<

all: test-copper

test-copper: test-copper.${OBJ_EXT}
	${LINKER} ${SAN} $^ -lpthread

clean:
	rm -f *~ *.${OBJ_EXT} test-copper

example report:

WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=2594197)
  Cycle in lock order graph: M12 (0x7fff99869c60) => M14 (0x7fff99869d88) => M12

  Mutex M14 acquired here while holding mutex M12 in thread T1:
    #0 pthread_mutex_lock <null> (test-copper+0x444ab6)
    #1 __gthread_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10/bits/gthr-default.h:749:12 (test-copper+0x4bb59d)
    #2 __gthread_recursive_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10/bits/gthr-default.h:811:10 (test-copper+0x4bb59d)
    #3 std::recursive_mutex::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/mutex:106:17 (test-copper+0x4bb59d)
    #4 std::unique_lock<std::recursive_mutex>::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/unique_lock.h:138:17 (test-copper+0x4bb59d)
    #5 std::unique_lock<std::recursive_mutex>::unique_lock(std::recursive_mutex&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/unique_lock.h:68:2 (test-copper+0x4bb59d)
    #6 copper::channel_op_status copper::channel<true, bool, std::queue, std::deque>::push_func_wt<(copper::wait_type)0, bool copper::channel<true, bool, std::queue, std::deque>::push_wt<(copper::wait_type)0, bool, bool, 0>(bool&&)::'lambda'()>(bool&&, bool&&...) /home/seth/src/test-copper/./copper.h:532:21 (test-copper+0x4bb59d)
    #7 bool copper::channel<true, bool, std::queue, std::deque>::push_wt<(copper::wait_type)0, bool, bool, 0>(bool&&) /home/seth/src/test-copper/./copper.h:510:19 (test-copper+0x4ba90d)
    #8 bool copper::channel<true, bool, std::queue, std::deque>::push<bool>(bool&&) /home/seth/src/test-copper/./copper.h:538:22 (test-copper+0x4ba90d)
    #9 b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0::operator()(bool) const /home/seth/src/test-copper/test-copper.cpp:12:34 (test-copper+0x4ba90d)
    #10 void std::__invoke_impl<void, b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0&, bool>(std::__invoke_other, b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0&, bool&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60:14 (test-copper+0x4ba90d)
    #11 std::__invoke_result<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0&, bool>::type std::__invoke<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0&, bool>(b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0&, bool&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:95:14 (test-copper+0x4ba90d)
    #12 std::invoke_result<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0&, bool>::type std::invoke<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0&, bool>(b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0&, bool&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/functional:88:14 (test-copper+0x4ba90d)
    #13 copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque>::operator()(bool&&) /home/seth/src/test-copper/./copper.h:893:36 (test-copper+0x4ba90d)
    #14 copper::channel_op_status copper::channel<true, bool, std::queue, std::deque>::_pop_func_with_lock_general<(copper::wait_type)0, copper::_detail::popper_base<true, bool, std::queue, std::deque>&, std::unique_lock<std::recursive_mutex> >(copper::_detail::popper_base<true, bool, std::queue, std::deque>&, std::unique_lock<std::recursive_mutex>&) /home/seth/src/test-copper/./copper.h:687:21 (test-copper+0x4bf350)
    #15 copper::channel_op_status copper::channel<true, bool, std::queue, std::deque>::pop_func_wt<(copper::wait_type)0, copper::_detail::popper_base<true, bool, std::queue, std::deque>&>(copper::_detail::popper_base<true, bool, std::queue, std::deque>&) /home/seth/src/test-copper/./copper.h:455:22 (test-copper+0x4bf057)
    #16 copper::channel_op_status copper::_detail::popper_base<true, bool, std::queue, std::deque>::select<(copper::wait_type)0>() /home/seth/src/test-copper/./copper.h:865:41 (test-copper+0x4ba61b)
    #17 copper::channel_op_status copper::_detail::select_manager<copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque> >::select<(copper::wait_type)0>() /home/seth/src/test-copper/./copper.h:1510:36 (test-copper+0x4ba61b)
    #18 copper::channel_op_status copper::_detail::opselect<(copper::wait_type)0, copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque>&>(std::tuple<copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque>&>&&) /home/seth/src/test-copper/./copper.h:1335:29 (test-copper+0x4ba61b)
    #19 std::enable_if<((copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque>::is_op_base) && ...), copper::channel_op_status>::type copper::select<copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque> >(copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque>&&) /home/seth/src/test-copper/./copper.h:1444:12 (test-copper+0x4ba61b)
    #20 b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*) /home/seth/src/test-copper/test-copper.cpp:9:50 (test-copper+0x4ba61b)
    #21 void std::__invoke_impl<void, void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*>(std::__invoke_other, void (*&&)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*&&, copper::channel<true, bool, std::queue, std::deque>*&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60:14 (test-copper+0x4c21b7)
    #22 std::__invoke_result<void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*>::type std::__invoke<void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*>(void (*&&)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*&&, copper::channel<true, bool, std::queue, std::deque>*&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:95:14 (test-copper+0x4c21b7)
    #23 void std::thread::_Invoker<std::tuple<void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/thread:264:13 (test-copper+0x4c21b7)
    #24 std::thread::_Invoker<std::tuple<void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*> >::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/thread:271:11 (test-copper+0x4c21b7)
    #25 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*> > >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/thread:215:13 (test-copper+0x4c21b7)
    #26 <null> <null> (libstdc++.so.6+0xda693)

  Mutex M12 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null> (test-copper+0x444ab6)
    #1 __gthread_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10/bits/gthr-default.h:749:12 (test-copper+0x4bf03e)
    #2 __gthread_recursive_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10/bits/gthr-default.h:811:10 (test-copper+0x4bf03e)
    #3 std::recursive_mutex::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/mutex:106:17 (test-copper+0x4bf03e)
    #4 std::unique_lock<std::recursive_mutex>::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/unique_lock.h:138:17 (test-copper+0x4bf03e)
    #5 std::unique_lock<std::recursive_mutex>::unique_lock(std::recursive_mutex&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/unique_lock.h:68:2 (test-copper+0x4bf03e)
    #6 copper::channel_op_status copper::channel<true, bool, std::queue, std::deque>::pop_func_wt<(copper::wait_type)0, copper::_detail::popper_base<true, bool, std::queue, std::deque>&>(copper::_detail::popper_base<true, bool, std::queue, std::deque>&) /home/seth/src/test-copper/./copper.h:454:21 (test-copper+0x4bf03e)
    #7 copper::channel_op_status copper::_detail::popper_base<true, bool, std::queue, std::deque>::select<(copper::wait_type)0>() /home/seth/src/test-copper/./copper.h:865:41 (test-copper+0x4ba61b)
    #8 copper::channel_op_status copper::_detail::select_manager<copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque> >::select<(copper::wait_type)0>() /home/seth/src/test-copper/./copper.h:1510:36 (test-copper+0x4ba61b)
    #9 copper::channel_op_status copper::_detail::opselect<(copper::wait_type)0, copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque>&>(std::tuple<copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque>&>&&) /home/seth/src/test-copper/./copper.h:1335:29 (test-copper+0x4ba61b)
    #10 std::enable_if<((copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque>::is_op_base) && ...), copper::channel_op_status>::type copper::select<copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque> >(copper::_detail::popper<b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*)::$_0, true, bool, std::queue, std::deque>&&) /home/seth/src/test-copper/./copper.h:1444:12 (test-copper+0x4ba61b)
    #11 b_run(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*) /home/seth/src/test-copper/test-copper.cpp:9:50 (test-copper+0x4ba61b)
    #12 void std::__invoke_impl<void, void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*>(std::__invoke_other, void (*&&)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*&&, copper::channel<true, bool, std::queue, std::deque>*&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60:14 (test-copper+0x4c21b7)
    #13 std::__invoke_result<void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*>::type std::__invoke<void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*>(void (*&&)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*&&, copper::channel<true, bool, std::queue, std::deque>*&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:95:14 (test-copper+0x4c21b7)
    #14 void std::thread::_Invoker<std::tuple<void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/thread:264:13 (test-copper+0x4c21b7)
    #15 std::thread::_Invoker<std::tuple<void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*> >::operator()() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/thread:271:11 (test-copper+0x4c21b7)
    #16 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*), copper::channel<true, bool, std::queue, std::deque>*, copper::channel<true, bool, std::queue, std::deque>*> > >::_M_run() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/thread:215:13 (test-copper+0x4c21b7)
    #17 <null> <null> (libstdc++.so.6+0xda693)

  Mutex M12 acquired here while holding mutex M14 in main thread:
    #0 pthread_mutex_lock <null> (test-copper+0x444ab6)
    #1 __gthread_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10/bits/gthr-default.h:749:12 (test-copper+0x4bb59d)
    #2 __gthread_recursive_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10/bits/gthr-default.h:811:10 (test-copper+0x4bb59d)
    #3 std::recursive_mutex::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/mutex:106:17 (test-copper+0x4bb59d)
    #4 std::unique_lock<std::recursive_mutex>::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/unique_lock.h:138:17 (test-copper+0x4bb59d)
    #5 std::unique_lock<std::recursive_mutex>::unique_lock(std::recursive_mutex&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/unique_lock.h:68:2 (test-copper+0x4bb59d)
    #6 copper::channel_op_status copper::channel<true, bool, std::queue, std::deque>::push_func_wt<(copper::wait_type)0, bool copper::channel<true, bool, std::queue, std::deque>::push_wt<(copper::wait_type)0, bool, bool, 0>(bool&&)::'lambda'()>(bool&&, bool&&...) /home/seth/src/test-copper/./copper.h:532:21 (test-copper+0x4bb59d)
    #7 bool copper::channel<true, bool, std::queue, std::deque>::push_wt<(copper::wait_type)0, bool, bool, 0>(bool&&) /home/seth/src/test-copper/./copper.h:510:19 (test-copper+0x4baa12)
    #8 bool copper::channel<true, bool, std::queue, std::deque>::push<bool>(bool&&) /home/seth/src/test-copper/./copper.h:538:22 (test-copper+0x4baa12)
    #9 main::$_1::operator()(bool) const /home/seth/src/test-copper/test-copper.cpp:29:33 (test-copper+0x4baa12)
    #10 void std::__invoke_impl<void, main::$_1&, bool>(std::__invoke_other, main::$_1&, bool&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60:14 (test-copper+0x4baa12)
    #11 std::__invoke_result<main::$_1&, bool>::type std::__invoke<main::$_1&, bool>(main::$_1&, bool&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:95:14 (test-copper+0x4baa12)
    #12 std::invoke_result<main::$_1&, bool>::type std::invoke<main::$_1&, bool>(main::$_1&, bool&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/functional:88:14 (test-copper+0x4baa12)
    #13 copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque>::operator()(bool&&) /home/seth/src/test-copper/./copper.h:893:36 (test-copper+0x4baa12)
    #14 copper::channel_op_status copper::channel<true, bool, std::queue, std::deque>::_pop_func_with_lock_general<(copper::wait_type)0, copper::_detail::popper_base<true, bool, std::queue, std::deque>&, std::unique_lock<std::recursive_mutex> >(copper::_detail::popper_base<true, bool, std::queue, std::deque>&, std::unique_lock<std::recursive_mutex>&) /home/seth/src/test-copper/./copper.h:687:21 (test-copper+0x4bf350)
    #15 copper::channel_op_status copper::channel<true, bool, std::queue, std::deque>::pop_func_wt<(copper::wait_type)0, copper::_detail::popper_base<true, bool, std::queue, std::deque>&>(copper::_detail::popper_base<true, bool, std::queue, std::deque>&) /home/seth/src/test-copper/./copper.h:455:22 (test-copper+0x4bf057)
    #16 copper::channel_op_status copper::_detail::popper_base<true, bool, std::queue, std::deque>::select<(copper::wait_type)0>() /home/seth/src/test-copper/./copper.h:865:41 (test-copper+0x4ba77b)
    #17 copper::channel_op_status copper::_detail::select_manager<copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque> >::select<(copper::wait_type)0>() /home/seth/src/test-copper/./copper.h:1510:36 (test-copper+0x4ba77b)
    #18 copper::channel_op_status copper::_detail::opselect<(copper::wait_type)0, copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque>&>(std::tuple<copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque>&>&&) /home/seth/src/test-copper/./copper.h:1335:29 (test-copper+0x4ba77b)
    #19 std::enable_if<((copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque>::is_op_base) && ...), copper::channel_op_status>::type copper::select<copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque> >(copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque>&&) /home/seth/src/test-copper/./copper.h:1444:12 (test-copper+0x4ba77b)
    #20 main /home/seth/src/test-copper/test-copper.cpp:26:50 (test-copper+0x4ba77b)

  Mutex M14 previously acquired by the same thread here:
    #0 pthread_mutex_lock <null> (test-copper+0x444ab6)
    #1 __gthread_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10/bits/gthr-default.h:749:12 (test-copper+0x4bf03e)
    #2 __gthread_recursive_mutex_lock(pthread_mutex_t*) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10/bits/gthr-default.h:811:10 (test-copper+0x4bf03e)
    #3 std::recursive_mutex::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/mutex:106:17 (test-copper+0x4bf03e)
    #4 std::unique_lock<std::recursive_mutex>::lock() /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/unique_lock.h:138:17 (test-copper+0x4bf03e)
    #5 std::unique_lock<std::recursive_mutex>::unique_lock(std::recursive_mutex&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/unique_lock.h:68:2 (test-copper+0x4bf03e)
    #6 copper::channel_op_status copper::channel<true, bool, std::queue, std::deque>::pop_func_wt<(copper::wait_type)0, copper::_detail::popper_base<true, bool, std::queue, std::deque>&>(copper::_detail::popper_base<true, bool, std::queue, std::deque>&) /home/seth/src/test-copper/./copper.h:454:21 (test-copper+0x4bf03e)
    #7 copper::channel_op_status copper::_detail::popper_base<true, bool, std::queue, std::deque>::select<(copper::wait_type)0>() /home/seth/src/test-copper/./copper.h:865:41 (test-copper+0x4ba77b)
    #8 copper::channel_op_status copper::_detail::select_manager<copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque> >::select<(copper::wait_type)0>() /home/seth/src/test-copper/./copper.h:1510:36 (test-copper+0x4ba77b)

    #9 copper::channel_op_status copper::_detail::opselect<(copper::wait_type)0, copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque>&>(std::tuple<copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque>&>&&) /home/seth/src/test-copper/./copper.h:1335:29 (test-copper+0x4ba77b)
    #10 std::enable_if<((copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque>::is_op_base) && ...), copper::channel_op_status>::type copper::select<copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque> >(copper::_detail::popper<main::$_1, true, bool, std::queue, std::deque>&&) /home/seth/src/test-copper/./copper.h:1444:12 (test-copper+0x4ba77b)
    #11 main /home/seth/src/test-copper/test-copper.cpp:26:50 (test-copper+0x4ba77b)

  Thread T1 (tid=2594199, running) created by main thread at:
    #0 pthread_create <null> (test-copper+0x426d0b)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda969)
    #2 __libc_start_main csu/../csu/libc-start.c:332:16 (libc.so.6+0x28564)

SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/home/seth/src/test-copper/test-copper+0x444ab6) in pthread_mutex_lock
==================

Platform

$ uname -a
Linux chirp 5.11.0-22-generic #23-Ubuntu SMP Thu Jun 17 00:34:23 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ clang --version
Ubuntu clang version 12.0.0-1ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin