actor-framework / incubator

Testing ground for libraries and tools that might one day find their way into the CAF main repository

CAF: the C++ Actor Framework

CAF is an open source implementation of the actor model for C++ featuring lightweight & fast actor implementations, pattern matching for messages, network transparent messaging, and more.

Online Resources

• Homepage: https://www.actor-framework.org
• Developer Blog: https://www.actor-framework.org/blog
• Guides and Tutorials: https://www.cafcademy.com/articles
• Manual: https://actor-framework.readthedocs.io
• Doxygen: https://www.actor-framework.org/doxygen/

Report Bugs / Get Help

• Open Issues on GitHub: https://github.com/actor-framework/actor-framework/issues/new
• Ask Questions on StackOverflow: https://stackoverflow.com/questions/ask?tags=c%2b%2b-actor-framework

Community

• Chat: https://gitter.im/actor-framework/chat
• Mastodon: https://fosstodon.org/@caf
• Twitter: https://twitter.com/actor_framework
• User Mailing List: https://groups.google.com/d/forum/actor-framework

Get CAF

We do not officially maintain packages for CAF. However, some of our community members made packages available for these package managers:

• Conan
• FreeBSD Ports
• Homebrew.
• VCPKG

Get the Sources

git clone https://github.com/actor-framework/actor-framework.git
cd actor-framework

Build CAF from Source

CAF uses CMake as its build system of choice. To make building CAF more convenient from the command line, we provide a configure script that wraps the CMake invocation. The script only works on UNIX systems. On Windows, we recommend generating an MSVC project file via CMake for native builds.

Using the configure Script

The script is a convenient frontend for CMake. See configure -h for a list of available options. By default, the script creates a build directory and asks CMake to generate a Makefile. A build with default settings generally follows these steps:

./configure
cd build
make
make test [optional]
make install [as root, optional]

Using CMake

To generate a Makefile for building CAF with default settings, either use a CMake GUI or perform these steps on the command line:

mkdir build
cd build
cmake ..

After this step, cmake -LH prints the most useful configuration options for CAF, their default value, and a helptext.

Other CMake projects can add CAF as a dependency by using find_package and listing the required modules (e.g., core or io) . When installing CAF to a non-standard location, set CAF_ROOT prior to calling find_package.

Dependencies

• CMake (for building CAF)
• OpenSSL (only when building the OpenSSL module)

Supported Platforms

C++ is an evolving language. Compiler vendors constantly add more language and standard library features. Since CAF runs on many platforms, this means we need a policy that on the one hand ensures that we only use a widely supported subset of C++ and on the other hand that we naturally progress with the shifting landscape to eventually catch up to more recent C++ additions (in order to not "get stuck").

So instead of singling out individual compiler versions, we build CAF for each commit on all platforms that we currently deem relevant. Everything that passes our CI is "fair game".

Our CI covers Windows (latest MSVC release), macOS (latest Xcode release), FreeBSD (latest) and several Linux distributions (via the Docker images found here). For Linux, we aim to support the current releases (that still receive active support) for the major distributions. Note that we do not build on Linux distributions with rolling releases, because they generally provide more recent build tools than distributions with traditional release schedules and thus would only add redundancy.

Build Documentation Locally

• Building an offline version of the manual requires Sphinx:

  cd manual
  sphinx-build . html

• Building an offline version of the API reference documentation requires Doxygen (simply run the doxygen command at the root directory of the repository).

Scientific Use

If you use CAF in a scientific context, please use one of the following citations:

@inproceedings{cshw-nassp-13,
  author = {Dominik Charousset and Thomas C. Schmidt and Raphael Hiesgen and Matthias W{\"a}hlisch},
  title = {{Native Actors -- A Scalable Software Platform for Distributed, Heterogeneous Environments}},
  booktitle = {Proc. of the 4rd ACM SIGPLAN Conference on Systems, Programming, and Applications (SPLASH '13), Workshop AGERE!},
  pages = {87--96},
  month = {Oct.},
  year = {2013},
  publisher = {ACM},
  address = {New York, NY, USA}
}

@article{chs-rapc-16,
  author = {Dominik Charousset and Raphael Hiesgen and Thomas C. Schmidt},
  title = {{Revisiting Actor Programming in C++}},
  journal = {Computer Languages, Systems \& Structures},
  volume = {45},
  year = {2016},
  month = {April},
  pages = {105--131},
  publisher = {Elsevier}
}

You can find the papers online at http://dx.doi.org/10.1145/2541329.2541336 and http://dx.doi.org/10.1016/j.cl.2016.01.002.

incubator's Issues

Add application base-class

Implementing Application protocols currently requires a lot of boilerplate code that is identical across all application protocols. The stacking approach requires dispatching functions to the following layers even if the function is not needed for the application itself. This leads to a lot of code similar to:

void some_func(Args args) {
  next_layer_.some_func(args);
}

We should think about providing an application_base, that implements a default implementation for this. In the case that future applications require a different behaviour, these functions could simply be overridden.

Slight race-condition in datagram_transport test

It seems that there is a slight race-condition in the datagram_transport test. Every couple hundred runs of that test fail.

for i in {0..1000}; do ./incubator-test -s datagram_transport -v5; done

Message oriented BASP

The application layer in the current design is message-oriented, while BASP is stream-oriented. We should think about reworking the current stream-oriented design to be message oriented, so that the underlying communication model stays
consistent.
An easy way to do so would be an adaptor that is situated directly on top of the transport abstraction. This adaptor could process the received data and create a message that is passed on further up the stack. This way, working with datagrams is less of a problem, while this change would not affect stream and message oriented transport protocols.

What do you think about this idea @Neverlord ?

Datagram_transport::packet unnecessarily allocates memory when accessing buffers

The function get_buffer_ptrs in datagram_transport::packet will always create a new vector with the ptrs of the contained buffers. A span would fully suffice in the use case.

Furthermore, in the transport_base the buffer_cache_type is a std::vector<byte_buffer>, while a std::deque would fit much better for the use-case of only accessing the cache at begin and end.

Resolve returns non existent ipv6 address for localhost

The net::ip::resolve() call resolves "localhost" to an ipv4 and an ipv6 address even though the actual interface does not have an ipv6 address.

I would expect that resolve() only returns an ipv4 address.

I've tested this on mobi7 with the caf-jenkins-fedora-28 docker image. ip a gives me:

[root@110ca512e5fc incubator]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
4586: eth0@if4587: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ac:11:00:04 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.17.0.4/16 brd 172.17.255.255 scope global eth0
       valid_lft forever preferred_lft forever

The result of resolve is a vector containing the addresses:

  -> 127.0.0.1 [line 47]
  -> ::1 [line 47]

caf.net SSL support

make_udp_datagram_socket returns wrong port

make_udp_datagram_socket() returns std::pair<udp_datagram_socket, uint16_t>, which contains the socket and the port the socket was bound on. The port is wrong though.

This can be checked by calling local_port() with the returned socket, which will return the actually bound port.

Test results are inconsistent

After adding CAF_REQUIRE(false); to a test case in libcaf_net the tests still passed (ninja -C build test) while the test executable itself (./build/libcaf_net/caf-net-test) fails. This should be consistent.

web-socket-feed example consumes 100% CPU

When running the web-socket-feed example and connecting a client, the example starts spinning at 100% CPU after some time. The example only should show activity once per second and otherwise sit idle.

Code duplication in different transport implementations

While going through the different transport implementations I noticed that we currently duplicate a lot of code.
I think we can clean up those implementations by supplying something like a transport_base, which encapsulates the shared state and functions. The different transport-implementations would only need to implement their individual init(), handle_read_event() and handle_write_event() functions + special state/functions.

Would that be a good idea? Or is there something that speaks against such a change @Neverlord ?

Test multiplexer shutdown

We need to implement tests for multiplexer shutdown. Currently we only test shutting down the multiplexer when its running in test setups - hence it does not run in its own thread when its being shut down.
To test the routine properly we need a test that starts a multiplexer using a dedicated thread for it. After that it should add an endpoint manager and then shuts the whole setup down.

Allow recycling of buffers from the application layer

Some applications such as the reliability option delivery from #79 need to use buffers from the buffer-caches in the transport. In the case of delivery, these buffers have to be discarded if a corresponding ACK is received, which is currently implemented by simply discarding the whole buffer.
A better solution would be to allow the application to recycle the buffer, so that it can be reused for the processing of future messages. Thus, we should think about adding a public recycle function to the interface of the transport.

Rename Transport/Application API

We should think about using more general naming for the transport and application protocol APIs. The Application layer is using a message-oriented communication model while the transport layer is using all different types depending on the transport protocol in use.
Hence, it would be more clear to rename write_packet in the transport abstraction to something more general such as send with some clear documentation about the underlying model (UDP->packet, SCTP->message, TCP->stream).
The same goes for handle_data in the application abstraction, which could be renamed to something along the lines of receive.

Do you have an opinion on that @Neverlord @josephnoir ?

Port read and write function to span

Current, we use (const) char* and size_t arguments to read/write functions and in some places span<char>.

Instead, we should consequently use span<const byte> wherever we currently use const char* pointers and span<byte> wherever we currently use char* pointers.

Separate `quic` from `transport`

This issue is only for documentation.

We should try to separate the quic library from the transport layer. Possibly a quic_application on top of the datagram_transport would be a good solution.

Undefined behavior when running incubator tests

When running applications using the incubator or the incubator tests themself, the UBSan emits runtime errors.

./build/libcaf_net/caf-net-test -v5
/Users/boss/code/actor-framework/libcaf_core/src/config_option.cpp:156:10: runtime error: call to function caf::expected<caf::config_value> caf::detail::parse_impl_delegate<int>(void*, caf::string_view) through pointer to incorrect function type 'caf::expected<caf::config_value> (*)(void *, caf::string_view)'
make_config_option.hpp:73: note: caf::expected<caf::config_value> caf::detail::parse_impl_delegate<int>(void*, caf::string_view) defined here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/boss/code/actor-framework/libcaf_core/src/config_option.cpp:156:10 in
/Users/boss/code/actor-framework/libcaf_core/caf/detail/unique_function.hpp:159:14: runtime error: member call on address 0x6040000006d0 which does not point to an object of type 'caf::detail::unique_function<caf::behavior (caf::local_actor *)>::wrapper'
0x6040000006d0: note: object is of type 'caf::detail::init_fun_factory_helper<caf::stateful_actor<caf::(anonymous namespace)::spawn_serv_state, caf::event_based_actor>, caf::behavior (*)(caf::stateful_actor<caf::(anonymous namespace)::spawn_serv_state, caf::event_based_actor>*), std::__1::tuple<>, true, true>'
 03 00 80 27  d0 34 03 0e 01 00 00 00  00 be be be be be be be  00 00 00 00 00 00 00 00  00 7d 8f 0c
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for 'caf::detail::init_fun_factory_helper<caf::stateful_actor<caf::(anonymous namespace)::spawn_serv_state, caf::event_based_actor>, caf::behavior (*)(caf::stateful_actor<caf::(anonymous namespace)::spawn_serv_state, caf::event_based_actor>*), std::__1::tuple<>, true, true>'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/boss/code/actor-framework/libcaf_core/caf/detail/unique_function.hpp:159:14 in
/Users/boss/code/actor-framework/libcaf_core/caf/detail/unique_function.hpp:179:7: runtime error: member call on address 0x6040000006d0 which does not point to an object of type 'caf::detail::unique_function<caf::behavior (caf::local_actor *)>::wrapper'
0x6040000006d0: note: object is of type 'caf::detail::init_fun_factory_helper<caf::stateful_actor<caf::(anonymous namespace)::spawn_serv_state, caf::event_based_actor>, caf::behavior (*)(caf::stateful_actor<caf::(anonymous namespace)::spawn_serv_state, caf::event_based_actor>*), std::__1::tuple<>, true, true>'

The tests run fine even with these errors, but we should look into this behavior.

Implement retries

Curently the transports try to read once per read_event which is not that performant. For example if only part of a packet was received during a read_event the transport will return and wait for another event to be triggered.
It would be much more performant to retry a couple of times during a read_event.

Implement Datagram layer for applications

Currently it is only possible to use stream oriented transport protocols to communicate between different nodes. If we want to be able to use datagram oriented protocols we will have to implement some kind of datagram layer, which extracts the data and passes it piecewise into the application.

Add shutdown function to the socket manager

Many work flows open a socket, read some amount of data, then write to the socket until completion before closing the connection. HTTP is a prime example. Currently, the socket manager has no shutdown function for closing a connection after flushing all buffers. It "kinda works" at the moment for most cases, because the socket managers ultimately get cleaned up when their reference count drops to 0. That is unless some other part of the application holds a reference to the manager in question.

Implement high-level API

This issue is solely meant for getting this off my mind.
We need to think of a high level API that handles connecting to other nodes, creating sockets and so on.
For this we need to implementi a proper middleman and fitting middleman_backends for specific protocols like TCP, UDP and so on.

Dispatcher never removes unused application stacks

The current design for stateless transport protocols, intends to use a dispatching layer to dispatch incoming data to the corresponding application stack. The problem is, no stack will ever be removed from the dispatcher, which will eventually lead to many unused stacks.
We should think of a way of closing such a "connection", so that the unused stacks can be cleaned up, when they are not needed anymore.