infinit / elle

The Elle coroutine-based asynchronous C++ development framework.

Elle, the coroutine-based asynchronous C++ development framework

Elle is a collection of libraries, written in modern C++ (C++14). It contains a rich set of highly reusable concepts, algorithms, API wrappers, ...

Elle is split into different specialized sub-libraries to provide elegant ways to approach asynchronism (using coroutines), networking, formatting, serialization, logging, RPCs, etc.

Notes:

• Elle is under development, used and maintained by Infinit as a core library. APIs, structures and concepts may change over time. You can use it as is but we don't guarantee any API backward compatibility.
• Elle has a sub-library also called elle, which might change name in a near future.

Example

Here is an example showing an asynchronous HTTP operation in a natural form (no callbacks) and basic JSON serialization.

// Initialize the HTTP Request.
elle::reactor::http::Request r("https://en.wikipedia.org/w/api.php",
                               elle::reactor::http::Method::GET);
r.query_string({
  {"format", "json"},
  {"action", "query"},
  {"prop", "extracts"},
  {"explaintext", ""},
  {"exintro", ""},
  {"titles", "JSON"}
});
// Perform the HTTP request and yield until response is available.
r.finalize();
// Deserialize the json response.
std::cout << elle::json::pretty_print(elle::json::read(r)) << std::endl;

Getting Elle

To download the source code and build Elle by yourself, get it from GitHub.

git clone https://github.com/infinit/elle --recursive # Clone elle and its submodules.

Note: If you cloned it using the GitHub "clone" button, do not forget to run git submodule update --init --recursive!

Note: If you forked Elle, you'll also need to fork the following repositories too: drake, dokan, libutp, miniupnp. The reason why is detailed on following GitHub issue.

Structure

As mentioned earlier, Elle is composed of a set of sub-libraries, designed to ease C++ development through robust and flexible implementations, including:

• elle: Utilities including serialization, logs, buffer, formatting, ...
• reactor: An asynchronous framework using a coroutines scheduler
• cryptography: Object-oriented cryptography wrapper around OpenSSL
• protocol: Network communication designed to support RPCs
• das: Symbol-based introspection
• athena: Byzantine environment algorithms (Paxos)
• service/aws: reactorified AWS API wrapper
• service/dropbox: reactorified Dropbox API wrapper
• nbd: A Network Block Device implementation.

How to build Elle (or parts of Elle)

Requirements

• gcc (>= 4.9.2) or clang (>= 3.5.0) or mingw (>= 5.3.0).
• python3 (>= 3.4.0) and pip3.

Build system

Elle uses Drake and has it as a submodule.

How to compile

For a detailed procedure, visit our wiki: How to build.

First you need to install drakes requirements.

sudo pip3 install -r elle/drake/requirements.txt # Install Drake dependencies.

Note: If you don't want Drake dependencies to be installed system-wide, you should consider using virtualenv.

Change directory to elle/_build/<architecture> where you can find a generic Drake configuration script.

GNU/Linux

cd elle/_build/linux64
./drake //build -j 2 # Build all libraries using 2 jobs.

macOS

cd elle/_build/osx
./drake //build -j 2 # Build all libraries using 2 jobs.

Because Elle was designed to be modular, you can build specific parts of Elle by running ./drake //<module>/build:

./drake //src/elle/cryptography/build -j 2 # To build libcryptography and its dependencies.
./drake //src/elle/reactor/build -j 2 # To build the libreactor and its dependencies.
./drake //src/elle/protocol/build -j 2 # To build the libprotocol and its dependencies.
./drake //...

It will result on <module>/lib/libelle_<module>.so and its dependencies on GNU/Linux, <module>/lib/lib<module>.dylib on macOS, ...

Dependencies

Elle depends on a few libraries which are automatically downloaded and built for your system by Drake if needed.

• Boost (v1.60.0, patched)
• PatchELF (v0.9)
• zlib (v1.2.11)
• OpenSSL (v1.0.2g)
• curl (v7.48.0, patched)
• libarchive (v3.1.2, patched)
• OpenLDAP (v2.4.44)

List of projects using Elle

• Infinit

Wiki

Consult Elle's wiki for more information about Elle.

Maintainers

• Website: https://infinit.sh/open-source
• Email: [email protected]

elle's Issues

Readme: typo

The command indicated to get the submodules dependencies is: git submodules update --init --recursive
instead of git submodule update --init --recursive

Submodules

In the .gitmodules file, the path for the drake dependency is ../drake. Shouldn't it be the path towards your drake's fork repository on github?

Same for miniupnp, libutp and dokan.

Remove unnecessary null pointer checks

An extra null pointer check is not needed in functions like the following.

• Reachability
• reactor_pthread_mutex_destroy

failed to build with gcc 7.3.0

Unable to build with fresh version of g++ compiler:

g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 7.3.0-11ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-7 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 7.3.0 (Ubuntu 7.3.0-11ubuntu1)

errors look like:
In file included from ../../src/elle/reactor/Channel.hh:163:0,
from ../../src/elle/reactor/Generator.hh:5,
from ../../src/elle/reactor/Generator.cc:1:
../../src/elle/reactor/Channel.hxx: In member function ‘const T& elle::reactor::Channel<T, Container>::peek() const’:
../../src/elle/reactor/Channel.hxx:193:42: error: no matching function for call to ‘wait(const elle::reactor::Barrier&)’
reactor::wait(this->_read_barrier);
^
In file included from ../../src/elle/reactor/Channel.hxx:3:0,
from ../../src/elle/reactor/Channel.hh:163,
from ../../src/elle/reactor/Generator.hh:5,
from ../../src/elle/reactor/Generator.cc:1:
../../src/elle/reactor/scheduler.hh:332:5: note: candidate: bool elle::reactor::wait(elle::reactor::Waitable&, elle::DurationOpt)
wait(Waitable& waitable, DurationOpt timeout = {});
^~~~
../../src/elle/reactor/scheduler.hh:332:5: note: conversion of argument 1 would be ill-formed:
../../src/elle/reactor/scheduler.hh:335:5: note: candidate: bool elle::reactor::wait(const Waitables&, elle::DurationOpt)
wait(Waitables const& waitables, DurationOpt timeout = {});
^~~~
../../src/elle/reactor/scheduler.hh:335:5: note: no known conversion for argument 1 from ‘const elle::reactor::Barrier’ to ‘const Waitables& {aka const std::vectorelle::reactor::Waitable*&}’
../../src/elle/reactor/scheduler.hh:338:5: note: candidate: bool elle::reactor::wait(const std::vector<std::reference_wrapperelle::reactor::Waitable >&, elle::DurationOpt)
wait(std::vector<std::reference_wrapper> const& waitables,
^~~~
../../src/elle/reactor/scheduler.hh:338:5: note: no known conversion for argument 1 from ‘const elle::reactor::Barrier’ to ‘const std::vector<std::reference_wrapperelle::reactor::Waitable >&’
../../src/elle/reactor/scheduler.hh:342:5: note: candidate: void elle::reactor::wait(boost::signals2::signal<void()>&)
wait(boost::signals2::signal<void ()>& signal);
^~~~
../../src/elle/reactor/scheduler.hh:342:5: note: no known conversion for argument 1 from ‘const elle::reactor::Barrier’ to ‘boost::signals2::signal<void()>&’
In file included from ../../src/elle/reactor/scheduler.hh:379:0,
from ../../src/elle/reactor/Channel.hxx:3,
from ../../src/elle/reactor/Channel.hh:163,
from ../../src/elle/reactor/Generator.hh:5,
from ../../src/elle/reactor/Generator.cc:1:
../../src/elle/reactor/scheduler.hxx:122:5: note: candidate: template<class Prototype, class ... Args> void elle::reactor::wait(boost::signals2::signal&, Args&& ...)
wait(boost::signals2::signal& signal, Args&& ... args)
^~~~
../../src/elle/reactor/scheduler.hxx:122:5: note: template argument deduction/substitution failed:
In file included from ../../src/elle/reactor/Channel.hh:163:0,
from ../../src/elle/reactor/Generator.hh:5,
from ../../src/elle/reactor/Generator.cc:1:
../../src/elle/reactor/Channel.hxx:193:42: note: types ‘boost::signals2::signal’ and ‘const elle::reactor::Barrier’ have incompatible cv-qualifiers
reactor::wait(this->_read_barrier);
^
and lots of warnings like:
In file included from ../../src/elle/serialization.hh:4:0,
from ../../src/elle/Exception.hh:9,
from ../../src/elle/Error.hh:3,
from ../../src/elle/cryptography/Error.hh:3,
from ../../src/elle/cryptography/dsa/PrivateKey.cc:9:
../../src/elle/Version.hh:75:13: warning: In the GNU C Library, "minor" is defined
by <sys/sysmacros.h>. For historical compatibility, it is
currently defined by <sys/types.h> as well, but we plan to
remove this soon. To use "minor", include <sys/sysmacros.h>
directly. If you did not intend to use a system-defined macro
"minor", you should undefine it after including <sys/types.h>.
ELLE_ATTRIBUTE_R(uint8_t, minor);
^~~~~~~~~~~~~~~~~~~~~~~~~

Production readiness of athena

I‘m looking for a stable paxos implementation in C++. Since the library is comprised of different modules, how would you describe the production readiness of athena? Is it used in bigger software projects, does it handle the nasty edge cases of paxos etc?

May be There Exists a DoS Attack Opportunity

I do not know anything about the internals of the elle library, but
at echo_server.cc, line 61 there is the following code:

while (true)
    {
    // Yield until reading "<...>\n".
    elle::Buffer line = socket->read_until("\n");
    // Write the line we just get (this yields too).
    socket->write(line);
    }

Depending on the implementation and possible limits set somewhere in the
implementation of the elle library, the line

    elle::Buffer line = socket->read_until("\n");

might allow an attack, where the input stream contains about
1TB worth of characters without containing a single line break.
That might exhaust the RAM of the computer or just crash the
application, if the limits are met.
If the read_until(...) is part of the infinit, then may be in some corner case
it might also crash the infinit server.