solidoss / solidframe

Cross-platform C++ framework for asynchronous, distributed applications.

SolidFrame

Cross-platform C++ framework for asynchronous, distributed applications.

Copyright

Copyright © 2007-present Valentin Palade (vipalade @ gmail.com).

All rights reserved.

License

Boost Software License - Version 1.0 - August 17th, 2003

Resources

• Tutorials
• Release Notes
• MPRPC library
• API Reference - TODO
• Wiki - TODO

Prerequisites

• C++14 enabled compiler
• CMake: for build system
• CxxOpts: needed by SolidFrame examples.
• OpenSSL/BoringSSL: needed by solid_frame_aio_openssl library.

Supported platforms

• Linux - gcc, llvm
• FreeBSD - llvm
• Darwin/macOS - llvm - (starting with XCode 8 which has support for thread_local)
• iOS - llvm + CocoaPods - example: Bubbles)
• Android - llvm/gcc - (starting with Android Studio 2.2 - example: Bubbles)
• Windows - MSVC - tested on Windows 10 with Visual Studio 2017

Libraries

Focused:

• solid_frame_mprpc: Message Passing - Remote Procedure Call over secure/plain TCP (MPRPC library)
  • mprpc::Service - pass messages to/from multiple peers.
• solid_frame_aio: asynchronous communication library using epoll on Linux and kqueue on FreeBSD/macOS
  • Actor - reactive object with support for Asynchronous IO, timers and evens
  • Reactor - reactor with support for Asynchronous IO
  • Listener - asynchronous TCP listener/server socket
  • Stream - asynchronous TCP socket
  • Datagram - asynchronous UDP socket
  • Timer - allows objects to schedule time based events
• solid_serialization_v2: binary serialization/marshaling
  • TypeMap
  • binary::Serializer
  • binary::Deserializer

All:

• solid_system:
  • Wrappers for socket/file devices, socket address, directory access
  • Log engine
• solid_utility:
  • Any - similar to boost::any
  • Event - Event class containing an ID a solid::Any object and a Category (similar to std::error_category)
  • InnerList - bidirectional list mapped over a vector/deque
  • Stack - alternative to std::stack
  • Queue - alternative to std:queue
  • ThreadPool - generic thread pool
• solid_serialization_v2: binary serialization/marshalling
  • TypeMap
  • binary::Serializer
  • binary::Deserializer
• solid_frame:
  • Actor - reactive object with support for timers and events
  • Manager - store services and notifies objects within services
  • Service - store and notifies objects
  • Reactor - active store of actors - allows actors to asynchronously react on events
  • Scheduler - a thread pool of reactors
  • Timer - allows actors to schedule time based events
  • shared::Store - generic store of shared objects that need either multiple read or single write access
• solid_frame_aio: asynchronous communication library using epoll on Linux and kqueue on FreeBSD/macOS
  • Actor - reactive object with support for Asynchronous IO, timers and evens
  • Reactor - reactor with support for Asynchronous IO
  • Listener - asynchronous TCP listener/server socket
  • Stream - asynchronous TCP socket
  • Datagram - asynchronous UDP socket
  • Timer - allows actors to schedule time based events
• solid_frame_aio_openssl: SSL support via OpenSSL
• solid_frame_file
  • file::Store - a shared store for files
• solid_frame_mprpc: Message Passing - Remote Procedure Call over TCP (MPRPC library)
  • mprpc::Service - pass messages to/from multiple peers.

Installation

Following are the steps for:

• fetching the SolidFrame code
• building the prerequisites folder
• building and installing SolidFrame
• using SolidFrame in your projects

Please note that boost framework is only used for building SolidFrame. Normally, SolidFrame libraries would not depend on boost.

Linux/macOS/FreeBSD

System prerequisites:

• C++14 enabled compiler: gcc-c++ on Linux and clang on FreeBSD and macOS (minimum: XCode 8/Clang 8).
• CMake

Bash commands for installing SolidFrame:

$ mkdir ~/work
$ cd ~/work
$ git clone [email protected]:vipalade/solidframe.git
$ mkdir external
$ cd extern
$ ../solidframe/prerequisites/build.sh
# ... wait until the prerequisites are built
$ cd ../solidframe
$ ./configure -e ~/work/external --prefix ~/work/external
$ cd build/release
$ make install
# ... when finished, the header files will be located in ~/work/extern/include/solid
# and the libraries at ~/work/external/lib/libsolid_*.a

If debug build is needed, the configure line will be:

$ ./configure -b debug -e ~/work/external --prefix ~/work/external
$ cd build/debug

Also if, on Linux clang toolchain is wanted for build, the configure line will be:

$ CXX=clang++ CC=clang ./configure -e ~/work/external --prefix ~/work/external

For more information about ./configure script use:

$ ./configure --help

Android

SolidFrame can be intergrated into any Android project by enabling C++ and CMake support and build SolidFrame from the project's CMakeLists.txt file, either as a git submodule or as CMake external project.

Bubbles Android Studio project, exemplifies how to integrate SolidFrame as a git submodule.

The CMakeLists.txt file also exemplifies how to use Google Snappy library as an ExternalProject which can be used as a starting point for how to integrate SolidFrame as a CMake ExternalProject too.

iOS

SolidFrame can be integrated into any iOS (Swift) project as an Cocoa Pod by adding lines as the following, to the project's Podfile:

  pod 'SolidFrame/frame_mprpc', :configuration => 'Debug', :git => 'https://github.com/vipalade/solidframe.git', :branch => 'master'
  pod 'SolidFrame/serialization_v2', :configuration => 'Debug', :git => 'https://github.com/vipalade/solidframe.git', :branch => 'master'
  pod 'SolidFrame/frame_aio_openssl', :configuration => 'Debug', :git => 'https://github.com/vipalade/solidframe.git', :branch => 'master'

Bubbles is a sample XCode Swift CocoaPod enabled project using SolidFrame.

Windows

System prerequisites:

• Visual Studio 2017 - e.g. Community Edition.
• CMake
• Git for Windows - the build flow uses Git Bash
• Perl for Windows - needed for building OpenSSL. Windows Git Bash installation also comes with perl but it won't work with OpenSSL build.

The Windows build flow, only relies on the prebuilt external folder of dependencies (on Linux/macOS/FreeBSD it is also available a build flow based on cmake's ExternalProject_Add, which automatically downdloads and builds the external prerequisites). To prepare the external folder one should use the prerequisites/build.sh script from Git Bash console as described below:

$ mkdir ~/work
$ cd ~/work
$ git clone https://github.com/vipalade/solidframe.git
$ mkdir external
$ cd external
$ ../solidframe/prerequisites/run_in_visual_studio_2017_environment.sh amd64 bash
# a new Git Bash instance will be created with Visual Studio 2017 environment setup.
# build OpenSSL
$ ../solidframe/prerequisites/build.sh --64bit
$ cd ../solidframe
$ ./configure -b release -f vsrls64 -e ~/work/external -g "Visual Studio 15 2017 Win64"
$ cd build/vsrls64
# the current folder contains SolidFrame.sln solution which can be opened in Visual Studio 2017
#
# issue a full build from from command line
$ cmake --build . --config release
# the following command can be used on build only the libraries:
# cmake --build . --config release --target libraries
#
# command used for running the tests:
$ cmake --build . --config release --target RUN_TESTS

Use SolidFrame in your projects

With CMake - the recommended way:

In CMakeLists.txt add something like:

set(SolidFrame_DIR "${EXTERNAL_DIR}/lib/cmake/SolidFrame" CACHE PATH "SolidFrame CMake configuration dir")
find_package(SolidFrame)

Where EXTERNAL_DIR points to where SolidFrame was installed (e.g. ~/work/external).

You can also directly use SolidFrame libraries from SolidFrame's build directory by specifying the SolidFrame_DIR when running cmake for your project:

$ cd my_project/build
$ cmake -DEXTERNAL_DIR=~/work/external -DSolidFrame_DIR=~/work/solidframe/build/release -DCMAKE_BUILD_TYPE=debug ..

Without CMake:

You need to specify the location for SolidFrame includes:

-I~/work/extern/include

and the location for SolidFrame libraries:

-L~/work/extern/lib

Maintainance

SolidFrame's build system has built-in support for:

• code reformatting via clang-format
• code static analisys via clang-tidy

source reformating via clang-format

For now the support is only available on Linux systems where "clang-format" package should be installed.

There is a "code-auto-format" build target which will scan and reformat all source code files using clang-format, configured via .clang-format file.

static analisys via clang-tidy

For now the support is only available on Linux systems where "clang-tidy" package should be installed.

The static analisys via clang-tidy is intergrated into the cmake build system. I can be enabled by setting "CMake_RUN_CLANG_TIDY" boolean via cmake-gui after a build configuration was created, or by specifing as parameter when creating a build configuration:

./configure -b debug -e ~/work/external -P "-DCMake_RUN_CLANG_TIDY:BOOLEAN=true"

clang-tidy is controlled via .clang-tidy configuration file.

Overview

SolidFrame is an experimental framework to be used for implementing cross-platform C++ network enabled applications or modules.

The consisting libraries only depend on C++ STL with one exception:

• solid_frame_aio_openssl: which depends on either OpenSSL or BoringSSL

In order to keep the framework as dependency-free as possible some components that can be found in other libraries/frameworks were re-implemented here too, most of them being locate in either solid_utility or solid_system libraries.

The next paragraphs will briefly present every library.

solid_system

The library consists of wrappers around system calls for:

• socketaddress.hpp: socket addresses, synchronous socket address resolver
• nanotime.hpp: high precision clock
• log.hpp: log engine
• device.hpp, seekabledevice.hpp, filedevice.hpp, socketdevice.hpp: devices (aka descriptors/handles) for: files, sockets etc.
• directory.hpp: basic file-system directory operations

The most notable component is the log engine which allows sending log records to different locations:

• file (with support for rotation)
• stderr/stdlog
• socket endpoint

The log engine defines two macros for specify log lines:

• solid_log: always consider logging the given log line
• solid_dbg: consider logging the line only on debug builds or when SOLID_HAS_DEBUG is defined (otherwise, the macros are empty - i.e. no code is generated)

    solid::log_start(std::cerr, {".*:VIEWS"});
    //...
    //a log line on the generic logger
    solid_log(generic_logger, Info, "starting aio server scheduler: "<<err.message());
    //...
    //a debug log line on a given logger:
    solid_dbg(logger, Verbose, this<<" enqueue message "<<_rpool_msg_id<<" to connection "<<this<<" retval = "<<success);

the "logger" in the above code is usually created like this:

namespace{
    const LoggerT logger("solid::frame::aio::openssl");
}

Notes on the above two blocks of code:

• The last parameter for log_start is a list of strings with the format [Regular Expression]:[V|I|E|W|S|v|i|e|w|s]
  • the regular expression is used for matching logger names
  • the flag part is used for enabling (the upper letters) or disabling (the lower letters) log levels.
• The meaning of letters that can be used in the flag part, correnspond to log levels:
  • V|v: Verbose
  • I|i: Info
  • E|e: Error
  • W|w: Warning
  • S|s: Statistics

solid_utility

The library consists of tools needed by upper level libraries:

• any.hpp: A variation on boost::any / experimental std::any with storage for emplacement new so it is faster when the majority of sizeof objects that get stored in any<> fall under a given value.
• event.hpp: Definition of an Event - a combination between something like std::error_code and an solid::Any<>.
• innerlist.hpp: A container wrapper which allows implementing bidirectional lists over a std::vector/std::deque (extensively used by the solid_frame_ipc library).
• memoryfile.hpp: A data store with file like interface.
• threadpool.hpp: Generic thread pool.
• dynamictype.hpp: Base for objects with alternative support to dynamic_cast
• dynamicpointer.hpp: Smart pointer to "dynamic" objects - objects with alternative support to dynamic_cast.
• queue.hpp: An alternative to std::queue
• stack.hpp: An alternative to std::stack
• algorithm.hpp: Some inline algorithms
• common.hpp: Some bits related algorithms

Here are some sample code for some of the above tools:

Any: Sample code

using AnyT = solid::Any<>;
//...
AnyT    a;

a = std::string("Some text");

//...
std::string *pstr = a.cast<std::string>();
if(pstr){
    cout<<*pstr<<endl;
}

Some code to show the difference from boost::any:

using AnyT = solid::Any<128>;

struct A{
    uint64_t    a;
    uint64_t    b;
    double      c;
};

struct B{
    uint64_t    a;
    uint64_t    b;
    double      c;
    char        buf[64];
};

struct C{
    uint64_t    a;
    uint64_t    b;
    double      c;
    char        buf[128];
};

AnyT    a;
//...
a = std::string("Some text");//as long as sizeof(std::string) <= 128 no allocation is made - Any uses placement new()
//...
a = A();//as long as sizeof(A) <= 128 no allocation is made - Any uses placement new()
//...
a = B();//as long as sizeof(B) <= 128 no allocation is made - Any uses placement new()
//...
a = C();//sizeof(C) > 128 so new allocation is made - Any uses new

Event: Sample code

Create a new event category:

enum struct AlphaEvents{
    First,
    Second,
    Third,
};
using AlphaEventCategory = EventCategory<AlphaEvents>;
const AlphaEventCategory    alpha_event_category{
    "::alpha_event_category",
    [](const AlphaEvents _evt){
        switch(_evt){
            case AlphaEvents::First:
                return "first";
            case AlphaEvents::Second:
                return "second";
            case AlphaEvents::Third:
                return "third";
            default:
                return "unknown";
        }
    }
};

Handle events:

void Actor::handleEvent(Event &&_revt){
    static const EventHandler<void, Actor&> event_handler = {
        [](Event &_revt, Actor &_robj){cout<<"handle unknown event "<<_revt<<" on "<<&_robj<<endl;},//fallback
        {
            {
                alpha_event_category.event(AlphaEvents::First),
                [](Event &_revt, Actor &_robj){cout<<"handle event "<<_revt<<" on "<<&_robj<<endl;}
            },
            {
                alpha_event_category.event(AlphaEvents::Second),
                [](Event &_revt, Actor &_robj){cout<<"handle event "<<_revt<<" on "<<&_robj<<endl;}
            },
            {
                generic_event_category.event(GenericEventE::Message),
                [](Event &_revt, Actor &_robj){cout<<"handle event "<<_revt<<"("<<*_revt.cast<std::string>()<<") on "<<&_robj<<endl;}
            }
        }
    };

    event_handler.handle(_revt, *this);
}

Creating events:

//...
rbase.handleEvent(alpha_event_category.event(AlphaEvents::Second));
rbase.handleEvent(generic_event_category.event(GenericEventE::Message, std::string("Some text")));
//...

InnerList: Sample code

MemoryFile: Sample code

solid_serialization_v2

• typemap.hpp: Enable polimorphism support in serializers/deserializers.
• binaryserializer.hpp: Binary "asynchronous" serializer.
• binarydeserializer.hpp: Binary "asynchronous" deserializer.
• binarybasic.hpp: Some "synchronous" load/store functions for basic types.
• typeidmap.hpp: Class for helping "asynchronous" serializer/deserializer support polymorphism: serialize pointers to base classes.

The majority of serializers/deserializers offers the following functionality:

• Synchronously serialize a data structure to a stream (e.g. std::ostringstream)
• Synchronously deserialize a data structure from a stream (e.g. std::istringstream)

This means that at a certain moment, one will have the data structure twice in memory: the initial one and the one from the stream.

The first version of the solid serialization library had two distinct steps for serilization:

• schedule items for serialization in a stack like callback structure
• do the actual item serialzation/deserialization when output space or input data is available

In order to improve speed, the second version of the library tries to overlap the above two steps - i.e. the items are scheduled only if the serialization/deserialization cannot be done inplace - e.g. the buffer is either full or empty.

Notable are two other abilities of the serialization engine:

• The support serializing streams - see ipc file tutorial especially messages definition
• The support for imposing limits on items: string, container, stream - i.e. serialization/deserialization terminates with an error if an item exceeds the limit set for the item category (either string, container, stream). This is very important when usend in an online protocol.

Sample code

A C++ structure with serialization support:

struct Test{
    using KeyValueVectorT = std::vector<std::pair<std::string, std::string>>;
    using MapT = std::map<std::string, uint64_t>;

    std::string         str;
    KeyValueVectorT     kv_vec;
    MapT                kv_map;
    uint32_t            v32;

    SOLID_SERIALIZE_V2(_s, _rthis, _rctx, _name){
        _s.add(str, _rctx, "Test::str");
        _s.add(kv_vec, _rctx, "Test::kv_vec").add(kv_map, _rctx, "Test::kv_map");
        _s.add(v32, _rctx, "Test::v32");
    }
};

Defining the typemap/serializer/deserializer:

#include "solid/serialization/serialization.hpp"

struct Context{};
struct TypeData{};

using TypeIdT       = uint8_t;//the type that is serialized and used to identify the registered type.
using TypeMapT      = serialization::TypeIdMap<TypeIdT, Context, solid::serialization::Serializer, solid::serialization::Deserializer, TypeData>;
using SerializerT   = TypeMapT::SerializerT;
using DeserializerT = TypeMapT::DeserializerT;

Prepare the typemap:

TypeMapT  typemap;
typemap.null(0);//null typeid
typemap.registerType<Test>(1/*type id*/);

Serialize and deserialize a Test structure:

std::string     data;
{//serialize
    Context         ctx;
    SerializerT     ser = typemap.createSerializer();
    const int       bufcp = 64;
    char            buf[bufcp];
    int             rv;

    std::shared_ptr<Test>   test_ptr = Test::create();
    test_ptr->init();

    ser.add(test_ptr, ctx, "test_ptr");

    while((rv = ser.run(buf, bufcp, ctx)) > 0){
        data.append(buf, rv);
    }
}
{//deserialize
    Context                 ctx;
    DeserializerT           des = typemap.createDeserializer();

    std::shared_ptr<Test>   test_ptr;

    des.add(test_ptr, ctx, "test_ptr");

    size_t                  rv = des.run(data.data(), data.size(), ctx);
    SOLID_CHECK(rv == data.size());
}

solid_frame

The library offers the base support for an asynchronous actor model with support for notification and timer events.

• manager.hpp: A synchronous, passive store of ActorBase grouped by services.
• actor.hpp: An active object with support for events: notification events and timer events.
• reactor.hpp: An active store of Actors with support for notification events and timer events.
• reactorcontext.hpp: A context class given as parameter to every callback called from the reactor.
• scheduler.hpp: A generic pool of threads running reactors.
• service.hpp: A way of grouping related Actors.
• timer.hpp: Used by Actors needing timer events.
• sharedstore.hpp: A store of shared object with synchronized non-conflicting read/read-write access.
• reactorbase.hpp: Base for all reactors
• timestore.hpp: Used by reactors for timer events support.
• schedulerbase.hpp: Base for all schedulers.
• actorbase.hpp: Base for all Actors

Usefull links

• An overview of the asynchronous actor model

solid_frame_aio

The library extends solid_frame with actors supporting IO, notification and timer events.

• aiodatagram.hpp: Used by aio::Actors to support asynchronous UDP communication.
• aiostream.hpp: Used by aio::Actors to support asynchronous TCP communication.
• aiotimer.hpp: Used by aio::Actors needing timer events.
• aiolistener.hpp: Used by aio::Actors listening for TCP connections.
• aiosocket.hpp: Plain socket access used by Listener/Stream and Datagram
• aioresolver.hpp: Asynchronous address resolver.
• aioreactorcontext.hpp: A context class given as parameter to every callback called from the aio::Reactor.
• aioreactor.hpp: An active store of aio::Actors with support for IO, notification and timer events.

Usefull links

• An overview of the asynchronous actor model
• Tutorial: aio_echo

solid_frame_aio_openssl

Work in progress: The library extends solid_frame_aio with support for Secure Sockets based on OpenSSL/BoringSSL libraries.

• aiosecuresocket.hpp: Used by aio::Stream for SSL.
• aiosecurecontext.hpp: OpenSSL context wrapper.

solid_frame_mprpc

Message Passing - Remote Procedure Call library:

• Pluggable - i.e. header only - protocol based on solid_serialization.
• Pluggable - i.e. header only - support for secure communication via solid_frame_aio_openssl.
• Pluggable - i.e. header only - support for communication compression via Snappy.

The header only plugins ensure that solid_frame_mprpc itself does not depend on the libraries the plugins depend on.

• mprpcservice.hpp: Main interface of the library. Sends mprpc::Messages to different recipients and receives mprpc::Messages.
• mprpcmessage.hpp: Base class for all messages sent through mprpc::Service.
• mprpccontext.hpp: A context class given to all callbacks called by the mprpc library.
• mprpcconfiguration.hpp: Configuration data for mprpc::Service.

Usefull links

• MPRPC README
• MPRPC Relay
• Tutorial: aio_echo
• Tutorial: mprpc_echo
• Tutorial: mprpc_request
• Tutorial: mprpc_request_ssl
• Tutorial: mprpc_file
• Tutorial: mprpc_echo_relay

solid_frame_file

The library offers a specialization of frame::ShareStore for files.

• filestore.hpp: specialization of frame::SharedStore for files with support for temporary files.
• filestream.hpp: std::stream support
• tempbase.hpp: Base class for temporary files: either in memory or disk files.

solidframe's Issues

solid_frame_mpipc: SOCKS5 support

solid_frame_aio_openssl: enhance SSL support

solid_serialization: test against protobuf & others

solid_serialization: more tests required especially for containers

solid_frame_mpipc: SSL support