Every node should have a JSON-RPC REST server to store data.

the REST protocol shall accept a header for a password. The password shall be generated by the RSV/PUT/GET/DELETE contracts

Current state

The current implementation of kad::Node has too many responsabilities, making the class big, harder to grasp and a bit messy (ok, currently it's not that terrible, but it will be the case soon as we add more feature (latency, simulation/real mode, use of NTTEC, …).

Proposed solution

Before adding more feature (latency, erasure coding) my plan is to rework kad::Node a little bit.
Concretely, I want to extract the Kademlia implementation & logic into dedicated classes (in the namespace dht or kad::dht maybe).

As I see it, we would need two entities: a DHT entry (which represents an Key-Value pair that is stored in the DHT) and a DHT node (which stores a subset of the DHT and participate to the routing).

The existing classes would be impacted as follow:

• kad::File would inherit from dht::Entry: for now, that would be the only change, but we will add more stuff in kad::File when we introduce erasure coding.
• kad::Node would inherith from dht::Node in order to be part of the Kademlia network.
• kad::Routable will be removed: it's not a really good abstraction, and its blurs the difference between a node and an entry (most of the functions that works on a kad::Routable expect a kad::File or can work with a CBignum directly: actually, I noticed that a lot of "fake" kad::Routable were created only to wrap a CBignum in order to use the API).

Overview

A node would be composed of two sub-component:

• the kad::Node: the public interface for the client, it will expose an API based on the standard GET, PUT, DELETE, … and will perform the encoding.
• the dht::Node: that will be used for the communication inter-node (routing, key dispatching, …)

The kad::Node will expose a JSON-RPC server to communicate with the client.
The dht::Node will use dht::NodeCom module (temporary name) that will be used for the communication inter-node (using Kademlia messages, such as FIND_NODE, STORE, …).
The dht::NodeCom would be a template parameter that would be used by dht::Node to communicate. Then we could have different implementation of dht::NodeCom:

• dht::NodeRpcCom that would communicate over the network (for "real" use case).
• dht::NodeLocalCom that would call the function directly (for "simulation" mode).

Here is an ugly schema to visualize how it looks like:

Current state of the work

I have a PR almost finished (compile but crash) that creates dht::Node and dht::Entry and adapt kad::Node and kad::File to rely on them.

In order to:

• see what's happen during a simulation
• ease the troubleshooting when things go awry

We should have a proper logging implemented in Quadiron.

After a quick look at the library available, I think we can go with easyloggingpp. It has minimal dependencies and a good feature set, amongst which:

• loggers can be setup from a configuration file
• loggers can be set independantly (normal case => enable simulator logging, disable DHT logging; debugging => enable tracing for the DHT module).

See https://github.com/vrancurel/ntl

The goal here is to have a standalone node which can interact with other nodes or with simulator(s).

Currently, we handle bignum by using a wrapper around OpenSSL implementation.
I saw several drawbacks to that:

• we have to maintain the wrapper
• no support for 256-bit integer (it is commented out), and this is the default integer type in the blockchain.
• we have to handle different version of OpenSSL (see https://github.com/vrancurel/quadiron/pull/4)
• seems to rely a lot on memory allocation (not a big deal for now)

An alternative would be cpp_int from the "multiprecision" package of Boost.
Advantage:

• almost no maintenance on our side
• native support for 128 and 256-bit integer
• support for 160-bit integer (address in the blockchain) is as easy as a typedef
• efficient code (it's Boost + it seems like default types are implemented with no new/malloc)

When we will have a functionnal independant node, we will probably want to write "feature" tests or test scenario that won"t fit in unit test.
Cumcumber/Behave/… can help with that.

Quadiron should be able to use NTTEC to split a file into chunks and distribute them to the nodes.
It should also be able to retrieve the chunks to rebuild the original file.

What we need is:

• to have an erasure coding module, that relies on NTTEC
• to have two new method (for now in the simulator shell, later in the RPC interface):
  • "put filename" (when using REST we will give the data directly instead of the filepath): read the file, apply erasure coding on it and store the chunks.
  • "get KEY": retrieve the data identified by KEY (find the chunks, fetch them and rebuild the file, send back the file to the user).

We should also be able to retrieve the file, even if some parts are missing (that's the reason for erasure coding after all).

Clang (and GCC) comes with a set of sanitizers that are useful (asan for Valgrind-like check, ubsan for undefined behavior, tsan for multithread code, …)

It would be good to be able to enable them on demand (like coverage), run test with them, …

In order to have a consistent code (this improve the readability) we should setup an auto-formatting (that way, we don't need to bother/argue about coding style during review) using clang-format.

Currently, you can control the simulator by executing command on its prompt.
It's good for interactive use, but not that much for playing "demo scenario" or "test script".

Ideally, the simulatior should be totally controllable through a REST API (exposing all the command that exists in the shell, plus some other such as "add node", "remove node", …).

The goal of this issue is to design and implement a REST API for the simulator (TBC: it will probably rely on Swagger).

This issue is a prerequisite for #36.