aeternity / aeternity

æternity blockchain - scalable blockchain for the people - smart contracts, state channels, names, tokens

Aeternity node

A new blockchain for æpps.

Optimized for scalability via smart contracts inside state-channels.

Has a built-in oracle for integration with real-world data.

Comes with a naming system, for developerability.

Written in Erlang.

To install and run the Aeternity node, see the instructions below or just follow the progress of the project via GitHub Issues.

If you have discovered a bug or security vulnerability please get in touch. The Aeternity Crypto Foundation pays bug bounties up to 100.000 AE Tokens for critical vulnerabilities. Please get in touch via [email protected].

Documentation

For an overview of the installation process for different platforms, building the package from source, configuration and operation of the Aeternity node please refer to Aeternity node documentation.

We keep our protocol, APIs and research spec in separate protocol repository.

How to start

We publish packages for major platforms on GitHub. Each release comes with release notes describing the changes of the Aeternity node in each particular version.

Please use the latest published stable release rather than the master branch. The master branch tracks the ongoing efforts towards the next stable release to be published though it is not guaranteed to be stable.

Quick Install

Linux / Mac

By using the installer to install the latest stable version:

bash <(curl -s https://install.aeternity.io/install.sh)

See the documentation for starting and configuring the node.

Docker

Alternatively, you can run the node client as a docker container:

Linux / Mac

Or running a docker container (latest tag):

mkdir -p ~/.aeternity/maindb
docker pull aeternity/aeternity
docker run -p 3013:3013 -p 3015:3015 \
    -v ~/.aeternity/maindb:/home/aeternity/node/data/mnesia \
    aeternity/aeternity

Windows

mkdir %APPDATA%\aeternity\maindb
docker pull aeternity/aeternity
docker run -p 3013:3013 -p 3015:3015 -v %APPDATA%/aeternity/maindb:/home/aeternity/node/data/mnesia aeternity/aeternity

Restore from snapshot

To speed up the initial blockchain synchronization the node database can be restored from a snapshot following the below steps:

• delete the contents of the database if the node has been started already
• download the database snapshot
• verify if the snapshot checksum matches the downloaded file
• unarchive the database snapshot

Note that the docker container must be stopped before replacing the database

The following snippet can be used to replace the current database with the latest mainnet snapshot assuming the database path is ~/.aeternity/maindb:

rm -rf ~/.aeternity/maindb/ && mkdir -p ~/.aeternity/maindb/
curl -o ~/.aeternity/mnesia_main_v-1_latest.tar.zst https://aeternity-database-backups.s3.eu-central-1.amazonaws.com/main_backup_v1_full_latest.tar.zst
CHECKSUM=$(curl https://aeternity-database-backups.s3.eu-central-1.amazonaws.com/main_backup_v1_full_latest.tar.zst.md5)
diff -qs <(echo $CHECKSUM) <(openssl md5 -r ~/.aeternity/mnesia_main_v-1_latest.tar.zst | awk '{ print $1; }')
test $? -eq 0 && tar --use-compress-program=unzstd -xf ~/.aeternity/mnesia_main_v-1_latest.tar.zst -C ~/.aeternity/maindb/

Additional resources

• Threat Model

aeternity's Issues

Peers utility - basic code

Utility to talk with other peers.
I propose there should be a function for every request. This way we will limit typos in request names (to those functions) and we will be able to check the types of data provided to requests.
This depends on having any external API methods.

This issue contains only the basic code. No complex functions. Just functions common to all requests type and probably one request as an example of implementation.

Secure the HTTP API

Enable SSL in Cowboy.

Implement peering

• Keep track of peers [simply]!(#37 (comment)).
• Hardcode the initial list of peers.
• Implement the dead peer scavenger.
• Add unit and integration tests.

Define governance constants

Allow the node to run in attached mode

I was working on the Dockerfile PR for the previous version, and the fact that the node was running in the background only was making things a bit tricky with Docker.

If you decide to keep it this way, can we make this behavior optional? It might also simplify integration with systemd and other tools that manage processes.

As an example, nginx runs in the background by default, but allows to configure it so it can run in the foreground, which is how the Docker image is running it.

Broadcast transaction

• Check that the nonce of the transaction is the account nonce + 1.
• Sign the transaction and include the public key. Avoid the transaction malleability problem.
• Do not check the sender's account balance as it will be done when creating the block proposal.

Validate received block

Header checks...

• Protocol version should match.
• Fields should not be empty.
• PoW evidence should be correct.
• Difficulty should be no lower than target.
• Time should not be too far into the future. Should be checked against the machine UTC time, not previous block time.

Block checks...

• Transaction digest.
• State digest.

HTTP API

Implement internal and external HTTP APIs using Swagger. This is the tracking ticket that we'll add other APIs tickets to.

• One API
• Two API

Oracles

Parent ticket for oracles.

Javascript light wallet

Investigate using LevelDB for storage

Investigate using LevelDB or RocksDB for transaction and block storage.

We should try to use an existing storage layer as much as we can and avoid rolling a home-grown one.

Exonum is using LevelDB and we should look into whether we should too. There's a LevelDB API for Erlang that we can piggyback on.

Handle orphaned blocks

We may find a higher block in a chain that does not include our current block. We'll need to go with the longest chain and orphan our current block. Its transactions need to be returned to the mempool if not part of the longest chain. See #16.

Serialize Merkle trees, blocks and headers for sending over the network

See #29 for the Merkle tree implementation.

• The serialization format should be easily parsed by JavaScript clients while being reasonably compact.

Add RocksDB to the project skeleton

Persist state of the world between node restarts

Each full node will rebuild the state of the world by processing the blockchain since the genesis block. This is a time-consuming operation so periodically saving the world to disk (RocksDB?) would be helpful. The node can restore the world after a restart and bring it up to update it based on the latest information received from peers.

Mine block using Cuckoo Cycle PoW

Depends on

• #5

Consensus

Tracking ticket for consensus.

• GHOST & DECOR+ consensus #244.
• Add block to the top of the chain #231.
• Handle orphaned blocks #59. This implements consensus, basically.
• Warn when merging a chain that's more than N blocks ahead #287.
• Handle orphaned blocks #59.
• Block with no parent #64.
• Return transactions to the mempool when discarding a block #233.

Send block

Block with no parent

We may receive a block that has no parent, e.g. where the full ancestry of the block cannot be determined and the block cannot be validated.

We need to

• Ask peers to fill in the history of this block.
• Keep the block around until history is filled.
• Validate the block.

Receive block

Depends on #45.

Peers and sync design

Peers

Summary

Everyone will have a list of live peers in the network. This list will never be fully in sync as peers come and go all the time. Peers will advertise themselves to network to get on the list.

Properties

• speed - estimated speed metric
• size - amount of data downloaded from peer, used for updating speed
• trust - metric of up-to-dateness and measure of the success and error rate
• last_success_time - updated to current time when
  • we make a successful request to peer
  • we learn of the peer from syncing
  • the peer advertises itself to us
• last_fail_time - updated when we fail to make a request to the peer.

Lifetime of peer in memory

• alive - there was a recent connection, i.e. last_success_time > last_fail_time
• dead - last_fail_time > last_success_time. This peer is not used except when telling other peers that it's dead.
• known - There was no recent update. This peer is not used but we keep the info around in case it comes back online.
• delete - peer should be deleted if we don't receive any updates about it for a long time.

Definitions

FastPeer: a peer with enough trust and speed to be in the top N peers. Should sync fast.

Peer: we use random peers to be sure we get the longest chain.

RandomFastPeer: chosen randomly from the list of FastPeers that is sorted by speed.

Notes

When notified of new peers we should randomly test some.

Light nodes will only ask for 100-1000 random live peers, it doesn't need to know them all.

We want to hit the sweetspot between fully trusting the list of peers we receive and checking every single peer. The former may get us spammed with random IP addresses. The latter would create a lot of traffic and DDoS a peer that just connected to the network.

Sync

Summary

When starting the node, sync will start off by one-time execution of functions to get us up-to-date with blockchain state as fast as possible. First Peers, then Headers, then Blocks. In the meen time it starts routines to keep the data up-to-date. By using one-time functions specially crafted for the task we are able to tell when the node is ready.

Phases

Syntax

[f] = a blocking function
[+] = start of an ongoing task
-> = result of our action (what we have after executing the step)

Sequence

1. [f] fast peers - Asks random FastPeers for some RandomFastPeers -> RandomFastPeers
2. [f] fast headers - Asks RandomFastPeers about headers. When we don't get much new stuff we continue -> low trust (fairly up-to-date) headers
3. [f] fast blocks (full only) - Asks RandomFastPeers about blocks. When we have all we continue -> blocks for the low trust headers
4. [f] random peers - ask random Peers about random Peers -> random Peers
5. [+] sync peers - routine to synchronize peers. In light mode downloads more only if we don't have enough FastPeers or Peers. Downloads all peers in full mode.
6. [f] good headers - ask random Peers about top block and get headers -> trustworthy (most probably the longest chain of) headers
7. [+] sync headers (also blocks in full mode) - routine to sync headers. Asks random Peer about top headers once in while. Downloads blocks from FastPeers.
8. [f] ready lite - set status to ready in light mode.

Everything below is for full mode only.

9. [f] good blocks - ask RandomFastPeers about blocks until headers and blocks are in sync -> blocks for trustworthy headers
10. [+] external api - we turn on external api now. External api should be on only for peers that are fully in sync. Being out of sync and having your external api on will result in low trust.
11. [+] advertiser - Once in a while we advertise ourselves to random peers. We decide if we should advertise more if nodes say they don't know us.
12. [f] ready full - set status to ready in full mode.

First sync

Before first sync peer list will be populated with some hardcoded ones plus maybe some downloaded from an API provided by Aethernity Anstalt.

Notes and issues

What if we get some peers with very high speed to mess with RandomGoodPeer??? Probably we shouldn't sync speed or do it to a limited extent.

We don't try to make sure we have all the peers, we try to have a list that's good enough. Peers should make sure they advertise themselves to be on the list (and get new blocks fast).

When we ask other peers about top block we store that information and when we establish information what's the true top block we make trustworthyness of peers that answer correctly higher and those that where significantly off lower. High trustworthyness peers are only used to get as good as possible initial sync in future. Low trustworthyness peers may be ignored as they provide false information.

How to store peers and how to synchronize them between two nodes? It would be good if we could work around sending whole lists.

The mentioned algorithm of spreading info about new peer in network may not be the best. There should be some existing very good solutions. (probably one's used by torrents networks?)

Implement the Cuckoo Cycle PoW

• Figure out how to plug in the Cuckoo Cycle PoW into Erlang.
• Implement using ports or NIF.

Upgrade of protocol version and governance

From @lucafavatella...

Two things I think we need before the launch - even more than features - are: the process of upgrade of protocol version, and the process of upgrade of governance.

• The protocol version is in the block and its header.
• I understand the governance tree shall be hashed in the block as all other state trees (e.g. accounts). (Currently the codebase does not include governance in the block.)

Configuration shall be classified:

• Items that are in governance since genesis block, hence these items are represented in the genesis block because governance is (TO-BE) hashed in block;
  • Question: Can oracles only change value of existing governance items, or also introduce new governance items?
• Items that are not in governance, and are represented by the protocol version;
  • I understand these include:
    • Fields of block header and block;
    • Serialization of block header and block;
    • Known transactions and their processing;
    • Peer-to-peer protocol (?)
  • Question: Is it possible to link a protocol version to the presence in governance of a certain item?
• Anything else?

The result shall be processes established for upgrading the protocol version (e.g. https://github.com/bitcoin/bips/blob/master/bip-0035.mediawiki) and governance. This may enable us to postpone after the launch of the mainnet items that are not urgent.

Implement SHA256 PoW

We can start with the SHA256 PoW in case there are delays with the implementation of the Cuckoo Cycle PoW.

Smart Contracts

Tracking ticket for smart contracts.

Mine block using SHA256

Depends on #43.

Create the project skeleton

Create the rebar configuration, directories, etc. to allow us to start adding blockchain components.

Validate block headers

Design the block data structure

• Figure out how transactions from issue #3 fit into a block.
• Define data structures in a single or multiple .hrl files, naming them clearly.

Let users generate a unique address whenever they want to receive tokens

We should be able to do with a single private key and multiple receive addresses. See Hierarchical Deterministic keys using safe curves

Design the block header structure

• Define data structures in a single or multiple .hrl files, naming them clearly.

Investigate storing proof with transactions

We don't have UTXOs like Bitcoin does. Need to prove that an account has enough money to spend and store this proof.

crypto:rand_uniform/2 is deprecated

crypto:rand_uniform/2 is deprecated and produces a warning. I think we should treat warning as errors. Is there some legitimate reason deprecated function has to be used?

Code warning introduced by @akorosmezey, please comment on that.

Build log:

apps/aecore/src/pow_sha256.erl:111: Warning: crypto:rand_uniform/2 is deprecated and will be removed in a future release; use rand:uniform/1

Implement Merkle trees

• Rewrite this library in Erlang.
• Add unit tests.

Describe and implement the gossip algorithm for sending out blocks

Identify transaction types and define data structures

• Collect the different transaction types from the old repo.
• Figure out if we can discard any.
• Define data structures in a single or multiple .hrl files, naming them clearly.

Source code updates as part of governance

Figure out a way to add source code updates to the governance system.

Select transaction to be added to a block (highest fee?)

Store blocks in RocksDB and fetch them

• Investigate RocksDB access patterns. How do we indexing blocks? Fetch previous block?
• What metadata do we keep in memory, e.g. to cross-check transactions for duplicates?

Add transaction to block

Handle orphaned transactions

Orphaned blocks may be accepted by the the network and later rejected when proof of a longer blockchain is received.

The user may see a transaction with 1 confirmation and then see the number of confirmations revert to 0 if a transaction has been orphaned.

See #59 and #62.

Propose an implementation of Merkle Patricia trees

We need Merkle Patricia trees, just like Ethereum.

We already researched various packages available.

We need to:

• Ignore Elixir solutions as we don't want to complicate our dev toolchain with an Elixir dependency.
• Find 2-3 Erlang packages and explain the cost of extending them for our needs.
• Check and report on any suitable C libraries and their integration cost.

Governance

Tracking ticket for governance issues.

• #18

Check that each block has no more than one coinbase transaction

Transactions

This is the tracking ticket for our transactions.

• Coinbase transaction #43.
• Spend transaction #185.
• Calculate transaction fee #246.
• Process received transaction #279.
• Broadcast transaction #62.
• Add new transactions to block while mining #68.
• Avoid transaction malleability issues #180.

State Channels

Tracking ticket for state channels.

Auto-generate Python Swagger client

Generate the Python Swagger client using swagger-codegen.

Implement block syncing

Depends on

• #11
• #12
• #13

Coinbase transaction

Depends on #15.

Implement the mempool

1. Push transactions into the mempool, e.g. when received by a node.
2. Pop transactions from the mempool, highest fee first.
3. Delete transactions from the mempool when a block that includes these transactions is received.