bigchaindb / bigchaindb

Meet BigchainDB. The blockchain database.

BigchainDB Server

BigchainDB is the blockchain database. This repository is for BigchainDB Server.

The Basics

• Try the Quickstart
• Read the BigchainDB 2.0 whitepaper
• Check out the Hitchiker's Guide to BigchainDB

Run and Test BigchainDB Server from the master Branch

Running and testing the latest version of BigchainDB Server is easy. Make sure you have a recent version of Docker Compose installed. When you are ready, fire up a terminal and run:

git clone https://github.com/bigchaindb/bigchaindb.git
cd bigchaindb
make run

BigchainDB should be reachable now on http://localhost:9984/.

There are also other commands you can execute:

• make start: Run BigchainDB from source and daemonize it (stop it with make stop).
• make stop: Stop BigchainDB.
• make logs: Attach to the logs.
• make test: Run all unit and acceptance tests.
• make test-unit-watch: Run all tests and wait. Every time you change code, tests will be run again.
• make cov: Check code coverage and open the result in the browser.
• make doc: Generate HTML documentation and open it in the browser.
• make clean: Remove all build, test, coverage and Python artifacts.
• make reset: Stop and REMOVE all containers. WARNING: you will LOSE all data stored in BigchainDB.

To view all commands available, run make.

Links for Everyone

• BigchainDB.com - the main BigchainDB website, including newsletter signup
• Roadmap
• Blog
• Twitter

Links for Developers

• All BigchainDB Documentation
• BigchainDB Server Documentation
• CONTRIBUTING.md - how to contribute
• Community guidelines
• Open issues
• Open pull requests
• Gitter chatroom

Legal

• Licenses - open source & open content
• Imprint
• Contact Us

bigchaindb's Issues

Change instructions for installing pip3+ on Ubuntu 14.04

The current instructions for installing pip3+ on Ubuntu 14.04 (in Section 1.3.1 of the docs) do install pip3+, but it's a really old version of pip3 (which causes problems), and there are issue with upgrading it. We found that the following is a much more reliable way to install a recent version of pip3+:

sudo apt-get install python3-setuptools
sudo easy_install3 pip

Bonus: Using this, the pip command is actually the Python 3 version of pip, which is the version we want to use. (You can check the pip version using pip -V.)

Refactor the structure of a transaction

The current schema of a transaction is the following:

{
    "id": "<sha3 hash>",
    "transaction": {
        "current_owner": "<pub-key>",
        "new_owner": "<pub-key>",
        "input": "<sha3 hash>",
        "operation": "<string>",
        "timestamp": "<timestamp>",
        "data": {
            "hash": "<SHA3-256 hash hexdigest of payload>",
            "payload": { }
        }
    },
    "signature": "<signature of the transaction>"
}

(for more information about the fields check out the docs)

We can reorganize some keys in the schema to have a simpler structure, like:

{
    "id": "<sha3 hash>",
    "payload_id": "<sha3 hash of transaction.payload>",

    "transaction": {
        "from": "<pub-key>",
        "to": "<pub-key>",
        "input": "<sha3 hash>",
        "operation": "<string>",
        "timestamp": "<timestamp>",
        "payload": { }
    },

    "signature": "<signature of the transaction>"
}

The idea is to have an easier to use, less nested dictionary.

The proposed changes are:

• move transaction.data.hash to payload_id to make clearer that it is an index
• move transaction.data.payload to transaction.payload to keep the structure less nested
• remove transaction.data since it'd be empty
• rename transaction.current_owner to transaction.from to keep the name of the field time-agnostic
• rename transaction.new_owner to transaction.to to keep the name of the field time-agnostic

Having some Difficulty Installing under Docker Windows

After the successful retrieval of:
$ git clone [email protected]:bigchaindb/bigchaindb.git

the attempted build gives:

$ docker-compose build
Building bigchaindb
Traceback (most recent call last):
File "", line 3, in
File "C:\projects\compose\compose\cli\main.py", line 55, in main
File "C:\projects\compose\compose\cli\docopt_command.py", line 23, in sys_dispatch
File "C:\projects\compose\compose\cli\docopt_command.py", line 26, in dispatch
File "C:\projects\compose\compose\cli\main.py", line 174, in perform_command
File "C:\projects\compose\compose\cli\main.py", line 195, in build
File "C:\projects\compose\compose\project.py", line 246, in build
File "C:\projects\compose\compose\service.py", line 678, in build
File "c:\projects\compose\venv\lib\site-packages\docker\api\build.py", line 49, in build
File "c:\projects\compose\venv\lib\site-packages\docker\utils\utils.py", line 103, in tar
File "c:\python27-x64\Lib\tarfile.py", line 2021, in add
IOError: [Errno 13] Permission denied: 'C:\Users\whoami\AppData\Local\Google\Chrome\User Data\Default\Cache\data_0'
docker-compose returned -1

Add installation instructions for Ubuntu 14.04 on AWS

Many users will want to install BigchainDB to run on an Ubuntu 14.04 instance on AWS, so make sure the installation instructions cover anything special they need to do.

Ease the life of Windows users

related:

• http://bugs.python.org/issue2943
• pypa/pip#942
• https://github.com/cython/cython/wiki/CythonExtensionsOnWindows

provide a RESTful API

in order to use the bigchaindb remotely in a fully language agnostic way, there should be a RESTful interface with JSON as serialization format.

this way, 3rd party developers can provide wrappers for various programming languages.

Create a BigchainDB Glossary of Terms

Feel free to comment with terms you want included.

AttributeError: module 'bigchaindb.db' has no attribute 'DatabaseAlreadyExistsException'

in bigchaindb.commands.bigchain.py

    def run_start(args):
        """Start the processes to run the node"""
        run_configure(args, skip_if_exists=True)
        bigchaindb.config_utils.file_config(args.config)
        try:
            db.init()
>       except db.DatabaseAlreadyExistsException:
E       AttributeError: module 'bigchaindb.db' has no attribute 'DatabaseAlreadyExistsException'

Consider using secondary indexes in some queries

Consider using secondary indexes for

• get_spent
• get_owned_ids
• get_transaction

Design Specs List

It would be great to have a file in which design principles and specifications are listed. Just to understand the basic assumptions on which the project is built.
For example:

• How is it supposed querying to be implemented?
• about transaction-validation: Do you want to use the blockchain design to leverage the distributed content or to let multiple owners of data to share/modify it safely? Basically, is it meant to be a p2p service among clients or a p2p among servers beyond a service?
• It is not clearly stated anywhere: the idea is to represent a blockchain using a JSON/NoSql storage/datastore?

I can be helpful with wiki and documentation.

Great idea guys!

Make a diagram to illustrate monitoring setup

At first glance, the monitoring system looks rather complicated. Three Docker containers? Why? I've drawn a simple diagram to illustrate how the monitoring system fits together, to add to the Monitoring section of the docs.

I will also do some minor copy-editing in that section, to bring it in line with the rest of the documentation.

Consider removing __init__.py files under tests/ dir

See pyest Good Integration Practices

avoid __init__.py files in your test directories. This way your tests can run easily against an installed version of mypkg, independently from the installed package if it contains the tests or not.

Unified place to set the version number

Right now, the BigchainDB version number is stored in at least two places:

1. /setup.py inside the setup(), there is e.g. version='0.1.3'
2. /docs/source/conf.py in the variables named version (e.g. "0.1") and release (e.g. "0.1.3-rc4").

Maybe there will be more places in the future.

It would be nice if there was one central place (file?) where the long version name could be stored, and then a script could read that central place to modify conf.py and setup.py accordingly, so the version number stays in sync across all files.

BigchainDB container doesn't exit gracefully

The docker container works! Thank you!

$ docker-compose up
Creating bigchaindb_rethinkdb-data_1
Creating bigchaindb_rethinkdb_1
Creating bigchaindb_bigchaindb_1
Attaching to bigchaindb_rethinkdb-data_1, bigchaindb_rethinkdb_1, bigchaindb_bigchaindb_1
rethinkdb_1      | Recursively removing directory /data/rethinkdb_data/tmp
rethinkdb_1      | Initializing directory /data/rethinkdb_data
rethinkdb_1      | Running rethinkdb 2.2.4~0jessie (GCC 4.9.2)...
rethinkdb_1      | Running on Linux 3.13.0-68-generic x86_64
rethinkdb_1      | Loading data from directory /data/rethinkdb_data
bigchaindb_rethinkdb-data_1 exited with code 0
rethinkdb_1      | Listening for intracluster connections on port 29015
rethinkdb_1      | Listening for client driver connections on port 28015
rethinkdb_1      | Listening for administrative HTTP connections on port 8080
rethinkdb_1      | Listening on addresses: 127.0.0.1, 172.17.0.3, ::1, fe80::42:acff:fe11:3%1640
rethinkdb_1      | Server ready, "643d16d5c461_92r" 140afe00-9284-40b0-8c35-a6b5c291106f
bigchaindb_1     | INFO:bigchaindb.config_utils:Configuration loaded from `/root/.bigchaindb_docker/config`
bigchaindb_1     | INFO:bigchaindb.db.utils:Create:
bigchaindb_1     | INFO:bigchaindb.db.utils: - database `bigchain`
bigchaindb_1     | INFO:bigchaindb.db.utils: - tables
bigchaindb_1     | INFO:bigchaindb.db.utils: - indexes
bigchaindb_1     | INFO:bigchaindb.db.utils: - genesis block
bigchaindb_1     | INFO:bigchaindb.db.utils:Done, have fun!
bigchaindb_1     | INFO:bigchaindb.commands.bigchain:Start bigchaindb main process
bigchaindb_1     | INFO:bigchaindb.processes:starting bigchain mapper
bigchaindb_1     | INFO:bigchaindb.processes:starting backlog mapper
bigchaindb_1     | INFO:bigchaindb.processes:starting block
bigchaindb_1     | INFO:bigchaindb.processes:starting voter
bigchaindb_1     | INFO:bigchaindb.block:bootstraping block module...
bigchaindb_1     | INFO:bigchaindb.block:finished reading past transactions
bigchaindb_1     | INFO:bigchaindb.voter:voter waiting for new blocks
bigchaindb_1     | INFO:bigchaindb.block:finished bootstraping block module...
bigchaindb_1     | INFO:bigchaindb.block:starting block module...

However, the exit can be messy when I kill the container:

^CGracefully stopping... (press Ctrl+C again to force)
Stopping bigchaindb_bigchaindb_1 ... done
Stopping bigchaindb_rethinkdb_1 ... done
rethinkdb_1      | Server got SIGTERM from pid 0, uid 0; shutting down...
rethinkdb_1      | Shutting down client connections...
rethinkdb_1      | All client connections closed.
rethinkdb_1      | Shutting down storage engine... (This may take a while if you had a lot of unflushed data in the writeback cache.)
bigchaindb_1     | Process backlog_mapper:
bigchaindb_1     | Traceback (most recent call last):
bigchaindb_1     |   File "/usr/local/lib/python3.5/multiprocessing/process.py", line 254, in _bootstrap
bigchaindb_1     |     self.run()
bigchaindb_1     |   File "/usr/local/lib/python3.5/multiprocessing/process.py", line 93, in run
bigchaindb_1     |     self._target(*self._args, **self._kwargs)
bigchaindb_1     |   File "/usr/src/app/bigchaindb/processes.py", line 28, in map_backlog
bigchaindb_1     |     for change in r.table('backlog').changes().run(b.conn):
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 240, in __next__
bigchaindb_1     |     return self._get_next(None)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 251, in _get_next
bigchaindb_1     |     self.conn._read_response(self.query, deadline)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 470, in _read_response
bigchaindb_1     |     = self._socket.recvall(12, deadline)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 369, in recvall
bigchaindb_1     |     raise ReqlDriverError("Connection is closed.")
bigchaindb_1     | rethinkdb.errors.ReqlDriverError: Connection is closed.
bigchaindb_1     | Process bigchain_mapper:
bigchaindb_1     | Traceback (most recent call last):
bigchaindb_1     |   File "/usr/local/lib/python3.5/multiprocessing/process.py", line 254, in _bootstrap
bigchaindb_1     |     self.run()
bigchaindb_1     |   File "/usr/local/lib/python3.5/multiprocessing/process.py", line 93, in run
bigchaindb_1     |     self._target(*self._args, **self._kwargs)
bigchaindb_1     |   File "/usr/src/app/bigchaindb/processes.py", line 49, in map_bigchain
bigchaindb_1     |     for change in r.table('bigchain').changes().run(b.conn):
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 240, in __next__
bigchaindb_1     |     return self._get_next(None)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 251, in _get_next
bigchaindb_1     |     self.conn._read_response(self.query, deadline)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 470, in _read_response
bigchaindb_1     |     = self._socket.recvall(12, deadline)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 369, in recvall
bigchaindb_1     |     raise ReqlDriverError("Connection is closed.")
bigchaindb_1     | rethinkdb.errors.ReqlDriverError: Connection is closed.
rethinkdb_1      | Storage engine shut down.
bigchaindb_rethinkdb_1 exited with code 0
bigchaindb_bigchaindb_1 exited with code 137

and if I try to bring it up again, it fails:

$ docker-compose up
Starting bigchaindb_rethinkdb-data_1
Starting bigchaindb_rethinkdb_1
Starting bigchaindb_bigchaindb_1
Attaching to bigchaindb_rethinkdb-data_1, bigchaindb_rethinkdb_1, bigchaindb_bigchaindb_1
rethinkdb_1      | Running rethinkdb 2.2.4~0jessie (GCC 4.9.2)...
rethinkdb_1      | Running on Linux 3.13.0-68-generic x86_64
rethinkdb_1      | Loading data from directory /data/rethinkdb_data
rethinkdb_1      | Listening for intracluster connections on port 29015
rethinkdb_1      | Listening for client driver connections on port 28015
rethinkdb_1      | Listening for administrative HTTP connections on port 8080
rethinkdb_1      | Listening on addresses: 127.0.0.1, 172.17.0.3, ::1, fe80::42:acff:fe11:3%1646
rethinkdb_1      | Server ready, "643d16d5c461_92r" 140afe00-9284-40b0-8c35-a6b5c291106f
bigchaindb_rethinkdb-data_1 exited with code 0
bigchaindb_1     | INFO:bigchaindb.config_utils:Configuration loaded from `/root/.bigchaindb_docker/config`
bigchaindb_1     | INFO:bigchaindb.commands.bigchain:Start bigchaindb main process
bigchaindb_1     | INFO:bigchaindb.processes:starting bigchain mapper
bigchaindb_1     | INFO:bigchaindb.processes:starting backlog mapper
bigchaindb_1     | INFO:bigchaindb.processes:starting block
bigchaindb_1     | INFO:bigchaindb.processes:starting voter
bigchaindb_1     | INFO:bigchaindb.block:bootstraping block module...
bigchaindb_1     | Process bigchain_mapper:
bigchaindb_1     | Traceback (most recent call last):
bigchaindb_1     |   File "/usr/local/lib/python3.5/multiprocessing/process.py", line 254, in _bootstrap
bigchaindb_1     |     self.run()
bigchaindb_1     |   File "/usr/local/lib/python3.5/multiprocessing/process.py", line 93, in run
bigchaindb_1     |     self._target(*self._args, **self._kwargs)
bigchaindb_1     |   File "/usr/src/app/bigchaindb/processes.py", line 49, in map_bigchain
bigchaindb_1     |     for change in r.table('bigchain').changes().run(b.conn):
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/ast.py", line 118, in run
bigchaindb_1     |     return c._start(self, **global_optargs)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 595, in _start
bigchaindb_1     |     return self._instance.run_query(q, global_optargs.get('noreply', False))
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 457, in run_query
bigchaindb_1     |     raise res.make_error(query)
bigchaindb_1     | rethinkdb.errors.ReqlOpFailedError: cannot subscribe to table `bigchain.bigchain`: primary replica for shard ["", +inf) not available in:
bigchaindb_1     | r.table('bigchain').changes()
bigchaindb_1     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
bigchaindb_1     | Process backlog_mapper:
bigchaindb_1     | Traceback (most recent call last):
bigchaindb_1     |   File "/usr/local/lib/python3.5/multiprocessing/process.py", line 254, in _bootstrap
bigchaindb_1     |     self.run()
bigchaindb_1     |   File "/usr/local/lib/python3.5/multiprocessing/process.py", line 93, in run
bigchaindb_1     |     self._target(*self._args, **self._kwargs)
bigchaindb_1     |   File "/usr/src/app/bigchaindb/processes.py", line 28, in map_backlog
bigchaindb_1     |     for change in r.table('backlog').changes().run(b.conn):
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/ast.py", line 118, in run
bigchaindb_1     |     return c._start(self, **global_optargs)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 595, in _start
bigchaindb_1     |     return self._instance.run_query(q, global_optargs.get('noreply', False))
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 457, in run_query
bigchaindb_1     |     raise res.make_error(query)
bigchaindb_1     | rethinkdb.errors.ReqlOpFailedError: cannot subscribe to table `bigchain.backlog`: primary replica for shard ["", +inf) not available in:
bigchaindb_1     | r.table('backlog').changes()
bigchaindb_1     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
bigchaindb_1     | Process block:
bigchaindb_1     | Traceback (most recent call last):
bigchaindb_1     |   File "/usr/local/lib/python3.5/multiprocessing/process.py", line 254, in _bootstrap
bigchaindb_1     |     self.run()
bigchaindb_1     |   File "/usr/local/lib/python3.5/multiprocessing/process.py", line 93, in run
bigchaindb_1     |     self._target(*self._args, **self._kwargs)
bigchaindb_1     |   File "/usr/src/app/bigchaindb/block.py", line 180, in start
bigchaindb_1     |     self.q_new_transaction = self.bootstrap()
bigchaindb_1     |   File "/usr/src/app/bigchaindb/block.py", line 166, in bootstrap
bigchaindb_1     |     .run(b.conn)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/ast.py", line 118, in run
bigchaindb_1     |     return c._start(self, **global_optargs)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 595, in _start
bigchaindb_1     |     return self._instance.run_query(q, global_optargs.get('noreply', False))
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 457, in run_query
bigchaindb_1     |     raise res.make_error(query)
bigchaindb_1     | rethinkdb.errors.ReqlOpFailedError: Cannot perform read: primary replica for shard ["", +inf) not available in:
bigchaindb_1     | r.table('backlog').between(['rww8pSM3SDPLZrDoDjhoAWu8nds9FqHcqYJpuUPi3Q3h', r.minval], ['rww8pSM3SDPLZrDoDjhoAWu8nds9FqHcqYJpuUPi3Q3h', r.maxval], index='assignee__transaction_timestamp').order_by(index=r.asc('assignee__transaction_timestamp'))
bigchaindb_1     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
bigchaindb_1     | Traceback (most recent call last):
bigchaindb_1     |   File "/usr/local/bin/bigchaindb", line 9, in <module>
bigchaindb_1     |     load_entry_point('BigchainDB==0.1.2', 'console_scripts', 'bigchaindb')()
bigchaindb_1     |   File "/usr/src/app/bigchaindb/commands/bigchain.py", line 123, in main
bigchaindb_1     |     start(parser, globals())
bigchaindb_1     |   File "/usr/src/app/bigchaindb/commands/utils.py", line 41, in start
bigchaindb_1     |     func(args)
bigchaindb_1     |   File "/usr/src/app/bigchaindb/commands/bigchain.py", line 92, in run_start
bigchaindb_1     |     processes.start()
bigchaindb_1     |   File "/usr/src/app/bigchaindb/processes.py", line 82, in start
bigchaindb_1     |     p_voter.start()
bigchaindb_1     |   File "/usr/src/app/bigchaindb/voter.py", line 176, in start
bigchaindb_1     |     self.bootstrap()
bigchaindb_1     |   File "/usr/src/app/bigchaindb/voter.py", line 160, in bootstrap
bigchaindb_1     |     last_voted = b.get_last_voted_block()
bigchaindb_1     |   File "/usr/src/app/bigchaindb/core.py", line 580, in get_last_voted_block
bigchaindb_1     |     .run(self.conn)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/ast.py", line 118, in run
bigchaindb_1     |     return c._start(self, **global_optargs)
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 595, in _start
bigchaindb_1     |     return self._instance.run_query(q, global_optargs.get('noreply', False))
bigchaindb_1     |   File "/usr/local/lib/python3.5/site-packages/rethinkdb/net.py", line 457, in run_query
bigchaindb_1     |     raise res.make_error(query)
bigchaindb_1     | rethinkdb.errors.ReqlOpFailedError: Cannot perform read: primary replica for shard ["", +inf) not available in:
bigchaindb_1     | r.table('bigchain').filter(lambda var_1: r.row['block']['voters'].contains('rww8pSM3SDPLZrDoDjhoAWu8nds9FqHcqYJpuUPi3Q3h')).filter(lambda var_2: var_2['votes'].contains(lambda var_3: (var_3['node_pubkey'] == r.expr('rww8pSM3SDPLZrDoDjhoAWu8nds9FqHcqYJpuUPi3Q3h')))).order_by(r.desc('block_number')).limit(1)
bigchaindb_1     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                          
bigchaindb_bigchaindb_1 exited with code 1

Design of a REST API

I take here some notes about the REST API.

Taking the first look to the core API I noted down these basic operations to be mapped into endpoints:

• Read Transaction (Read)
  • by id
  • by payload
• Write transaction (Update)
• ... (add anything to have a complete list)

Next I will write algorithms to accomplish each of these operations. Like:

• Write Transaction:

bigchaindb.core.create_transaction
bigchaindb.core.sign_transaction
bigchaindb.core.write_transaction

• Read Transaction:

And creating a development server:
Is it needed to embed the server in the RethinkDB Web-interface one, or a parallel one has to be started? Is it the first option possible (reading the RDB doc now for the first time) or advisable? If the second, I would suggest Falcon Framework.

Non interactive first start

Is there a way to start the bigchaindb for the first time in non interactive mode? "-y" does not work in this case it seems

Non Deterministic assignment of transactions in S is a DoS vulnerability

listing 3 line 8 on page 24 of the white paper is:
i ∼ U ({0 , 1 , . . ., k -1 , k +1 , . . ., N -1}) # Stochastic . Any
node but k avoids duplicate vote .

This means that if the same transaction is sent to two nodes, most of the time it will be placed into S twice, one of those entries will be rejected from C. This is a DoS vector.

If I can find the place in the actual source that handles this I will mention it.

Better organization of transaction-related code

Move all transaction related code to bigchaindb.transactions instead of bigchaindb.util

Validate transaction before writing it to the backlog

Right now, bigchaindb.web.views only verifies the signature. We should call validate_transaction instead so that we can catch double spends and other errors before writing the transaction to the backlog, and send whatever exception back to the client

Make the required version of rethinkdb more general

Currently, setup.py specifies a very specific version of rethinkdb (Python package):

'rethinkdb==2.2.0.post1',

That's causing some issues already because when I try to install rethinkdb=2.2.0.post1, I get this message: Requested rethinkdb==2.2.0.post1, but installing version 2.2.0-1

Then, when I go to run bigchaindb start, it fails because it requires rethinkdb==2.2.0.post1 but 2.2.0-1 is installed.

@vrde suggested changing the setup.py requirement to something more general:

'rethinkdb>=2.2,<2.3',

I guess that bigchaindb should also be changed to have the same, looser requirement.

Add a documentation page to run a Vagrant based container on Windows or Linux

It's quite easy to provision and run a Vagrant VM:

• Install Vagrant in your host system following instructions here
• In your VM-to-be's home directory create the VagrantFile and the bash script to provision the machine (config.sh)

VagrantFile

# -*- mode: ruby -*-
# vi: set ft=ruby :

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.provision "shell", path: "config.sh"
  config.vm.box = "ubuntu/trusty64"
  config.vm.network "forwarded_port", guest: 7000, host: 7000
  config.vm.network "forwarded_port", guest: 5000, host: 5000
end

config.sh

apt-get -qqy update
apt-get -qqy upgrade
apt-get -qqy install build-essential
apt-get -qqy install curl
apt-get -qqy install git
apt-get -qqy install libssl-dev
apt-get -qqy install python3-pip
apt-get -qqy install rethinkdb
python3 -m pip install --upgrade pip
python3 -m pip install bigchaindb

• run vagrant up to start provisioning (SO and libraries get installed)
• run vagrant ssh to log in the newly created machine
• test if working, from VM's command line:

$> python3
>>> from bigchaindb.core import Bigchain
>>> bigchain = Bigchain()
>>> print(bigchain.host)
localhost
>>> print(bigchain.me)
<print the public key of the instance>

Update the documentation after code re-organization

relates to #108

Update the documentation so that code that was moved out bigchaindb.core still appears under Developer Interface section

Why the default branch is called `develop` and not `master` ?

Knowing that master doesn't exists, and you claim following github flow, which assumes the default branch to be named master.

Third-party libraries get installed in repository's root

After installation all the libraries' directories and eggs get created in the root of the repository creating some messy environment.

The libraries should be saved in a proper lib directory during installation.

Make the configuration easier for docker-based setup

Since containers are ephemeral, the config file that gets created under ~/.bigchaindb will be wiped out along with the container each time a new container is started.

We need a way to mount this file, or to only rely on environment variables, which would be probably easier.

Plugin or hook architecture e.g. for validate_transaction

In the bigchaindb/bigchaindb chat room on Gitter, @thedoctor asked an interesting question:

...the question I was getting at is whether folks thought a plugin or hook architecture would be of general utility? I'm imagining hooks in key places in Bigchain – at least validate_transaction and validate_block. It would be easy for the community to contribute arbitrary consensus plugins. Make sense?

I know there has been some internal discussion about that idea (i.e. at ascribe GmbH) and said as much on Gitter. I wonder how doable it is, or if there are any big gotchas. I said I'd open an issue about it on GitHub (i.e. this issue). Any thoughts @r-marques @diminator @vrde @andreasATascribe ?

@thedoctor replied:

Cool – I'll wait for some design discussion there [on GitHub] then I'm happy to implement it and PR.

Great!

Example integrations

The description mentions how bigchaindb is complimentary, it also mentions smart contracts. An example of how/where an EVM or Ethereum Ledger could integrate would be beneficial.

handle pep8 tool complaint about E712

See pep8 https://www.python.org/dev/peps/pep-0008/#programming-recommendations

Don't compare boolean values to True or False using == .

Yes: if greeting:
No: if greeting == True:
Worse: if greeting is True:

EDIT

Thanks to @Andygmb for asking clarifications, and to @genecyber for commenting. The above pep8 excerpt may not be the ideal one. This one may be better:

Comparisons to singletons like None should always be done with is or is not , never the equality operators.

although I am not fully sure, such that if one wishes to insist on referring to pep8, then the section A Foolish Consistency is the Hobgoblin of Little Minds is perhaps the best one for this case. E.g.:

[...] But most importantly: know when to be inconsistent -- sometimes the style guide just doesn't apply. When in doubt, use your best judgment. Look at other examples and decide what looks best. And don't hesitate to ask! [...]

Context for this issue

We have some tests that check:

assert x == True

the pep8 tool (version 1.7.0) will complain with:

E712 comparison to True should be 'if cond is True:' or 'if cond:'
E712 comparison to False should be 'if cond is False:' or 'if not cond:'

The following post is related: http://stackoverflow.com/questions/21019797/pep8-minor-annoyance, along with the answer http://stackoverflow.com/a/21019934/2769475, which helps understand the issue.

So in order to solve this issue, one may run the pep8 tool against the tests directory, and for each E712 error, one may first check whether it is necessary at all to make an explicit verification. That is, a simple

assert x

or

assert not x

may be sufficient.

On the other hand, if the case requires an explicit check, such as y must exactly be True, it seems that the check:

assert y is True

would be better ... since y = 1 should evaluate to False.

Said differently, one should keep in mind what the above mentioned stackoverflow answer shows:

>>> assert 1
>>> assert 1 == True
>>> assert 1 is True
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

>>> assert not 0
>>> assert 0 == False
>>> assert 0 is False
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

Please document that libffi-dev is a recommended package

Debian, Ubuntu

apt-get install libffi-dev

bigchain_benchmark.py doesn't load transactions properly in virtualenv

the bigchain-benchmark load command doesn't seem to respect virtualenv fully. For instance, after setting everything up (rethinkdb running) the following fails:

(env)ryan@ryan: bigchaindb (develop) $ bigchain-benchmark load
Traceback (most recent call last):
  File "/home/ryan/anaconda/bin/bigchain-benchmark", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/home/ryan/anaconda/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3141, in <module>
    @_call_aside
  File "/home/ryan/anaconda/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3127, in _call_aside
    f(*args, **kwargs)
  File "/home/ryan/anaconda/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3154, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/ryan/anaconda/lib/python2.7/site-packages/pkg_resources/__init__.py", line 640, in _build_master
    ws.require(__requires__)
  File "/home/ryan/anaconda/lib/python2.7/site-packages/pkg_resources/__init__.py", line 941, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/ryan/anaconda/lib/python2.7/site-packages/pkg_resources/__init__.py", line 828, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'Bigchain' distribution was not found and is required by the application

The virtualenv is python 3.4, but the traceback references my Anaconda 2.7 installation. On the other hand, this works:

(env)ryan@ryan: bigchaindb (develop) $ python bigchaindb/commands/bigchain_benchmark.py load
INFO:bigchaindb.config_utils:Configuration loaded from `/home/ryan/.bigchaindb`
INFO:__main__:Starting 1 processes
INFO:logstats.base:transactions: 100, transactions.speed: 100
INFO:logstats.base:transactions: 240, transactions.speed: 140
INFO:logstats.base:transactions: 374, transactions.speed: 134
INFO:logstats.base:transactions: 503, transactions.speed: 129
...

Improve documentation for non-linux users

We need to state in the documentation that we are currently only supporting linux and point windows users to the vagrant instructions an mac osx users to the docker deloyment

Add a CHANGELOG.md file to track changes

It'd be nice to have a document where the changes are listed, with attribution as appropriate. It's usually useful to note even small configuration changes, such as #41, or for version bumps, such as #42, so people can find out what's changed between versions.

Those same notes made in the CHANGELOG.md file can be copied over for each release that's done as well.

For example, see how React does this:

• CHANGELOG.md
• Releases

in tests, handle pep8 check E711 comparison to None should be 'if cond is None:'

pep8 (https://www.python.org/dev/peps/pep-0008/#programming-recommendations).states:

Comparisons to singletons like None should always be done with is or is not, never the equality operators.

which suggests that one should write

assert is None

instead of

assert == None

AttributeError: module 'bigchaindb' has no attribute 'config_utils'

i see the following error trying to run bigchaindb example:

Python 3.5.1 (v3.5.1:37a07cee5969, Dec 5 2015, 21:12:44)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

from bigchaindb import Bigchain
b = Bigchain()
Traceback (most recent call last):
File "", line 1, in
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/bigchaindb/core.py", line 47, in init
bigchaindb.config_utils.autoconfigure()
AttributeError: module 'bigchaindb' has no attribute 'config_utils'
b
Traceback (most recent call last):
File "", line 1, in
NameError: name 'b' is not defined

i am on mac os.

Tools & features to support benchmarking under realistic loads

To test and benchmark BigchainDB under realistic loads, we need some additional tools & features:

• tools to generate realistic transactions, for various applications
• tool to read files of test transactions from files
• tool to deploy a large number of testing clients and have them send requests to a test cluster
• ? a tool to bulk-upload realistic transactions directly into the backlog table of a test cluster?
• Integration of the above tools with our existing benchmarking tools and features

When "bigchaindb start", give user hints on what they can do next

When you start RethinkDB, it gives some hinds on what you can do next:

Listening for intracluster connections on port 29015
Listening for client driver connections on port 28015
Listening for administrative HTTP connections on port 8080
Listening on addresses: 127.0.0.1, 127.0.1.1, ::1
To fully expose RethinkDB on the network, bind to all addresses by running rethinkdb with the `--bind all` command line option.
Server ready, "ascribe3_XPS_8700_5nu" fafe8d60-0494-4ea9-ad78-5305b6fc97b5

Brett suggested we could have some hints like that when the user runs bigchaindb start. Here's what it says now:

Generating keypair
Database host? (default `localhost`): 
Database port? (default `28015`): 
Database name? (default `bigchain`): 
Ready to go!
INFO:bigchaindb.config_utils:Configuration loaded from `/home/troy/.bigchaindb`
INFO:bigchaindb.db.utils:Create:
INFO:bigchaindb.db.utils: - database `bigchain`
INFO:bigchaindb.db.utils: - tables
INFO:bigchaindb.db.utils: - indexes
INFO:bigchaindb.db.utils: - genesis block
INFO:bigchaindb.db.utils:Done, have fun!
INFO:bigchaindb.commands.bigchain:Start bigchaindb main process
INFO:bigchaindb.processes:starting bigchain mapper
INFO:bigchaindb.processes:starting backlog mapper
INFO:bigchaindb.processes:starting block
INFO:bigchaindb.processes:starting voter
INFO:bigchaindb.block:bootstraping block module...
INFO:bigchaindb.block:finished reading past transactions
INFO:bigchaindb.voter:voter waiting for new blocks
INFO:bigchaindb.block:finished bootstraping block module...
INFO:bigchaindb.block:starting block module...

That is fairly informative, but then it feels like it maybe got stuck. Is it still starting block module? Is it "ready" in some sense? If so, in what sense?

Handle and test installation on various OSes+distros

The BigchainDB installation requirements will differ slightly from OS to OS, especially the list of OS-level dependencies and how to install them. The setup.py file can take care of the Python dependencies, but not the OS-level dependencies.

How shall we handle OS-specific installation differences? Maybe write a Python script that checks the OS+distro and then installs the appropriate dependencies? Maybe create a Conda installer? Docker? Saltstack?

Would it make sense to test installation on various OS+distro combinations with Travis CI?

• Make list of OSes + versions we want to support now
• Write an installer (or installers) to install BigchainDB, and all OS-level dependencies, on each of those OSes + versions
• Add installation tests in Travis CI

Secure node-to-node communications

Some BigchainDB federations will want secure node-to-node communications, e.g. with encryption or two-way authentication or both.

Right now, the nodes only communicate with each other via RethinkDB, using a RethinkDB wire protocol (quite powerful: it can do all sorts of database operations). Can those communications be secured? See "Using SSH tunneling" on this page:

https://www.rethinkdb.com/docs/security/

Review the current configuration mechanism

The main goal of this issue is to gather key points about the configuration of bigchaindb.

Some points to consider:

• Identify the configuration needs for development, staging, and production environments.
• Identify environments we wish to support (e.g.: bare metal servers, container-based infrastructure, a laptop, etc), along with their specific requirements with respect to configuration and security. (For example, if sensitive data is required as part of the configuration, how should it be handled?)
• Evaluate the pros & cons of the possible approaches, considering both file formats (e.g.: .py, .ini, .json) and the configuration parser used (e.g.: Lib/configparser.py, custom approach).
• Consider whether having a ready to use, out-of-the-box configuration is feasible and worth providing for developers who wish to use bigchaindb for "experimental" purposes.

Resources

• http://martinfowler.com/articles/injection.html (found via SO Q.)

Figure out how to interoperate with IPFS

It would be nice for BigchainDB to interoperate with IPFS by @jbenet et al.

For example, it would be great if we both use the same standard way of referencing "objects" in the world (e.g. files or BigchainDB transactions).

IPFS references objects using IPLD (including multihash for all hashes).

We could start by changing our hashes (which currently assume sha3-256 always) to multihash.

There are probably other things we could do as well. It would be great to iterate with @jbenet on this.

Getting Started (Tutorial) - b.get_transaction get None

I followed "Getting Started (Tutorial)", and was running "3.3. Read the Creation Transaction from the DB"
try tx_retrieved = b.get_transaction(tx_signed['id']) but get None. May I miss anything?

Problem on installation of package: distutils.errors.DistutilsError

When trying python3.4 -m pip install bigchaindb this error display on Ubuntu Trusty 64:

 File "/usr/lib/python3.4/distutils/command/build_ext.py", line 448, in build_extensions
        self.build_extension(ext)
      File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 178, in build_exte
nsion
        _build_ext.build_extension(self,ext)
      File "/usr/lib/python3.4/distutils/command/build_ext.py", line 503, in build_extension
        depends=ext.depends)
      File "/usr/lib/python3.4/distutils/ccompiler.py", line 574, in compile
        self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
      File "/usr/lib/python3.4/distutils/unixccompiler.py", line 118, in _compile
        raise CompileError(msg)
    distutils.errors.CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1025, in run_se
tup
        run_setup(setup_script, args)
      File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 50, in run_setup
        lambda: execfile(
      File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 100, in run
        return func()
      File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 52, in <lambda>
        {'__file__':setup_script, '__name__':'__main__'}
      File "/usr/lib/python3/dist-packages/setuptools/compat.py", line 78, in execfile
        exec(compile(source, fn, 'exec'), globs, locs)
      File "setup.py", line 192, in <module>
        elif argv[i - 1:i] == ['--egg-base']:
      File "/usr/lib/python3.4/distutils/core.py", line 163, in setup
        raise SystemExit("error: " + str(msg))
    SystemExit: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip_build_root/cryptography/setup.py", line 318, in <module>
        **keywords_with_side_effects(sys.argv)
      File "/usr/lib/python3.4/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 239, in __init__
        self.fetch_build_eggs(attrs.pop('setup_requires'))
      File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 264, in fetch_build_eggs
        replace_conflicting=True
      File "/usr/lib/python3/dist-packages/pkg_resources.py", line 620, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/usr/lib/python3/dist-packages/pkg_resources.py", line 858, in best_match
        return self.obtain(req, installer) # try and download/install
      File "/usr/lib/python3/dist-packages/pkg_resources.py", line 870, in obtain
        return installer(requirement)
      File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 314, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 616, in easy_in
stall
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 646, in install
_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 834, in install
_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1040, in build_
and_install
        self.run_setup(setup_script, setup_base, args)
      File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1028, in run_se
tup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc'
failed with exit status 1
    Complete output from command /usr/bin/python3.4 -c "import setuptools, tokenize;__file__='/tmp/p
ip_build_root/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().r
eplace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-t7ka_7vj-record/install-record.t
xt --single-version-externally-managed --compile:
    c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory

 #include <ffi.h>

                 ^

compilation terminated.

Traceback (most recent call last):

  File "/usr/lib/python3.4/distutils/unixccompiler.py", line 116, in _compile

    extra_postargs)

  File "/usr/lib/python3.4/distutils/ccompiler.py", line 909, in spawn

    spawn(cmd, dry_run=self.dry_run)

  File "/usr/lib/python3.4/distutils/spawn.py", line 36, in spawn

    _spawn_posix(cmd, search_path, dry_run=dry_run)

  File "/usr/lib/python3.4/distutils/spawn.py", line 162, in _spawn_posix

    % (cmd, exit_status))

distutils.errors.DistutilsExecError: command 'x86_64-linux-gnu-gcc' failed with exit status 1



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

  File "/usr/lib/python3.4/distutils/core.py", line 148, in setup

    dist.run_commands()

  File "/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands

    self.run_command(cmd)

  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command

    cmd_obj.run()

  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 185, in run

    cmd = self.call_command('install_lib', warn_dir=0)

  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 171, in call_command

    self.run_command(cmdname)

  File "/usr/lib/python3.4/distutils/cmd.py", line 313, in run_command

    self.distribution.run_command(command)

  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command

    cmd_obj.run()

  File "/usr/lib/python3/dist-packages/setuptools/command/install_lib.py", line 21, in run

    self.build()

  File "/usr/lib/python3.4/distutils/command/install_lib.py", line 109, in build

    self.run_command('build_ext')

  File "/usr/lib/python3.4/distutils/cmd.py", line 313, in run_command

    self.distribution.run_command(command)

  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command

    cmd_obj.run()

  File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 49, in run

    _build_ext.run(self)

  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 339, in run

    self.build_extensions()

  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 448, in build_extensions

    self.build_extension(ext)

  File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 178, in build_extensio
n

    _build_ext.build_extension(self,ext)

  File "/usr/lib/python3.4/distutils/command/build_ext.py", line 503, in build_extension

    depends=ext.depends)

  File "/usr/lib/python3.4/distutils/ccompiler.py", line 574, in compile

    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)

  File "/usr/lib/python3.4/distutils/unixccompiler.py", line 118, in _compile

    raise CompileError(msg)

distutils.errors.CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1025, in run_setup

    run_setup(setup_script, args)

  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 50, in run_setup

    lambda: execfile(

  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 100, in run

    return func()

  File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 52, in <lambda>

    {'__file__':setup_script, '__name__':'__main__'}

  File "/usr/lib/python3/dist-packages/setuptools/compat.py", line 78, in execfile

    exec(compile(source, fn, 'exec'), globs, locs)

  File "setup.py", line 192, in <module>

    elif argv[i - 1:i] == ['--egg-base']:

  File "/usr/lib/python3.4/distutils/core.py", line 163, in setup

    raise SystemExit("error: " + str(msg))

SystemExit: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/tmp/pip_build_root/cryptography/setup.py", line 318, in <module>

    **keywords_with_side_effects(sys.argv)

  File "/usr/lib/python3.4/distutils/core.py", line 108, in setup

    _setup_distribution = dist = klass(attrs)

  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 239, in __init__

    self.fetch_build_eggs(attrs.pop('setup_requires'))

  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 264, in fetch_build_eggs

    replace_conflicting=True

  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 620, in resolve

    dist = best[req.key] = env.best_match(req, ws, installer)

  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 858, in best_match

    return self.obtain(req, installer) # try and download/install

  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 870, in obtain

    return installer(requirement)

  File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 314, in fetch_build_egg

    return cmd.easy_install(req)

  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 616, in easy_instal
l

    return self.install_item(spec, dist.location, tmpdir, deps)

  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 646, in install_ite
m

    dists = self.install_eggs(spec, download, tmpdir)

  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 834, in install_egg
s

    return self.build_and_install(setup_script, setup_base)

  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1040, in build_and_
install

    self.run_setup(setup_script, setup_base, args)

  File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1028, in run_setup

    raise DistutilsError("Setup script exited with %s" % (v.args[0],))

distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc' fail
ed with exit status 1

Command /usr/bin/python3.4 -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/cryptograph
y/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __f
ile__, 'exec'))" install --record /tmp/pip-t7ka_7vj-record/install-record.txt --single-version-exter
nally-managed --compile failed with error code 1 in /tmp/pip_build_root/cryptography
Exception information:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/pip/req.py", line 707, in install
    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
  File "/usr/lib/python3/dist-packages/pip/util.py", line 715, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command /usr/bin/python3.4 -c "import setuptools, tokenize;__file_
_='/tmp/pip_build_root/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__)
.read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-t7ka_7vj-record/install
-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build
_root/cryptography

Compiling works properly for other software and packages.
GCC version: Ubuntu 4.8.4-2ubuntu1~14.04.1

Write tool(s) to generate realistic transactions for testing purposes

We need some way to generate lots of realistic-looking transactions, for testing and benchmarking purposes.

Thoughts:

• The meaning of "realistic transaction" depends on the use case. Maybe we can have one tool with parameters to change the kind of transactions it generates, or maybe separate tools?
• I suspect this tool (or these tools) could be stand-alone
• Maybe one option could be for the transactions to write to a file or to stdout? Then other tools could be used to read-from-file into wherever. (Some customers might bring their own files-of-test-transactions, from their own sources or tools.)

Update CONTRIBUTING.py with our new Git workflow

We decided to go with a simpler Git workflow for now (based on the GitHub flow) so the CONTRIBUTING.md file should be updated.

Write tool to read transactions from files for testing

If we want to test or benchmark a cluster with realistic transactions, then one way to do that will be to read a bunch of test transactions from a file (or set of files). We'll need a tool to read those files and use the transactions as appropriate for the test (e.g. maybe to simulate a bunch of clients, or maybe just to fill up a bunch of backlog tables).

Write tool to deploy a large number of *clients* for testing purposes

To test and benchmark a live BigchainDB cluster, we will want a way to simulate real traffic from real clients, so we need a way to deploy a bunch of clients to simulate various load scenarios. (The clients will need a way to get realistic-looking transactions - see Issue #111 and Issue #112 .)

Voter is not validating blocks correctly when checking for double spends

When a voter is validating a transaction it checks if the input of a transaction was already spent. To check if the input was already spent it checks if there is any transaction in the bigchain that already used that input. Since when voting the block is already in the bigchain it considers the first transaction using that input has a double spend.

We need to add a check so that the voter does not consider the current block when checking if there is already a transaction in the bigchain with that input.

Clean up docker containers for monitoring

Suggested by @sbellem, move custom containers used for monitoring under the bigchainDB umbrella on dockerhub. Additionally, I'd like to clean up the containers themselves as they are somewhat brittley assembled. Related to #72

Create Python Style Guide (v1)

A Python Style Guide was already discussed in the old (private) repo:

https://github.com/ascribe/bigchain/issues/59
https://github.com/ascribe/bigchain/issues/28

I will try to synthesize the discussion that we had there, as a starting point.

It will include a section on writing tests; the basics anyway.

Re-add licenses to docs

The recent change in the theme used for the documentation caused the custom HTML sidebar (on the left) to stop working. The main consequence was that the notice about the docs license went away. Figure out a way to add it back. Maybe just add a section on licenses.

If you do that, you may as well delete all the custom sidebar stuff from the docs, so it doesn't surprise us in the future.