v4lproik / simple-blockchain-quickstart

This is an experimental repository which aims at shipping a decent POW blockchain skeleton for anyone who wants to get into the blockchain world via the Golang programming language

Simple-blockchain-quickstart

This is merely a skeleton that helps you quickly set up a simplified version of a blockchain app written in Golang.

Getting started

Install Golang & Useful packages

1. Install goenv
2. Install make
3. Install protoc
4. Install golang => 1.18.3
5. Download dependencies

make dep

Set env variables

source config/local.env

Building

make build

Hot reload

1. Install air

go install github.com/cosmtrek/air@latest

2. (Optional) Export the GOPATH set by goenv if you are using goenv

export PATH=$(go env GOPATH)/bin:$PATH

Run as client

make build && ./bin/simple-blockchain-quickstart -d ./testdata/node1/blocks.db -g ./testdata/node1/genesis.json -k ./testdata/node1/keystore/ -u ./testdata/node1/users.toml -n ./testdata/node1/network_nodes.toml -r 19:51:41
go build -o ./bin/simple-blockchain-quickstart
1.657907504219829e+09	info	Transactions file: ./testdata/node1/blocks.db
1.6579075042198799e+09	info	Genesis file: ./testdata/node1/genesis.json
1.657907504219883e+09	info	Users file: ./testdata/node1/users.toml
1.657907504219885e+09	info	Nodes file: ./testdata/node1/network_nodes.toml
1.657907504219887e+09	info	Keystore dir: ./testdata/node1/keystore/
1.657907504219889e+09	info	Output: console

[GIN-debug] POST   /api/auth/login           --> github.com/v4lproik/simple-blockchain-quickstart/domains/auth.AuthEnv.Login-fm (5 handlers)
[GIN-debug] GET    /api/auth/.well-known/jwks.json --> github.com/v4lproik/gin-jwks-rsa.Jkws.func1 (5 handlers)
[GIN-debug] POST   /api/balances/            --> github.com/v4lproik/simple-blockchain-quickstart/domains/balances.(*BalancesEnv).ListBalances-fm (6 handlers)
[GIN-debug] GET    /api/healthz              --> github.com/v4lproik/simple-blockchain-quickstart/domains/healthz.RunDomain.func1 (5 handlers)
[GIN-debug] GET    /api/nodes/status         --> github.com/v4lproik/simple-blockchain-quickstart/domains/nodes.NodesEnv.NodeStatus-fm (5 handlers)
[GIN-debug] POST   /api/nodes/blocks         --> github.com/v4lproik/simple-blockchain-quickstart/domains/nodes.NodesEnv.NodeListBlocks-fm (5 handlers)
[GIN-debug] PUT    /api/transactions/        --> github.com/v4lproik/simple-blockchain-quickstart/domains/transactions.TransactionsEnv.AddTransaction-fm (6 handlers)
[GIN-debug] PUT    /api/wallets/             --> github.com/v4lproik/simple-blockchain-quickstart/domains/wallets.(*WalletsEnv).CreateWallet-fm (6 handlers)

Run in container

The docker image has been built so the mandatory options are passed in an env file. The extra options are passed through the variable cmd. To sum up cmd is responsible for switching from running the app as a client or as a node. The options related to the app itself are stored in config/local.conf.

#first remove all images locally if you rebuild from this folder
docker-compose down --rmi all
#eg. run as a node for fish shell users
set -l cmd -r; docker-compose -f docker-compose-local.yml up
#eg. run as cli for bash users
cmd="transaction list" docker-compose -f docker-compose-local.yml up

Testing

chmod +x ./deployment_script/test.sh
make test -B

Format the code

make format
> gofumpt -l -w .
args.go
commands/password.go

Generate coverage

make test-coverage-install
make test-coverage

Generate doc

1. Install swag
2. (Optional) Export the GOPATH set by goenv if you are using goenv

export PATH=$(go env GOPATH)/bin:$PATH

3. Run the swagger

swag init

Authentication

JWK authentication is optional, you can activate a verification for each endpoint if necessary. You need to activate it through the environment variable.

export SBQ_IS_AUTHENTICATION_ACTIVATED="true";
export SBQ_JWT_KEY_PATH="./testdata/node1/private.pem";
export SBQ_JWT_KEY_ID="sbq-auth-key-id";
export SBQ_JWT_EXPIRES_IN_HOURS="24";
export SBQ_JWT_DOMAIN="localhost";
export SBQ_JWT_AUDIENCE="localhost:8080";
export SBQ_JWT_ISSUER="sbq-local";
export SBQ_JWT_ALGO="HS256";

JWT tokens are being signed with the private key passed as an environment variable. Also, you need to provide the url containing the JKMS derived from the private key. You can use this project JKMS service exposing the public key parameters needed to verify a JWT token.

export SBQ_IS_JKMS_ACTIVATED="true";
export SBQ_JWT_JKMS_REFRESH_CACHE_INTERVAL_IN_MIN="1";
export SBQ_JWT_JKMS_REFRESH_CACHE_RATE_LIMIT_IN_MIN="1000";
export SBQ_JWT_JKMS_REFRESH_CACHE_TIMEOUT_IN_SEC="1";

The users are declared in ./testdata/node1/users.toml. See the Test data section for the test accounts.

curl localhost:8080/api/balances/ -X POST                                                                                                                          15:03:11
{"error":{"code":401,"status":"Unauthorized","message":"authentication token cannot be found","context":[]}}

> curl localhost:8080/api/auth/login -X POST -d '{"username": "v4lproik", "password":"P@assword-to-access-api1"}' -H 'Content-type: application/json'
{"access_token":"eyJhbGciOiJSUzI1NiIsImtpZCI6InNicS1hdXRoLWtleS1pZCIsInR5cCI6IkpXVCJ9.eyJkYXQiOnsiTmFtZSI6InY0bHByb2lrIiwiSGFzaCI6IiRhcmdvbjJpZCR2PTE5JG09NjU1MzYsdD0zLHA9MiRGdVNVWlEwbXJUTTl1SXBQOHFwTlV3JG5kMjFqdGdVWmpKanowNzhqZGxTREt4cWFqdjVwYWl4bG9HR05nVE1KSXcifSwiZXhwIjoxNjU2NTk0MDE1LCJpYXQiOjE2NTY1MDc2MTUsIm5iZiI6MTY1NjUwNzYxNX0.4VBqD9Cg2KH96CioyRtSIlM2edGneXxZLrxG46Qub4Pol-NWOXI9_PAmIL_DmQEvF95x44m9Vl8VF2RZdO42B03cxKZPKIzjZjalHqyEl3YPyz27kP7d_YCCMjSzKMbx8Np7u9orWjlC5MayCB2rtgefag3DkKGJWUAIH5OfDPy6B-XLsgL8caWN0aM4TCelC-geo2bC488Xk79YffhfNLJPuvgKuuUeWaWLz-YHcALbguqRP_ehqDvn5vzBBWAS_aCYN3W9-dsOHttfSRKaxmxQm-hxcp01T7ezXgNO3gnJmfuWff-96UKZVb0QPzG1ltPWInqheKRypviuAEIHUg"}

> curl localhost:8080/api/balances/ -X POST -H "X-API-TOKEN: eyJhbGciOiJSUzI1NiIsImtpZCI6InNicS1hdXRoLWtleS1pZCIsInR5cCI6IkpXVCJ9.eyJkYXQiOnsiTmFtZSI6InY0bHByb2lrIiwiSGFzaCI6IiRhcmdvbjJpZCR2PTE5JG09NjU1MzYsdD0zLHA9MiRGdVNVWlEwbXJUTTl1SXBQOHFwTlV3JG5kMjFqdGdVWmpKanowNzhqZGxTREt4cWFqdjVwYWl4bG9HR05nVE1KSXcifSwiZXhwIjoxNjU2NTk0MDE1LCJpYXQiOjE2NTY1MDc2MTUsIm5iZiI6MTY1NjUwNzYxNX0.4VBqD9Cg2KH96CioyRtSIlM2edGneXxZLrxG46Qub4Pol-NWOXI9_PAmIL_DmQEvF95x44m9Vl8VF2RZdO42B03cxKZPKIzjZjalHqyEl3YPyz27kP7d_YCCMjSzKMbx8Np7u9orWjlC5MayCB2rtgefag3DkKGJWUAIH5OfDPy6B-XLsgL8caWN0aM4TCelC-geo2bC488Xk79YffhfNLJPuvgKuuUeWaWLz-YHcALbguqRP_ehqDvn5vzBBWAS_aCYN3W9-dsOHttfSRKaxmxQm-hxcp01T7ezXgNO3gnJmfuWff-96UKZVb0QPzG1ltPWInqheKRypviuAEIHUg"
{"balances":[{"account":"0x7b65a12633dbe9a413b17db515732d69e684ebe2","value":998000},{"account":"0xa6aa1c9106f0c0d0895bb72f40cfc830180ebeaf","value":1003000}]}

Test data

In the folder ./testdata/node*/ you can find some data that could be used to test the application.

Api users

Username: v4lproik
Password: P@assword-to-access-api1
Hash    : $argon2id$v=19$m=65536,t=3,p=2$FuSUZQ0mrTM9uIpP8qpNUw$nd21jtgUZjJjz078jdlSDKxqajv5paixloGGNgTMJIw

Account : 0x7b65a12633dbe9a413b17db515732d69e684ebe2
Password: P@assword-to-access-keystore1
Keystore: testdata/node1/keystore/UTC--2022-06-26T13-49-16.552956900Z--7b65a12633dbe9a413b17db515732d69e684ebe2

Username: cloudvenger
Password: P@assword-to-access-api2
Hash    : $argon2id$v=19$m=65536,t=3,p=2$j2yd8FWqhApKrrqmkkLMQA$Lfh/7K+oP3IWdTrQSjURBS6PFttzlksmozz8kuGBCqk

Account : 0x7b65a12633dbe9a413b17db515732d69e684ebe2
Password: P@assword-to-access-keystore2
Keystore: testdata/node1/keystore/UTC--2022-06-26T13-50-53.976229800Z--a6aa1c9106f0c0d0895bb72f40cfc830180ebeaf

Miners

Account : 0x01fc1af4a56cde68675dc44cabd486e8d3559f07
Password: P@assword-to-access-keystore3
Keystore: testdata/node1/keystore/UTC--2022-07-19T22-42-22.558797000Z--01fc1af4a56cde68675dc44cabd486e8d3559f07

simple-blockchain-quickstart's Issues

Add active nodes / remove unactive from nodes database

Add the active nodes found during the sychronisation phase into the nodes database. If a node goes offline, it'd be nice to have more nodes to choose from until the node goes back online again.

Implement the ability for a node to discover other nodes within the network

Implement the ability for a node to discover other nodes within the network so they can update their own database. This can be done through exposing the node status including its block height.

Expose the node sync time interval as env variable

Run tests from CI/CD directly in container

We want parity even for testing. Thus we need to run test in our container as the project has yet been containerised.

Move io.ReadFile to io.Copy

Big files should not be parsed with io.Readfile as it'll be stored in memory. Situation becomes even worse when above file is being requested by multiple users in parallel. You will endup with multiple copies of file in memory and will eventually crash due to insufficient memory.

Expose node status

Node should be able to expose their block height so the nodes from the network can synchronise their own database from one another.

Add print blockchain version after creating the command print

Add querying language for toml v2

toml v2 doesn't provide a querying language. Right now we are extracting all the user file into an array and then loop over it to find the right user. We need to find a way to avoid storing too much in memory by introducing a querying language like dasel.

Containerize the project

Parallelise node discovery

We shouldn't sequentially update our database if the freshly found node has a heigher height than our current state. Instead we should update our database from the node that has the highest height amongst them all.

Replace toml v1 library to v2

Toml v2 is almost stable and is a massive improve in terms of performance and stability over the v1.

Gracefully shutdown server

Remove default http client when sync blocks from other network nodes

Add genesis and transaction files integrity check

Bootstraping the node should check the format of the genesis and transaction files. The node should not be able to run if the content doesn't comply with a defined format.

Write a simple sync database alrgorithm

Use the node status endpoint to gather information about unknown nodes and see their block height.
If the block height is superior, it then means that we need to collect their blocks and insert them into our database.

• Write a simple task manager (we'll need to pull information until we have a 'push' system)
• Write the pulling algorithm scanning the other nodes
• Write the algorithm updating the local database

Add swagger doc for the HTTP endpoints

Add benchmark for node sync goroutines

Would be nice to have some banchmark for the goroutines synchronising the database.
Also we need to take into the case where the time needed for synching the database is longer than the next occurence.
This would create a memory leak situation.

Add more Golang code syntax checks

Add a better linter and code checks.

Add Dockerfile test

Use container structure test.
Good ref. https://emmer.dev/blog/testing-docker-images-with-container-structure-test/
https://github.com/ory/dockertest

Add PoW consensus mechanism

Expose swagger via Gin Gonic handler

Add hot reload for http server

Improve Golang coverage

We need to be able to improve the coverage by skipping some lines of code (eg. return error passthrough etc).