rubykube / microkube

Minimal stack for deploying peatio (open source crypto-currencies trading exchange platform) on a virtual machine

MicroKube

Please note, that this project is deprecated and moved to https://github.com/openware/opendax

Minimal stack for VM deployment.

Getting started

Install ruby with rvm

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
curl -sSL https://get.rvm.io | bash -s stable

Bundle install depedencies

bundle
rake -T

Run everything

rake service:all

Insert in file /etc/hosts

0.0.0.0 www.app.local
0.0.0.0 monitor.app.local

You can login on www.app.local with the following default users from seeds.yaml

Seeded users:
Email: [email protected], password: 0lDHd9ufs9t@
Email: [email protected], password: Am8icnzEI3d!

Usage

Initial configuration

All the MicroKube deployment files have their confguration stored in config/app.yml.

Feel free to fill it out with correct values:

Parameter	Description
app.name	Global application name
app.domain	Base domain name to be used
ssl.enabled	Enable SSL certificate generation
ssl.email	Email address to use for SSL generation requests
images	Application images tags
vendor	Frontend application Git repo URL

Once you're done with the configuration, render the files using rake render:config. You can easily apply your changes at any time by running this command.

Note: be sure to append all the subdomains based on app.domain to your
/etc/hosts file if you're running MicroKube locally

Bringing up the stack

The MicroKube stack can be brought up using two ways:

1. Bootstrap all the components at once using rake service:all[start]
2. Start every component one by one using rake service:*component*[start]

The components included in the stack are:

• proxy - Traefik, a robust cloud native edge router/reverse proxy written in Go
• backend - Vault, MySQL, Redis and RabbitMQ grouped together
• cryptonodes - cryptocurrency nodes such as Geth [Optional]
• daemons - Peatio daemons and Ranger [Optional]
• setup - setup hooks for Peatio and Barong to run before the application starts(DB migration etc.)
• app - Peatio, Barong and the Ambassador API gateway
• frontend - the frontend application located at vendor/frontend
• tower - the Tower admin panel application located at vendor/tower

For example, to start the backend services, you'll simply need to run rake service:backend[start]

Note: all the components marked as [Optional] need to be installed using
rake service:*component*[start] explicitly

Go ahead and try your deployment on www.your.domain!

Stopping and restarting components

Any component from the stack can be easily stopped or restarted using rake service:*component*[stop] and rake service:*component*[restart].

For example, rake service:frontend[stop] would stop the frontend application container and rake service:proxy[restart] would completely restart the reverse proxy container.

Accessing the deployment

Note: Make sure your VM of choice has its firewall rules configured to let in
HTTP and/or HTTPS traffic and your DNS entries are pointing at its external IP.

All the components with external endpoints are accessible by their respective subdomains based on the domain provided in the configuration:

• www.base.domain - frontend application and Peatio and Barong APIs mounted on /api
• peatio.base.domain - Peatio UI and API
• barong.base.domain - Barong API
• tower.base.domain - the Tower admin panel application
• monitor.base.domain - Traefik's dashboard useful for monitoring which components are enabled
• ws.ranger.base.domain - Ranger's WebSocket endpoint [Optional]
• eth.base.domain - Geth JSON RPC API [Optional]

Using Vendor

Fill in the list of vendor to clone in app.yaml

Render compose file

rm compose/vendor.yaml
rake render:config

Review the generated file

Clone the vendors and start

source ./bin/set-env.sh
rake vendor:clone
docker-compose -f compose/vendor.yaml up -d

Terraform Infrastructure as Code Provisioning

You can easily bring up Microkube from scratch on Google Cloud Platform using Terraform!

To do this, just follow these simple steps:

• Fill app.yml with correct values
• Run rake terraform:apply
• Access your VM from the GCP Cloud Console
• Have fun using it!

To destroy the provisioned infrastructure, just run rake terraform:destroy

Installer tool

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/rubykube/microkube/master/bin/install)"

microkube's Issues

rake service:daemons[start] fails starting ranger due to port in use

I've installed microcube 0.1.5 with no configuration changes and wish to try using some of the APIs but have not figured out how to do this. I assume I need ranger running however I get an error when trying to start daemons because frontend is already using 8080:

rake service:daemons[start]
----- Starting peatio daemons -----
docker-compose up -d ranger withdraw_audit blockchain global_state deposit_collection deposit_collection_fees deposit_coin_address slave_book pusher_market pusher_member matching order_processor trade_executor withdraw_coin k
microkube_proxy_1 is up-to-date
microkube_rabbitmq_1 is up-to-date
microkube_db_1 is up-to-date
microkube_redis_1 is up-to-date
microkube_peatio_1 is up-to-date
Creating microkube_ranger_1 ... error

ERROR: for microkube_ranger_1 Cannot start service ranger: driver failed programming external connectivity on endpoint microkube_ranger_1 (f36efa6e50a2d7f975e927695a55389aac68f6d4041bbf433c18a1631d7d971b): Error starting userland proxy: Bind for 0.0.0.0:8080 failed: port is already allocated

ERROR: for ranger Cannot start service ranger: driver failed programming external connectivity on endpoint microkube_ranger_1 (f36efa6e50a2d7f975e927695a55389aac68f6d4041bbf433c18a1631d7d971b): Error starting userland proxy: Bind for 0.0.0.0:8080 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.
rake aborted!
Command failed with status (1): [docker-compose up -d ranger withdraw_audit...]
lib/tasks/service.rake:67:in start' lib/tasks/service.rake:7:in call'
lib/tasks/service.rake:7:in block (2 levels) in <top (required)>' lib/tasks/service.rake:75:in block (2 levels) in <top (required)>'
/Users/build/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/exe/rake:27:in <top (required)>' /Users/build/.rvm/gems/ruby-2.6.0/bin/ruby_executable_hooks:24:in eval'
/Users/build/.rvm/gems/ruby-2.6.0/bin/ruby_executable_hooks:24:in `

'
Tasks: TOP => service:daemons
(See full trace by running task with --trace)

Error 404 , page not found

how to solve this

Build Failed Using Latest Docker Image

Hi, Everything works fine without editing the app.yml file.

When I change the value on app.yml#L9 to peatio: rubykube/peatio:latest And run rake service:all build fail

/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => db:seed
(See full trace by running task with --trace)
rake aborted!
Command failed with status (1): [docker-compose run --rm barong bash -c "./...]
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/file_utils.rb:67:in `block in create_shell_runner'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/file_utils.rb:57:in `sh'
lib/tasks/service.rake:120:in `start'
lib/tasks/service.rake:7:in `call'
lib/tasks/service.rake:7:in `block (2 levels) in <top (required)>'
lib/tasks/service.rake:126:in `block (2 levels) in <top (required)>'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `block in execute'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `each'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `execute'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:213:in `block in invoke_with_call_chain'
/home/gsrwallet/.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:193:in `invoke_with_call_chain'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:182:in `invoke'
lib/tasks/service.rake:206:in `start'
lib/tasks/service.rake:7:in `call'
lib/tasks/service.rake:7:in `block (2 levels) in <top (required)>'
lib/tasks/service.rake:223:in `block (2 levels) in <top (required)>'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `block in execute'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `each'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `execute'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:213:in `block in invoke_with_call_chain'
/home/gsrwallet/.rvm/rubies/ruby-2.6.0/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:193:in `invoke_with_call_chain'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/task.rb:182:in `invoke'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/application.rb:160:in `invoke_task'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/application.rb:116:in `each'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/application.rb:116:in `block in top_level'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/application.rb:125:in `run_with_threads'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/application.rb:110:in `top_level'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/application.rb:83:in `block in run'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/application.rb:186:in `standard_exception_handling'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/lib/rake/application.rb:80:in `run'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/bin/rake:23:in `load'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/bin/rake:23:in `<main>'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/bin/ruby_executable_hooks:24:in `eval'
/home/gsrwallet/.rvm/gems/ruby-2.6.0/bin/ruby_executable_hooks:24:in `<main>'
Tasks: TOP => service:setup
instance-1:~/microkube$ 

Install working but URL issues

Hello. I am working on a deployment of Peatio for an IT project (Academic type 4 coursework). I managed to install and deploy peatio with microkube (after lots of frustration with workbench).

All services start and run, and I am able to access www.mydomain.com and domain.com/admin, moniter.mydomain.com etc. So far so good. However, some of the services do not seem to work.

I have modified the etc/hosts file with the necessary information:
0.0.0.0 peatio.mydomain.com
0.0.0.0 barong.mydomain.com

However, when I try to access there in my browser, I get 404 page not found. Please, can someone assist?

This happens right after rake service:all is used.

cryptonodes.yaml won't get compiled

Hi guys,

you forget to include cryptonodes.yaml to .env file. Thus it won't compile..

Is microkube really a development environment

Great job moving to the minimalistic version. Its a huge advancement from workbench.

In Mikrokubes and Peatios documentation this minimalistic environment is called as a development environment.
But I dont see how one can edit the code and see changes in the live application.

Is this really a development environment ? If yes, can you please advice me for example how to do the changes.
Perhaps im missing something here.

Thanks.

Private Channel Ranger

hi, i have a problem to authenticate a private channel on ranger, i look documentation on peatio still using bearer to auth private channel

Microkube debugging

Hi,

wonderful job with microkube!

Can you please try to make some more documentation available? I would be interested how to debug changes to specific seed files and/or the backend code.

Thanks and cheers!

Change password endpoint action should be `put` instead of `post`

According to REST:

How to config smtp?

hi i dont know how to config smtp :|

zcash coins how to add?

Hi,

what is the basic procedure to add zcash like coins such as zclassic, btcp, etc?

Thanks!

frontend router errors

the provider comes up (in monitor.) but when I try to get to www/peatio, results in 404.

when I try tower., error, Bad Gateway.

No errors were present when starting components. Any ideas?

Can't logout

Hi, I believe I have everything on the stack up and running:

But when I go to click on the top right, Exit which leads to a logout endpoint, I don't seem to get logged out. If I refresh I'm still logged in.

How to use SSL certs with microkube?

Hello, I wonder if you could please guide about using SSL certs with Microkube? Perhaps LetsEncrypt?

Option redirect from app.local to www.app.local for http & https

Performance issue peatio/lib/daemons/blockchain.rb

when running about 15 blockchains the sole blockchain.rb process cannot keep up.

when starting multiple threads of it, it can occur that a deposit is credited multiple times

It seems to be happening on the first time deposit only to new wallet deposit addresses

please advise how to solve this performance issue
Not a server problem, server is high end i9

config/gateway/auth.yaml is causing JWT auth failed

"Authorization" Header is somehow filtered. Have to remove this config file to make the Bearer Token work

How to add a new coin to peatio with microkube

Hi,

what are the files that need to be edited to add a new coin to peatio (running in the microkube framework)?
I have peatio local repo already, just want to know which files to change for a new coin.

Thanks!

Switch from envs to templates in compose files

• Remove envs form docker compose files.
• Add configurations to app.yaml and use power of ruby as much as possible.

Suggestions:

• Make a loop and generate code for each peatio daemon, instead of copy paste.

deposit_coin_address error

The deposit_coin_address daemon logs the following error

#<ActiveRecord::StatementInvalid: Mysql2::Error: Table 'peatio_development.authentications' doesn't exist: SHOW FULL FIELDS FROM authentications>

Changing the image from rubykube/peatio:1.9.1-rc.20 to rubykube/peatio:2.0.0.alpha in file compose/daemons.yaml fixes the error and the daemon works fine.

Open SSL error

Some funny errors, which need to be fixed:

OpenSSL::OpenSSLError: password must be at least 4 bytes

Fixed with

gem update openssl

Not working -- 404

404 errors - not working errors
checked everything, routing, docker, dns, all start and configured correctly as described in the readme (which is missing a lot, I had to go through lots of tweaks, requires some testing from your side please)
please make installation script automated or test and update then and tell details so all modules work seamlessly together.

Changing front end microservice

How can we change the front end micro service ?
Config/app.yml mentions two instances of frontend :

vendor:
  frontend: https://github.com/rubykube/mikroapp.git

images:
  frontend: rubykube/mikroapp:0.1.5

I understand the GitHub setting but not sure how to go about the docker cloud image.
Do we need to update both variables ?

Thanks.

docker compose unsupported

ERROR: Version in "./compose/app.yaml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a version of "2" (or "2.0") and place your service definitions under the services key, or omit the version key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/

Docker version 18.09.1, build 4c52b90
docker-compose version 1.8.0, build unknown

any help?

Can not login after `rake service:all`

Issue Description

Follow the READEM, deploy the all services via rake service:all, but can not login on www.<domain> with the default seeds user/password, it reports error message Oups! Error occurs, please try again later.

Version

0.1.19

ambassador responses with 404 for /api requests

App responses with 404 for valid api endpoints, removing cors: section helped me, maybe smt is wrong with cors config?

NameError: uninitialized constant Pathname

issue:

~/microkube$ rake render:config
rake aborted!
NameError: uninitialized constant Pathname
lib/tasks/render.rake:3:in `<top (required)>'
/home/dragon/microkube/Rakefile:9:in `load'
/home/dragon/microkube/Rakefile:9:in `block in <top (required)>'
/home/dragon/microkube/Rakefile:8:in `each'
/home/dragon/microkube/Rakefile:8:in `<top (required)>'
(See full trace by running task with --trace)

Docker login

Prior to start you should login into docker with docker login.
This should be in script as well.
Please add.

After login you have to put your username (not email) and password to docker hub.

No Trading Page ?

Trading page is coming soon. Is microkube an incomplete project ?

Error after starting vendor

docker-compose -f compose/vendor.yaml up -d

Render Config regenerates secret keys even if they are already present

The render config should check if the keys are present and not overwrite the keys already present.

Also we should find a better way to store vault data like if we run rake docker:down and thenrake service:all the vault totp data will be lost and all the users with 2FA enabled will be locked

Ability to configure cryptonodes

Now all currencies are enabled, but we don't have crypto-nodes for them.

I suggest:

• Add geth and bitcoind to cryptonodes.yaml.

• Add ability to enable geth and bitcoind in app.yaml.

• Update seed files for peatio from default one.

not able to verify via SMS

Something wrong with twilio client

Whitelists peatio public route.

The public api should not require a session to return info.

How to upgrade new version

I see Peatio has v2.3.3 and how do I upgrade Microkube to v2.3.3 according to Peatio?

Critical issue with Open DB port

The DB ports are open and of course, we are not supposed to use the default password BUT for those who are using default passwords, you guys are in danger as someone is exploiting this and encrypting your databases.

If you are using my fork https://github.com/martianatwork/microkube then I will push a fix for 2.1 on a stable-2-1 branch.

Documentation of postmaster / email capability

Is there any documentation how the postmaster container is supposed to work for sending real emails into the real world?

500 errror

Internal Server Error shows after rake service:all

Deposits are not appearing in history

Configured everything. Got deposit address. All daemons are running. Sent some eth to address. Deposits are not there. Not in balance, not in deposits history. Which container logs should I check to find the issue?

Can't login with seed username/password

It stays at the same page with login form, while location in address bar changes to /wallets

How to connect websocket?

rake service:all[start]
rake service:daemons[start]
ws://ws.ranger.app.local not working
on monitor cant finds ranger,... http://monitor.app.local/dashboard/

TKS FOR YOUR HELP