luxoft / fabric-skeleton Goto Github PK

Currently, for multiple different cluster configuration, ops_cli fails when performing the 'Run Fabric configurator' operation.. Partial output below.

To reproduce
Once fabric-skeleton is install on a block chain ec2 instance steps to reproduce are

1. cd ~/fabric-skeleton/ops
2. export FABRIC_CFG_PATH=fabric-skeleton/ops/network_dist
3. export AWS_DEFAULT_REGION=us-west-1
4. export AWS_REGION=us-west-1
5. ./ops-cli -i ~/.ssh/Blockchain-controller.pem -c cluster-config
   or
   ./ops-cli -i ~/.ssh/Blockchain-controller.pem -c single_sample

software configuration
A list of the software environment is found here SWlist.txt (it was generated by the showSWConfig.sh command currently in the Bootstrap branch)

Attached are the logs and configuration files for two test runs which presented essentially the same problem
cluster_config.log
single_sample.log

cluster-config.yaml
single_sample.yaml

PLAY [Run Fabric configurator] *********************************************************************

TASK [Gathering Facts] *****************************************************************************
ok: [localhost]

TASK [command] *************************************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "gradle networkConfigure", "delta": "0:00:14.839140", "end": "2018-04-02 20:24:03.369610", "msg": "non-zero return code", "rc": 1, "start": "2018-04-02 20:23:48.530470", "stderr": "\nFAILURE: Build failed with an exception.\n\n* Where:\nBuild file '/home/ubuntu/fabric-skeleton/build.gradle' line: 8\n\n* What went wrong:\nA problem occurred evaluating root project 'fabric-skeleton'.\n> Failed to apply plugin [id 'com.google.protobuf']\n > Could not create an instance of type com.google.protobuf.gradle.ProtobufSourceDirectorySet_Decorated.\n\n* Try:\nRun with --stacktrace option
Currently, for multiple different cluster configuration, ops-cli fails when performing the 'Run Fabric configurator' operation.. Partial output below.

Add step for downloading this file from some binary repository instead of keeping it in the repository
fabric-skeleton/ops/configurator/fabric-configurator-fat-1.3.2.jar

Steps to reproduce:

1. Start cluster
   ./ops-cli -i ~/.ssh/Blockchain-controller.pem -c single_sample
2. Kill cluster
   ./ops-cli -i ~/.ssh/Blockchain-controller.pem -c single_sample -k
3. Start cluster again
   ./ops-cli -i ~/.ssh/Blockchain-controller.pem -c single_sample

Expected result:
Cluster starts normally
Actual result:
Cluster fails to start
Root cause
When cluster is starting it checks whether it was already started or not by checking availablility of AWS hosts. When we execute kill and hosts are in progress of termination, it thinks that cluster is alive and this leads to errors.
Workaround
After killing cluster, open AWS EC2 website and manually check that everything is down.
Proposed solution:
Enhance killing script so that it waits until actual cluster stop synchronously

Have some step for executing test chaincode in ansible as the last task

Currently, if bootStrap.sh script fail at some point while executing commands on remote controller - it does not catch this fact, and tells that everything was okay.

Need to add error-codes checks after each ssh invokation

For example check that all processes coreesponding to network structure are running