hyperledger / cello

Operating System for Enterprise Blockchain

Hyperledger Cello is a blockchain provision and operation system, which helps manage blockchain networks in an efficient way.

1. Introduction
2. Main Features
3. Quick Start
4. Documentation
5. Why named cello?
6. Notice

Introduction

Using Cello, everyone can easily:

• Build up a Blockchain as a Service (BaaS) platform quickly from scratch.
• Provision customizable Blockchains instantly, e.g., a Hyperledger fabric network v1.0.
• Maintain a pool of running blockchain networks on top of baremetals, Virtual Clouds (e.g., virtual machines, vsphere Clouds), Container clusters (e.g., Docker, Swarm, Kubernetes).
• Check the system status, adjust the chain numbers, scale resources... through dashboards.

A typical usage scenario is illustrated as:

Quick Start

Environmental preparation:

1. docker how install
2. docker compose(we switched to Docker Compose V2) how install
3. make all script for cello service management is written in Makefile
4. kubernetes (optional) how install
5. node how install

If environment is prepared, then we can start cello service.

• Set local storage environment variable, e.g. Use current path as storage path

  $  export CELLO_STORAGE_PATH=$(pwd)/cello

• Start service locally

  $ make local

• Optional: Build essential images for cello service (the docker hub image auto build haven't ready, in the future you can ignore this step.)

  • Build docker images

    $ make docker

  • Then run services locally then

    $ make start

• After service started up, if use docker-compose method, you can see output:

  CONTAINER ID   IMAGE                            COMMAND                  CREATED         STATUS         PORTS                                                                                  NAMES
  81e6459965ec   hyperledger/cello-agent-docker   "gunicorn server:app…"   4 seconds ago   Up 2 seconds   0.0.0.0:2375->2375/tcp, :::2375->2375/tcp, 0.0.0.0:5001->5001/tcp, :::5001->5001/tcp   cello.docker.agent
  04367ab6bd5e   postgres:11.1                    "docker-entrypoint.s…"   4 seconds ago   Up 2 seconds   0.0.0.0:5432->5432/tcp, :::5432->5432/tcp                                              cello-postgres
  29b56a279893   hyperledger/cello-api-engine     "/bin/sh -c 'bash /e…"   4 seconds ago   Up 2 seconds   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp                                              cello-api-engine
  a272a06d8280   hyperledger/cello-dashboard      "bash -c 'nginx -g '…"   4 seconds ago   Up 2 seconds   80/tcp, 0.0.0.0:8081->8081/tcp, :::8081->8081/tcp                                      cello-dashboard

• Stop cello service.

  $ make stop

• Clean all containers

  $ make clean

• Check available make rules

  $ make help

• Visit Cello dashboard at localhost:8081

• Check troubleshoot section if you get any question.

Main Features

• Manage the lifecycle of blockchains, e.g., create/start/stop/delete/keep health automatically.

• Support customized (e.g., size, consensus) blockchains request, currently we mainly support Hyperledger fabric.

• Support native Docker host, swarm or Kubernetes as the worker nodes. More supports on the way.

• Support heterogeneous architecture, e.g., X86, POWER and Z, from bare-metal servers to virtual machines.

• Extend with monitor, log, health and analytics features by employing additional components.

Documentation, Getting Started and Develop Guideline

For new users, it is highly recommended to read the documentation first.

And feel free to visit the online documentation for more information. You can also run make doc to start a local documentation website (Listen at localhost:8000.

Why named Cello?

Can you find anyone better at playing chains? :)

Incubation Notice

This project is a Hyperledger project in Incubation. It was proposed to the community and documented here, and was approved by Hyperledger TSC at 2017-01-07. Information on what Incubation entails can be found in the Hyperledger Project Lifecycle document.

Inclusive Language Statement

These guiding principles are very important to the maintainers and therefore we respectfully ask all contributors to abide by them as well:

• Consider that users who will read the docs are from different backgrounds and cultures and that they have different preferences.
• Avoid potential offensive terms and, for instance, prefer "allow list and deny list" to "white list and black list".
• We believe that we all have a role to play to improve our world, and even if writing inclusive documentation might not look like a huge improvement, it's a first step in the right direction.
• We suggest to refer to Microsoft bias free writing guidelines and Google inclusive doc writing guide as starting points.

This work is licensed under a Creative Commons Attribution 4.0 International License.

cello's Issues

ansible agent should support load balancer to expose endpoints

Is your feature request related to a problem? Please describe.
Currently ansible agent uses NodePort to expose fabric endpoints, it is just one options,
Load Balancer is another, ansible agent should support the agent.

Describe the solution you'd like
Add load balancer support

Describe alternatives you've considered

Additional context
Add any other context or screenshots about the feature request here.

cello-user-dashboard 意外终止

cello-user-dashboard | 2020-02-26 02:45:19,512 INFO 124 [egg-multipart] stream mode enable
cello-user-dashboard | /node_modules/egg/lib/application.js:70
cello-user-dashboard | throw e;
cello-user-dashboard | ^
cello-user-dashboard |
cello-user-dashboard | AssertionError [ERR_ASSERTION]: [egg-passport-oauth2] config.passportOauth2.key required
cello-user-dashboard | at module.exports.app (/node_modules/egg-passport-oauth2/app.js:10:3)
cello-user-dashboard | at Hook.configDidLoad (/node_modules/egg-core/lib/lifecycle.js:92:9)
cello-user-dashboard | at Lifecycle.triggerConfigDidLoad (/node_modules/egg-core/lib/lifecycle.js:143:14)
cello-user-dashboard | at Lifecycle.triggerConfigWillLoad (/node_modules/egg-core/lib/lifecycle.js:137:10)
cello-user-dashboard | at AppWorkerLoader.loadCustomApp (/node_modules/egg-core/lib/loader/mixin/custom.js:37:20)
cello-user-dashboard | at AppWorkerLoader.load (/node_modules/egg/lib/loader/app_worker_loader.js:33:10)
cello-user-dashboard | at new Application (/node_modules/egg/lib/application.js:66:19)
cello-user-dashboard | at Object. (/node_modules/egg-cluster/lib/app_worker.js:19:13)
cello-user-dashboard | at Module._compile (module.js:643:30)
cello-user-dashboard | at Object.Module._extensions..js (module.js:654:10)
cello-user-dashboard | [2020-02-26 02:45:19.532] [cfork:master:108] worker:124 disconnect (exitedAfterDisconnect: false, state: disconnected, isDead: false, worker.disableRefork: true)
cello-user-dashboard | [2020-02-26 02:45:19.533] [cfork:master:108] don't fork, because worker:124 will be kill soon
cello-user-dashboard | 2020-02-26 02:45:19,533 INFO 108 [master] app_worker#1:124 disconnect, suicide: false, state: disconnected, current workers: ["1"]
cello-user-dashboard | [2020-02-26 02:45:19.537] [cfork:master:108] worker:124 exit (code: 1, exitedAfterDisconnect: false, state: dead, isDead: true, isExpected: false, worker.disableRefork: true)
cello-user-dashboard | 2020-02-26 02:45:19,539 ERROR 108 nodejs.AppWorkerDiedError: [master] app_worker#1:124 died (code: 1, signal: null, suicide: false, state: dead), current workers: []
cello-user-dashboard | at Master.onAppExit (/node_modules/egg-cluster/lib/master.js:408:21)
cello-user-dashboard | at emitOne (events.js:116:13)
cello-user-dashboard | at Master.emit (events.js:211:7)
cello-user-dashboard | at Messenger.sendToMaster (/node_modules/egg-cluster/lib/utils/messenger.js:137:17)
cello-user-dashboard | at Messenger.send (/node_modules/egg-cluster/lib/utils/messenger.js:102:12)
cello-user-dashboard | at EventEmitter.cluster.on (/node_modules/egg-cluster/lib/master.js:277:22)
cello-user-dashboard | at emitThree (events.js:141:20)
cello-user-dashboard | at EventEmitter.emit (events.js:217:7)
cello-user-dashboard | at ChildProcess.worker.process.once (internal/cluster/master.js:185:13)
cello-user-dashboard | at Object.onceWrapper (events.js:317:30)
cello-user-dashboard | name: "AppWorkerDiedError"
cello-user-dashboard | pid: 108
cello-user-dashboard | hostname: cello-user-dashboard
cello-user-dashboard |
cello-user-dashboard | 2020-02-26 02:45:19,545 ERROR 108 [master] app_worker#1:124 start fail, exiting with code:1
cello-user-dashboard | 2020-02-26 02:45:19,549 ERROR 108 [master] exit with code:1
cello-user-dashboard | 2020-02-26 02:45:19,555 ERROR 114 [agent_worker] receive disconnect event on child_process fork mode, exiting with code:110
cello-user-dashboard | ⚠️ Error: /node_modules/egg-bin/lib/start-cluster {"stricky":true,"port":8081,"baseDir":"/var/www","workers":1,"framework":"/var/www/node_modules/egg"} exit with code 1
cello-user-dashboard | 2020-02-26 02:45:19,598 INFO 114 [agent_worker] beforeExit success
cello-user-dashboard | 2020-02-26 02:45:19,599 ERROR 114 [agent_worker] exit with code:110
cello-user-dashboard | ⚠️ Command Error, enable DEBUG=common-bin for detail
cello-user-dashboard | npm ERR! code ELIFECYCLE
cello-user-dashboard | npm ERR! errno 1
cello-user-dashboard | npm ERR! [email protected] start:dev: egg-bin dev --stricky --port 8081
cello-user-dashboard | npm ERR! Exit status 1
cello-user-dashboard | npm ERR!
cello-user-dashboard | npm ERR! Failed at the [email protected] start:dev script.
cello-user-dashboard | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
cello-user-dashboard |
cello-user-dashboard | npm ERR! A complete log of this run can be found in:
cello-user-dashboard | npm ERR! /root/.npm/_logs/2020-02-26T02_45_19_670Z-debug.log
cello-user-dashboard | npm ERR! code ELIFECYCLE
cello-user-dashboard | npm ERR! errno 1
cello-user-dashboard | npm ERR! [email protected] start: if-env NODE_ENV=production && npm run start:prod || npm run start:dev
cello-user-dashboard | npm ERR! Exit status 1
cello-user-dashboard | npm ERR!
cello-user-dashboard | npm ERR! Failed at the [email protected] start script.
cello-user-dashboard | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
cello-user-dashboard |
cello-user-dashboard | npm ERR! A complete log of this run can be found in:
cello-user-dashboard | npm ERR! /root/.npm/_logs/2020-02-26T02_45_19_722Z-debug.log

Build fails with the following error

Build failed.

Expected Behavior

Build should succeed.

Current Behavior

Build now fails with the following error.

Successfully built uwsgi pygraphviz watchdog django-allauth django-rest-auth coreschema inflection PyYAML validate-email strict-rfc3339 billiard dockerpty docopt texttable pathtools itypes pyrsistent
ERROR: docker-compose 1.23.2 has requirement jsonschema<3,>=2.5.1, but you'll have jsonschema 3.1.1 which is incompatible.
Installing collected packages: pytz, Django, uwsgi, djangorestframework, django-filter, itypes, uritemplate, idna, urllib3, chardet, certifi, requests, MarkupSafe, jinja2, coreschema, coreapi, ruamel.yaml.clib, ruamel.yaml, six, inflection, drf-yasg, PyYAML, jsonpointer, rfc3987, click, validate-email, strict-rfc3339, flex, pyrsistent, more-itertools, zipp, importlib-metadata, attrs, jsonschema, swagger-spec-validator, psycopg2-binary, holdup, django-extensions, pygraphviz, billiard, vine, amqp, kombu, celery, redis, dockerpty, cached-property, websocket-client, docopt, docker-pycreds, docker, texttable, docker-compose, argh, pathtools, watchdog, defusedxml, python3-openid, oauthlib, requests-oauthlib, django-allauth, django-rest-auth, PyJWT, djangorestframework-jwt, pyasn1, rsa, pyasn1-modules, cachetools, google-auth, python-dateutil, kubernetes, django-cors-headers
ERROR: docker-compose 1.23.2 has requirement requests!=2.11.0,!=2.12.2,!=2.18.0,<2.21,>=2.6.1, but you'll have requests 2.22.0 which is incompatible.

Possible Solution

Steps to Reproduce

Context (Environment)

• OS:
• Architecture:
• Docker version:
• Docker compose version:
• Kubernetes version:

Detailed Description and log

Possible Implementation

Does it support fabric version 1.4

I want to know does it support hyperledger fabric version 1.4 with raft as my consensus.

Broken/Missing Setup Doc Links

Setup Doc links are not available or there are broken links

Expected Behavior

User should be able to navigate to the doc links

Current Behavior

User is not able to navigate to the doc links due to broken links or unavailable links

Possible Solution

Please update the doc links

Steps to Reproduce

Navigate to

1. https://github.com/hyperledger/cello/blob/master/docs/tutorial.md
2. https://github.com/hyperledger/cello/blob/master/docs/setup/server.md#21-clone-source-code
3. https://github.com/hyperledger/cello/blob/master/docs/setup/server.md#22-prepare-environment
4. https://github.com/hyperledger/cello/blob/master/docs/setup/server.md#23-start-service

Improve Kubernetes Operator Agent Docs

Improves the kubernetes agent docs

docker hub dashboard image built failed

because yarn install rxjs-6.5.5.tgz: ESOCKETTIMEDOUT, will try to use npm install.

Unable to create agent or node in cello

When I use cello to create agent or node, I am facing the following issue:

1. Try to create a new agent, receive the Success response and Error 400 response together;
   
   Image 1 Try to create a new agent

2. Try to create a new node, receive Error 400 response as I chose type "CA", receive NOTHING as I chose another types(Front-end seems don't make any request to service).
   
   Image 2 Try to create CA Node
   
   Image 3 Try to create another types Node(Peer or Order)

Though I tried ,but still can't solve this problem.

I guess there was something wrong in my deployment of cello, maybe I miss some key steps or uninstall some dependencies?

---

Here is the environment Info:

• OS: Ubuntu 16.04.4 LTS

• Kernal: Linux 4.4.0-145-generic

• Architecture: x86-64

• docker version: 18.03.0-ce, build 0520e24

• docker componse version: 1.24.1, build 4667896b

Here is the steps I done:

1. make setup-master

2. make setup-worker

3. make start

And here is cello start logs:

Image 4 start script output

Image 5 api-engine log(Unable to load uWSGI plugin, maybe caused by this?)

Image 6 api-engine-task log

Image 7 redis log

Image 8 postgress log

wget: bad address 'nexus.hyperledger.org'

Expected Behavior

make docker should work.

Current Behavior

Upon running make docker, it fails at getting the address "nexus.hyperledger.org"

Possible Solution

"nexus.hyperledger.or" has been retired. Check this link for proof.

Artifacts have been moved to JFrog

Possible solution is to update the links that points to the correct repo.

Context (Environment)

• OS: Ubuntu Linux 16.04
• Architecture: x64
• Docker version: 18.06.1-ce
• Docker compose version: 1.24.1

login user and passowrd?

What user and passowrd when login MASTER_IP:8080?

answer:

https://github.com/hyperledger/cello/blob/master/docs/scenario.md

Thx.

Design the API spec for user dashboard

Need to re-arch the code following the new governing model.

Feel free to review the drafted spec

Contribute the H3C release

Need to create a branch based on v0.9.0 for the H3C release.

• Update the documentation, add deployment steps in English;
• Cleanup the code, add comments.
• Deadline: End of this April.

fabric k8s operator ca example file has few extra spaces in the yaml file

fabric_v1alpha1_ca_cr.yaml has one extra space at the end. Which renders the yaml file invalid

Expected Behavior

no extra spaces.

Current Behavior

invalid yaml file

Possible Solution

remove the extra spaces

Steps to Reproduce

Context (Environment)

• OS:
• Architecture:
• Docker version:
• Docker compose version:
• Kubernetes version:

Detailed Description and log

Extra spaces made the yaml file invalid. need to remove them.

Possible Implementation

Remove the extra spaces

can't create node in dashboard

when I create node at dashboard occur some error. check the container log of hyperledger/cello-api-engine :

[uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/server.ini
open("./python_plugin.so"): No such file or directory [core/utils.c line 3721]
!!! UNABLE to load uWSGI plugin: ./python_plugin.so: cannot open shared object file: No such file or directory !!!

---

check the container log of hyperledger/cello-nginx:

"POST /api/nodes HTTP/1.1" 400 53 "http://62.234.106.181:8081/operator/node/new/node-info?network_type=fabric&agent_type=kubernetes&network_version=1.4.2&type=ca" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36" "-"

In product mode, Nginx server not mount media path into container

Expected Behavior

In product mode, nginx server should support media url. such as: http://127.0.0.1:8085/engine/media/xxx

Current Behavior

nginx.conf not set media url mapping.

Possible Solution

1. set media url config in nginx.conf.default
2. mount media volume into nginx container.

Steps to Reproduce

1. make start
2. create new kubernetes agent with config file
3. wget http://127.0.0.1:8085/engine/media/config_files/c41c3c5e-b963-47f5-b927-c4d4f501165e/79396cc13be39fda0c39edf3da8bfa5d.zip will got 404 not found

Context (Environment)

Detailed Description

Possible Implementation

Move make check into step template

Move make check into step template, and add condition check, only if PR triggered.

Netlify build function failed

Expected Behavior

Build function successfully

Current Behavior

Build function failed

Possible Solution

Add netlify.tolm under dashboard folder

Detailed Description and log

10:00:38 PM: No netlify.toml found. This is needed to configure the function settings. For more info: https://github.com/netlify/netlify-lambda#installation
10:00:38 PM: npm ERR! code ELIFECYCLE
10:00:38 PM: npm ERR! errno 1
10:00:38 PM: npm
10:00:38 PM: ERR! [email protected] functions:build: npm run generateMock && netlify-lambda build ./lambda
10:00:38 PM: npm
10:00:38 PM: ERR! Exit status 1
10:00:38 PM: npm
10:00:38 PM: ERR!
10:00:38 PM: npm
10:00:38 PM: ERR!
10:00:38 PM: Failed at the [email protected] functions:build script.
10:00:38 PM: npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
10:00:38 PM: npm
10:00:38 PM: ERR! A complete log of this run can be found in:
10:00:38 PM: npm ERR! /opt/buildhome/.npm/_logs/2019-11-21T14_00_38_805Z-debug.log
10:00:38 PM: Function Dir: /opt/build/repo/src/dashboard/functions

Possible Implementation

Make sure all k8s related agent describe supported k8s versions

Make sure all k8s related agent describe supported k8s versions, add related doc

• Deployment template file;
• Documentation

Operator should be not able to create node in dashboard

Is your feature request related to a problem? Please describe.
If user want to create a new node, must join a organization at first, because not belong to any organization, and can see all the deployed node, so in operator dashboard, can list all deployed node, but can not create new node.

Describe the solution you'd like
Maybe can remove or disable new button in operator dashboard.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

The dockerfile of kubernetes agent need to revise to support corev1api of kubernetes

The dockerfile of kubernetes agent need to revise to support corev1api of kubernetes

Expected Behavior

when we create node,error will report like this:

#_root@rtxie-Standard-PC-i440FX-PIIX-1996:~/cello# docker logs4834991f2f59

docker: 'logs4834991f2f59' is not a docker command.
See 'docker --help'
root@rtxie-Standard-PC-i440FX-PIIX-1996:~/cello# docker logs 4834991f2f59
Exception when call CoreV1Api: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Wed, 23 Oct 2019 03:37:38 GMT', 'Content-Length': '264'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services "service-23be5dac-6ef5-4282-92f9-5e36781c16de" not found","reason":"NotFound","details":{"name":"service-23be5dac-6ef5-4282-92f9-5e36781c16de","kind":"services"},"code":404}

Exception when call AppsV1beta1Api: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Wed, 23 Oct 2019 03:37:38 GMT', 'Content-Length': '361'})
HTTP response body: {
"kind": "Status",
"apiVersion": "v1",
"metadata": {

},
"status": "Failure",
"message": "deployments.extensions "deploy-23be5dac-6ef5-4282-92f9-5e36781c16de" not found",
"reason": "NotFound",
"details": {
"name": "deploy-23be5dac-6ef5-4282-92f9-5e36781c16de",
"group": "extensions",
"kind": "deployments"
},
"code": 404
}_

Current Behavior

Possible Solution

Steps to Reproduce

Context (Environment)

Detailed Description

Possible Implementation

The dockerfile need to revise like this

ENV KUBECONFIG /app/.kube/config
CMD export PYTHON_VER=${PYTHON_VERSION:0:3}
ENV PYTHONPATH /app:/usr/local/lib/python${PYTHON_VER}:/usr/local/lib/python${PTYHON_VER}/lib-dynload:/usr/local/lib/python${PATHON_VER}/site-packages

Ansible agent fails create daemonset on k8s 1.16

Ansible agent fails create daemonset on k8s 1.16

Expected Behavior

ansible agent should succeed

Current Behavior

fails

Possible Solution

Ansible agent uses v1beta for daemonset which has been removed in k8s 1.16

Steps to Reproduce

Context (Environment)

• OS:
• Architecture:
• Docker version:
• Docker compose version:
• Kubernetes version:

Detailed Description and log

Possible Implementation

api-engine-tasks pod in CrashLoopBackOff: No module named 'api_engine'

I am trying to spin up your latest version of Cello on a kubernetes EKS cluster. I followed all the steps but I get your api-engine-tasks pod in CrashLoopBackOff with the following error:

Expected Behavior

All pods successfully deployed and running.

Current Behavior

default api-engine-99c857b95-6sgc7 1/1 Running 0 16s
default api-engine-tasks-785f8cc46f-cwjtt 0/1 CrashLoopBackOff 1 17s
default cello-nginx-85c45dcd6c-n6wpt 1/1 Running 0 14s
default postgres-58d84fcb94-rwx8g 1/1 Running 0 19s
default redis-6fbb95f5c8-9csf8 1/1 Running 0 18s

$ kubectl logs -f api-engine-tasks-785f8cc46f-cwjtt
/scripts/change_local_settings.sh: line 5: /var/www/server/api_engine/settings.py.example: No such file or directory
All initial Done!!
Traceback (most recent call last):
File "/usr/local/bin/celery", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/celery/main.py", line 16, in main
_main()
File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 322, in main
cmd.execute_from_commandline(argv)
File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 496, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 273, in execute_from_commandline
argv = self.setup_app_from_commandline(argv)
File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 479, in setup_app_from_commandline
self.app = self.find_app(app)
File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 501, in find_app
return find_app(app, symbol_by_name=self.symbol_by_name)
File "/usr/local/lib/python3.6/site-packages/celery/app/utils.py", line 359, in find_app
sym = symbol_by_name(app, imp=imp)
File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 504, in symbol_by_name
return imports.symbol_by_name(name, imp=imp)
File "/usr/local/lib/python3.6/site-packages/kombu/utils/imports.py", line 57, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "/usr/local/lib/python3.6/site-packages/celery/utils/imports.py", line 104, in import_from_cwd
return imp(module, package=package)
File "/usr/local/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'api_engine'

Possible Solution

Steps to Reproduce

1. export DEPLOY_METHOD=k8s
2. export KUBECONFIG=/path/to/kubeconfig
3. export AWS_PROFILE=yourAWSprofile
4. make start

Context (Environment)

• OS: Mac
• Docker version:
  Client: Docker Engine - Community
  Version: 19.03.4
  API version: 1.40
  Go version: go1.12.10
  Git commit: 9013bf5
  Built: Thu Oct 17 23:44:48 2019
  OS/Arch: darwin/amd64
  Experimental: false

Server: Docker Engine - Community
Engine:
Version: 19.03.4
API version: 1.40 (minimum version 1.12)
Go version: go1.12.10
Git commit: 9013bf5
Built: Thu Oct 17 23:50:38 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
GitCommit: fec3683

• Docker compose version:
  docker-compose version 1.24.1, build 4667896b
  docker-py version: 3.7.3
  CPython version: 3.6.8
  OpenSSL version: OpenSSL 1.1.0j 20 Nov 2018
• Kubernetes version:
  Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.7", GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4", GitTreeState:"clean", BuildDate:"2019-09-18T14:47:22Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"}
  Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.6-eks-5047ed", GitCommit:"5047edce664593832e9b889e447ac75ab104f527", GitTreeState:"clean", BuildDate:"2019-08-21T22:32:40Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
• Python version:
  $ python --version
  Python 2.7.10
  $ python3 --version
  Python 3.7.3

extensions/v1beta1 api version deprecated after Kubernetes 1.16

Expected Behavior

For kubernetes v1.16 server, can deploy fabric node successfully.

Current Behavior

Can not deploy fabric node on kubernetes 1.16 later version.

HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'Thu, 24 Oct 2019 08:27:21 GMT', 'Content-Length': '222'})
HTTP response body: {
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "the server could not find the requested resource",
  "reason": "NotFound",
  "details": {

  },
  "code": 404
}

Traceback (most recent call last):
  File "main.py", line 9, in <module>
    create_node()
  File "/app/operations/create_node.py", line 161, in create_node
    _create_fabric_node()
  File "/app/operations/create_node.py", line 119, in _create_fabric_node
    k8s_client.create_deployment(AGENT_ID, **deployment)
  File "/app/utils/client.py", line 183, in create_deployment
    raise e
  File "/app/utils/client.py", line 179, in create_deployment
    namespace=namespace, body=body, pretty="true"
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/apis/extensions_v1beta1_api.py", line 175, in create_namespaced_deployment
    (data) = self.create_namespaced_deployment_with_http_info(namespace, body, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/apis/extensions_v1beta1_api.py", line 266, in create_namespaced_deployment_with_http_info
    collection_formats=collection_formats)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 334, in call_api
    _return_http_data_only, collection_formats, _preload_content, _request_timeout)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 168, in __call_api
    _request_timeout=_request_timeout)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 377, in request
    body=body)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 266, in POST
    body=body)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 222, in request
    raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'Thu, 24 Oct 2019 08:27:21 GMT', 'Content-Length': '222'})
HTTP response body: {
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "the server could not find the requested resource",
  "reason": "NotFound",
  "details": {

  },
  "code": 404
}

Possible Solution

try get k8s server version before use it. and use different function and api version to create deployment.

Steps to Reproduce

1. make start
2. create organization & admin for org.
3. create kubernetes agent with upload config file.
4. login org admin and apply the agent.
5. create ca node.
6. will shown node is deploying

Context (Environment)

Detailed Description

Possible Implementation

Add Kuberentes Operator Agent Dockerfile

The Dockerfile creates a container for the deploying peer/ca/orderer. This is updated Dockerfile as per required by dockerhub

我改怎样新建代理

新的代码版本中，8080于8081的两个登陆面板是否应该合并？

Use api-engine API to create a orderer node

Let api-engine provide a RESTful API to bootup a orderer node (1.4 or 2.0).

Add netlify auto build support

Add netlify configuration file, for auto deploy dashboard and mock api on netlify

Kubernetes agent create node, when check deployment is alive if pod is not created will cause crash

Expected Behavior

Can update node status after node is deployed.

Current Behavior

After node is deployed, cause crashed when check node status.

Traceback (most recent call last):
  File "main.py", line 9, in <module>
    create_node()
  File "/app/operations/create_node.py", line 162, in create_node
    _create_fabric_node()
  File "/app/operations/create_node.py", line 140, in _create_fabric_node
    print(pod, pod.status)
AttributeError: 'NoneType' object has no attribute 'status'

Possible Solution

Check if pod is not null object at first.

Steps to Reproduce

1. make start
2. MOCK=False PROXY=http://9.59.196.20:8085/engine make start-dashboard-dev run at your computer
3. create new agent
4. apply agent
5. create new node

Context (Environment)

OS: Ubuntu 18.04
Arch: x86_64
Kubernetes version: 1.13
Docker version: 19.03.3
Docker compose version: 1.24.1

Detailed Description

Possible Implementation

Ansible agent can not support fabric 1.4.3

Fabric 1.4.3 cryptogen does not place each org admin cert in the org msp admincerts directory. This change caused the configtxgen not be able to include admin certs in the config admin certs, that eventually caused genesis block not include admin certs, eventually channel creation will fail.

Expected Behavior

Admin certs should be included msp admincerts directory and channel creation wont fail.

Current Behavior

Admin cert is missing from org msp/admincerts directory

Possible Solution

Add admin cert in users/Admin@org/signcerts to the msp/admincert directory

Steps to Reproduce

Use an example ansible agent network spec file and change the fabric release to 1.4.3, then produce a fabric network,then run a channel create test, the test will fail.

Context (Environment)

• OS:
• Architecture:
• Docker version:
• Docker compose version:
• Kubernetes version:

Detailed Description and log

Possible Implementation

Incorrectly list the project's license as "ISC".

LF report: https://lfscanning.org/reports/hyperledger/cello-2020-01-03-d9c03bc4-e231-4bff-9638-41c8ae123390.html.

Some package.json files incorrectly list the project's license as "ISC". These should be corrected to "Apache-2.0"

cello/build_image/dockerhub/v0.9.0/user-dashboard/package.json

Expected Behavior

Not use ISC license packages.

Current Behavior

Some packages are using ISC license.

Possible Solution

Replace the package that uses ISC license.

Steps to Reproduce

Context (Environment)

• OS:
• Architecture:
• Docker version:
• Docker compose version:
• Kubernetes version:

Detailed Description and log

Possible Implementation

Add bug issue template and pull request template

Add bug issue template and pull request template

health go fail when add chain

hello,

when i add new chain in added host (for example host1 by docker),

cello-watchdog | [2020-02-04 16:04:52,940] INFO [modules.cluster] [cluster.py:900 db_update_one()] - filter {'id': '96c93fc42a0448768d732f44b211feaa', 'state': 'active'} operations {'health': 'FAIL'}

why?
thanks.

Dashboard api proxy to api engine, proxy_header host doesn't include port info

Expected Behavior

Api proxy from dashboard to api engine, should include port info, so that when api engine use build_absolute_uri can get the port info, and generate the right url.

Current Behavior

Such as get agent config url path, http://9.59.196.20/engine/media/config_files/63a4e2b6-a61b-4537-9558-4a33789b2222/bf08eecabf0b7afee5b25e0363e9c061.zip
port is ignored ,but the server is not running on 80 port, it's really port is 8085, so agent can't download the config file.

Possible Solution

Add api engine service port into proxy_set_header Host $host:SERVICE_PORT

Steps to Reproduce

1. make start
2. create new agent
3. create a ca node, will keep deploying.

Context (Environment)

Detailed Description or log

Traceback (most recent call last):
  File "main.py", line 9, in <module>
    create_node()
  File "/app/operations/create_node.py", line 161, in create_node
    _create_fabric_node()
  File "/app/operations/create_node.py", line 84, in _create_fabric_node
    k8s_client = get_k8s_client()
  File "/app/utils/__init__.py", line 30, in get_k8s_client
    k8s_client = KubernetesClient(config_file=k8s_config)
  File "/app/utils/client.py", line 16, in __init__
    config.load_kube_config(config_file)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 546, in load_kube_config
    config_persister=config_persister)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 503, in _get_kube_config_loader_for_yaml_file
    with open(filename) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/app/.kube/config'

Possible Implementation

capability started in 1.4.x must use V1_4_x

Current capability can use two digits, but started in 1.4, all digits have to be used
such as V1_4_3. V1_4 wont work

Expected Behavior

All orderer nodes should start up correctly

Current Behavior

All orderer nodes fails with errors like the following:

panic: [channel orderersystemchannel] config requires unsupported channel capabilities: Channel capability V1_4 is required but not supported: Channel capability V1_4 is required but not supported

Possible Solution

Use the three digit capability string

Steps to Reproduce

Context (Environment)

• OS:
• Architecture:
• Docker version:
• Docker compose version:
• Kubernetes version:

Detailed Description and log

Possible Implementation

PR broke CI

PR #120 was merged, and there are two repos that have not built since then:

https://hub.docker.com/repository/docker/hyperledger/cello-api-engine/builds
https://hub.docker.com/repository/docker/hyperledger/cello-dashboard/builds

no 8080 port

When I run make setup-master,setup-worker and make run.

How to proceed like docs

there is an error "UNABLE to load uWSGI plugin " when api-engine started

After run the command "make start-k8s", it reponse an error-"Internal error" when I send a api request to the api-engine service.

curl -H 'Content-type: application/json' -XPOST http://192.168.16.101:32001/engine/auth/login/ -d '{"username":"admin", "password": "pass", "type": "account"}'

Can someone help me solve the problem?

The logs of api-engine pod are:

All initial Done!!
Traceback (most recent call last):
File "manage.py", line 18, in
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/init.py", line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.6/site-packages/django/core/management/init.py", line 325, in execute
settings.INSTALLED_APPS
File "/usr/local/lib/python3.6/site-packages/django/conf/init.py", line 57, in getattr
self._setup(name)
File "/usr/local/lib/python3.6/site-packages/django/conf/init.py", line 44, in _setup
self._wrapped = Settings(settings_module)
File "/usr/local/lib/python3.6/site-packages/django/conf/init.py", line 107, in init
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/local/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 674, in exec_module
File "", line 781, in get_code
File "", line 741, in source_to_code
File "", line 219, in _call_with_frames_removed
File "/var/www/server/api_engine/settings.py", line 27
DEBUG =
^
SyntaxError: invalid syntax
Traceback (most recent call last):
File "manage.py", line 18, in
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/init.py", line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.6/site-packages/django/core/management/init.py", line 325, in execute
settings.INSTALLED_APPS
File "/usr/local/lib/python3.6/site-packages/django/conf/init.py", line 57, in getattr
self._setup(name)
File "/usr/local/lib/python3.6/site-packages/django/conf/init.py", line 44, in _setup
self._wrapped = Settings(settings_module)
File "/usr/local/lib/python3.6/site-packages/django/conf/init.py", line 107, in init
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/local/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 674, in exec_module
File "", line 781, in get_code
File "", line 741, in source_to_code
File "", line 219, in _call_with_frames_removed
File "/var/www/server/api_engine/settings.py", line 27
DEBUG =
^
SyntaxError: invalid syntax
[uWSGI] getting INI configuration from /etc/uwsgi/apps-enabled/server.ini
open("./python_plugin.so"): No such file or directory [core/utils.c line 3721]
!!! UNABLE to load uWSGI plugin: ./python_plugin.so: cannot open shared object file: No such file or directory !!!

Refine bug & feature issue template

Refine bug & feature issue template

Clean docker build image for api-engine

The current building process involves several useless steps.

And it will trigger security alerts with the old version of django packages.

Need to clean unused code.

disable upload file support for api engine swagger doc

because enable upload file support for api engine swagger doc, will cause cannot add form parameters when the request has a request body in api engine service. So the temporary method is disable it.

"cello-nginx" is restarting always

Expected Behavior

All containers should be up and should show them when docker ps is executed.
Then, going to localhost:8081 and see the Dashboard.

Current Behavior

Output of make start

if [ "docker-compose" = "docker-compose" ]; then \
	make start-docker-compose; \
else \
	make start-k8s; \
fi
make[1]: Entering directory '/home/genesis3g/cello'
docker-compose -f bootup/docker-compose-files/docker-compose.yml up -d --force-recreate
Creating cello-postgres-server ... done
Creating cello-redis           ... done
Creating cello-api-engine-tasks ... done
Creating cello-api-engine       ... done
Creating cello-nginx            ... done
Creating cello-dashboard        ... done
make[1]: Leaving directory '/home/genesis3g/cello'

Aparently everthing is OK. But...

Output of docker ps

CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS                          PORTS               NAMES
1a696c6fba92        hyperledger/cello-nginx        "bash -c '/config-ng…"   6 minutes ago       Restarting (1) 3 seconds ago                        cello-nginx
7b93d6716b09        hyperledger/cello-api-engine   "/bin/sh -c 'bash /e…"   6 minutes ago       Up 6 seconds                                        cello-api-engine
11d3a77557df        hyperledger/cello-api-engine   "/bin/sh -c 'bash /e…"   6 minutes ago       Up 6 minutes                                        cello-api-engine-tasks
6de08c232e12        postgres:11.1   

It doesn't show cello-redis or cello-dashboard containers, AND cello-nginx is always restarting, due that cello-postgres-server too.

Of course, I can't go to localhost:8081 to see the dashboard.

I don't know what is happening!

Context (Environment)

• OS: Deepin 15.11
• Architecture: 64
• Docker version: 18.09.6, build 481bc77
• Docker compose version: 1.25.4, build 8d51620a

Detailed Description and log

Output of docker logs --tail 50 --follow --timestamps cello-nginx is:

2020-02-27T21:18:39.531216212Z 2020/02/27 21:18:39 [emerg] 1#1: host not found in upstream "api-engine:8080" in /etc/nginx/nginx.conf:36
2020-02-27T21:18:39.531255817Z nginx: [emerg] host not found in upstream "api-engine:8080" in /etc/nginx/nginx.conf:36

Add actions file for PR

Add Pull request workflow

Fix setup-master script, remove download operation of unused images

Expected Behavior

• should download hyperledger/cello-dashboard, hyperledger/cello-nginx, hyperledger/cello-api-engine images

Current Behavior

• download images include: baseimage engine mongo operator-dashboard user-dashboard watchdog parse-server

Possible Solution

• remove theme

Steps to Reproduce

1. make setup-master

Context (Environment)

Detailed Description

Possible Implementation

部署完，运行报错

部署完 cello 之后，监控 dashboard 日志：make logs service=dashboard，会有这样的报错：
dashbard | Traceback (most recent call last):
dashbard | File "dashboard.py", line 4, in
dashbard | from resources import bp_index,
dashbard | File "/app/resources/init.py", line 1, in
dashbard | from .index import bp_index
dashbard | File "/app/resources/index.py", line 17, in
dashbard | from modules import cluster_handler, host_handler
dashbard | File "/app/modules/init.py", line 1, in
dashbard | from .cluster import cluster_handler
dashbard | File "/app/modules/cluster.py", line 14, in
dashbard | from agent import get_swarm_node_ip,
dashbard | File "/app/agent/init.py", line 1, in
dashbard | from .docker_swarm import get_project,
dashbard | File "/app/agent/docker_swarm.py", line 10, in
dashbard | from docker import Client
dashbard | ImportError: cannot import name 'Client'

How do I add nodes?

Fabric k8s operator CA example cr file has wrong parameters

Fabric k8s operator CA example cr file uses wrong configParams format

Expected Behavior

`
configParams:

• name: FABRIC_CA_SERVER_PORT
  value: "7054"
• name: FABRIC_CA_SERVER_DEBUG
  value: false
  `

Current Behavior

configParams:
FABRIC_CA_SERVER_PORT: 7054
FABRIC_CA_SERVER_DEBUG: false

Possible Solution

Steps to Reproduce

Context (Environment)

• OS:
• Architecture:
• Docker version:
• Docker compose version:
• Kubernetes version:

Detailed Description and log

Possible Implementation

Imporve Kuberentes Agent to work with Dashboard and deploy nodes

The change let's the agent to create/delete node while taking the
parameters from the config.json. This will be provided to node after uploading the file

The agent updates the UI for the changes made

Peer/CA Certs can also be handled now.

Documentation of the Operator Agent

The requirement for PersistentVolumes for each peer/ca/orderer is not apparent as well as the AccessType for the volume

我该怎样清理 之前在面板上创建的链等数据

Agent ansible docker image build failed

Expected Behavior

cello-agent-ansible docker build successfully.

Current Behavior

Possible Solution

Install curl package.

Steps to Reproduce

1. make agent-docker-ansible

Context (Environment)

Detailed Description

Possible Implementation