hyperledger / indy-node

The server portion of a distributed ledger purpose-built for decentralized identity.

Indy Node

• Indy Node
  • Announcements
    • April 12 2023
  • About Indy Node - Relationship with Sovrin - Getting Started Guide - Hyperledger Wiki-Indy
  • Technical Overview of Indy Blockchain
  • Indy Node Repository Structure
  • Dependent Projects
  • Contact us
  • How to Contribute
  • How to Install a Test Network
  • How to Start Working with the Code
  • Continuous Integration and Delivery
  • How to send a PR
    • How to send a PR to both plenum and node
  • Docs and links

Announcements

April 12 2023

The project branches have changed.

The main branch now contains the Ubuntu 20.04 work stream, and the previous main branch containing the Ubuntu 16.04 work stream has been moved to the ubuntu-16.04 branch. We encourage everyone to switch to using the new code and appreciate your patience while we stabilize the work flows and documentation on this new branch.

The following changes were made to the branches:

• main (default) renamed to ubuntu-16.04
  • This retargeted the associated PRs.
• ubuntu-20.04-upgrade set as the default branch.
• ubuntu-20.04-upgrade (default) renamed to main

About Indy Node

This codebase embodies all the functionality to run nodes (validators and/or observers) that provide a self-sovereign identity ecosystem on top of a distributed ledger. It is the core project for Indy; over time, all other indy-* projects may collapse into this one, except for indy-sdk.

Indy has its own distributed ledger based on RBFT.

Relationship with Sovrin

This code is independent from but commonly associated with The Sovrin Foundation. The Sovrin Foundation is a public utility for identity, built on top of this codebase. People who install sovrin packages (e.g., with sudo apt install sovrin) get prepackaged genesis transactions that integrate with an Indy validator pool using Sovrin's governance and trust framework. However, it is possible to use Indy Node with a different network, using whatever conventions a community chooses.

Getting Started Guide

We recommend that developers should explore Indy Walk through to learn about Indy basics or Getting Started Guide with VCX and Getting Started Notebook.

Hyperledger Wiki-Indy

If you haven't done so already, please visit the main resource for all things "Indy" to get acquainted with the code base, helpful resources, and up-to-date information: Hyperledger Wiki-Indy.

Technical Overview of Indy Blockchain

• Refer to our documentation site at indy.readthedocs.io for the most current documentation and walkthroughs
  • indy.readthedocs.io/projects/plenum
  • indy.readthedocs.io/projects/node
• Please visit Technical Overview of Plenum.
• More documentation can be found in indy-plenum-docs and indy-node-docs.

Indy Node Repository Structure

Indy Node repo consists of the following parts:

• indy-node:
  • indy-plenum-based implementation of distributed ledger
  • Extends plenum's base pool functionality with specific transactions support (CLAIM_DEF, SCHEMA, POOL_UPGRADE, etc.)
• indy-common
  • Common code for indy-node
• scripts
  • Some scripts that can be run for installed Node (in particular, scripts to start Nodes, generate keys, prepare test Network, etc.)
• doc
  • A folder with documentation
• dev-setup
  • A folder with scripts helping to configure development environment (python, dependencies, projects, virtual environment)

Dependent Projects

• indy-plenum
  • The heart of the distributed ledger technology inside Hyperledger Indy.
  • Most probably you will need to make changes in Plenum if you want to contribute to Indy. So, if you want to work with Indy Node, you will need to have the Plenum code as well in most of the cases and work with two projects at the same time (see How to Start Working with the Code below).
• indy-sdk
  • An official SDK for Indy.
  • It contains client and anoncreds implementation
  • You don't need it to contribute to Indy-Node. But please use indy-sdk for your own applications dealing with Indy ecosystem.
• ursa
  • Hyperledger's shared crypto library
  • In particular, it contains BLS multi-signature crypto needed for state proofs support in Indy.

Contact us

• Bugs, stories, and backlog for this codebase are managed in Hyperledger's Jira. Use project name INDY.

• Join us on Hyperledger's chat in or to discuss.

How to Contribute

• We'd love your help; see these instructions on how to contribute.
• You may also want to read this info about maintainers.
• See How to send a PR below.

How to Install a Test Network

You can have a look at Start Nodes to understand what needs to be done to create a Network, initialize and start Nodes, and what scripts are provided for this.

The described process is automated in one of the ways below (it allow to install a test Network):

• Docker Start Pool with Docker

• Docker-based pool using with new libindy-based CLI:

  • Start Pool Locally
  • Get Started with Libindy

• Also coming soon: Create virtual machines in AWS.

How to Start Working with the Code

Please have a look at Dev Setup

Continuous Integration and Delivery

Please have a look at Continuous integration/delivery

How to send a PR

• Make sure that you followed write code guideline before sending a PR
• Do not create big PRs; send a PR for one feature or bug fix only. If a feature is too big, consider splitting a big PR to a number of small ones.
• Consider sending a design doc into design folder (as markdown or PlantUML diagram) for a new feature before implementing it
• Make sure that a new feature or fix is covered by tests (try following TDD)
• Make sure that documentation is updated according to your changes
• Provide a full description of changes in the PR including Jira ticket number if any
• Make sure that all your commits have a DCO sign-off from the author
• Make sure that static code validation passed (you can run flake8 . on the project root to check it; you can install flake8 from pypi: pip install flake8)
• Put the link to the PR into #indy-pr-review channel in Rocket.Chat
• A reviewer needs to start your tests first (add test this please comment to the PR)
• You need to make sure that all the tests pass
• A reviewer needs to review the code and approve the PR. If there are review comments, they will be put into the PR itself.
• You must process them (feel free to reply in the PR threads, or have a discussion in Rocket.Chat if needed)
• A reviewer or maintainer will merge the PR (we usually use Squash)

How to send a PR to both plenum and node

If you made changes in both indy-plenum and indy-node, you need to do the following:

• Raise a PR to indy-plenum's master and wait until code is reviewed and merged (see above)
  • So, a new build of indy-plenum is created
• Note a just built version X.Y.Z.devB of indy-plenum (you can check it in pypi or on CI server).
• Change indy-plenum's dependency version to the new one in indy-node's setup.py.
• Raise PR to indy-node's master and wait until code is reviewed and merged (see above)
  • So, a new build of indy-node is created

Docs and links

• Indy-plenum is based on RBFT protocol
• Please have a look at documents and diagrams in docs/source folder
• Please have a look at documents and diagrams in Plenum's docs folder, or on https://indy.readthedocs.io/projects/plenum :
  • Technical Overview of Plenum
  • Plenum Consensus Algorithm Diagram
  • Glossary
  • Storages
  • Request Handling
  • Catchup
  • Catchup Diagram
  • Audit Ledger
  • Plugins
• Relationship between Entities and Transactions: relationship diagram
• Supported transactions and their format: transactions
• Supported requests (write, read) and their format: requests
• Network roles and permissions
• Indy file folder structure guideline
• Helper Scripts
• Pool Upgrade
• Node Addition

indy-node's Issues

Remove pip imports in favor of importlib_metadata

See hyperledger/indy-plenum#1548 for details.

Make the same changes to indy-node to remove the dependency on pip.

Define the anoncreds 2 revocation ledger transactions, content and work required (from JIRA INDY-2365)

Discussions were held about anoncreds revocation 2.0 (aka Merkle Tree-based revocation) based on the capabilities exposed in Ursa.  We propose that the following attributes and approaches be used in the Indy credential revocation 2.0 instance.

• Tree leaves will be indexed bits (vs. bytes) that are either 0 (not revoked) or 1 (revoked).
• The Merkle tree will be an 8-ary tree (vs. binary or 4-ary).
• Nodes holding the tree leaves will be 64-bit unsigned integers,
• As such, a minimum revocation registry is one layer deep with a root hash and eight 64 bit ints, and hence would be capable of handling 512 credentials.
• On creation of a revocation registry, a parameter will be the depth of the tree, with a value of 1 to 7, supporting up to ~134 million credentials in a single registry.
• The table below gives the number of credentials and MByte size of the leaves by the number of layers of the registry.
• The two existing revocation-related ledger transactions will continue to be used - REVOC_REG_DEF and REVOC_REG_ENTRY
• REVOC_REG_DEF will contain at least a link to the cred def and the number of layers of the Merkle Tree. Since all the leaves are 0 (all credentials unrevoked), the Merkle Tree itself is not needed.
• REVOC_REG_ENTRY will contain at least a signed Merkle Tree root and a gzip compressed set of leaf nodes in a well-defined configuration.
  • Since the distribution of revoked credentials is likely to be skewed one way or the other, the compression rate in the common case is likely to be high.
  • For a 7-layer ledger with (unlikely) ineffective compression the worst ledger entry size would be ~16Mb.
  • The interior nodes of the Merkle Tree must be calculated from the leaves and the calculated root verified against that in the ledger transaction.
    • It may be possible to organize the leaves such that a prover can stream the compressed data, decompressing only the section necessary to construct the proof--the path from the credential leaf to the root.
  • A "domain separation" value is needed as well, for reasons to be added here.
• As is necessary today, the holder/prover and verifier must read from the ledger the REVOC_REG_DEF and appropriate REVOC_REG_ENTRY values in creating and verifying the proof.
• Unlike today, no tails file is needed by the prover.

To be determined is whether any change is needed to Indy Node to support the transaction content, or whether the current Indy Node code can support the same transaction with different content. Note that Indy Node does not have to do anything with the transaction content other than write it to the ledger and pass it back to any readers.

_load_cdll: Can't load libursa: libursa.so: cannot open shared object file: No such file or directory

Guide From: https://github.com/ecred-tech/indy-node/tree/master/environment/docker/pool

Tried following the above guide for me to be able to setup indy-node on my local machine.

root@0f39a420545b:/home/indy# init_indy_node Alpha 0.0.0.0 9701 0.0.0.0 9702
Node-stack name is Alpha
Client-stack name is AlphaC
Generating keys for random seed b'7207Fb5eDBf4527902215CE661FD22a9'
Init local keys for client-stack
Public key is HDS86XZcXYKCPkrQRzLLB72QmB6Ao9hAvka8zRLScMcL
Verification key is 3f27Ksu62d6zbJqCJsYrywkRSjckguGPeDgGvFXkKyj2
Init local keys for node-stack
Public key is HDS86XZcXYKCPkrQRzLLB72QmB6Ao9hAvka8zRLScMcL
Verification key is 3f27Ksu62d6zbJqCJsYrywkRSjckguGPeDgGvFXkKyj2
_load_cdll: Can't load libursa: libursa.so: cannot open shared object file: No such file or directory
libursa.so: cannot open shared object file: No such file or directory
root@0f39a420545b:/home/indy#

Complete work on the removal of indy-node's indy-crypto dependency

The work to transition from indy-crypto to ursa is complete in the code (as far as we know), but the dependency on indy-crypto remains. A PR #1607 has removed it from the code, but the work to remove it from the build pipeline is not complete.

This task is to have that PR merged and then fix as needed the build pipeline to work and complete the tests. In doing that, the indy-plenum dependency will be updated to the version that uses ursa.

Ubuntu 20.04: create CD for Ubuntu 20.04

Create CD for Ubuntu 20.04 for a feature brunch ubuntu-20.04-upgrade

Acceptance criteria:

• All existing tests are launched via GitHub Actions, and they pass (if pass locally)
• All jobs(steps) pass
• Reports from launches are available

Indy Node deb package are published to a public repository (focal like a channel name for Ubuntu 20.04 Focal Fossa)

Error while creating pool using indy-cli

I have ran the indy_pool network and it is up using the command : docker run -itd -p 9701-9708:9701-9708 indy_pool

Now, I have installed indy-cli in my system

Now when I am trying to create a pool using the command:
pool create local_pool gen_text_file=/<PATH_TO_INDY_SDK>/indy-sdk/cli/docker_pool_transactions_genesis

It is giving me the error :
Unknown "gen_text_file" parameter present

Can someone explain why is this issue coming and what can be done to resolve this?

Ensure technical policies already enable the extraction of additional metrics for `validator-info`

• Add more detailed hardware metrics to the collection and output of validator-info
• Add additional resource and system metrics to the collection and output of validator-info

Implement a health, diversity, and compliance analysis plug-in

As a companion to #1670, integrate the analysis portion(s) of the Technical Verification Script into a new plugin to facilitate continuous monitoring.

The implementation should build on the analysis plugin once merged; hyperledger/indy-node-monitor#26

This is to address the discussions regarding incorporating the collection of the additional data analyzed by the Technical Verification Script, in order to facilitate continuous compliance monitoring; hyperledger/indy-node-monitor#24 (comment)

Building script issue.

Dear all, I got an issue while using the script build-indy-node.sh to build indy-node. It stopped at processing generating manifest in prepare-package. I attached the screenshot.

My environment is under pure Ubuntu 16.04.

Cannot build dockers

Running pool_start kicks off a docker build for the nodes that fails.

Running on Ubuntu 20.04

The following packages have unmet dependencies:
indy-node : Depends: indy-plenum (= 1.13.0~dev1024) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

Formatting with Black?

In other python projects in the Hyperledger Ecosystem, black is used to help keep coding style consistent. I think Node would benefit from a similar treatment. This would however mean that we would need to essentially reformat every file in the project which is less than ideal. I think the improved formatting would be worth it but I'm interested to hear thoughts from others.

Multiple Dependency Installs in GHA

Noticed, that when slicing the modules for the tests each slice is installing the pip dependencies.
Would it be beneficial to include the dependency install into the Dockerfile?
For example add the following pip command from .github/workflows/build.yaml:232 to the .github/workflows/build/Dockerfile?

        run: |
          # Explicitly use the existing pip cache location in the node-build image.
          pip --cache-dir /root/.cache/pip install .[tests]

Update default auth_rules to reflect deprecation of ATTRIB

As a message to those creating new networks that the ATTRIB functionality is being removed from indy-node I suggest that we change the default auth_rules table to mark ATTRIB transactions as FORBIDDEN.

Currently the default auth_rules give some permissions to those that would like to perform ATTRIB transactions. Changing it to FORBIDDEN would make it so that, by default, a user would no longer be able to perform an ATTRIB transaction, thus demonstrating that ATTRIB has been deprecated. The setting of FORBIDDEN can easily be reverted to the previous default value, if desired, so that a new network can still allow ATTRIB transactions as needed for backward compatibility. (I tested changing a value that defaults to FORBIDDEN to a different value and then back and all worked well).

Again, this suggestion is only to change the default value of ATTRIB transactions in the auth_rule table. This will not affect upgrading existing networks, nor will it change anything except the default value (which is still alterable, if desired).

Determine ACLs on who is allowed to request, receive and store additional metrics for `validator-info`

• Add more detailed hardware metrics to the collection and output of validator-info
• Add additional resource and system metrics to the collection and output of validator-info

Error in "Setting up a new network" document

There have been at least 2 people that have recently asked me questions related to this issue, so I looked into it and discovered the following:

In step IV of https://github.com/hyperledger/indy-node/blob/master/docs/source/NewNetwork/NewNetwork.md there is a step which says that the user must create a directory (using new network name) in order to copy in the genesis files. But the directory it mentions must already have been created during step III.2 before the Steward ran init_indy_node, or their keys will not have been created in the right place. @pSchlarb I will have a deeper look at this tomorrow and see if I can get a PR put in that will help alleviate the confusion.
This confusion surrounding this issue has also expanded to another document. The Hands On Walkthrough also needs repaired to reflect the above. (Steps 8 and 9 must be moved to Before step 5) It will not work as it currently written.

Update remove_ledger.py to protect against removing a ledger that has been used

Update remove_ledger.py to protect against removing a ledger that has been used.

If a ledger has been used (it contains transactions) it's history is critical to the historical integrity if the network, even if the ledger has been frozen. Without the historical data for a given ledger the ability to audit the history of the ledger to prove that there was no tampering is not possible. Therefore it is important to protect such ledgers from being deleted.

Refer to the Drawbacks section of the 0162-frozen-ledgers HIPE in hyperledger/indy-hipe#162.

How does a chain node get the raw data from another node when ATTRIB transaction ?

in ATTRIB transaction:

raw (sha256 hash string; mutually exclusive with hash and enc):
Hash of the raw attribute data. Raw data is represented as JSON, where the key is the attribute name and the value is the attribute value. The ledger only stores a hash of the raw data; the real (unhashed) raw data is stored in a separate attribute store.

so, how does a chain node get the raw data from another node when ATTRIB transaction ? if needn't, then the separate attribute store in different node will differ ?

thanks!

Dev Setup Error: Python could not import the module virtualenvwrapper.hook_loader

I'm following this quick setup tutorial for Ubuntu 16.04. On the step source ~/.bashrc, I hit this error:

cj@cj-VirtualBox:~/projects/indy-node/dev-setup/ubuntu$ source ~/.bashrc
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/stevedore/_cache.py", line 28, in <module>
    import importlib.metadata as importlib_metadata
ImportError: No module named 'importlib.metadata'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.5/dist-packages/virtualenvwrapper/hook_loader.py", line 16, in <module>
    from stevedore import ExtensionManager
  File "/usr/local/lib/python3.5/dist-packages/stevedore/__init__.py", line 11, in <module>
    from .extension import ExtensionManager
  File "/usr/local/lib/python3.5/dist-packages/stevedore/extension.py", line 19, in <module>
    from . import _cache
  File "/usr/local/lib/python3.5/dist-packages/stevedore/_cache.py", line 31, in <module>
    import importlib_metadata
  File "/usr/local/lib/python3.5/dist-packages/importlib_metadata/__init__.py", line 88
    dist: Optional['Distribution'] = None
        ^
SyntaxError: invalid syntax
virtualenvwrapper.sh: There was a problem running the initialization hooks.

If Python could not import the module virtualenvwrapper.hook_loader,
check that virtualenvwrapper has been installed for
VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3 and that PATH is
set properly.

Has anyone seen this before?

Add a getting started guide for the new virtual development environments.

The following PRs add virtual development environments:

• #1718
• #1722

Please add a getting started guide to the top of the Dev Setup document about using the new virtual development environments as the preferred method to get started with the code?

Review current stable-master comparison to see if there is anything other than Rich Schema changes

We think that the difference between the current stable -- master branches is only about the new Rich Schema transactions that were added to the ledger. This task is to review the difference and identify anything that is (a) not Rich Schema and (b) code that could change the behaviour of indy-node. That should leave out the RS changes, documentation updates and test changes. Each identified change should be listed on this ticket so that we can discuss it at the next Indy Contributors call on Sept. 29, 2020.

Add a detailed tutorial on how to create a New Indy Network

Resources:

• Create New Indy Network
• Hyperledger Indy Custom Network with Indy Node & Plenum(Protocol & Ledger)

The resulting document should be placed in indy-node/docs.

Feedback:

• One user has indicated there where some steps in Hyperledger Indy Custom Network with Indy Node & Plenum(Protocol & Ledger) they were unable to follow. Although I don't have the specific details as to which steps.

suggestion in naming

Perhaps the second line makes it easier to define the concept

1- There is an indy ledger that contains several ledger included audit ledger , config ledger and ...
2- There is an indy ledger that contains several sub ledger included audit sub ledger , config sub ledger and ...

thank you

Known issue with running tests in the virtual development environments

Due to a bug in the microsoft/vscode-python extension, some until tests appear to fail with a "Test result not found" error. The tests do not actually fail.

Details can be found here;

• #1718 (comment)

This issue affects both:

• #1718
• #1722

Unable to start indy test network

Followed the instructions provided in the README.md file to start a test network,

Following command failed to execute ./pool_start.sh 4 10.0.0.2,10.0.0.3,10.0.0.4,10.0.0.5 10 9701, below is the console output,

Creating pool
4 10.0.0.2,10.0.0.3,10.0.0.4,10.0.0.5 10
Creating pool of 4 nodes with ips 10.0.0.2,10.0.0.3,10.0.0.4,10.0.0.5
Creating a new node Node1 0.0.0.0 9701 0.0.0.0 9702
Setting up docker with systemd
The systemd cgroup hierarchy is already mounted at /sys/fs/cgroup/systemd.
Your Docker host is now configured for running systemd containers!
Building indybase
Sending build context to Docker daemon   34.3kB
Step 1/1 : FROM solita/ubuntu-systemd:16.04
 ---> ebf71084f672
Successfully built ebf71084f672
Successfully tagged indybase:latest
Building indycore for user 1000
Sending build context to Docker daemon   34.3kB
Step 1/13 : FROM indybase
 ---> ebf71084f672
Step 2/13 : ARG uid=1000
 ---> Using cache
 ---> 5bb1984e1a34
Step 3/13 : RUN apt-get update -y && apt-get install -y         git     wget    python3.5       python3-pip     python-setuptools       python3-nacl    apt-transport-https     ca-certificates
 ---> Using cache
 ---> 5fd2a788e057
Step 4/13 : RUN pip3 install -U         'pip<10.0.0'    setuptools
 ---> Using cache
 ---> 9b350415b376
Step 5/13 : RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE7709D068DB5E88
 ---> Using cache
 ---> cf28ce36c515
Step 6/13 : RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BD33704C
 ---> Using cache
 ---> 245d6fad4bed
Step 7/13 : RUN echo "deb https://repo.sovrin.org/deb xenial master" >> /etc/apt/sources.list
 ---> Using cache
 ---> dd3840bbfdb3
Step 8/13 : RUN echo "deb https://repo.sovrin.org/sdk/deb xenial master" >> /etc/apt/sources.list
 ---> Using cache
 ---> 4e09de1da87e
Step 9/13 : RUN useradd -ms /bin/bash -l -u $uid indy
 ---> Using cache
 ---> ec5d8907883f
Step 10/13 : RUN apt-get update -y && apt-get install -y indy-node libindy
 ---> Running in 376e9ed88430
Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [109 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]
Get:5 http://archive.ubuntu.com/ubuntu xenial-updates/universe Sources [518 kB]
Get:6 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [2331 kB]
Get:7 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [1494 kB]
Get:8 https://repo.sovrin.org/deb xenial InRelease [28.4 kB]
Get:9 https://repo.sovrin.org/sdk/deb xenial InRelease [28.5 kB]
Get:10 https://repo.sovrin.org/deb xenial/master amd64 Packages [344 kB]
Get:11 https://repo.sovrin.org/sdk/deb xenial/master amd64 Packages [524 kB]
Fetched 5593 kB in 9s (614 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 indy-node : Depends: indy-plenum (= 1.13.0~dev1024) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c apt-get update -y && apt-get install -y indy-node libindy' returned a non-zero code: 100

Broken links

On this page:
https://hyperledger-indy.readthedocs.io/projects/node/en/latest/requests.html
there are several occurrences of this link:
https://github.com/hyperledger/indy-node/blob/master/docs/source/auth_rules
...and it does not exist.

ERROR:ursa.lib:_load_cdll: Can't load libursa: libursa.so: cannot open shared object file: No such file or directory

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"

$ docker version
Client: Docker Engine - Community
 Version:           20.10.2
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        2291f61
 Built:             Mon Dec 28 16:17:32 2020
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.2
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8891c58
  Built:            Mon Dec 28 16:15:09 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Error message:

Step 18/20 : RUN if [ ! -z "$ips" ] && [ ! -z "$nodenum" ] && [ ! -z "$nodecnt" ]; then generate_indy_pool_transactions --nodes $nodecnt --clients $clicnt --nodeNum $nodenum --ips "$ips"; fi
 ---> Running in bcea4ec6bdb2
ERROR:ursa.lib:_load_cdll: Can't load libursa: libursa.so: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/usr/local/bin/generate_indy_pool_transactions", line 19, in <module>
    getTxnOrderedFields(), ConfigHelper, NodeConfigHelper)
  File "/usr/local/lib/python3.5/dist-packages/plenum/common/test_network_setup.py", line 246, in bootstrapTestNodes
    config_helper_class, node_config_helper_class)
  File "/usr/local/lib/python3.5/dist-packages/plenum/common/test_network_setup.py", line 129, in bootstrapTestNodesCore
    _, verkey, blskey, key_proof = initNodeKeysForBothStacks(nd.name, keys_dir, nd.sigseed, override=True)
  File "/usr/local/lib/python3.5/dist-packages/plenum/common/keygen_utils.py", line 51, in initNodeKeysForBothStacks
    keys = initLocalKeys(node_stack_name, keys_dir, sigseed, use_bls=use_bls, override=override)
  File "/usr/local/lib/python3.5/dist-packages/plenum/common/keygen_utils.py", line 16, in initLocalKeys
    blspk, key_proof = init_bls_keys(keys_dir, name, sigseed) if use_bls \
  File "/usr/local/lib/python3.5/dist-packages/plenum/common/keygen_utils.py", line 30, in init_bls_keys
    stored_pk, key_proof = bls_factory.generate_and_store_bls_keys(seed)
  File "/usr/local/lib/python3.5/dist-packages/crypto/bls/bls_factory.py", line 23, in generate_and_store_bls_keys
    sk, pk, key_proof = self.generate_bls_keys(seed)
  File "/usr/local/lib/python3.5/dist-packages/plenum/bls/bls_crypto_factory.py", line 18, in generate_bls_keys
    sk, pk, key_proof = self._generate_raw_bls_keys(seed)
  File "/usr/local/lib/python3.5/dist-packages/crypto/bls/bls_factory.py", line 41, in _generate_raw_bls_keys
    seed)
  File "/usr/local/lib/python3.5/dist-packages/crypto/bls/indy_crypto/bls_crypto_indy_crypto.py", line 122, in generate_keys
    gen = IndyCryptoBlsUtils.bls_from_str(params.g, Generator)
  File "/usr/local/lib/python3.5/dist-packages/crypto/bls/indy_crypto/bls_crypto_indy_crypto.py", line 42, in bls_from_str
    return cls.from_bytes(bts)
  File "/usr/local/lib/python3.5/dist-packages/ursa/bls.py", line 32, in from_bytes
    do_call(cls.from_bytes_handler, xbytes, len(xbytes), byref(c_instance))
  File "/usr/local/lib/python3.5/dist-packages/ursa/lib.py", line 17, in do_call
    err = getattr(_cdll(), name)(*args)
  File "/usr/local/lib/python3.5/dist-packages/ursa/lib.py", line 27, in _cdll
    _cdll.cdll = _load_cdll()
  File "/usr/local/lib/python3.5/dist-packages/ursa/lib.py", line 62, in _load_cdll
    raise e
  File "/usr/local/lib/python3.5/dist-packages/ursa/lib.py", line 54, in _load_cdll
    res = CDLL(lib_name)
  File "/usr/lib/python3.5/ctypes/__init__.py", line 347, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libursa.so: cannot open shared object file: No such file or directory

This was done using ./pool_start.sh located in  /indy-node/environment/docker/pool.

From my understanding, this has to do with the build image and not the host itself but I could be wrong. Any ideas?

Cheers

Add more detailed hardware metrics to the collection and output of `validator-info`

The output of validator-info currently returns limited hardware metrics for a given node. To better facilitate node monitoring as well as continuous node compliance monitoring, validator-info should report a more complete set of hardware metrics including:

• Hard Disk Stats
  • Total size (amount), available (amount), and used (amount and percent) for all volumes.
  • Total size (amount), available (amount), and used (amount and percent) for indy-node specifically.
• Memory and CPU
  • The CPU and memory use of the node processes is reported, however,
  • The output should be updated to provide more details regarding overall system memory and CPU usage and load.
• Network Interface IP Binding
  • Information regarding NIC and IP address bindings.
  • To be used to ensure the node has been configured with separate IP addresses, bound to separate NICs, and assigned to different subnets.

All information should be included in the output of validator-info on the node itself, and authenticated calls to the get-validator-info transaction. It appears the results for each are different, with the results of validator-info containing more information.

This is to address HDD, Memory, and CPU resource discussions here; hyperledger/indy-node-monitor#24 (comment)

Requirements:

• #1672
• #1673

Pool upgrades retry indefinitely when unsuccessful

When a pool upgrade fails on a particular node, the upgrade process is rescheduled and restarted on the node following the timeout period specified in the pool-upgrade command used to schedule the initial upgrade. If the upgrade process is continuously unsuccessful the upgrade is rescheduled on the node and retried indefinitely until an upgrade cancel command is issued to stop the process.

This was observed during a network upgrade over the last two days. The upgrade on one node failed on Friday. That upgrade was canceled, the node was restarted and a targeted upgrade was scheduled on that node for 2020-08-28T21:00:00-00:00 with a longer timeout. That upgrade also failed, so we'll be investigating that issue with the Steward further. However, a cancel command was not issued right away and the upgrade process continued to retry until the cancel command was finally issued this morning.

Here is the upgrade log from the node:

"upgrade_log": [
  "2020-08-30 12:00:10.052972\tscheduled\t2020-08-29 14:00:00+00:00\t1.1.89\t1598644251746331800\tsovrin\n",
  "2020-08-30 12:00:10.053393\tstarted\t2020-08-29 14:00:00+00:00\t1.1.89\t1598644251746331800\tsovrin\n",
  "2020-08-30 12:30:10.278094\tscheduled\t2020-08-29 14:00:00+00:00\t1.1.89\t1598644251746331800\tsovrin\n",
  "2020-08-30 12:30:10.278418\tstarted\t2020-08-29 14:00:00+00:00\t1.1.89\t1598644251746331800\tsovrin\n",
  "2020-08-30 13:00:10.491240\tscheduled\t2020-08-29 14:00:00+00:00\t1.1.89\t1598644251746331800\tsovrin\n",
  "2020-08-30 13:00:10.491587\tstarted\t2020-08-29 14:00:00+00:00\t1.1.89\t1598644251746331800\tsovrin\n",
  "2020-08-30 13:30:10.743418\tscheduled\t2020-08-29 14:00:00+00:00\t1.1.89\t1598644251746331800\tsovrin\n",
  "2020-08-30 13:30:10.743969\tstarted\t2020-08-29 14:00:00+00:00\t1.1.89\t1598644251746331800\tsovrin\n",
  "2020-08-30 14:00:10.965996\tscheduled\t2020-08-29 14:00:00+00:00\t1.1.89\t1598644251746331800\tsovrin\n",
  "2020-08-30 14:00:10.966318\tstarted\t2020-08-29 14:00:00+00:00\t1.1.89\t1598644251746331800\tsovrin\n"

The cancel command was issued before the upgrade started at 14:00:10+00:00 would have timed out. The log was taken at 14:45:00+00:00. The upgrade did not get rescheduled, however it does not show the upgrade was canceled, or failed. On Friday after the upgrade was canceled and the node was restarted the upgrade log indicated the upgrade had failed (but not before being restarted).

The upgrade process should automatically stop after a number of failed retry attempts (say 3 by default).

The node continued to operate and perform normally during this period; it continued to participate and stayed in consensus.

Add feature flag control over the Rich Schema transactions to indy-node

A number of transactions to support Rich Schema (RS) have been added to indy-node and merged into the stable branch. We'd like to merge them into the master branch, but we don't have a good way to test them, as indy-sdk support for RS has not been built out. The changes of interest can be seen in this comparison of the current (as I type this) commits for stable and master.

The approach we have decided upon in order to move forward with the merge is to feature flag the new transactions so that they are treated as "unknown" by indy-node until we are ready to activate them.

Recover when Audit Ledger is out of sync with other ledgers

config / domain / plugins ledgers -> audit (holds latest hash of above three)
Part 1 of config txn types is permissions (did the writer have the correct permission ledger?)

• Issue: if killing indy-node after writing latest hash on audit ledger, but not the first three, so node says we can't run
• Desired behavior: discard newer transactions and then play catchup

Comment origination

Indy-Test-Automation workflow inputs are static

The input parameters of the workflow dispatch in build.yaml that triggers the indy-test-automation workflow is currently hardcoded.
To improve the releasing process of indy-node, all parameters should be set dynamically.

The code is at: https://github.com/hyperledger/indy-node/blob/master/.github/workflows/build.yaml#L429

Setup Hyperledger Artifactory repositories for publishing release artifacts

indy-node and indy-plenum artifacts have up until now been published to https://repo.sovrin.org/, using the Sovrin hosted Jenkins CI/CD environment. As part of the migration to the GitHub actions based CI/CD process we are also migrating over to the Hyperledger Artifactory hosted environment for publishing the artifacts generated by the new pipelines.

Only Hyperledger artifacts will be hosted in the new repository. Related Sovrin packages will continue to live in repo.sovrin.org

References:

• Debian Repositories
• Signing Keys Management
• Setup JFrog CLI
• JFrog Access Tokens
• Repository Layouts
• https://hyperledger.jfrog.io/ui/repos/tree/General/artifactory-build-info

Example for current repo structure:

• https://repo.sovrin.org/deb/pool/xenial/rc/i/indy-node/
• https://repo.sovrin.org/deb/pool/xenial/stable/i/indy-node/
• https://repo.sovrin.org/lib/apt/xenial/rc/
• https://repo.sovrin.org/lib/apt/xenial/stable/
• https://repo.sovrin.org/lib/apt/xenial/master/
• http://ftp.debian.org/debian/

Additional Resources:

• Example of publishing a Docker image to Hyperledger Artifactory: https://github.com/hyperledger/besu/blob/a58c472480a46a4020383e264fed61e20a05bee4/.circleci/config.yml#L224-L248
• webfactory/ssh-agent
• Import GPG
• Hosting your own PPA repository on GitHub

Debian Documntion:

• SecureApt
• 7.5. Package signing in Debian

New Hyperledger Indy Repository:

• Hyperledger Indy Repository

Process Summary for setting up the deployment configurations for use with Setup JFrog CLI:

• Created a new indy-cd user with only profile edit permissions.
• Created an indy-publishers group, indy-cd user is a member.
• Created an indy-deploy permission that provides the indy-publishers group with Deploy permissions to the indy Artifactory Repository.
• Generated a private api-key for the indy-cd user.
• Created a new config using the jfrog-cli containing the indy-cd user's token.
• Exported the resulting configuration token to be mounted in GitHub as a secret (INDY_ARTIFACTORY_REPO_CONFIG).

Ubuntu 20.04: create CI for Ubuntu 20.04

Create CI for Ubuntu 20.04 for a feature brunch ubuntu-20.04-upgrade

Acceptance criteria:

• All existing tests are launched via GitHub Actions, and they pass (if pass locally)
• All jobs(steps) pass
• Reports from launches are available.

Documentation to run a production indy network with governance in place.

There should be proper documentation provided to run production capable indy network and guidance on governance capabilities. It should be structured in such a way that any entity should be equipped to run their own indy network. Current documentation is very much scattered, that an entity would have to rely on third parties to understand the steps. Proper documentation is very much required if we have to spread the SSI movement across boundaries.

Nodes get lost during view change (infinite loop)

I am unable to determine the cause yet, but I think I have seen the same thing happen for 2 different networks in the last week. Here is what I think happens (more details might be coming later, when I know more)

Symptom: All nodes go "Out of consensus"

1. In the logs I see a View change is requested near the beginning of the problems.
2. Logs show view change requests coming at a rapid pace and the view number moving up rapidly with no actual view change happening (or sometimes happening, but not as fast as the numbers are going up)
3. Logs fill up rapidly with view change requests
4. Looks like an infinite loop (Logs filled rapidly with view change requests for at least an hour and a half)
5. A restart of the network fixed one of the networks, but not the other.
   Logs available on request (48Mb)

Add additional resource and system metrics to the collection and output of `validator-info`

The Technical Verification Script collects and analyses node resource and and system information in order to determine and report on a given node's level of compliance. To facilitate continuous health, diversity, and compliance monitoring the collection and analysis of the data should be separated. The metrics should be collected and reported by the node itself through validator-info, and the analysis should be performed within hyperledger/indy-node-monitor as discussed here; hyperledger/indy-node-monitor#24 (comment). This would allow an organization to develop their own set of compliance metrics and write an indy-node-monitor plugin to analyze and report on a given node or network's level of compliance based on those rules.

As such the scope of this ticket would be to incorporate the metrics collected by the Technical Verification Script into validator-info so they become more readily available for continuous monitoring solutions.

While designing/implementing give some thought to any additional metrics that would support continuous security monitoring as discussed here; hyperledger/indy-node-monitor#24 (comment)

All information should be included in the output of validator-info on the node itself, and authenticated calls to the get-validator-info transaction.

This is to address the discussions regarding incorporating the collection of the additional data analyzed by the Technical Verification Script, in order to facilitate continuous compliance monitoring; hyperledger/indy-node-monitor#24 (comment)

Requirements:

• #1672
• #1673

Ubuntu 20.04: migrate the codebase from 16.04 to 20.04 Ubuntu version.

Migrate Indy Node codebase to Ubuntu20.04 and new Python.

Acceptance criteria:

• Bump Python version from 3.6 to 3.8
• Bump all dependencies where it's possible without risky code changes
• All existing tests should pass

Missing PYPI_API_TOKEN for indy-node repository

We're missing the PYPI_API_TOKEN in the indy-node repository, which is causing the python publishing to fail here; https://github.com/hyperledger/indy-node/runs/3579864618?check_suite_focus=true

The token was setup on indy-plenum:

Fix broken Indy Node CD pipeline

Merging changes with moving from Indy Crypto to Ursa (see this PR: #1607) has broken Indy Node CD pipeline. This needs to be fixed, because it would block any further releases (and frankly doing general development work is also quite hard in this situation)

Acceptance criteria

• Indy Node should have green CD pipeline in master

Update base docker images to 20.04

Currently, indy node bases its docker images off ubuntu 16.04. this version is no longer supported.

Please update them to use 20.04

Updated Dev Setup

Is there a more up-to-date setup guide for developers?
The one here seems like it's very outdate. (e.g., tells you to use indy-crypto but it looks like Ursa is used instead)

Node loses consensus, but does not regain it

Note: Summary might need changed to add "for a very long time". I just saw a case that might be similar where it took 3 hours to regain consensus. (Will include more logs from Uphold if/when they arrive)

Environment:
Indicio Indy Networks running indy-node version 1.12.4

Steps to replicate: (the following general behavior was seen in log files for 2 nodes that lost consensus on 2 different networks, a week apart.)

1. Node loses connectivity to 3f+1 nodes (4 or 5 nodes, on Indicio networks) for 30 seconds or more.
2. Nodes reconnect within 5 minutes.
3. Node reports "Out of Consensus" and sends request for VIEW change every 5 minutes.
4. Node does not return to consensus within 45 minutes (in anonyome node example), so node was restarted, at which point it did return to consensus and regained normal operation.

Expected Behavior: Node returns to consensus quickly after conditions which caused it to go out are restored.

Actual Behavior: Node did not return to consensus for a long time (at least 45 minutes in one case).

Notes: In Anonyome log, start at about 2021-09-17 15:32:53 to see above behaviour. OOC messages (reporting) started at about 2021-09-17 15:41:00 plus or minus 1 minute.
In Opsnode-dn log, start at about 2021-09-10 01:59:15, and OOC occurred at about 2021-09-10 02:23:00.
Uphold log does not display a similar pattern(only disconnected from one node, the primary, at 15:24:15), but it went out of consensus at about 2021-9-28 15:45:00 and returned to consensus about 3 hours later at about 18:43:00. This one is possibly unrelated, but also took a very long time to return to consensus. (Regained connection to primary at 2021-09-28 15:24:54)

What is the resource usage of calculating the interior nodes of the revocation 2 merkle tree?

See #1595 for details of the plan for revocation 2.

We need a technical spike done to verify our assumptions about the proposed approach to storing the leaves of the merkle tree on the ledger. Notable, the leaves, a list of 64-bit unsigned ints with each bit representing 1 credential's revocation status and the ability for a prover to generate the interior nodes of the merkle tree. The request is to do something like:

• generate an arbitrary merkle tree based on a parameter of the number of layers (1-7)
  • the code from #1596 could/should be used for creating the test merkle tree
• run a test to determine the resulting size of the data structure holding the full tree (leaves and interior nodes), and the time to generate the tree.
• from each test, output the parameters and the resource consumption of the process
• consider other strategies for generating ONLY the interior nodes necessary to generate the proof - only the nodes needed to provide a path from the credential ID of interest to the root.

The code is purely a tech spike and need not be production worthy. It should be pushed to the repo for review by the maintainers and for possible use in the future - for example, playing with different ways of storing the set of leaves, and testing different node generation mechanisms.

Using docker swarm

Is it possible to to run indy nods in docker swarm for multi host pool ?

Test Pool Documentation Outdated

The documentation of the Testpool and the IndyStory Walkthrough seems to be outdated.
Following environment/docker/pool/StartIndyAgents.md there is no indy command in the container.

What is the compression profile of the proposed handling of revocation 2 merkle tree leaves

See #1595 for details of the plan for revocation 2.

We need a technical spike done to verify our assumptions about the proposed approach to storing the leaves of the merkle tree on the ledger. Notable, the leaves, a list of 64-bit unsigned ints with each bit representing 1 credential's revocation status. The request is to do something like:

• define a way to store the leaves of the merkle tree based on a parameter of the number of layers (1-7).
• For a test, define a percent of the number of credentials that will be revoked, ranging from say 10 to 50%.
• Randomly generate the revocation status of all the credentials (the bit) based on the percent.
• gzip the result.
• output the parameters of the test and the size of the resulting compressed chunk of data.

The code is purely a tech spike and need not be production worthy. It should probably be pushed to the repo for review by the maintainers and for possible use in the future - for example, playing with different ways of storing the (uncompressed) set of leaves, and testing different compression algorithms.

Perfomance test

Hi

Error whem I m trying to execute perf_process.py :

Traceback (most recent call last):
File "/usr/local/bin/perf_processes.py", line 4, in
import('pkg_resources').run_script('indy-perf-load==1.2.0', 'perf_processes.py')
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 658, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 1445, in run_script
exec(script_code, namespace, namespace)
File "/usr/local/lib/python3.6/dist-packages/indy_perf_load-1.2.0-py3.6.egg/EGG-INFO/scripts/perf_processes.py", line 465, in
File "/usr/local/lib/python3.6/dist-packages/indy_perf_load-1.2.0-py3.6.egg/perf_load/perf_utils.py", line 143, in call
File "/usr/local/lib/python3.6/dist-packages/indy_perf_load-1.2.0-py3.6.egg/perf_load/perf_utils.py", line 160, in init
File "/usr/local/lib/python3.6/dist-packages/indy_perf_load-1.2.0-py3.6.egg/perf_load/perf_utils.py", line 160, in
File "/usr/lib/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Replace dependency on Indy-SDK with Indy-VDR

Indy-VDR is receiving more development than Indy-SDK, so we should replace calls to libindy with calls to indy-vdr.

Steps:

• Update unit tests.
• Update integration tests.
• Update package dependencies.

Notes:

• Related issue in indy-plenum

Define the transactions for the Revocation 2 RevReg and RevEntry transactions

See #1595 for details of the plan for revocation 2.

We need to understand the RevReg and RevEntry transactions for revocation 2. What needs to be in each transaction? Once determined, what verification should be defined for each?

This task requires knowledge of the Ursa revocation 2 implementation and the information needed by the prover and the verifier to carry out their roles in providing a proof of non-revocation. #1595 mentions a couple of things, but likely there are more items needed.

rename master branch

Summary

We should rename the branch master to main and use that going forward for our work.

From Problematic Terminology in Open-Source on master/slave terminology in software:

Use of this term is problematic. It references slavery to convey meaning about the relationship between two entities.

Removing this terminology from our workflow is a small gesture to include more people from marginalized groups in this project.

(I’m open to names other than main)

Technical Steps

• create main branch from master
• make main the default GitHub branch
• modify github/central to use main for release notes reloading
• redirect PRs to main in hyperledger/indy-node
• move branch protections from master to main
• modify docs to reference main instead of master
• delete master branch to avoid confusion?

Feedback?