polkascan / py-sr25519-bindings

Python bindings for the RUST implementation of the sr25519 cryptography library

Polkascan Open-Source

Polkascan Open-Source Application

Quick deployment (Use hosted Polkascan API endpoints)

Step 1: Clone repository:

git clone https://github.com/polkascan/polkascan-os.git

Step 2: Change directory:

cd polkascan-os

Step 3: Check available releases:

git tag

Step 4: Checkout latest releases:

git checkout v0.x.x

Step 5: Make sure to also clone submodules within the cloned directory:

git submodule update --init --recursive

Step 6: Then build the other docker containers

docker-compose -p kusama -f docker-compose.kusama-quick.yml up --build

Use public Substrate RPC endpoints

Step 1: Clone repository:

git clone https://github.com/polkascan/polkascan-os.git

Step 2: Change directory:

cd polkascan-os

Step 3: Check available releases:

git tag

Step 4: Checkout latest releases:

git checkout v0.x.x

Step 5: Make sure to also clone submodules within the cloned directory:

git submodule update --init --recursive

Step 6: During the first run let MySQL initialize (wait for about a minute)

docker-compose -p kusama -f docker-compose.kusama-public.yml up -d mysql

Step 7: Then build the other docker containers

docker-compose -p kusama -f docker-compose.kusama-public.yml up --build

Full deployment

The following steps will run a full Polkascan-stack that harvests blocks from a new local network.

Step 1: Clone repository:

git clone https://github.com/polkascan/polkascan-os.git

Step 2: Change directory:

cd polkascan-os

Step 3: Check available releases:

git tag

Step 4: Checkout latest releases:

git checkout v0.x.x

Step 5: Make sure to also clone submodules within the cloned directory:

git submodule update --init --recursive

Step 6: During the first run let MySQL initialize (wait for about a minute)

docker-compose -p kusama -f docker-compose.kusama-full.yml up -d mysql

Step 7: Then build the other docker containers

docker-compose -p kusama -f docker-compose.kusama-full.yml up --build

Links to applications

• Polkascan Explorer GUI: http://127.0.0.1:8080
• Monitor harvester progress: http://127.0.0.1:8080/kusama/harvester/admin
• Harvester Task Monitor: http://127.0.0.1:5555
• Polkadot JS Apps: http://127.0.0.1:8081

Other networks

• Polkadot: Use docker-compose.polkadot-quick.yml, docker-compose.polkadot-public.yml and docker-compose.polkadot-full.yml
• Substrate Node Template (https://github.com/substrate-developer-hub/substrate-node-template): Use docker-compose.substrate-node-template.yml

Add custom types for Substrate Node Template

• Modify harvester/app/type_registry/substrate-node-template.json to match the introduced types of the custom chain
• Truncate runtime and runtime_* tables on database
• Start harvester
• Check http://127.0.0.1:8080/node-template/runtime-type if all type are now correctly supported
• Monitor http://127.0.0.1:8080/node-template/harvester/admin if blocks are being processed and try to restart by pressing "Process blocks in harvester queue" if process is interupted.

Cleanup Docker

Use the following commands with caution to cleanup your Docker environment.

Prune images

docker system prune

Prune images (force)

docker system prune -a

Prune volumes

docker volume prune

API specification

The Polkascan API implements the https://jsonapi.org/ specification. An overview of available endpoints can be found here: https://github.com/polkascan/polkascan-pre-explorer-api/blob/master/app/main.py#L60

Troubleshooting

When certain block are not being processed or no blocks at all then most likely there is a missing or invalid type definition in the type registry.

Some steps to check:

• The Substrate node should be started in archive mode (--pruning archive)
• Be sure to regularly update, also the Git submodules: git submodule update --init --recursive for latest version of py-scale-codec
• Check which blocks are not being processed and try to restart manually at: http://127.0.0.1:8080/<NETWORK_NAME>/harvester/admin
• Check Celery queue for more details about failing blocks: http://127.0.0.1:5555/tasks?state=FAILURE
• Check and match types in ./harvester/app/type_registry/custom_types.json with added types in Substrate

You can also dive into Python to pinpoint which types are failing to decode:

import json
from scalecodec.type_registry import load_type_registry_file
from substrateinterface import SubstrateInterface

substrate = SubstrateInterface(
    url='ws://127.0.0.1:9944',
    type_registry_preset='substrate-node-template',
    type_registry=load_type_registry_file('harvester/app/type_registry/custom_types.json'),
)

block_hash = substrate.get_block_hash(block_id=3899710)

extrinsics = substrate.get_block_extrinsics(block_hash=block_hash)

print('Extrinsincs:', json.dumps([e.value for e in extrinsics], indent=4))

events = substrate.get_events(block_hash)

print("Events:", json.dumps([e.value for e in events], indent=4))

py-sr25519-bindings's Issues

Please consider bumping PyO3 version and re-releasing

Hello, we had an upstream report of a bug from a user who has installed on PyPy 3.10 and run into an incompatibility between PyPy 3.10 and the aging PyO3 version bundled in your 0.2.0 release.

PyO3/pyo3#3590
https://foss.heptapod.net/pypy/pypy/-/issues/4015

I think if you update to latest PyO3 (0.20.0 at time of writing) and re-release, things should work ok.

Thanks in advance, and also for being a user of PyO3 🙏

Include Cargo.lock in repo

That would make it easier for distributions to package py-sr25519-bindings.
See also rust-lang/cargo#315.

Installation failure

Package can't be installed using just pip install py-sr25519-bindings on Python 3.11 and Python 3.11.1.

Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
     
      Cargo, the Rust package manager, is not installed or is not on PATH.
      This package requires Rust and Cargo to compile extensions. Install it through
      the system's package manager or via https://rustup.rs/
      
      Checking for Rust toolchain....
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

Generate Wheels for Python 3.12

Delete incorrect release 0.1.6 from PyPI

There is a 0.1.6 release on PyPI which seems to be a mistake (released at the same time as 0.2.0 and there is no 0.1.6 release on Github). Can you please delete it on PyPI so downstream software stops breaking?

Generate wheel for Python 3.10

Hi! It'd be nice if we could get a wheel in pypi for python 3.10

Missing functions: MuSig, pubkey.decompress and compress

Hi,
Thanks for the schnorrkel python bindings.

It would be very helpful if you could provide multi-sig functionality plus compress/decompress for Pubkey.

Generate wheel for aarch64

Hello! We at https://github.com/rotki are preparing a version of our app for aarch64 and would be nice if this project could also provided wheel files for this architecture at pypi.

I saw that you already have a github action to build in different platforms so maybe could be useful to use this project https://github.com/uraimo/run-on-arch-action

Thank you

py-sr25519-bindings wheels for aarch64 and armv7

Hello,

Would you kindly add pre-built wheels for this library for aarch64 (RPi 4) and armv7 (RPi 3) to avoid installing Rust and building library from source (quite time-consuming on RPi's)?

Regards

Issue with verifying correct externally generated signatures

Valid signatures generated with e.g. PolkadotJS (apps) doesn't seem to verify using this RUST binding. Signatures generated with the bindings itself verifies correctly, and also verifies in PolkadotJS.

Original issue: polkascan/py-substrate-interface#185