lnp-bp / client_side_validation

Standard implementation of client-side-validation APIs

Foundation Libraries for Client-side-validation

This is an implementation defining standard of client-side-validation representing a set of its Foundation libraries. The standards covered by the libraries include LNPBP-4, LNPBP-7, LNPBP-8, LNPBP-9, and LNPBP-81 (see Components section below for more details).

Client-side-validation is a paradigm for distributed computing, based on top of proof-of-publication/commitment medium layer, which may be a bitcoin blockchain or other type of distributed consensus system.

The development of the library is supported by LNP/BP Standards Association and is performed on its GitHub page.

The original idea of client-side-validation was proposed by Peter Todd with its possible applications designed by Giacomo Zucco. It was shaped into the protocol design by Dr Maxim Orlovsky with a big input from the community.

Minimum supported rust version for the library (MSRV) is 1.66 and 2021 rust edition.

Documentation

Detailed developer & API documentation for all libraries can be accessed at:

• https://docs.rs/client_side_validation/
• https://docs.rs/single_use_seals/
• https://docs.rs/commit_verify/

To learn about the technologies enabled by the library please check slides from our tech presentations and LNP/BP tech talks videos.

Components

This library consists of the following main three components, which define independent parts constituting together client-side-validation API and its core functionality. These are:

• Commit-verify client-side-validation-specific APIs:
  • consensus commitments (part of LNPBP-8 standard)
  • multi-commitments (LNPBP-4 standard)
  • merklization for client-side-validation (LNPBP-81 standard)
• Single-use-seals API (LNPBP-8 standard)
• Client-side-validation API from the library root, linking those components together according to LNPBP-9 standard.

Usage

The repository contains rust libraries for client-side validation.

Use library in other projects

To use libraries, you just need latest version of libraries, published to crates.io into [dependencies] section of your project Cargo.toml. Here is the full list of available libraries from this repository:

client_side_validation = "1" # "Umbrella" library including all of the tree libraries below
commit_verify = "1" # Consensus and multi-message commitments
single_use_seals = "1" # Generic (non-bitcoin-specific) API

"Umbrella" client_side_validation library is configured with default set of features enabling all of its functionality (and including all of other libraries from this repository, listed above). If you need to restrict this set, either use specific libraries - or configure main library with a set of features in the following way:

[dependencies.client_side_validation]
version = "1"
default-features = false
features = [] # Your set of features goes here

The library has four feature flags, all of which are not used by default:

• rand, providing support for generating random 32-byte sequences of Slice32 type, used in many LNP/BP applications (for instance as hash-lock preimages or during LNPBP-4 multi-commitments);
• serde, providing support for data structure serialization with serde across all library;
• async, providing async traits for projects using async rust.

For specific features which may be enabled for the libraries, please check library-specific guidelines, located in README.md files in each of library subdirectories.

Libraries based on client-side-validation

Most of the developers will be probably interested in a more high-level libraries based on client-side-validation, applying it to a specific commitment mediums (bitcoin transaction graph from blockchain or state channels, or more exotic systems like confidential bitcoin transactions used by elements & liquid, or mimblewimble-based systems). Here is (potentially incomplete) list of such libraries:

• Bitcoin: BP Core Lib, which contains bitcoin UTXO single-use-seal implementations for pay-to-contract and sign-to-contract types of seals, as well as a library for deterministic bitcoin commitments. This library is maintained by LNP/BP Association.
• Pandora sealchain, an experimental blockchain of Pandora Network, playing with client-side-validation using modified bitcoin consensus rules, extended with eltoo- and covenants-related functionality and with removed native blockchain-level coin.

Contributing

Contribution guidelines can be found in CONTRIBUTING

Licensing

The libraries are distributed on the terms of Apache 2.0 opensource license. See LICENCE file for the license details.

client_side_validation's Issues

Provide size-restricted encoders for writing the data

StrictEncoding issue with enum tuple struct field initialization

When fixing some clippy lints on the descriptor wallet, I ran into an issue here:

Docs on the linter rule are here:
https://rust-lang.github.io/rust-clippy/master/index.html#init_numbered_fields

Normally I wouldn't file a feature request just to satisfy a linter, but this might be indicative of some sort of oversight. Feel free to close if we're comfortable with using #![allow(clippy::init_numbered_fields)] in all projects using strict encoding.

Use `e: &mut E` instead of `mut e: E` encoder/decoder arguments

This reduces boilerplate in method implementation and avoids rust compiler crashes due to recursions in generic resolution

Finalize trust resolvers API

CommitmentId produces different results in Wasm32 target

Hi @dr-orlovsky,

I noticed that the CommitmentId::commitment_id function returns different values ​​according to the target compilation.

I discovered this while I was testing rgb-schemata, I even created a test that can help you with the bug investigation:

#[cfg(test)]
mod test {
    use super::*;

    #[test]
    fn show_schema_id() {
        let expected = "5YWfKW3CqANHsKqpxy3HaCpt5bgvsMXHUuXiHpoynEYG";
        let schema = nia_schema();
        let id = schema.schema_id().to_string();
        assert_eq!(expected, id); // OK!
    }
}

#[cfg(target_arch = "wasm32")]
mod test_wasm32 {
    use super::*;
    use wasm_bindgen_test::*;
    wasm_bindgen_test_configure!(run_in_browser);
    
    #[wasm_bindgen_test]
    fn show_schema_id() {
        let expected = "5YWfKW3CqANHsKqpxy3HaCpt5bgvsMXHUuXiHpoynEYG";
        let schema = nia_schema();
        let id = schema.schema_id().to_string();
        assert_eq!(expected, id); // Fail!
       /*
          panicked at 'assertion failed: `(left == right)`
          left: `"5YWfKW3CqANHsKqpxy3HaCpt5bgvsMXHUuXiHpoynEYG"`,
          right: `"12vPR9qthiLpPgoytGBbeerVYFKvdCXEUmbg891HVyHV"`', src/main.rs:143:9
      */

    }
}

Hope this helps.

Refactor strict_encoding crate

The original intent of strict_encoding was to provide encoding standard for the client-side-validation. That is why the crate is a part of this repository and used everywhere inside RGB and AluVM.

However, it appeared that the crate has become much more widely used, including commercial projects much outside of the scope of the client-side-validation. That happened due to the fact that in rust there are no good binary compact encodings guaranteeing determinism in data size or ordering.

Widespread use of strict_encoding creates pressure for adding more dependencies and types to it, since rust foreign type policies prohibits derivation of strict encoding traits on other dependencies downstream. While the new dependencies are optional, it still blows up codebase and requires permanent releases of new version.

On the other hand, client-side-validation requirements to the encoding are quite specific; for instance, we use bitcoin consensus encoding in many places, and such encoding is not consistent nor provides the same approach as encoding for other types.

So basically, it should not be advised to use strict encoding for anything else than client-side-validation and add more types here. Nevertheless, it is really hard to prevent that, especially taking into account existing adoption of the crate.

The solution I propose is the following:

• Fork the strict encoding crate into two different crates with different purposes: strict_encoding and confined_encoding;
• Move strict encoding outside of this repository, to https://github.com/Internet2-wg/ (where it will be used for instance by AluVM);
• Migrate all client-side-validation, BP Core and RGB Core libraries to use exclusively confined_encoding;
• Let any other project use strict_encoding (including descriptor wallet etc);
• Remove use of any custom encodings (like bitcoin consensus etc) from the strict_encoding;
• Remove TLV support from the confined_encoding;
• Move encoding_derive_helpers to https://github.com/rust-amplify/

This will restrict use of new crate confined_encoding to client-side-validation consensus only. Any non-consensus code (including descriptor wallet etc) MUST NOT use confined_encoding. The confined_encoding must be ossified and new types must not be added to it.

Increase multi-protocol commitment tree size limit

Currently, MPC trees (LNPBP-4) are limited to a maximum depth of 2^4 = 16, i.e. they may contain up to 2^16=65536 elements. The original design assumed that's enough to host all assets which may be allocated to a single UTXO.

However, tests do show that even 127 different assets may not UNIQUELY fit into a tree with width=655536 (the position of the asset is 256-bit asset id modulo size of the tree) and practically we can assume just around 64 assets to be assignable to the same UTXO.

Here I propose extending MPC tree depth to 32 and tree width to 2^32 such that the tree may host up to 16000 separate assets (no of assets < 2^(width / 2 - 2))

Sign-to-contract support in DBCs and signle-use-seals

Prerequisites:

• S2C standard LNP-BP/LNPBPs#118
• S2C commitments in Schnorr signatures inside secp256k1 library.
  Implementation WIP: https://github.com/LNP-BP/secp256k1/tree/schnorr-commitments
  Previous work: bitcoin-core/secp256k1#589
• S2C commitment scheme for MuSig2 and its implementation in secp256k1-zkp
• Rust code for ECDSA S2C from secp256k1-zkp
• Rust code for Schnorr S2C from secp256k1

Write fuzzify tests

The most required parts are LNPBP-4 code and merklization

Prohibid master pushes & require PR review

Two anchors randomly fail to merge with ~10% probability

Details: RGB-WG/rgb-node#208

Use Box<&dyn Error> in `Error::DataIntegrityError` type

Set up CI

Implement sign-to-contract single-use-seals

Replace on bitcoin encoding crate with 0.29 release

client_side_validation/strict_encoding/Cargo.toml

Line 23 in dc9b15e

bitcoin = "0.26" # TODO #195: Replace on bitcoin encoding crate with 0.27 release

Make bitcoin crate dependency optional in strict encoding library and everywhere else

This will require rewriting strict encoding for primitive data types instead of using bitcoin consensus encoding for them.

Complete documenting crate

Encode miniscript structures in a binary way in strict_encoding

https://github.com/LNP-BP/rust-lnpbp/blob/95e66c5297fa0222b9a29c2e9b4fe82edeae2e49/strict_encoding/src/miniscript.rs#L37

Complete async APIs for client-side-validation and single-use-seals

Pending async trait to become part of MSRV (still years)

Do not return usize from encoding methods

usize makes sense when not all of the data are written. Otherwise, the size of encoded data can be checked with the planned amplify::io::ByteCounter

Complete test coverage

In must be above 75%

Strict encoding should provide explicit guarantees on collection sizes

The main strict encoding rule was to restrict the size of any collection (string, vector of data, maps) to 2^16 elements (i.e. have a length encoded by 2 bytes). This happens implicitly, which leads to a non-obvious bugs missed in code reviews.

To address the issue it is proposed first to perform #97 and than, additionally to that, do the following:

• Abandon confined_encoding for unordered hash collections (HashSet, HashMap);
• Remove confined_encoding from all other collection types;
• Introduce new types for collections, specifically ConfinedString, ConfinedVec, ConfinedSet, ConfinedMap, backed by inner types String, Vec, BTreeSet, BTreeMap) and require all consensus code to explicitly use them.

Re-implement strict encoding for arrays with const generics once MSRV > 1.50

Strict encoding for arrays with more than 2^16 members

This will require:

• LongVec helper wrapper type, defined at strict_encoding library and used internally
• ... by strict_encode_derive, which will read special #[strict_encoding(length_bytes = 1|2|3|4|8)] attribute and encode/decode underlying array via converting it through LongVec
• Uint24 type to represent arrays of size 2^24

Downstream dependencies:

• RGB-WG/rgb-core#59
• RGB-WG/rgb-core#68

Do grin_secp256k1zkp::Error strict encoding test once this error type will be `Ord`

Propose a PR to grin_secp256k1zkp to make Error type Ord

Provide a dedicated hash type for LNPBP-4 multi-cmmitments

Run & enforce linting

Move network address uniform encoding (LNPBP-47) to a dedicated repo in @internet2-org

WASM support

stens fails to compile to wasm32-unknown-unknown due to the dependency, amplify_syn.

Make library no_std

Refactor commit_encode to work through hashers provided as io::Write

Write library-specific READMEs

Decide on compressed pubkey/signature strict encoding

Currently, bitcoin public key/signatures allow encoding in compressed and uncompressed form, depending on the compression flag value, and do not encode the flag itself. Secp256k1 keys and ECDSA signatures do not allow uncompressed encoding.

This

• makes two types incompatible
• creates unnecessary space consumption with uncompressed option

It is proposed to prohibit uncompressed keys and make bitcoin and secp256k1 key/signature implementations to be identical. If it is required to use uncompressed flag, it must be presented as a separate field withing the structure.

Alternative: use bitcoin-based data types to always encode compression flag, but serialize even uncompressed keys in a compressed version.

broken MerkleBlock conceal procedure

Executing rgb-lib tests we found an issue that sometimes happens and seems very similar to RGB-WG/rgb-node#208, but I'm not sure it's the same.

The error we receive is MerkleBlock conceal procedure is broken and comes from ~/.cargo/git/checkouts/client_side_validation-8dff74d720902144/46cb8a7/commit_verify/src/mpc/block.rs:418:21.

@dr-orlovsky could you please investigate this?

Update with bitcoin 0.26.1 release

• test cases for Bech32m addresses (strict encoding)
• strict decode for address::Payload (use WitnessVersion type)

Extract strict encoding test helpers into a separate crate

Remove PSBT BIP-174 encoding since it is not deterministic

Downstream crates should start using modern PSBT implementation from psbt(https://crates.io/crate/psbt) crate, supporting also strict encoding and both v0 and v2 PSBT versions (requires use of string_encoding flag in that crate).

When does `strict_encoding` reaches stable release?

I've been building some functionality to master branch of lnp-node, and I encountered some nasty build error in dependency strict_encoding 0.9.0.
Something like this

error[E0432]: unresolved import `bitcoin::XpubIdentifier`
  --> /home/joemphilips/.cargo/registry/src/github.com-1ecc6299db9ec823/strict_encoding-0.9.0/src/bitcoin.rs:32:21
   |
32 |     XOnlyPublicKey, XpubIdentifier,
   |                     ^^^^^^^^^^^^^^
   |                     |
   |                     no `XpubIdentifier` in the root
   |                     help: a similar name exists in the module: `PubIdentifier`

Not sure where this PubIdentifier came from, I could not find the name with git log -p in rust-bitcoin.

Anyway, I tried to debug by forking strict_encoding. which existed in this repository.
But I suppose it has been moved to https://github.com/strict-types/strict-encoding now?
If so, forking this repository seems not a good idea.
Probably I should wait until the strict-encoding in the new repository gets stable and other lnp-bp-rgb libraries depends on it? How long should I wait for it?

Modify LNPBP4 commitment arithmetics as per implemented `MultimsgCommitment`

https://github.com/LNP-BP/rust-lnpbp/blob/95e66c5297fa0222b9a29c2e9b4fe82edeae2e49/src/seals/lnpbp4.rs#L99-L100

Implement resolve_seal_issues

client_side_validation/src/api.rs

Lines 239 to 251 in 5e87205

	/// Method used to check whether seal resolution issues are resolved, for
	/// instance allowing to try a different resolver, re-check after restored
	/// network connectivity or after some time allowing unmined transactions to
	/// be mined.
	pub fn resolve_seal_issues<Resolver>(&mut self, _resolver: &mut Resolver)
	where
	Resolver: SealResolver<
	<R::SealIssue as SealIssue>::SingleUseSeal,
	Error = R::SealIssue,
	>,
	{
	todo!()
	}

Add `#[allow(named_arguments_used_positionally)]` to strict_encoding_derive

Otherwise it breaks CI with nightly copiler: https://github.com/LNP-BP/client_side_validation/runs/7593625092?check_suite_focus=true

Implement merklization of LNPBP-4 multimessage commitments

After LNP-BP/LNPBPs#103

Refactor tests & complete test coverage

After v0.10 refactoring some of the tests has to be updated

Implement strict encoding of BIP340 KeyPair

Implement strict encoding for KeyPair type once there will be a way to serialize its inner data in Secpk256k1 lib (see rust-bitcoin/rust-secp256k1#298)

Strict encoding derives must use general TLV attributes

To prevent difference in TLV implementation for structs implementing both lightning and strict encoding the tlv attribute must have universal name (like #[tlv] instead of #[network_encoding(tlv)]

Add type system reflection

Add `from_slice` method in tagged hash

https://github.com/LNP-BP/rust-lnpbp/blob/95e66c5297fa0222b9a29c2e9b4fe82edeae2e49/src/tagged_hash.rs#L83

Verify byte order for `HashFixedByte` strategy

https://github.com/LNP-BP/rust-lnpbp/blob/95e66c5297fa0222b9a29c2e9b4fe82edeae2e49/strict_encoding/src/strategies.rs#L81

Complete client-side validation trust resolvers

And use them throughout RGB validation

Revert 538417ba00a5e98e9b1d7881162fe5f0c46c9397

538417b was a temporarily fix