oasisprotocol / oasis-sdk

Official SDK for the Oasis Network.

Oasis SDK

Directories

• client-sdk: Client libraries for interacting with the Oasis consensus layer and runtimes in different languages.
• runtime-sdk: Oasis Runtime SDK that makes it easy to develop new runtimes.
• runtime-sdk/modules: A collection of larger modules for the Runtime SDK.
• contract-sdk: Oasis Contract SDK that can be used to develop Rust smart contracts that compile to WASM and can be executed inside the contracts Runtime SDK module.

License

All Oasis SDK software components are licensed under Apache 2.0.

The content of the documentation (the /docs folder in any of the components) including the media (e.g. images and diagrams) is licensed under Creative Commons Attribution 4.0 International.

oasis-sdk's Issues

Add storage tests

There should be a set of tests for the high-level storage helpers.

ts-web: add an automated test to check if typedefs go out of sync with other codebases

goes hand in hand with manual maintenance

cc @peterjgilbert

Runtime-sdk extend testing framework to support advancing rounds

Extend the mock testing framework to support advancing rounds.

Context: #92 (comment)

Benchmarks

We should implement some end-to-end benchmarks for various modules that are part of the SDK and run them periodically (e.g. daily) on CI.

ts-web/core release roadmap

prereqs

• "documentation" 🤭 #4
• license #6

intermediate release: share with our developer partners

• add read permission on github

middle phase

• (add further discovered tasks here)

public release

• bump version to 1.0.0
• flip this repo public
• create npm repos
• build and upload to npm
• clean up allowed readers

ts-web/rt: retrying a transaction etc?

There are some drawbacks to having a single helper that does the signing and submission, one of which is that it reduces the flexibility if you want to have the user sign it once and then send it either later or multiple times, etc. if, for example, there are network issues.

For our records, the reason we have a draft doing it this way is to carry method type information between (i) constructing the transaction with the request body type, which happens before signing, and (ii) parsing the response data, which happens after submitting.

Maybe we should look into making some clearly container structs like UnverifiedTransaction and SignatureSigned have a type parameter instead of unknown content. It takes us farther away from the possibility of translating type definitions automatically, but seems worth it.

I generally like to use `unwrap()` with a comment to prevent useless strings from filling up my `.text` section.

I generally like to use unwrap() with a comment to prevent useless strings from filling up my .text section. I'm sure that's best processed using strip or something, so probably ignore me.

Originally posted by @nhynes in #94 (comment)

Update ts-web for oasis-core master branch

tasks breakdown forthcoming

but we'll be pursuing this mainly for its expanded runtime interfaces

Configure dependabot

Similar to the other repositories, we configure dependabot for keeping dependencies up to date

merging plan for 2021/03/25

could I have us do the following:

1. @kostko separate out 'Bump Oasis Core version' from #38 and merge that. I approve of that change.
2. @abukosek separate out 'w1p' (the runtime itself) from #37, make any minor changes, and merge that.
3. Reviewers please review #51, which is based on a branch that simulates the above steps, and I'll merge that.
4. Then #37 can continue without blocking anything, and we'll have the tests we need to finish off #38

ts-web: Standardize tsconfig.json

Since signer-ledger, rt, and core already use the same tsconfig.json, I propose having a base tsconfig.json inside ts-web that is then extended by the children packages.

Do not vote for obviously malformed transactions

Currently a bad proposer can include malformed transactions in a block and while they will be rejected during processing they will still be included.

If a batch includes obviously malformed transactions (not following the transaction format or having invalid signatures), the batch should be rejected as invalid by the runtime, returning an error from RuntimeExecuteTxBatchRequest (which will cause the workers to not vote for the proposal and indicate a failure instead).

Support modules without events

It should be possible to have a module that implements no events (i.e. passes () as the Event type).

Gas model and accounting

We should add a simple gas model and corresponding accounting, likely similar to what we do in the consensus layer with each operation costing a specific amount of "gas". The gas limit and fee then implicitly define a gas price and nodes can decide to set a minimum price they will accept.

There is the question on how to estimate gas cost? Could be done as a special query like core.EstimateGas.

Consensus layer interaction module

A module that supports interacting with the consensus layer via runtime messages:

• Transfering tokens from consensus layer (withdraw).
• Transfering tokens to consensus layer (transfer).
• Staking tokens in the consensus layer.

One consideration is that there is a maximum number of messages that can be emitted per round. So probably we would want to return additional metadata to the mempool in Oasis Core so that it could actually take this into account.

The denomination of the consensus layer token should be configurable (or we could use the token_symbol from the consensus layer's genesis file, but then we would need a way to query it).

Requires oasisprotocol/oasis-core#3564.

storage: shorter key lifetime requirement

    pub fn tx_value<V>(&mut self, key: &str) -> &mut V

I think that the static requirement is a bit heavy, but w/e

Originally posted by @nhynes in #94 (comment)

for dispatch context and tx context value and tx_value (latter forthcoming) key parameter, currently requires &'static str

Consensus module: support loading consensus denomination from consensus state

Requires: oasisprotocol/oasis-core#3868

Implement missing secp256k1 signer

Currently the secp256k1 signer implementation in the Go client SDK is missing and should be filled in. We could use the github.com/btcsuite/btcd/btcec library (unless there are any better alternatives).

ts-web: getting started guide

ideas on what to cover:

• getting envoy
• configuring envoy
• what if they need CORS
• a sane method whitelist
• link to setting up a non-validator node
• cross referencing oasis-core godoc
• slightly more comprehensive description of how types correlate between JS and Go
• transfer example snippets from playgrounds

Consider replacing `StoreKey` with `AsRef<[u8]>`

AsRef<[u8]> seems about what the StoreKey trait is going for. It also comes with a shiny new const generic impl. Whether this issue is relevant depends on whether StoreKey will ever not be just as_store_key().

Multisig accounts

The SDK should support multisig accounts.

ts-web/rt: consensus module

from #92

edit: also some changes to core module

Add procedural macro for module derivation

Currently there is quite some boilerplate involved with defining a module, it's internal API, callable method dispatch and query method dispatch. It would be cool if there was a procedural macro that made this simpler.

Rewards module

We should implement a simple module for disbursing rewards to node operators. Rewards should come from a specific module account and be disbursed accoording to a genesis configuration parameter.

Requires oasisprotocol/oasis-core#3892.

ts-web: Remove some elements that won't be used

An example, common.CODE_CERTIFICATE_ROTATION_FORBIDDEN:

The identity module is not exposed to clients so it probably doesn't make much sense to transfer it over?

Originally posted by @kostko in #4 (comment)

currently we've transferred a pretty broad set of methods, constants, etc., mostly by scanning the Go codebase for certain types and registration calls. let's come up with a more sophisticated way choosing what to model and what not to model in the typescript SDK

Setup release process

Similar to the other repositories, we should set up a release process. Note that this repository contains a bunch of different packages (Typescript, Rust, Go) that need to be taken into account. All the packages should use the same versioning scheme.

Bump Oasis Core to 21.1.x

Since a new version has been released we should bump the dependency to 21.1.x.

Add E2E tests

We should add some simple E2E tests (using the E2E test framework from Oasis Core) that make sure that a trivial runtime based on the runtime-sdk works correctly with the specified version of Oasis Core and the various client-sdk implementations.

Add consensus_accounts ts/go clients

Add ts/go clients for the consensus_accounts module added in #92

Add more test keys

Currently we only have three convenient test keys (Alice, Bob, Charlie). We should add some more for use in test cases.

Bump Oasis Core to 21.0

Now that Oasis Core 21.0 has been released the SDK should start using that instead of the master branch.

ts-web/core: move gRPC modules' methods into separate clients?

We could also split these into multiple classes so instead of nic.beaconGetBaseEpoch() you would do nic.beacon.getBaseEpoch()?

Originally posted by @kostko in #36 (comment)

---

Currently they're all methods in a single "node internal" client class. I hadn't split this up the way runtime modules have wrappers separated into modules because oasis-node is not mix-and-match in the way runtime modules are.

This might become more appealing when we encounter any of the following:

• if the current naming scheme of concatenating the module name and method name starts to become ambiguous
• if we want to pass a single module's set of methods around as an object

update readme when Oasis Network upgrades to oasis-core v21

Originally posted by @kostko in #81 (comment)

Expose the transport in LedgerContextSigner

In the constructor of LedgerContextSigner, we should probably expose the transport as public, or as private and proxy certain method to the transports.

As mentioned in https://github.com/LedgerHQ/ledgerjs/tree/master/packages/hw-transport-webusb of the two use cases :

1- Briefly open/do an operation (get the address, sign a transaction/close, each time
2- Keep it open, but listen for webusb events (device unplugged), and the user might want to "close" his wallet in the GUI and reopen it later

We currently cannot do either of those usecases since the transport is not exposed, so we cannot call close on it and we cannot listen to events

I propose simply adding transport to the constructor and making it public for now, but happy to discuss a more advanced API

oasis-sdk/client-sdk/ts-web/signer-ledger/src/index.ts

Lines 28 to 58 in c6cb460

	export class LedgerContextSigner implements oasis.signature.ContextSigner {
	app: OasisApp;
	path: number[];
	publicKey: Uint8Array;
	constructor(app: OasisApp, path: number[], publicKey: Uint8Array) {
	this.app = app;
	this.path = path;
	this.publicKey = publicKey;
	}
	public(): Uint8Array {
	return this.publicKey;
	}
	async sign(context: string, message: Uint8Array): Promise<Uint8Array> {
	const response = successOrThrow(
	await this.app.sign(this.path, context, bufFromU8(message)),
	'ledger sign',
	);
	return u8FromBuf(response.signature as Buffer);
	}
	static async fromWebUSB(keyNumber: number) {
	const transport = await TransportWebUSB.create();
	const app = new OasisApp(transport);
	// Specification forthcoming. See https://github.com/oasisprotocol/oasis-core/pull/3656.
	const path = [44, 474, 0, 0, keyNumber];
	const publicKeyResponse = successOrThrow(await app.publicKey(path), 'ledger public key');
	return new LedgerContextSigner(app, path, u8FromBuf(publicKeyResponse.pk as Buffer));
	}

Invariant checker

Each Runtime SDK module should be able to define an invariant checker (e.g., by introducing a new InvariantHandler trait) that can optionally be invoked on the state checking for any invariant violations. To invoke the invariant checker one would use a special query defined by the core module.

Examples of invariants for the accounts module include:

• All account balances should sum up to the total supply.

Examples of invariants for the consensus layer interaction module (#11) include:

• Total supply of the designated consensus layer token denomination should be less than or equal to the balance of the general runtime's account in the consensus layer.

In E2E tests we should then invoke the invariant checker query.

ts-web/core: update envoy instructions

we no longer ship a docker command. update the getting started doc to refer to the getenvoy distribution

Add derive macro for easier event derivation

To make it easier to define events:

#[derive(Debug, Serialize, Deserialize)]
#[serde(untagged)]
pub enum Event {
    Transfer {
        from: Address,
        to: Address,
        amount: token::BaseUnits,
    },

    Burn {
        owner: Address,
        amount: token::BaseUnits,
    },

    Mint {
        owner: Address,
        amount: token::BaseUnits,
    },
}

impl event::Event for Event {
    fn module(&self) -> &str {
        MODULE_NAME
    }

    fn code(&self) -> u32 {
        match self {
            Event::Transfer { .. } => 1,
            Event::Burn { .. } => 2,
            Event::Mint { .. } => 3,
        }
    }

    fn value(&self) -> cbor::Value {
        cbor::to_value(self)
    }
}

Add parameter for maximum number of transaction signers

We should add a parameter to the core module that defines the maximum number of transaction signers that can be specified in a transaction.

Add derive macro for easier error derivation

In addition to what the procedural macro for thiserror::Error does, it should also implement the SDK's Error trait (module and code), like this (where module name and error codes would be specified via struct/variant attributes):

impl error::Error for Error {
    fn module(&self) -> &str {
        MODULE_NAME
    }

    fn code(&self) -> u32 {
        match self {
            Error::InvalidArgument => 1,
            Error::InsufficientBalance => 2,
            Error::Forbidden => 3,
        }
    }
}

And a conversion to error::RuntimeError like this:

impl From<Error> for error::RuntimeError {
    fn from(err: Error) -> error::RuntimeError {
        error::RuntimeError::new(err.module(), err.code(), &err.msg())
    }
}

ts-web/signer-ledger: extra bytes after signature

It's reported that the Ledger signer is making signatures with extra bytes at the end, and that a workaround is to slice them off.

What are these bytes, and could we confirm that the slicing is correct?

cc @Esya

Governance module

We should add a governance module which would support voting on proposals where proposal types (and what happens after their execution) could be defined by other modules.

One of the basic proposals implemented by the governance module itself (or some consensus* module) should be to update the runtime descriptor in the consensus layer via a runtime message.

ts-web: marginal gRPC services

RegisterService calls that need some review

• control grpc_debug -> this one I'll add in #10
• IAS -> separate socket
• runtime enclaverpc -> done
• sentry -> separate socket
• worker storage -> internal socket #62

Consider adding a `Module::Migration` type

For doing user-level migrations. It's possible to stuff it into Genesis, with support from the module, but that's strange and may be better represented by a new type+method.

The use case is: a SQLite migration-only module (no methods, batch handling, or auth) has its own state to be migrated, and also would also usefully contain shared logic for migrating application-layer state. As the SQLite module is a module, it's not directly indexable by the Runtime, so the only way to address it is through the GenesisState. In this case, type GenesisState = Vec<Migration> would be appropriate, which is not what's intended, I think.

Maybe I'm misusing the module system? I don't think so, though. For instance, I could imagine db setup/teardown happen in block handlers. Everyone would have to do that, so why not make it a module?

ts-web/core: propagate playground errors to test runner

currently they end the test abruptly, and the test runner times out waiting for the test to finish. it would be better if the test runner could get the error somehow

Configure domain separation context

The runtime SDK should configure the domain separation context based on:

• The runtime ID.
• The consensus layer genesis document hash.

Add transaction and query fuzzing tests

We should have some transaction/query E2E fuzzing tests similar to what we have with the txsource workloads in Oasis Core. It should be made in such a way that can be easily reused in other (external) modules and runtimes.

Requires #21.

tests/e2e/simplekvtest.go needs update for signature context

from #38, there's a new parameter in AppendSign.

Over in https://github.com/oasisprotocol/oasis-bridge/pull/31, I'm adding a call to GetInfo to get the right context.

cc @abukosek

I think there's ongoing work to hook this up to CI, but I think we don't have a PR open for it, so I'm filing a standalone issue

New methods for playground: `GetChainContext` and get account nonce

You could use the new GetChainContext method instead of fetching the whole genesis document.

Originally posted by @kostko in #40 (comment)

ts-web/core: new cborg has types

let's update and erase that 'expect-error' line

Add tests for max messages per runtime context

Since we are adding a new message this should actually be:

        if self.messages.len() >= self.max_messages as usize {

Also would be cool to have a test for this.

Originally posted by @kostko in #92 (comment)