rollkit / go-da

Generic Data Availability Interface for Modular Blockchains

Rollkit

Rollkit is the first soveriegn rollup framework. For more in-depth information about Rollkit, please visit our website.

Using Rollkit

Rollkit CLI

The easiest way to use rollkit is via the rollkit CLI.

Requires Go version >= 1.22.

A cli tool that allows you to run different kinds of nodes for a rollkit network while also helping you generate the required configuration files

Install

To install rollkit, simply run the following command at the root of the rollkit repo

make install

The latest Rollkit is now installed. You can verify the installation by running:

rollkit version

Quick Start

You can spin up a local rollkit network with the following command:

rollkit start

Explore the CLI documentation here

Building with Rollkit

Rollkit is the first sovereign rollup framework that allows you to launch a sovereign, customizable blockchain as easily as a smart contract.

Check out our tutorials on our website.

Contributing

We welcome your contributions! Everyone is welcome to contribute, whether it's in the form of code, documentation, bug reports, feature requests, or anything else.

If you're looking for issues to work on, try looking at the good first issue list. Issues with this tag are suitable for a new external contributor and is a great way to find something you can help with!

See the contributing guide for more details.

Please join our Community Discord to ask questions, discuss your ideas, and connect with other contributors.

Helpful commands

# Run unit tests
make test

# Generate protobuf files (requires Docker)
make proto-gen

# Run linters (requires golangci-lint, markdownlint, hadolint, and yamllint)
make lint

# Lint protobuf files (requires Docker and buf)
make proto-lint

Tools

1. Install golangci-lint
2. Install markdownlint
3. Install hadolint
4. Install yamllint

Dependency graph

To see our progress and a possible future of Rollkit visit our Dependency Graph.

Audits

Date	Auditor	Version	Report
2024/01/12	Informal Systems	eccdd...bcb9d	informal-systems.pdf
2024/01/10	Binary Builders	eccdd...bcb9d	binary-builders.pdf

go-da's Issues

[Feature Request]: Add MaxBlobSize to the Da-Interface

Implementation ideas

Is blocking: rollkit/rollkit#1233

Also, adding the fact that the sum of blobs if submitting a list should be smaller than the MaxBlobSize. However there could be an edge case where the sum cannot actually = MaxBlobSize as there might be some overhead from the transaction bytes.

[Feature Request]: Create a test with large inputs

Implementation ideas

Currently all the test in test suite works on very small blobs. We need a test witch will cover handling of bigger blobs.

Data in the blobs should be randomized. There should be a helper function to run a test for given size of a blob, and a subtests for various sizes.

Related to #12.

What is the meaning of `gasPrice` in `Submit`?

What is the meaning of gasPrice and what is the supposed behavior of the DA adapter wrt it?

go-da/da.go

Lines 25 to 29 in 011ba69

	// Submit submits the Blobs to Data Availability layer.
	//
	// This method is synchronous. Upon successful submission to Data Availability layer, it returns the IDs identifying blobs
	// in DA.
	Submit(ctx context.Context, blobs []Blob, gasPrice float64, namespace Namespace) ([]ID, error)

Create gRPC mapping of DA interface

Define gRPC mapping (protobuf service) of Go DA interface.

Make `Submit` function async

Currently, our Submit function is async in Rollkit. An option to do that here would be to decouple the inclusion proof generation from the response of the submission.

The flow could be Submit -> get commitment as a response ( can be generated deterministically) and would be an instant response
Then something like getProof(ID) to get the proof which might happen Blocks (minutes) into the future

This would be the current interaction with celestias blob module.

Either way, I think we should merge this and then make changes afterwards , as is does not have to be perfect from the start and we can expect to break the API.

Originally posted by @nashqueue in #1 (comment)

Add dummy DA implementation for tests

Create as simple as possible DA implementation & corresponding tests.

[Feature Request]: configurable gas price

Implementation ideas

The latest release v0.0.1 allows users to pass in SubmitOptions which include GasLimit and Fee. Individually they are not very useful because both depend on blob size of the transaction they're submitting. If the client configures a fixed GasLimit and Fee, they risk either not getting their transaction processed for larger blobs or overpaying for smaller blobs.

Instead it would be better to allow users to configure a GasPrice per transaction so that GasLimit and Fee can be derived from it based on the blob size.

So, there should be a new minor version bump release which replaces *SubmitOptions with GasPrice float64. If the GasPrice is less than zero, the DA server can default to DefaultSubmitOptions.

[Feature Request]: Add namespace / appID as optional parameter to Submit function

Implementation ideas

In order to implement go-da directly into celestia-node, setting namespace as a global parameter is not a viable option. Additionally, it restricts the possibility for clients to use multiple namespaces without instantiating multiple clients. To fix this, we can add a field SubmitOptions to the Submit signature.

[Feature Request]: GetAll method

Implementation ideas

Currently there is no way to get blobs without knowing it's exact ID - this prohibits any form of scanning DA layer.

Proposed solution is to add following method:

type DA interface {
// ... 
    GetAll(daHeight uint64) ([]Blob, error)
// ...
}

Implementation of this method should return all the blobs found in DA layer at given height.

[Feature Request]: Add a SyncWait method

Implementation ideas

Implementation ideas
Currently there is no way to wait before the remote node is synced so that we can start submitting blobs.

Proposed solution is to add following method:

type DA interface {
// ... 
    SyncWait() (error)
// ...
}

Implementation of this method should transparently call Header.SyncWait

Initial code of DA interface

Draft initial version of DA interface code.

[Feature Request]: Async API

Implementation ideas

Because blob submission can take significant amount of time (for example to await for next block in DA chain) it would make sense to provide easy to use asynchronous API.

Add mutex lock around Dummy DA data

Make tests reusable

Test suite of this repository should be prepared in reusable, developer friendly way.
Probably the best way of achieving this is to create a package, with public methods that can be reused by DA implementers.
Existing tests (from #1) already expose methods with signature like:

func FunctionName(t *testing.T, da da.DA)

Extra function to call all the tests should be created.
Tests implemented in this repository should be implementation agnostic. DummyDA implementation should pass all the tests.

[Feature Request]: implement celestia-node's handling of go version in CI

Implementation ideas

REF: celestiaorg/celestia-node#2806

This use of a CI step that sets the go version as an output allows us to have a single go version defined for the entire CI.

Handling of forks

It's not clear how the forks are going to be handled. Given the following function:

go-da/da.go

Lines 16 to 17 in 011ba69

	// GetIDs returns IDs of all Blobs located in DA at given height.
	GetIDs(ctx context.Context, height uint64, namespace Namespace) ([]ID, error)

Two calls with the same height and namespace can return different IDs.

You can fit the definition by considering only finalized blocks for height. I would imagine it may be slightly annoying for Avail since it uses BABE & GRANDPA which is pretty fast most of the time. In Ethereum however finality time is way longer.

1. Am I missing something?
2. What is the suggested way to handle non-finalized chain heads?
3. What's the game of plan of Avail, if known.
4. Are there plans to change the API to accomodate non-single-slot-finality chains?

Setup go-da repository

Setup:

• LICENCE (Apache 2.0)
• NOTICE
• GitHub workflows for build and lint

Question regarding `DA.MaxBlobSize`

go-da/da.go

Lines 7 to 8 in 011ba69

	// MaxBlobSize returns the max blob size
	MaxBlobSize(ctx context.Context) (uint64, error)

It is not clear what is permited inside of the implementation. Both concrete implementations (namely, at the time of writing, avail and celestia) return a constant number.

However, error in the return type suggests it may be possible to implement this function via an RPC. Also, it's not entirely clear when this request is going to be called: once or before every submission? If the former, what would be recommendations to support the DA layers with [governance] configurable blob sizes.

[Feature Request]: standard error codes

Implementation ideas

Implementations should return standard error codes so that clients have a way to handle the error and retry if possible.

We can reuse JSONRPC error codes in the range --32000 to -32099 , clients should map their errors into the standard error codes for e.g.:

• 32001 - request timeout
• 32002 - mempool congested
• 32003 - insufficient fee
• 32004 - incorrect sequence / nonce
• 32005 - blob size too large

gRPC can reuse the JSONRPC error codes for simplicity.

Based on the error, the client can retry with a higher fee, different nonce or smaller blob.

Implementations should wrap their error with the corresponding typed error.

E.g.:

pkg someda

...
var (
	// Err32003 is returned when the blob transaction has insufficient fee
	Err32003 = errors.New("32003")
)
...


// Submit submits the Blobs to Data Availability layer.
func (c *SomeDA) Submit(ctx context.Context, blobs []da.Blob, gasPrice float64, ns da.Namespace) ([]da.ID, error) {
        ...
	height, err := c.Submit(ctx, blobs, gasPrice, ns)
	if err != nil {
		if errors.Is(err, ErrInsufficientFee) {
			return errors.Wrap(da.Err32003, err)
		}
		return nil, err
	}
        ...
}