rollkit / celestia-da

Celestia Implementation of Modular Data Availability Interface

Rollkit

Rollkit is the first soveriegn rollup framework. For more in-depth information about Rollkit, please visit our website.

Using Rollkit

Rollkit CLI

The easiest way to use rollkit is via the rollkit CLI.

Requires Go version >= 1.22.

A cli tool that allows you to run different kinds of nodes for a rollkit network while also helping you generate the required configuration files

Install

To install rollkit, simply run the following command at the root of the rollkit repo

make install

The latest Rollkit is now installed. You can verify the installation by running:

rollkit version

Quick Start

You can spin up a local rollkit network with the following command:

rollkit start

Explore the CLI documentation here

Building with Rollkit

Rollkit is the first sovereign rollup framework that allows you to launch a sovereign, customizable blockchain as easily as a smart contract.

Check out our tutorials on our website.

Contributing

We welcome your contributions! Everyone is welcome to contribute, whether it's in the form of code, documentation, bug reports, feature requests, or anything else.

If you're looking for issues to work on, try looking at the good first issue list. Issues with this tag are suitable for a new external contributor and is a great way to find something you can help with!

See the contributing guide for more details.

Please join our Community Discord to ask questions, discuss your ideas, and connect with other contributors.

Helpful commands

# Run unit tests
make test

# Generate protobuf files (requires Docker)
make proto-gen

# Run linters (requires golangci-lint, markdownlint, hadolint, and yamllint)
make lint

# Lint protobuf files (requires Docker and buf)
make proto-lint

Tools

1. Install golangci-lint
2. Install markdownlint
3. Install hadolint
4. Install yamllint

Dependency graph

To see our progress and a possible future of Rollkit visit our Dependency Graph.

Audits

Date	Auditor	Version	Report
2024/01/12	Informal Systems	eccdd...bcb9d	informal-systems.pdf
2024/01/10	Binary Builders	eccdd...bcb9d	binary-builders.pdf

celestia-da's Issues

[Feature Request]: Expose gRPC server

Implementation ideas

Create a binary so the DA interface can be served with gRPC.

Investigate OOM leak

Container using version celestia-da:v0.12.1-rc0

Flags used: --gateway --metrics --metrics.tls=false

Getting OOM with 2GB memory

running otel-collector in same pod to export otel-metrics for prometheus

we should make sure this version is using the celestia node version that has the fix: celestiaorg/celestia-node#2905 (comment), and if so, then debug memory leaks in celestia-da

[Feature Request]: upgrade to celestia-node v0.12.4

Implementation ideas

https://github.com/celestiaorg/celestia-node/releases/tag/v0.12.4

[Feature Request]: new release cycle and versioning

Implementation ideas

celestia-da versions were initially based on celestia-node and tightly coupled to it's releases.

As we have more breaking changes, this will be difficult to maintain. As such, starting from the next minor bump i.e. v0.13.x we should maintain celestia-da version independently from celestia-node version.

As we'll have different versions for celestia-da and celestia-node, there should be a version command that should output both versions.

[Feature Request]: properly handle `ErrBlobNotFound`

Implementation ideas

After celestiaorg/celestia-node@a7a17e4 is released in celestia-node, we can improve error handling using errors.Is instead of comparing error strings in GetIDs.

v0.12.9 Does not have a install make command in the makefile.

Version of Git SHA

using tag - v0.12.9

System OS

Linux

Installed tools

No response

Others

No response

Steps to reproduce it

after running make build succcessfully
make install fails with

make: go: No such file or directory
make: *** No rule to make target 'install'.  Stop.

### Expected result

celestia-da should be installed and made executable 

### Actual result

`make install` fails with 

make: go: No such file or directory
make: *** No rule to make target 'install'. Stop.

Relevant log output

No response

Notes

No response

[Feature Request][design]: DA error handling

Implementation ideas

This is a design discussion to coordinate error handling between celestia core/app and rollup clients like rollkit.

Some issues with the current model

• core/app accepts transactions which may not be mined see: celestiaorg/celestia-app#3081
• tx which are not mined return timed out waiting for tx to be included in block
  • should this be handled by core/node? see: rollkit/rollkit#1522
  • should core/node autoretry with higher fee
• if a tx is accepted by the mempool (e.g account being reused), rollup cannot do anything until it gets dropped
  • can core allow a faster ttl? or allow replace-by-fee?
• testing DA submission issues is dependent on mempool state (e.g. can be congested) which makes it hard to reproduce issues like rollkit/rollkit#1520

Overall this makes handling DA submission challenging because of the lack of congest control flow and lack of dynamic fee market.

[Feature Request]: Remove Celestia OpenRPC dependecy.

Implementation ideas

Instead of importing the Celestia openRPC repository, we should import celestia-node directly. This was the main motivation behind rollkit/rollkit#1122

[Feature Request]: Add Tests to CelestiaDA Implemenation

Implementation ideas

self-explanatory

[Feature Request]: NewCelestiaDA use consistent argument position for context

Implementation ideas

NewCelestiaDA should accept context as first param.

proof from method GetProof cannot be validated(unable to unmarshal proof) by method Validate

Version of Git SHA

v0.13.0

System OS

Ubuntu 22.04.3 LTS

Installed tools

docker image of celestia-node(v0.13.0)

Others

No response

Steps to reproduce it

1. launch a light celestia node: https://github.com/AnomalyFi/nodekit-relay/blob/with-streaming/scripts/start_celestia_light.sh
2. clone this repository: https://github.com/bianyuanop/celestiatest
3. type go run main.go in the folder of repository mentioned in step 2

Expected result

should validate without any error

Actual result

Cannot get validated because of cannot Unmarshal proof to pb.Proof, see image

Relevant log output

No response

Notes

No response

[Feature Request]: Handle big blobs

Implementation ideas

I like big blobs and I cannot lie

As pointed out in the code comments and review comments, there is high probability that bigger blobs can break the handling of Proofs.
The tests for this will be implemented in go-da (see: rollkit/go-da#17), but changes in the code will most probably be necessary.

[Feature Request]: implement celestia-node's go-version handling in CI

Implementation ideas

REF: celestiaorg/celestia-node#2806

This use of a CI step that sets the go version as an output allows us to have a single go version defined for the entire CI.

[Feature Request]: package release binaries

Implementation ideas

We should create a workflow to build and package binaries with every release.

[Feature Request]: allow configurable gas fee, rate options

Implementation ideas

We should be able to pass the gas fee, rate options from the client. This might need changes to the interface.

[Feature Request]: allow configuring global gas price

Implementation ideas

Currently, clients need to pass gasPrice with every Submit request. Instead, we can allow a global gasPrice to be configured from cli options and allow clients to override it per request.

This makes rollout easier because it's easier to configure celestia-da than to update each integration to add the gas price flag.

[Feature Request]: bundle celestia-node as binary

Implementation ideas

While the Data Availability Service implementation can be run as a standalone service and connect to a remote celestia-node over RPC, it is convenient to combine both into the same binary so that whenever server is started, it also starts celestia-node and connects to the same instance.

As the manager for the celestia-node server, the new binary can access the node auth token directly from the store and no longer require the auth token flag. Only the namespace flag will be required.

Also, as we'll be combining the binaries, it will be better to have a namespace for the flags so that they don't clash with node flags. To make things consistent, we can rename the flags like so:

Old Key	New Key	Required	Default Value
rpc.address	da.grpc.address	false	http://127.0.0.1:26658
rpc.token	da.grpc.token	false	derived/generated from node store
namespace	da.grpc.namespace	required	n/a
listen.address	da.grpc.listen	false	127.0.0.1:0
listen.network	da.grpc.network	false	tcp

It would be better to wrap the celestia-node commands and implement this as a PreRun command so that no internals from node need to be exposed.

[Feature Request]: package docker image

Implementation ideas

It would be good to package celestia-da as a docker image, very similar to celestia-node, except that the binary wraps node and also runs the da grpc service. This can be used by local-celestia-devnet to provide a devnet which can directly be used by grpc clients to submit blobs to.

Unimplemented rpc error for MaxBlobSize

something wrong rpc error: code = Unimplemented desc = unknown method MaxBlobSize for service da.DAService
10:12AM ERR DA layer submission failed error="unable to get DA max blob size" attempt=11 module=BlockManager

this is happening because celestia-da (main branch and last release) used go-da 57bc36006772 which does not have MaxBlobSize.

we should update this to use 93600f2 which has the api MaxBlobSize.