celestiaorg / networks

Celestia Networks

This repository contains the configuration files for Celestia networks.

Please refer to the Celestia Docs for guides on running your own node. The configuration files genesis.json and addrbook.json are intended for use by Consensus nodes and should be placed in the node's home directory (by default $HOME/.celestia-app/config).

Networks

Name	Type	Chain ID	Configs
Mocha	Testnet	mocha-4	mocha-4
Arabica	Testnet	arabica-11	arabica-11
Celestia	Mainnet Beta	celestia	celestia

Software versions

The celestia-app and celestia-node versions in use on each network are visible on this Grafana dashboard.

Public Dashboards

DA Full and Light nodes might have troubles connecting to the networks, so you can checkout this Grafana dashboard to see health/uptime status of DA Bootstrappers (now celestia network only).

networks's Issues

Format of `peers.txt` changes across testnets

Problem

Some scripts downstream of this repo assume that peers.txt has trailing commas. Some scripts assume no trailing commas. Since this repo doesn't enforce a consistent format, it's not clear what scripts should assume.

networks/mocha-3/peers.txt

Lines 1 to 2 in 3758740

	[email protected]:26656,
	[e-mail]:11656

contains trailing commas

networks/mocha-4/peers.txt

Lines 1 to 3 in 3758740

	[email protected]:26656
	[e-mail]:43656
	[e-mail]:43656

does not.

Consequently, I expect a command in the docs https://docs.celestia.org/nodes/full-consensus-node/#setup-the-p2p-networks to fail because

sed -i.bak -e "s/^persistent_peers =./persistent_peers = "$PERSISTENT_PEERS"/" $HOME/.celestia-app/config/config.toml

assumes that there are trailing commas.

Proposal

Describe a format for peers.txt. Enforce it consistently for all peers.txt files. IMO no trailing commas.

Version matrix in README

Problem

It isn't immediately obvious which version of celestia-app and celestia-node are running on which testnet

Proposal

It may be helpful to include a version matrix in the README. Something like

Network	celestia-app	celestia-node
Mamaki	https://github.com/celestiaorg/celestia-app/releases/tag/v0.6.0	...
Arabica	https://github.com/celestiaorg/celestia-app/releases/tag/v0.7.0	...

arabica-9 peers.txt is missing

celestiaorg/docs#996

node crashes with error

panic: open /home/mahof/.celestia-app/data/cs.wal: too many open files

fix CI failures

Tasks

Beta Give feedback

1. empty file in gentx dir causes failure (either by removing file or making the tool ignore .files) (see #367)
2. looks like bot not allowed to keep track of PRs when they come from forks (see https://github.com/celestiaorg/networks/actions/runs/6597567930/job/17924687535?pr=364; but weirdly https://github.com/celestiaorg/networks/actions/runs/6597322356/job/17924245542?pr=362 does not expose the same behavior)
3. bot can't comment (maybe also related to forks?): also see: https://github.com/celestiaorg/networks/actions/runs/6597322356/job/17924245542?pr=362
4. amount check should be utia instead of tia (e.g. expected between 1 and 9, got 8000000)

`auth failure: secret conn failed` when attempting to connect to mocha-4 peers

Context

I'm trying to run a mocha-4 consensus node. I'm using the persistent peers at https://github.com/celestiaorg/networks/blob/master/mocha-4/peers.txt

Problem

I'm observing

3:53PM INF Error reconnecting to peer. Trying again addr={"id":"f9e950870eccdb40e2386896d7b6a7687a103c99","ip":"88.99.219.120","port":43656} err="auth failure: secret conn failed: read tcp 10.239.124.37:64815->88.99.219.120:43656: i/o timeout" module=p2p tries=12
3:53PM INF Ensure peers module=pex numDialing=1 numInPeers=0 numOutPeers=0 numToDial=9
3:53PM INF Error reconnecting to peer. Trying again addr={"id":"34499b1ac473fbb03894c883178ecc83f0d6eaf6","ip":"64.227.18.169","port":26656} err="auth failure: secret conn failed: read tcp 10.239.124.37:64816->64.227.18.169:26656: i/o timeout" module=p2p tries=6
3:53PM INF Error reconnecting to peer. Trying again addr={"id":"43e9da043318a4ea0141259c17fcb06ecff816af","ip":"141.94.73.39","port":43656} err="auth failure: secret conn failed: read tcp 10.239.124.37:64820->141.94.73.39:43656: i/o timeout" module=p2p tries=6

Add CI task to verify peer connectivity

Context

There are several open PRs that add/remove/modify a peer address in peers.txt. Examples:

1. #525
2. #524
3. #521
4. #519
5. #509
6. #508
7. #507

Problem

It's not immediately obvious how to verify the correctness of these PRs

Proposal

Add a CI check that verifies if a peer is accessible and running on the network that corresponds to the peers.txt that was modified in the PR.

Instructions to use Docker image to add `celestia-node` full node to devnet

We should provide instructions for using the Docker image from celestia-node #295 to add a full-node to the existing devnet.

Add instructions for securely passing existing key(s) to Docker image(s)

Related to both #39 and #40

Inspired by this discussion: celestiaorg/celestia-node#293 (review)

There should be instructions for how a Celestia app or node can be setup using existing private keys in a secure manner for both Docker and K8s. This should utilize Docker Swarm Secrets or K8s Secrets.

bug: formatting output is not useful

When echoing the output here, it removes the whitespaces (incl. all useful formatting):

networks/.github/workflows/validate-gentx.yml

Line 37 in a1981e6

echo $formatted

easy fix: echo "$formatted" instead.

Also, for some files, there are only some whitespace diff which is not visible from the output even when logged correctly as per above. To improve that, we can hexdump the first few bytes:

echo "Hexdump of file:"
head -c 60 "$file" | hexdump -C
echo "Hexdump of jq output:"
echo "$formatted" | head -c 60 | hexdump -C

That would e.g. give a hint on the diff here #401

Consider removing peer: `celestia.xprv.io`

Context

I'm attempting to run a full consensus node on celestia

Problem

When I set the full consensus node's persistent peers to the list of peers in https://github.com/celestiaorg/networks/blob/master/celestia/peers.txt I encounter

11:59AM ERR Error in peer's address err="error looking up host (celestia.xprv.io): lookup celestia.xprv.io: no such host" module=p2p

Proposal

Consider removing

networks/celestia/peers.txt

Line 71 in d360932

[email protected]:30001

when I remove it, I no longer encounter the error

Improve docs on howto join devnet

Some feedback gathered from discord (mainly):

The relation between the different binaries is unclear. It already popped up on other channels and again on discord.
Ideally, there was a brief overview in the readme here and a pointer to more details; these details could be in the celestia-node repo:

What is the relation between celesita-app & celestia-node?

Newcomers start following instructions from the top and the most relevant parts are only at the very end of the documentation:

phase1,2,3 is for building up local dev environment. If you want to participate devnet-2, just follow up phaseX.

For new ppl this fact is not clear. A quick mention of this in the beginning of the document will probably save newcomers an hour time.

Blockspacerace folder name doesn't match chain id

Each folder should match the chain id - blockspacerace should be in this instance blockspacerace-0

doc: Mocha bootstrap-peers is not available on repo

https://raw.githubusercontent.com/celestiaorg/networks/master/mocha/bootstrap-peers.txt

This bootstrap-peers.txt does not exist. None of peers seems alive in https://raw.githubusercontent.com/celestiaorg/networks/master/mocha/peers.txt.

Would you please provide the valid peers to run the consensus full node?

Thank you.

docs: create a doc for running a devnet 2 node setup

For the community to try what we have achieved during devnet launch, we should transfer our doc to a proper guideline.
Existing PR from rene is great start

README that explains how to init the node and where to put these two files (genesis and addressbook)

I wonder if we want to add a little README.md that explains how to init the node and where to put these two files (genesis and addressbook)

Originally posted by @cmwaters in #514 (comment)

Remove gentxs folder from the root - it is confusing

Create `celestia-node` boostrap nodes

The devnet could use a few (start w/ 2) celestia-node bootstrap nodes that we ensure are kept in sync with the network.

Instructions to add `celestia-app` validator to devnet using docker image

We should provide instructions for using the Docker image from celestia-app #163 to add a new validator to the devnet.

scripts: add more scritps for scenario with 3 DO droplets

Most of the work from this guideline can be scripted for a faster deployment
#36

Generate celestia addrbook.json

Context

celestia-app has a command to convert peers.txt into an addrbook.json

$ celestia-appd addrbook --help
Convert a list of peers into an address book.
The first argument (peers.txt) should contain a new line separated list of peers. The format for a peer is `id@ip:port` or `id@domain:port`.
The second argument (addrbook.json) should contain the complete file path, including both the directory path and the desired output file name, in the following format: `path/to/filename`. The address book is saved to the output file in JSON format.

Usage:
  celestia-appd addrbook peers.txt addrbook.json [flags]

Flags:
  -h, --help   help for addrbook

Proposal

Once https://github.com/celestiaorg/networks/blob/master/celestia/peers.txt is finalized, run the command and save the addrbook.json in https://github.com/celestiaorg/networks/blob/master/celestia/

Request for arabica-10 peers.txt

there a invalid address

https://github.com/celestiaorg/networks/blob/3b60f38b3264d951f43b89782991b3e7337651fc/celestia/peers.txt#L27C28-L27C28

when add this peer and start validator show
'''ERR Error in peer's address err="invalid address ([email protected]:16656): encoding/hex: odd length hex string" module=p2p
'''

Peer Error - Wrong Network

7145da826bbf64f06aa4ad296b850fd697a211cc

There are warnings that this node is on the wrong network in the logs, it would be better if it was checked and deleted from the list.

https://github.com/celestiaorg/networks/blob/master/mamaki/peers.txt

Define `seeds.txt` for arabica-10

Either in this or the following testnet we should sping up a few seed nodes (tendermint nodes running in seed mode) and mention those seeds in the seed list. See: #47 (comment)

RoomIT Validator Consensus | Gentx and Peer

Hall Team,

Could I join as Validator Consensus?

I have cerate gentx.

Kindly for procedure and access for networks repository.

Best Regards

"Celestia Application Installation" link in ReadMe Broken

At the bottom of the page, the "Celestia Application Installation" instructions link is broken. Takes you to a 404 page.

Light Client

Audit arabica genesis.json

Motivation

celestiaorg/celestia-app#1363 was caused by arabica-4's genesis.json not containing an expected parameter. If we recreate a genesis.json via:

celestia-appd init $CHAINID --chain-id $CHAINID

then the genesis file will contain all the parameters expected by celestia-app. However, for arabica-4 and arabica-5 we've manually copied + pasted the genesis.json to retain a list of account addresses that we fund at genesis time (i.e. a faucet address).

Problem

If we continue copy + pasting the genesis files, then we will fail to pick up on changes to default parameters or new parameters.

Proposal

1. Audit the diff between arabica-5's genesis.json and a freshly created genesis.json
2. Before we start a new chain, have the gensis.json in version control and audited by celestia-app team

ci: run-network job does not get executed

Validate Genesis / run-network (pull_request) does not seem to run:
/home/runner/work/_temp/1310e5a5-7d32-4bf2-bf04-e2e09c49dac5.sh: line 1: [[-f: command not found

https://github.com/celestiaorg/networks/actions/runs/6606039623/job/17941835312?pr=397

Switch from FIFO to prioritized mempool

Context

We'd like validators to use the prioritized mempool see celestiaorg/celestia-app#590 (comment)

Problem

Arabica (and potentially most validators on Mamaki) use the default FIFO mempool.

Proposal

We'd like our testnets to be representative of the parameters used on mainnet so we should update the Arabica validator to use the prioritized mempool (i.e. version: "v1"). If possible, we should also update any default configuration so that as many validators as possible on Mamaki update to using the prioritized mempool.