Peers and sync design about aeternity

[joelreymont]

How is trust calculated?

How does a peer advertise itself to us?

When do we update the dead or alive flags?

What is enough the definition of GoodPeer? What is N?

How does using Peers ensure the longest chain?

In the sync sequence...

1. Where do we get the initial list of FastPeers to query? How many peers should we ask for?

2. When exactly do we continue? Why are headers low-trust and why fairly up to date? What headers do we ask for? Are these headers for the top block? How are they different from step 6 then?

3. Where do we get the initial list of peers?

4. How many peers is enough in light mode?

In the list of notes and issues...

Please clarify We use trust only for the initial download. Asking random Peers for the top block is used to build trust that's used later when we do the initial sync. It's usefull to have as good as possible initial sync so asking random Peers will result in the least new data possible.

[sennui]

1. the notion of dead or alive - is a peer forever in the dead? Is it open list or capped on some size? Probably we don't want to maintain a list of dead peers

2. Can you detail the force sync flow, for the freshly mined block?

3. When setting up some default numbers of random peers it should be relative, not hardcoded to (100-1000)

[sennui]

Re. point 10: you want to enable/disable API based on "state"? Lets approach it in a bit different way (or express it in different way). Node can start listening on API when its operational and it is operational after the first sync. Turning APIs on/off during regular operation is never a good idea

[sennui]

The API to exchange information has to be granular and extendable. You mention the exchanges of blocks and headers. Also advertising peers. Design the api in a way that it takes equivalents of maps. If we want to pull more data to sync it will be easy to add (like more transactions that are not mined)

[lucafavatella]

1. [f] fast peers - Asks random FastPeers for some RandomFastPeers -> RandomFastPeers

• I understand that when the node starts for the first time it will have one (or a bunch of) initial peers, so it shall read "Asks random FastPeers - or initial Peer specified by configuration if none known - ..."

• If it "Asks ... for some RandomFastPeers", does it mean that the network call returns not only the network connection info but also the statistics e.g. trust and speed/size?

2. [f] fast headers - Asks RandomFastPeers about headers. When we don't get much new stuff we continue -> low trust (fairly up-to-date) headers

• Is block header validation in this step?

• Assuming block header validation is in this step, considering that node aims at identifying "the" longest chain, how does node acquires the headers ("-> low trust (fairly up-to-date) headers")? Does it first ask for top header hash+height from a certain number of peers, then picks any one chain at greatest height and validates it sequentially, resulting in one chain of block headers that are valid?

• Assuming block header validation is in this step, considering that validation of block header depends on previous block headers for (1) validation of difficulty, (2) presence of block header identified by previous hash, (3) block header leading to genesis, does node parallelize computations?

[sennui]

There is always a list of known peers. We can host them and host the list on our site (like an static API)

[lucafavatella]

3. [f] fast blocks (full only) - Asks RandomFastPeers about blocks. When we have all we continue -> blocks for the low trust headers

This step is listed as being for both full and light mode of operation. Does it mean even in light mode all blocks shall be downloaded?

[joelreymont]

Shall we create a Google Doc and paste the link here?

[cytadela8]

To everyone:

How is trust calculated?

Formula for trust has to be defined. The idea is the trust represents whether according to previous responses the node tends to return the top header for the longest chain when asked. Aside from that the trust is lowered if node gives us any untrue or malformed information.

Aside from determining the nodes we should use for sync (only high trust nodes when caching up, all nodes except those with low trust in normal operation), trust can also be used to determine nodes that are trying to attack network and to drop connections from them.

How does a peer advertise itself to us?

My idea is the information about the existence of a new peer in network will be spread in limited extend in peers sync. Node will be able to use a special "advertise_peer" request to inform other of it's existence as fully operational full node. (I suggest peer sync will be pull only)

When do we update the dead or alive flags?

the notion of dead or alive - is a peer forever in the dead? Is it open list or capped on some size? Probably we don't want to maintain a list of dead peers

I don't think, I ever used the word "flag". (proposal text was edited by someone). I don't plan to update them in any special ways. The better, but still not good anough word for those is "state". The diffrent states are just some conditions depending of which we treat the peer diffrently.

"alive" is the standard state of peer. Only "alive" peer is used for synchronization.

"dead" peer is a peer that we failed to connect to recently. We informed others about that. (this is a possible suspect to change as the peers sync systems I haven't decided upon. I'm open to some thought-out proposals)

"known" that we failed to connect so long time ago we stop to inform other we failed to connect to it, but we still keep it in memorry with all of it's statistics (speed and trustworthyness) in case it comes back online. We will be informed about that by others or by the peer itself.

"delete" state is the condition for removal from our storage all information about the peer as we think it's not going to come back online.

What is enough the definition of GoodPeer? What is N?

N has to be decided. N may be a percentage. The idea is a GoodPeer is a peer that has a fast connection and according to our experience it tends to give good (up-to-date) correct chain.
GoodPeer is used for initial sync so it's fast.

How does using Peers ensure the longest chain?

To ensure we get the longest chain we ask random known to us Peers about the block. As oposed to asking only those with good connection. Generally my idea is we use GoodPeers to achieve
fairly good sync and then we asked some random peers if they have anythning better (a longer chain). This way we get a faster sync by asking peers with good connection, but we don't lose security by relaying on peers with good connection.

Where do we get the initial list of FastPeers to query?

Where do we get the initial list of peers?

That's the point I realized someone changed GoodPeer to FastPeer in my text and I got confused. To everyone FastPeer=GoodPeer=definition in proposal Like in every (I think) P2P system initial list of peers has to be hardcoded or downloaded from somewhere. After first execution of client system will self regulate and generate new lists based on responses from network.

How many (Fast)peers should we ask for?

How many peers is enough in light mode?

The number of peers to fetch in step 1 has to be decided on. It has to be big enought. Probably something like 100 is good. The same applies for step 4 and 5.

When exactly do we continue? Why are headers low-trust and why fairly up to date? What headers do we ask for? Are these headers for the top block? How are they different from step 6 then?

By "continue" I mean: proceed to next step. In step 2 we fetch from GoodPeers/FastPeers all the headers we don't have that are on the longest chain. I call those "low-trust" or "fairly up-to-date" becouse we use peers with good network connection to get them. Peers with good network connection I don't consider random and consider them a possible source of manimulation. Some big company/wealthy person may get many very-very fast nodes and try to influence what data do other peers get, most probably by providing some short fork. Becouse of that we can only establish a trustworthy information about longest chain in slower (if used for a lot of headers) step 6.

Please clarify:

We use trust only for the initial download. Asking random Peers for the top block is used to build trust that's used later when we do the initial sync. It's usefull to have as good as possible initial sync so asking random Peers will result in the least new data possible.

Sorry, that sentence is a little bit chaotic. When we ask other peers about top block we store that information and when we establish information what's the true top block we make trustworthyness of peers that answer correctly higher and those that where significantly off lower. High trustworthyness peers are only used to get as good as possible initial sync in future. Low trustworthyness peers may be ignored as they provide false information. (Please note "may be", I'm not sure about that idea. Is seems not needed.)

Can you detail the force sync flow, for the freshly mined block?

You mean the Gossip protocol? Gossip protocol will be used for spreading freshly mined block. It's described in the docs in "Gossip" PR on aeternity/testnet. Gossip protocol will use random peers as defined in this document.

When setting up some default numbers of random peers it should be relative, not hardcoded

Hmm, I'm not sure it's important. We can assume more then some fixed percentage of network is honest. If we have 100 trully random Peers there is a (high) fixed chance we hit any of those nodes no matter the actuall size of the network. (The chance doesn't change when network grows). Could you describe why do you think a fixed number of random peers to ask (for example) for the top block isn't good enough? Is my explanation clear or maybe I should provide some numbers?

Re. point 10: you want to enable/disable API based on "state"? Lets approach it in a bit different way (or express it in different way). Node can start listening on API when its operational and it is operational after the first sync. Turning APIs on/off during regular operation is never a good idea

I think we are thinking the same. External API is turned on when the node is synchronized with network and doesn't get turned off (obviously except when turning off whole node). Lite-node doesn't expose external API.

I consider the node synchronized with network and operational only after step 9.

The API to exchange information has to be granular and extendable. You mention the exchanges of blocks and headers. Also advertising peers. Design the api in a way that it takes equivalents of maps. If we want to pull more data to sync it will be easy to add (like more transactions that are not mined)

I think I'm not familiar with some concept that you are reffering to. (Maybe I don't know something) Please describe your concern.

I understand that when the node starts for the first time it will have one (or a bunch of) initial peers, so it shall read "Asks random FastPeers - or initial Peer specified by configuration if none known - ..."

You understand correctly. I consier the first sync a special situation. I will add some paragraph about diffrences in it.

If it "Asks ... for some RandomFastPeers", does it mean that the network call returns not only the network connection info but also the statistics e.g. trust and speed/size?

Yes, but probably only speed. Also as pointed out in the "notes and issues" this may be something that can be exploited and we shouldn't trust the returned information too much.

Assuming block header validation is in this (2) step, considering that node aims at identifying "the" longest chain, how does node acquires the headers ("-> low trust (fairly up-to-date) headers")? Does it first ask for top header hash+height from a certain number of peers, then picks any one chain at greatest height and validates it sequentially, resulting in one chain of block headers that are valid?

Yes, you are correct. One chain is good enough, becouse it's very-very-very expensive to create a long chain, becouse of the required proof of work, so the aquired chain shouldn't be very different from the truelly longest one.

Assuming block header validation is in this (2) step, considering that validation of block header depends on previous block headers for (1) validation of difficulty, (2) presence of block header identified by previous hash, (3) block header leading to genesis, does node parallelize computations?

Due to the nature of our blockchain node may have limmited ability to parallelize verification of headers. I didn't investigate possibilities of parallelizing verification of headers. It should be investigated, but I don't think that's important for mainnet readyness.

[f] fast blocks (full only) - Asks RandomFastPeers about blocks. When we have all we continue -> blocks for the low trust headers

This step is listed as being for both full and light mode of operation. Does it mean even in light mode all blocks shall be downloaded?

You cited a step with "(full only)" in it's description. By that I mean it will only by executed an a full-node / in full mode.

[cytadela8]

I think I answered all of the questions. If something is unclear I will provide further explanation of my ideas. I'm open to changes.

[joelreymont]

@cytadela8 Thanks for the time you put into writing this and clarifying your intent.

I'm going to close this ticket as too complex. All we need for the foreseeable future is

1. A list of peers where each peer is stamped with last successful network operation (last).
2. A scavenger process to remove peers that we haven't successfully interacted with for more than T minutes or seconds, as set by a constant.

Keeping track of peer speed, for example, is an optimization that we can make at a later time. Same for other features of this proposal. Please remember to keep it simple!

[cytadela8]

@wagerlabs I agree parts of this design aren't required for MVP, but whole thing as a principle I would keep. I think it's important to keep in mind the bigger picture when coding MVP. I consider the difference between this design and one for MVP to be:

• don't implement all "fast..." steps
• don't implement other things concerned with speed
• don't implement trustworthyness or have minimal implementation
• simpler peer lifetime as you described

[joelreymont]

Let's implement the 1 and 2 from my comments. That's all we need for now and there are other tasks to work on.