tox-docs / text Goto Github PK

How does tox decide if it should use TCP or UDP to contact a peer? When does it use a TCP replay to contact it indirectly? Maybe these questions will be answered in some of the docs that haven't been written yet?

Related question: Does tox ever use only one TCP node to relay all its traffic?

The purpose of the onion routing is to hide the IP of the node that's advertising itself. However, when a node (A) advertises itself to what I'll call an introducer it sends it its public key. This introducer node now can pretend to be attempting to add A to its friends list because it knows its public key. The only protection is the nospam, which is 4 bytes. The size and randomness of the nospam is the only thing protecting A from having its IP discovered by a peer. This is not obvious from the nospam name. I think this property needs to be documented more clearly somewhere. Maybe this should go in the not-yet-written DHT documentation?

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L40

Comment on

replayed response

found at column 626.

Can you explain what effect on the system would be if the ping id was absent?

https://github.com/Tox-Docs/Text/blob/master/src_text/net_crypto.txt#L3

Comment on

this information is known

found at column 1074.

Where is this information stored?

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L18

Comment on

Only the UDP numbers (2 and 10) are used in the DHT module.

found at column 1.

This effectively means: the protocol is always UDP (0), and only the address family can change.

https://github.com/Tox-Docs/Text/blob/master/src_text/net_crypto.txt#L20

Comment on

Random nonce

found at column 2.

What random source is this nonce generated from?

Please add some line breaks before [ in packet specifications. It's little effort and helps readability a lot.

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L73

Comment on

NAT ping packets:

found at column 1.

If I understand correctly, the packet structure below is in fact the payload that sits inside the encrypted part of the DHT request packets. Please clarify that.

https://github.com/Tox-Docs/Text/blob/master/src_text/net_crypto.txt#L22

Comment on

[Sender's real public key (32 bytes)]
[Sender's DHT public key (32 bytes)]

found at column 0.

Why is this information sent? Earlier in the document we established that they are already known.

https://github.com/Tox-Docs/Text/blob/master/src_text/TCP_server.txt#L9

Comment on

must wait for a sign that the client received his handshake

found at column 599.

How long does it usually need to wait for this sign?

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L54

Comment on

8 nodes closest to each of the public keys in its DHT friends list

found at column 63.

Can these n (n=friend list size) lists of 8 nodes overlap with each other? Can they overlap with the 32 nodes?

The repository README.md mention WIP, but for what does WIP stand for?

https://github.com/Tox-Docs/Text/blob/master/src_text/TCP_server.txt#L17

Comment on

packets sent to us

found at column 244.

The word "us" seems to mean two different parties at different points in the doc. Please change it to use consistent terms for things and don't overload terms.

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L48

Comment on

good (alive) nodes

found at column 109.

Can you describe what good/alive means exactly? I see you described it further down. If possible, try to define words before their use. If it's impossible, e.g. because there is a mutual dependency (should occur rarely), add a note that the word is explained in more detail later, in section XXX.

The current folder structure contains an src_txt, md and latex folder, but all documents are under the src_txt folder not under md. Why you choose the .txt file format for the documentation?
I mean the .txt files are hard to read and md would be easier to read. This leads to the second question: what is the long term plan for this documentation? Is there any plan to convert the .txt files in .md files? Is there any plan to make a latex documentation?

https://github.com/Tox-Docs/Text/blob/master/src_text/TCP_server.txt#L11

Comment on

send data to friends who may not know that we are connected to the current TCP server while we know they are

found at column 650.

In what situation does this occur? Also, can this in theory be used for all communications, and the data packets are an optimisation, or are there limits to what can be sent via OOB (apart from the size limit)? It might make sense to increase that limit so that TCP clients can choose to use the less efficient OOB method to establish a connection once it runs out of connection IDs. It can later, when another connection is closed, establish a real connection and continue there.

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L54

Comment on

ping response of send node packet

found at column 597.

What is a ping response of a send node packet?

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L48

Comment on

that the node receiving the get node knows

found at column 128.

Can you describe what "knowing" means exactly? What list are the nodes selected from?

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L16

Comment on

the numbers on my machine for ipv4 and ipv6

found at column 222.

What numbers are these? Where did you find them on your machine?

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L28

Comment on

Get nodes (Request):

found at column 1.

Move these packet structure docs down to sit together with the description of their semantics.

https://github.com/Tox-Docs/Text/blob/master/src_text/net_crypto.txt#L12

Comment on

If it was received from a TCP OOB packet it must be sent back by a TCP OOB packet via the same relay with the destination being the peer who sent the response.

found at column 207.

You mean the peer who sent the request?

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L54

Comment on

a random node

found at column 293.

Why is randomness used here instead of an order?

https://github.com/Tox-Docs/Text/blob/master/src_text/net_crypto.txt#L24

Comment on

number (must be sent back untouched in cookie response)

found at column 15.

Ensure that names are consistent. E.g. ping_id, or echo_id vs. "number". Unless this number actually serves a different purpose, in which case there may be a better name than "number".

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L77

What do you mean when you use the term "friend" in the context of the DHT? I thought that friend public keys are not DHT public keys, so there are no "friend" nodes in the DHT. Also, communicating to a friend requires finding their ip/port etc. over the onion protocol, doesn't it? I just don't understand the connection between the DHT and the concept of friends.

https://github.com/Tox-Docs/Text/blob/master/src_text/TCP_server.txt#L15

Comment on

following 128 bytes of data

found at column 61.

Can you give this packet a name? Most packets have names, like "ping request". It makes it easier to point at them in other places in the docs.

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L54

Comment on

Toxcore stores the 32 nodes

found at column 1.

Why 32? Why 8? For every number, please add a reason for it, and a reasonable range for other implementations. Describe the effect a change to those numbers has on the distributed system and on the node itself.

I did some calculations to see how well the current DHT algorithm scales in terms of bandwidth. I based the calculation on this:

x = (32 max nodes in DHT + 8 nodes close to friends * num_friends)
x * 2 * ping(66) + x / 3 * (getnodes(113) + sendnodes(ipv4 238 ipv6 286)) = ? bytes/s
The first term is the pings per second

For 0 friends with all ipv4 nodes:
632 Bps

For 0 friends with all ipv6 nodes:
708 Bps

For 100 friends with all ipv6 nodes:
18428 Bps -> 18 kBps

For 100 friends with all ipv4 nodes:
16432 Bps -> 16 kBps

For 1000 friends with all ipv4 nodes:
158632 Bps -> 158 kBps

The assumption that all your friends have unique 4 closest friends is reasonable if the public keys are uniformly distributed (they better be) and your friends make up a small percent of the network. Someone should do the math to figure out the exact number taking into account probabilities, etc.

My conclusion from this exercise is that the DHT could use tuning if we expect users to have a lot of friends, but it's perfectly fine for up to 50-100 friends right now. I don't think most users have more than 10 actually. I think the network traffic usage would be a useful metric for ToxCore to expose for debugging purposes and so we can check how close to reality these numbers are.

https://github.com/Tox-Docs/Text/blob/master/src_text/net_crypto.txt#L14

Comment on

simple packet flood

found at column 405.

How many keys does an attacker need to pre-generate to do an effective DoS on a single user?

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L21

Comment on

Ping(Request and response):

found at column 1.

All the packets below have a common structure that is not clear from the description. Every DHT packet looks the same, but with a different payload. In the case of a ping packet, the payload is the ping id. Repeating the entire DHT packet structure hides this commonality.

https://github.com/Tox-Docs/Text/blob/master/src_text/TCP_server.txt#L17

Comment on

base nonce we want the server to use

found at column 192.

What is the reason for a base nonce? Also, can you explain what exactly a base nonce is?

What actual handshake packet? There has been no previous mention of a handshake packet, why it would be needed, what its purpose would be. This section needs a heading and an introduction.

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L42

Comment on

This is then followed by the DHT public key of the one sending the response and a random nonce.

found at column 43.

This redundancy is unnecessary if you abstract out the "DHT packet format" and just talk about packet ids and payloads.

what should other implementations do?

Toxcore has a time out of 15 seconds for cookie packets.

^ similar for that

https://github.com/Tox-Docs/Text/blob/master/src_text/net_crypto.txt#L12

Comment on

The cookie response must be sent back using the exact same link the cookie request packet was sent from

found at column 1.

Why? Is it possible to send it via another link? What happens if you do?

https://github.com/Tox-Docs/Text/blob/master/src_text/net_crypto.txt#L27

Comment on

Encrypted message is encrypted with sender's DHT private key, receiver's DHT public key and the nonce.

found at column 1.

Please avoid this redundancy by abstracting out the cookie communication protocol basics. Redundancy means I need to manually check that the same keys are used for both packets, because I must by default assume they are different.

We came up with some suggestions with subliun. I'm putting this link here to make sure you see it:

https://docs.google.com/document/d/1k2ip7P3Nwh6C4ujubr4Edx_E1ShDzexgyHMJNF9CXeI/edit

https://github.com/Tox-Docs/Text/blob/master/src_text/TCP_server.txt#L17

Comment on

encrypted with the private key of the client

found at column 76.

Is it encrypted with the DHT private key or the long term private key?

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L81

Comment on

are on 3 types of NATs

found at column 52.

I assume they are only on 1 type of NAT at a time. Please clarify.

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L16

Comment on

family numbers

found at column 367.

TCP/UDP are not a family. TCP6 is not a family. IPv4 is a family and TCP is a protocol. See https://www.iana.org/assignments/address-family-numbers/address-family-numbers.xhtml.

what does this mean?

Some sentences and paragraphs where I can't work out what you mean due to grammar/ambiguity:

• line 163 ambiguous (does it contain sendback data AND a nonce?).
• line 189 Use of equal needs clarification in this paragraph (particularly first sentence). It's hard to know what key you're referring to at each point.
• line 205 bit explaining information for handling response needs to be clarified

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L92

Comment on

This is because since we send ping requests to every peer that is closer than what we have in our list it means we have sent at least 4 ping requests to those ip/ports but not received a response.

found at column 202.

This sentence is too long, and broken. I don't understand it, please restructure.

vague. consider revising.

https://github.com/Tox-Docs/Text/blob/master/src_text/net_crypto.txt#L12

Comment on

TCP OOB packet

found at column 233.

Are TCP OOB packets used in toxcore? If yes, for what?

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L12

Comment on

2 == IPv4, 10 == IPv6, 130 == TCP IPv4, 138 == TCP IPv6

found at column 18.

I suggest talking about bits here instead of numbers. You have a bit 0/1 for the protocol, then 3 bits nothing, and then 4 bits that can be 0b0010 or 0b1010 for the address family.

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L60

Comment on

The toxcore behaviour can be improved as it is currently very basic and there is room for much improvement.

found at column 1.

This sentence adds no information and can be removed.

https://github.com/Tox-Docs/Text/blob/master/src_text/net_crypto.txt#L5

Comment on

work over UDP it must account for possible packet loss, packets arriving in the wrong order and has to implement some kind of congestion control.

found at column 30.

A TCP- like implementation not at the network level but at the net_crypto level? Which level is this packet loss coping mechanism implemented at?

-Line 16 - Unsure how clear it is how family numbers are presented in packed node (may be my own lack of knowledge)
-Line 16 -What sort of numbers on your machine represent 2 and 10? Are they used on other machines? A better reason for the document might look better.
-Line 42 - In last sentence think you mean the ping id that was sent in the ping request (in which case it should be THE 8 byte ping id) but didn't change as uncertain.
-Line 56 - Needs criteria for adding nodes to lists (good to make it obvious).
-Line 96 - Probably should outline how you check the NAT ping request is from a friend.

I haven't read all of it yet, but Syncthing seems to be very well documented. Here's their protocol documentation:

• https://github.com/syncthing/specs/blob/master/DISCOVERYv2.md
• https://github.com/syncthing/specs/blob/master/BEPv1.md

If you know of other examples of good protocol documentation please post some more here. Bonus points of the protocols are similar.

https://github.com/Tox-Docs/Text/blob/master/src_text/DHT.txt#L34

Comment on

Send_nodes (response (for all addresses)):

found at column 1.

Ping (I'd like it to be called Echo, because that's exactly what it does: it replies with the exact same thing it received) and GetNodes are in concept an RPC service. I would like to see that reflected in the docs. Instead of a "GetNodes" and "SendNodes", make it a Nodes (RPC) service with a NodesRequest and a NodesResponse. This provides the reader with a higher level concept rather than just the packets. It also simplifies naming.