In-progress version of draft-ietf-quic-multipath
License: Other
This is the working area for the IETF QUIC Working Group Internet-Draft, "Multipath Extension for QUIC".
See the guidelines for contributions.
Contributions can be made by creating pull requests. The GitHub interface supports creating pull requests using the Edit (✏) button.
Formatted text and HTML versions of the draft can be built using make.
$ makeCommand line usage requires that you have the necessary software installed. See the instructions.
The draft is using old style references, such as "section 9 of {{RFC9000}}". Version 3 of XML2RFC defines a structure way to do that, such as: <xref section="9" sectionFormat="of" target="RFC9000"/>, which can be expressed in recent versions of Markdown as {{Section 9 of RFC9000}}. The new style references don't change the text version much, but in the HTML rendering they generate direct references to the target section, which is nice for the readers.
We mention in the introduction that if a node uses non null CID, it must use different CID for different paths. This is true whether the peers negotiated single or multiple packet number spaces. The text about connection ID in the introduction of the multiple PN space section should be moved to a common section.
I have a sentence in the intro that this draft provide some basic scheduling guidance. I was think about something like: Data belonging to the same stream should be sent over the same path. If a new stream is opened, a packet is selected based on the current load (e.g. number of stream/max congestion window). Or something else?
Or is the plan to really not say anything here?
Based on the discussed from the side meeting, we need to
While discussing single packet space vs multi packet space designs, @martinduke brought up the topic of hardware offloading. The effect of any design changes on packet encryption/decryption should take HW offloads into account. As I understand it, single packet space would not modify the packet encryption/decryption logic, but the multi packet space design would because of the difference nonce length. Additionally, since multi-path would be a negotiated feature/extension, that would mean all connections that don't negotiate the feature would have the "old" model/logic, where as the connections that do would have the new encryption/decryption logic. IMO, this could significantly complicated HW offloads.
We need some description for how to init new paths in this section.
@boucadair flagged in his PRs that there are many places using hyphenated "multi-path" and many using "multipath". We need to fix that.
In the "path abandon" frame, the path identifier is defined as:
Path Identifier {
Identifier Type (i) = 0x00..0x02,
Path Identifier Content (i),
}
But in the identifier type 2, the "path identifier content" is empty. We should fix that.
The text about computation of RTT was copied from the "simple option" draft to section 6.2.2. Computing Path RTT, but the issues around RTT computation are common to both single number space and multiple number spaces. Maybe move that text to a new section 6.5?
I opened issue to make sure we don forget to address this and will close #4 instead.
RFC9000 is cited massively in the draft. It would be helpful to indicate to the reader where to look by indicating the section.
Some change examples are provided below:
OLD:
`We assume that the reader is familiar with the terminology used in {{QUIC-TRANSPORT}}.`
NEW:
`We assume that the reader is familiar with the terminology used in {{QUIC-TRANSPORT}} (Section 1.2, in particular).`
OLD:
`RTT measurements and PMTU discovery should be per-path (following {{QUIC-TRANSPORT}})`
NEW:
`RTT measurements and PMTU discovery should `be` per-path (Section 14 of {{QUIC-TRANSPORT}})`
and so on.
Currently, the multiple packet number spaces proposal relies on the sequence number of the Connection ID when a non-zero connection ID is used. With non-zero-length Connection IDs at both ends, it is indeed possible that over a same path/5-tuple, the client reaches the server using a DCID with a sequence number that is different from the sequence number of the DCID used by the server to reach the client (e.g., Figure 2). In the current draft, we consider that if the connection ID is zero-length, the packet number space identifier is 0, and applying multiple packet number spaces is thus not possible.
I believe we can do better than this. Consider the situation where the client uses a non-zero length DCID to reach the server while the server uses zero-length DCID. In this case, it is the client that initiates the usage of a path with path validation, associating a server-provided non-zero-length DCID to reach the server. Once the first client's packet reaches the server on the new path, the server can associate the client's 5-tuple to the sequence number of the used Destination Connection ID, making a valid Packet Number Space Identifier. The client can easily map incoming packets on the 5-tuple to the associated DCID sequence number.
Does this sound reasonable?
In Section 2 you define a transport parameter and set of values. These are presented in a table with a column header "Allowed server responses", what does that mean? One way to read this is that it implies servers would tailor their Transport Parameters to the value sent by the client but that's conflicting with our philosophy that TPs are unilateral declarations, not active negotiation.
So the question for me is, is this just a wording issue or a design one?
In RFC9000 path migration is restricted to clients only, mainly because of problems with NATs which a) is the main use case to have migrations (address change at the client only) and b) could anyway cause failure for mitigation attempts from the server.
However, with multipath, failure of opening a new path is less critical as the old path(s) is not abandon at the same time. Therefore we should further discussion if that restriction is still technical useful or if it can be released easily.
This issue was initially discussed as part of issue #31
To estimate path delay accurately, loss recovery of QUIC utilizes the ACK Delay field of ACK frame (see RFC 9002; section 5.3.2).
However, when single packet number space is being used, it is difficult for the sender to obtain ACK Delay for all the paths being used, because ACK Delay is reported only for the packet carrying the largest packet number. It is the only the ACK delay of the path with shortest path-delay that can be reported.
I think it might be a good idea to acknowledge this shortcoming in the documentation, so that we do not miss the downside when we compare the approaches (or when we try to address the shortcomings of the single packet number space approach).
As we have 0-RTT transmission in QUIC-TRANSPORT, and path validation brings at lease one round-trip-time delay for data to be sent on the new path, can we add some mechanism to provide anti-amplification limitation and support this scenario?
In our testing, one thing worth noticing is that for different sim cards, the uplink/downlink ratio can be quite different. For example, with a China Mobile 4G card, we measured a downlink at 35Mbps and an uplink at 3Mbps, while with a China Telecom 4G card, we measured a downlink at 45Mbps and an uplink at 23Mbps at the same time in the same spot.
If we want to measure RTT more precisely, we let ACK return from the same path. However, because some sim cards are more asymmetrical than the others, it introduced a question, do we want ack frequency control per path?
In the QUIC ack-frequency draft, one of the motivations is to handle link asymmetry such that the link throughput won't be constraint by ack packets in the reverse direction. My understanding is that some 5G IoT cards can be more asymmetrical, so do we want to add a mechanism to address this issue? What do you think?
The current draft says that "Servers SHOULD stop sending traffic on paths through where no non-probing packet was received in the last 3 path RTTs". However, it is possible that only a few data was generated in the past few RTTs due to app-limit conditions, and as a result, only one path was used. So I think, in section 3.2.3, we should suggest the use of some keep-alive mechanisms such as sending ping frames on a particular path if one is interested in keeping that path alive. Let me know what you think?
The sentence above is part of the introduction, but the should is misleading for congestion control at least (we have a MUST for per-path congestion control later).
I would suggest to split the above sentence in two parts:
We should probably list somewhere in the document the state that must/should be maintained on a per-path basis and the state that is related to the connection. This could clarify some of the mechanisms.
Concerning PathMTU, I would suggest to stay with a should and not a must since an implementation could simply track the minimum MTU among the different paths without maintaining per-path MTU. Such an implementation would work and if the MTUs do not differ significantly, it would probably be simpler than an implementation than maintains per-path MTU and needs to adjust the packets based on the path where each packet is sent.
Concerning RTT, we discuss the problem of measuring the RTT later with the packet number spaces discussion. Furthermore, some congestion control schemes need an accurate per-path RTT while others do not strictly require rtt estimations.
This draft initially originates from a merging effort of previous Multipath proposals. While many points were found to be common between them, there remains one design point that still requires consensus: the number of packet number spaces that a Multipath QUIC connection should support (i.e., one for the whole connection vs. one per path).
The current draft enables experimentation with both variants, but in the final version we will certainly need to choose between one of the versions.
The draft states that
For example, assuming the IV value is 6b26114b9cba2b63a9e8dd4f, the connection ID sequence number is 3, and the packet number is aead, the nonce will be set to 6b2611489cba2b63a9a873e2.
Shouldn't be the result 6b2611489cba2b63a9e873e2 instead of 6b2611489cba2b63a9a873e2?
I copied this sentence from draft-liu:
Before an endpoint adds a new path, it SHOULD check whether there is at least one unused available Connection ID for each side.
Shouldn't this be a MUST instead of a SHOULD?
The current text on the defintion of ACK MP says:
ACK_MP frame is formatted as shown in Figure 6.
ACK_MP Frame {
Type (i) = TBD-00..TBD-01 (experiments use 0xbaba00..0xbaba01 or 0x42..x43),
Packet Number Space Identifier (i),
Largest Acknowledged (i),
ACK Delay (i),
ACK Range Count (i),
First ACK Range (i),
ACK Range (..) ...,
[ECN Counts (..)],
}
The figure is too wide, and I am fixing that with issue #39, but really we should pick just one set of identifiers, not 2. I understand that draft-liu and draft-deconinck were using these two values, but we should pick one of them, or a third one.
In order to be consistent with RFC9000, use "QUIC version 1" instead of "QUIC v1" through the document.
In the current draft, we are having two names of multi-path ACK. I think we should stick to just one.
QUIC defines different ways that enable a host to control the amount of data that the other host can transmit at the connection level (MAX_DATA) and at the stream level (MAX_STREAMS and MAX_STREAMS_DATA).
The MAX_STREAMS and MAX_STREAMS_DATA relate to the connection and should not be affected by the fact that it uses multiple paths.
For MAX_DATA, the situation is a bit different. There are deployments and applications that would like to restrict the amount of data or bandwidth that is sent on a given path. A typical example is a smartphone that does not want to use too much data on cellular but has no restriction on WiFi. As in MPTCP, we need a connection-level MAX_DATA that limits the amount of data which can be in flight. In MPTCP, this corresponds to the receive window and the same window is advertised over all subflows. We could not do better than this within the IETF.
In MPQUIC, there is an opportunity to send a per-path MAX_DATA to restrict the utilization of a path or define other control frames to specify the maximum allowed bandwidth per path.
We might discuss whether we should include this functionality in version 1 or leave it to another draft.
In 3.2.3, we have the following paragraph:
When more than one path is available, servers shall monitor the arrival of non-probing packets on the available paths. Servers
SHOULD stop sending traffic on paths through where no non-probing packet was received in the last 3 path RTTs, but MAY
ignore that rule if it would disqualify all available paths. To avoid idle timeout of a path, endpoints can send ack-eliciting packets
such as packets containing PING frames Section 19.2 of [QUIC-TRANSPORT] on that path to keep it alive. Sending periodic
PING frames also helps prevent middlebox timeout, as discussed in Section 10.1.2 of [QUIC-TRANSPORT].
This paragraph implicitly indicates that we consider a path to be active if we send and receive packets on this path. This definition works well for MPTCP where each data packet triggers an ACK on the same path. In MPQUIC, a server could have two paths that it uses to send data. However, the client might decide to always send the ACKs on only one path (e.g. the lowest delay one). In this case, the two paths are clearly active, but the server could consider the higher delay path to be inactive since it does not receive packets over this path. It could send PING frames to trigger data packets, but then it would need to send one PING every 3 RTT, which seams excessive.
I would suggest to rewrite the text as follows (I removed Server, because I think the recommendation should be generic):
When more than one path is available, hosts shall monitor both the arrival of non-probing packets and the acknowledgements
for the packets sent over each path. Hosts SHOULD stop sending traffic on paths whether either: (a) no non-probing packet
was received in the last 3 path RTT or (b) no non-probing packet sent over this path was acknowledged during the last 3 path
RTT, but MAY ignore that rule if it would disqualify all available paths. To avoid idle timeout of a path, endpoints can send
ack-eliciting packets such as packets containing PING frames Section 19.2 of [QUIC-TRANSPORT] on that path to keep
it alive. Sending periodic PING frames also helps prevent middlebox timeout, as discussed in Section 10.1.2 of [QUIC-TRANSPORT].
There are many isntances in the markdown source where a whole paragraph is entered in a single line of text. This makes the reviewing harder, and messes up the "what changed" view when reviewing PRs.
While separate packet number spaces allow for more efficient ACK encoding, especially when paths have highly different latencies, this approach requires the use of a connection ID
Not sure to understand why the last part of the sentence is an issue. The use of zero-length connection IDs is problematic anyway in setups with NATs, for example.
Due to the copying&pasting from different documents, we use sometimes client/server and sometimes endpoint/peer in the draft. If I understand everything correctly, both endpoint can open (start validation) and close paths, correct?
There I would propose to remove all instances of client/server from the draft (expect for the handshake negotiation part). I can do that but wanted to check first if that's the correct thing to do
Should we trigger an connection closure with a MP_CONNECTION_ERROR in the case an endpoint receives an ACK_MP frame acknowledging packets on a non-existing packet number space ID?
Scheduling and re-ordering mechanism are usually out of scope of a multi-path standard and is up to implementers and/or separate IETF work. However, there might be the wish to align between sender and receiver the priority and usage of paths. Multi-path protocols like MPTCP and MP-DCCP, see this in scope and define therefore a MP_PRIO option.
So far, MP-QUIC does not define anything in this direction.
Beyond path scheduling MP-QUIC also offers stream multiplexing, where it can be useful to have a prioritization indication for stream schedulers at both ends. Per https://datatracker.ietf.org/doc/html/rfc9000#section-2.3, stream prioritization is so far not not part of QUIC v1.
Define a prioritization exchange for paths and streams within the MP-QUIC draft.
This facilitates scenarios where both sides should apply the same path/stream prioritization (e.g. trigger a handover or respect cost) or the information about the sender prioritization is useful for e.g. re-ordering mechanisms on receiver side. While this can be applied in end-to-end scenarios it will be in particular useful for Hybrid Access and 3GPP ATSSS.
Btw. that was the original question I intended to ask in #81 , but probably misleading 😀
If we want an MP specific error type we need to also add this to the IANA section.
I took the following text from draft-liu:
Note: Even if use of multiple packet number spaces is negotiated but a peer uses an zero length connection ID, then all packets sent to that peer MUST be numbered in a single number space (as specified in the previous section), because the packet level decryption implementation will only see one Connection ID sequence number (the default number 0).
This basically makes the one packet number handling mandatory to implement as you also need it if multiple packet number spaces were negotiated but zero length CID is used.
The alternative would be to say that you can't used zero length CID if multiple packet number spaces are negotiated.
Also I'm currently proposing enable_multipath=1 for multiple number space and enable_multipath=2 for one number space to be compatible with the draft-liu and the alibaba implementation. However if we think one packet number space is mandatory to implement, it would make more sense to have it the other way around and allow fallback to one packet number space, namely if the client ask for two packet number spaces but the server supports only one but would also lead to a successful negotiation.
What do people think about this?
Consider adding this entry to the terminology section:
- Path: refers to the 4-tuple {source IP address, source port number, destination IP address, destination port number}. A path refers to "network path" used in {{QUIC-TRANSPORT}}.
This also clarifies the link with "network path" in RFC9000.
The current draft says "Servers SHOULD stop sending traffic on paths through which non-probing packet was received in the last 3 path RTTs, but MAY ignore that rule if it would disqualify all available paths". First, I believe there is a typo, where the "non-probing packet" should be "no non-probing packet". Second, 3 RTT is too short for our applications and we prefer a longer timeout. Maybe we want to add a new parameter (e.g. max_path_idle_timeout) to allow the server and client to negotiate?
As RFC9000 provides the handling of streams, I wonder if a MP-QUIC has to consider this as well, at least in a section similar to the exisiting "Packet scheduling" one?
To become more precise, assuming within one MP-QUIC connection the availability of different streams with different demands. Maybe one stream is "best effort", while another is latency critical and expects in-order devilery. In a multipath scenario using a DSL and a cellular access, there might be the wish to send the latency critical stream through DSL prioritized over other streams, while the "best effort" stream then keeps access to the remaining DSL capacity + the cellular capacity.
RFC9000 defines a list of transport parameters.
0x01 | max_idle_timeout | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x02 | stateless_reset_token | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x03 | max_udp_payload_size | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x04 | initial_max_data | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x05 | initial_max_stream_data_bidi_local | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x06 | initial_max_stream_data_bidi_remote | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x07 | initial_max_stream_data_uni | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x08 | initial_max_streams_bidi | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x09 | initial_max_streams_uni | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x0a | ack_delay_exponent | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x0b | max_ack_delay | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x0c | disable_active_migration | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x0d | preferred_address | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x0e | active_connection_id_limit | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x0f | initial_source_connection_id | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG] |
0x10 | retry_source_connection_id | permanent | [RFC9000, Section 18.2] | 2021-02-11 | IETF | [QUIC_WG]
These transport parameters apply to all the paths that compose a Multipath QUIC connection. We should probably confirm this in the mpquic draft to avoid any misinterpretation (e.g. for max_idle_timeout or ack delays). There are two transport parameters that could warrant some discussion:
disable_active_migration : This transport parameter is incompatible with enable_multipath. An endpoint that receives both this transport parameter and a non zero enable_multipath parameter MUST treat this as a connection error of type
MP_CONNECTION_ERROR and close the connection.
preferred_address: RFC9000, section 9.2.1 contains the following:
Once the handshake is confirmed, the client SHOULD select one of the two addresses provided by the server and initiate path
validation (see Section 8.2). A client constructs packets using any previously unused active connection ID, taken from either the
preferred_address transport parameter or a NEW_CONNECTION_ID frame.
As soon as path validation succeeds, the client SHOULD begin sending all future packets to the new server address using the
new connection ID and discontinue use of the old server address. If path validation fails, the client MUST continue sending all
future packets to the server's original IP address.
For multipath, do we want to be stronger than SHOULD select one ? Can a client create news paths towards the initial address or only to one of the preferred addresses ? Should the client use both addresses (IPv4 and IPv6) if non-empty ?
I believe stand-by (or backup) has been useful in MPTCP. This is not really about scheduling because all it says is: I won't use this path actively(yet) but please leave it open.
There is one part I am afraid people might feel a little bit hard to follow: the three types of path identifiers in the PATH_STATUS frame. Those three types of path identifiers were introduced in order to be compatible with zero-length CID use cases, so I think we might need two additional example figures (like figure 3 in the current draft), one for (non-zero length CID, zero-length CID) and the other one for (zero-length CID & zero-length CID). Do you think that is a good idea? I can work on those figures if it doesn't complicate the flow of the draft.
With a packet number space per path, the ACK_MP frame provides the per path ECN feedback that enables a sender to detect congestion using ECN on a specific path. The per-path congestion control can adjust the congestion window based on the received feedback.
With a single packet number space shared among the different paths, the ECN fields of the ACK frame provide ECN information that is common to all paths. This makes it impossible for a sender to determine which path is affected by congestion when it receives an ACK frame with positive ECN counters. A per-path congestion control cannot easily react to such an ambiguous ECN feedback.
In section 3.1, we wrote
When the multipath option is negotiated, clients that want to use an additional path MUST first initiate the Address Validation procedure with PATH_CHALLENGE and PATH_RESPONSE frames described in Section 8 of [QUIC-TRANSPORT]. After receiving packets from the client on the new paths, the servers MAY in turn attempt to validate these paths using the same mechanisms.¶
With multipath, we cannot simply write that the server MAY validate the new path. There is a risk of a malicious client that operates as follows:
We probably need to start working on the security considerations section of the draft.
I wonder whether this behavior would nullify the activation of multipath in some cases (e.g., backup purposes or when MP_ACKs are not sent over the same path).
Servers
SHOULD stop sending traffic on paths through where no non-probing
packet was received in the last 3 path RTTs, but MAY ignore that rule
if it would disqualify all available paths.
The implication on the client should be discussed, especially that the I-D says:
ACK_MP frame Section 10.2 can be returned via either a different
path, or the same path identified by the Path Identifier, based on
different strategies of sending ACK_MP frames.
(BTW, s/ACK_MP frame Section 10.2/ACK_MP frame (Section 10.2)
Given that paths are initiating/validating/active/closed, it looks like we will need at some point to specify some path state (finite state machine?) and what behavior is acceptable in each of them (e.g., we cannot send packets on a closed state,...).
When negotiating the use of a single packet number space, it is not clear whether the use of ACK_MP frame is acceptable or not. Should we only stick to ACK frames and consider reception of ACK_MP frames to be MP_PROTOCOL_VIOLATION error? Or should we tolerate their reception if the packet number space identifier is 0?
When I try to produce the draft, I get a number of warnings:
PS C:\Users\huite\GitHub\draft-lmbdhk-quic-multipath> kramdown-rfc2629 --v3 .\draft-lmbdhk-quic-multipath.md > draft.xml
*** Input contains HT ("tab") characters. Undefined behavior will ensue.
*** overwriting RFC2119
*** overwriting QUIC-TLS
PS C:\Users\huite\GitHub\draft-lmbdhk-quic-multipath> xml2rfc --text draft.xml
file:/C:/Users/huite/GitHub/draft-lmbdhk-quic-multipath/draft.xml(12): Warning: Setting consensus="true" for IETF STD document (this is not the schema default, but is the only value permitted for this type of document)
file:/C:/Users/huite/GitHub/draft-lmbdhk-quic-multipath/draft.xml(935): Warning: Unused reference: There seems to be no reference to [QUIC-Invariants] in the document
file:/C:/Users/huite/GitHub/draft-lmbdhk-quic-multipath/draft.xml(553): Warning: Artwork too wide, reducing indentation from 0 to 0
file:/C:/Users/huite/GitHub/draft-lmbdhk-quic-multipath/draft.xml(553): Warning: Figure too wide, reducing indentation from 3 to 0
file:/C:/Users/huite/GitHub/draft-lmbdhk-quic-multipath/draft.xml(675): Warning: Artwork too wide, reducing indentation from 0 to 0
file:/C:/Users/huite/GitHub/draft-lmbdhk-quic-multipath/draft.xml(675): Warning: Figure too wide, reducing indentation from 3 to 0
file:/C:/Users/huite/GitHub/draft-lmbdhk-quic-multipath/draft.xml(665): Warning: Section too wide, reducing indentation from 0 to 0
file:/C:/Users/huite/GitHub/draft-lmbdhk-quic-multipath/draft.xml(574): Warning: Section too wide, reducing indentation from 0 to 0
file:/C:/Users/huite/GitHub/draft-lmbdhk-quic-multipath/draft.xml(72): Warning: Middle too wide, reducing indentation from 0 to 0
file:/C:/Users/huite/GitHub/draft-lmbdhk-quic-multipath/draft.xml(675): Warning: Too long line found (L940), 8 characters longer than 72 characters:
Type (i) = TBD-00..TBD-01 (experiments use 0xbaba00..0xbaba01 or 0x42..x43),
Created file draft.txt
When the multipath option is negotiated, clients that want to use an additional path MUST first initiate the Address Validation procedure with PATH_CHALLENGE and PATH_RESPONSE frames described in Section 8 of {{QUIC-TRANSPORT}}.
There are two cases:
I think some elaboration is needed.
Lack of the reference to Section 7.4/Section 7.4.2 of 9000 where the general handling of (new) transport parameters is discussed.
When the new transport parameter is not present, this does not necessarily mean multipath is not supported as it can be supported, but disabled (for this connection as a function of local policy). I would update this text
OLD:
If the peer does not carry the enable_multipath transport parameter, which means the peer does not
support multipath
The text says that both endpoint can send an abandon frame. However, if I understand the current text correctly it only says that the receiving peer of that frame can remove its state as the sender of the abandon frame indicated it will not send any more packets ion that path. However, the sender of the abandon frame can't remove its state until it also receives an abandon frame from the other end. That would mean a path can "half-open"...
Do we need to say something that that the receiver of an abandon frame SHOULD also send an abandon frame in response? Or MAY? Or MUST?
Or is the path considered completely closed as soon as an ACK for the packet containing the abandon frame is received? In this case we need to say that when an abandon frame is received, the receiver MUST not use that path anymore.
I think we need to clarify that!
Please provide a PR or let me know what the intention was and I can write something!
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.