Subject: Re: [quicwg/base-drafts] Update draft-ietf-quic-transport.md (#105)

From: Martin Thomson <[email-masked]>
Date: Mon, 02 Jan 2017 20:54:30 -0800

martinthomson commented on this pull request.

> -The maximum packet size for QUIC is the maximum size of the encrypted payload of
-the resulting UDP datagram.  All QUIC packets SHOULD be sized to fit within the
-path's MTU to avoid IP fragmentation.  The recommended default maximum packet
-size is 1350 bytes for IPv6 and 1370 bytes for IPv4.  To optimize better,
-endpoints MAY use PLPMTUD {{!RFC4821}} for detecting the path's MTU and setting
-the maximum packet size appropriately.
+The maximum packet size for QUIC is the maximum size of the entire UDP payload,
+including the public header, encrypted payload, and any authentication fields.
+All QUIC packets SHOULD be sized to fit within the path's MTU to avoid IP
+fragmentation. To optimize bandwidth efficiency, endpoints SHOULD use Path MTU
+Discovery ({{!RFC1191}}, {{!RFC1981}}) for detecting the path's MTU, setting the
+maximum packet size appropriately, and storing the result of previous PMTU
+determinations.
+
+(TODO: Should there be a high minimum MTU for QUIC to avoid ICMP attacks? If so,
+does the endpoint fail over to TCP or simply allow fragmentation?)

Rather than log this here, open a separate issue.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/105#pullrequestreview-14885089

---

From: Martin Thomson <[email-masked]>
Date: Mon, 02 Jan 2017 21:00:14 -0800

+The maximum packet size for QUIC is the maximum size of the entire UDP payload,
+including the public header, encrypted payload, and any authentication fields.
+All QUIC packets SHOULD be sized to fit within the path's MTU to avoid IP
+fragmentation. To optimize bandwidth efficiency, endpoints SHOULD use Path MTU
+Discovery ({{!RFC1191}}, {{!RFC1981}}) for detecting the path's MTU, setting the
+maximum packet size appropriately, and storing the result of previous PMTU
+determinations.
+
+(TODO: Should there be a high minimum MTU for QUIC to avoid ICMP attacks? If so,
+does the endpoint fail over to TCP or simply allow fragmentation?)
+
+QUIC endpoints MUST maintain a separate PMTU estimate for each IP address the
+peer is using in the connection. Endpoints SHOULD maintain an estimate for each
+combination of local and remote IP addresses (as each pairing may have a
+different minimum MTU in the path).

I think that we're going to need a definition for "path", which leads to this being simply: Endpoints MUST maintain a separate PMTU estimate for each path.

The implication of your construction is that different remote IP addresses get a different PMTU estimate, but different local ones might not.  That's probably fine if the endpoint knows that outgoing packets from different local addresses go to the same remote address over the same path, but I'd rather assume nothing in any mandate we make. 

That said, if an endpoint has really good information that paths are the "same", then estimates can be easily copied from one path to another.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/105#pullrequestreview-14885271

---

From: Martin Thomson <[email-masked]>
Date: Mon, 02 Jan 2017 21:00:29 -0800

+## Packet MTU Determination

Path MTU Discovery?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/105#pullrequestreview-14885283

---

From: Martin Thomson <[email-masked]>
Date: Mon, 02 Jan 2017 21:01:10 -0800

+does the endpoint fail over to TCP or simply allow fragmentation?)
+
+QUIC endpoints MUST maintain a separate PMTU estimate for each IP address the
+peer is using in the connection. Endpoints SHOULD maintain an estimate for each
+combination of local and remote IP addresses (as each pairing may have a
+different minimum MTU in the path).
+
+## Packet MTU Determination
+
+Path MTU Discovery is optional, but endpoints that implement it observe the
+following requirements.
+
+### DF Marking {#dfmarking}
+
+QUIC endpoints set the Don't Fragment (DF) bit in the IP header of selected

IPv4

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/105#pullrequestreview-14885298

---

From: Martin Thomson <[email-masked]>
Date: Mon, 02 Jan 2017 21:33:15 -0800

martinthomson requested changes on this pull request.

Path MTU Discovery?

You need to point out that this only applies to IPv4.

+combination of local and remote IP addresses (as each pairing may have a
+different minimum MTU in the path).
+
+## Packet MTU Determination
+
+Path MTU Discovery is optional, but endpoints that implement it observe the
+following requirements.
+
+### DF Marking {#dfmarking}
+
+QUIC endpoints set the Don't Fragment (DF) bit in the IP header of selected
+QUIC datagrams. These packets MUST use PADDING frames, as necessary, to raise the
+overall packet size to the expected maximum.
+
+The first packet from the client MUST be maximum-size and set the DF bit.

Why?  What is the maximum size?  Can a client reduce the size if it discovers that the packet doesn't fit?  How far can it reduce the size?

Is that an RFC6919 "MUST (but we know that you won't)"?  I say that because clients will have to balance the probability of packets arriving with the desire to avoid amplification attack.  I appreciate that fragmentation here is hazardous for numerous reasons, so I agree with the DF marking, but the actual value we choose here is going to be better if it is fixed somehow so that clients can be compliant.

+
+## Packet MTU Determination
+
+Path MTU Discovery is optional, but endpoints that implement it observe the
+following requirements.
+
+### DF Marking {#dfmarking}
+
+QUIC endpoints set the Don't Fragment (DF) bit in the IP header of selected
+QUIC datagrams. These packets MUST use PADDING frames, as necessary, to raise the
+overall packet size to the expected maximum.
+
+The first packet from the client MUST be maximum-size and set the DF bit.
+
+The last server-generated packet in the Transport Handshake MUST be padded to

I'm not sure what message you are referring to here.  The transport handshake doesn't really have concrete messages.

If you are referring to the TLS handshake, then that only has one message from the server in many cases.  If you pad that message, that could be an issue.  We're already pushing the limits of what is reasonable given that it usually contains a server certificate.

What problem are you attempting to solve by requiring DF on the server messages?

+
+### DF Marking {#dfmarking}
+
+QUIC endpoints set the Don't Fragment (DF) bit in the IP header of selected
+QUIC datagrams. These packets MUST use PADDING frames, as necessary, to raise the
+overall packet size to the expected maximum.
+
+The first packet from the client MUST be maximum-size and set the DF bit.
+
+The last server-generated packet in the Transport Handshake MUST be padded to
+maximum-size with the DF bit set. Earlier packets may be smaller to save server
+resources until various handshake mechanisms have validated the client.
+
+QUIC endpoints MUST set the DF bit on the first packet sent to or from an IP
+address new to the QUIC connection, and pad the payload appropriately.

Again, please ensure that you are clear about why you are requiring this.

+QUIC endpoints set the Don't Fragment (DF) bit in the IP header of selected
+QUIC datagrams. These packets MUST use PADDING frames, as necessary, to raise the
+overall packet size to the expected maximum.
+
+The first packet from the client MUST be maximum-size and set the DF bit.
+
+The last server-generated packet in the Transport Handshake MUST be padded to
+maximum-size with the DF bit set. Earlier packets may be smaller to save server
+resources until various handshake mechanisms have validated the client.
+
+QUIC endpoints MUST set the DF bit on the first packet sent to or from an IP
+address new to the QUIC connection, and pad the payload appropriately.
+
+QUIC endpoints MAY set DF and pad packets when it has evidence that the path
+between two previously used IP addresses has changed.

Rathole alert: we probably need at least an existence proof here.  Name - by way of example - one way that an endpoint would learn this.

+The last server-generated packet in the Transport Handshake MUST be padded to
+maximum-size with the DF bit set. Earlier packets may be smaller to save server
+resources until various handshake mechanisms have validated the client.
+
+QUIC endpoints MUST set the DF bit on the first packet sent to or from an IP
+address new to the QUIC connection, and pad the payload appropriately.
+
+QUIC endpoints MAY set DF and pad packets when it has evidence that the path
+between two previously used IP addresses has changed.
+
+QUIC endpoints SHOULD periodically send a DF-marked, padded packet exceeding the
+recorded PMTU to probe for a change in the PMTU. In a change from {{!RFC1191}}
+and {{!RFC1981}}, these packets MUST NOT be sent on a path more often than
+every 20 RTTs (implying that the chance of a blind attack succeeding is less than
+5% without a storm of ICMP packets, given the mechanisms below).

Is the assumption that an attacker won't be able to a) observe the initiation of the connection, and b) guess the RTT?  Because both of those are not true in the general case.  Randomizing the interval and recommending a minimum idle period might work, but only if you also levy some sort of penalty on an attacker.  For instance, spurious ICMP/PTB messages can cause future messages to be ignored.

Given that you have the rationale for this in the next section, I think that you should move this requirement down there.

+PMTUD performs poorly with misbehaving routers that do not send
+ICMP Packet Too Big messages {{!RFC2923}}. To mitigate these "black holes", QUIC
+endpoints MAY use PLPMTUD {{!RFC4821}} in addition to PMTUD when repeated losses
+of DF-marked packets indicate this problem.
+
+PMTUD is potentially vulnerable to off-path attacks that successfully guess the
+IP/port 4-tuple and reduce the MTU to a bandwidth-inefficient value.
+TCP connections mitigate this risk by using the (at minimum) 8 bytes of transport
+header echoed in the ICMP message to validate the echoed TCP sequence number in
+the current connection context. However, as QUIC operates over UDP, this echoed
+information may consist only of the IP and UDP headers, which hold no connection
+context except the IP/port 4-tuple. The following requirements (in addition to
+limits on DF marking above) mitigate these problems:
+
+The endpoint MUST maintain a list of all unacknowledged packet numbers with the

This list needs bullets.

> +of DF-marked packets indicate this problem.
+
+PMTUD is potentially vulnerable to off-path attacks that successfully guess the
+IP/port 4-tuple and reduce the MTU to a bandwidth-inefficient value.
+TCP connections mitigate this risk by using the (at minimum) 8 bytes of transport
+header echoed in the ICMP message to validate the echoed TCP sequence number in
+the current connection context. However, as QUIC operates over UDP, this echoed
+information may consist only of the IP and UDP headers, which hold no connection
+context except the IP/port 4-tuple. The following requirements (in addition to
+limits on DF marking above) mitigate these problems:
+
+The endpoint MUST maintain a list of all unacknowledged packet numbers with the
+DF flag set.
+
+The endpoint MUST remove packet numbers from the list when the loss detection
+algorithm declares the packet lost, or the packet is acknowledged.

Would it be reasonable to trim this list more aggressively than otherwise? Several reasons:
1. It only results in lost PMTUD feedback
2. It reduces exposure to those off-path attacks
3. Path elements are able to respond in less than 1RTT

> +information may consist only of the IP and UDP headers, which hold no connection
+context except the IP/port 4-tuple. The following requirements (in addition to
+limits on DF marking above) mitigate these problems:
+
+The endpoint MUST maintain a list of all unacknowledged packet numbers with the
+DF flag set.
+
+The endpoint MUST remove packet numbers from the list when the loss detection
+algorithm declares the packet lost, or the packet is acknowledged.
+
+The endpoint MUST ignore any Packet Too Big message that arrives when there are
+no unacknowledged packet numbers in the list.
+
+Incoming Packet Too Big messages SHOULD be applied to all QUIC connections that
+share the same local and remote IP addresses, assuming they are valid for one
+connection.

We should refer to a path here.  How we identify a path locally is probably something we need to include in the definition.

> +The endpoint MUST maintain a list of all unacknowledged packet numbers with the
+DF flag set.
+
+The endpoint MUST remove packet numbers from the list when the loss detection
+algorithm declares the packet lost, or the packet is acknowledged.
+
+The endpoint MUST ignore any Packet Too Big message that arrives when there are
+no unacknowledged packet numbers in the list.
+
+Incoming Packet Too Big messages SHOULD be applied to all QUIC connections that
+share the same local and remote IP addresses, assuming they are valid for one
+connection.
+
+Valid Packet Too Big messages MUST trigger immediate retransmission of
+retransmittable data from packet numbers in the list, with no adjustment in
+congestion control parameters consistent with a congestion-induced loss.

I'm not an expert but this seems wrong: if the signal originates from the remote side of a congested link, isn't this a great way to add to that congestion?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/105#pullrequestreview-14885337

---

From: martinduke <[email-masked]>
Date: Tue, 10 Jan 2017 15:39:20 -0800

martinduke commented on this pull request.

+

Padding the server message verifies that the MTU is the same on the reverse path.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/105

---

From: martinduke <[email-masked]>
Date: Tue, 10 Jan 2017 15:42:21 -0800

Closed #105.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/105#event-918363369

---

From: martinduke <[email-masked]>
Date: Tue, 10 Jan 2017 15:42:20 -0800

Thanks for the comments. I'm closing this PR in favor of a significantly modified #106 that incorporates the comments that are still relevant.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/105#issuecomment-271733961

---

From: martinduke <[email-masked]>
Date: Tue, 10 Jan 2017 15:43:18 -0800

> Is that an RFC6919 "MUST (but we know that you won't)"?

It may very well be. I thought there was consensus that QUIC endpoints MUST pad handshake packets, but if not we can fight it out in #106.