Subject: Re: [quicwg/base-drafts] PMTUD (kitchen-sink version) (#106)

From: Martin Thomson <[email-masked]>
Date: Mon, 02 Jan 2017 21:43:12 -0800

martinthomson commented on this pull request.

> +
+QUIC endpoints MAY set DF and pad packets when implementing probes as described in
+{{!RFC1191}}, {{!RFC1981}}, or {{!RFC4821}}.
+
+QUIC endpoints MAY set DF and pad packets when it has evidence that the path between two
+previously used IP addresses has changed.
+
+### Special Considerations for Path MTU Discovery 
+
+Traditional ICMP-based path MTU discovery ({{!RFC1191}}, {{!RFC1981}}) is
+potentially vulnerable to off-path attacks that succesfully guess the IP/port 4-tuple
+and reduce the MTU to a bandwidth-inefficient value. TCP connections mitigate this
+risk by using the (at minimum) 8 bytes of transport header echoed in the ICMP message
+to validate the TCP sequence number as valid for the current connection. However, as
+QUIC operates over UDP, this echoed information may consist only of the IP and UDP
+headers, which hold no connection-specific context except the IP/port 4-tuple.

This isn't strictly correct.  There are 4 16-bit values (source port, destination port, length, checksum).  The problem is that not many implementations are willing to remember length and checksum for the packets that they sent.  Also, including length means that the packet contains very little entropy, meaning that it's relatively easy to guess from off-path.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#pullrequestreview-14886653

---

From: martinduke <[email-masked]>
Date: Tue, 03 Jan 2017 10:13:24 -0800

martinduke commented on this pull request.

That's a good point. How about "this echoed information may consist only of the IP and UDP headers, which usually has insufficient entropy to mitigate off-path attacks." ?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106

---

From: Martin Thomson <[email-masked]>
Date: Tue, 03 Jan 2017 14:35:50 -0800

Yeah, that works.

---

From: janaiyengar <[email-masked]>
Date: Thu, 05 Jan 2017 22:06:02 -0800

janaiyengar requested changes on this pull request.

A few high-order thoughts. I think this addition boils down to a few things. (i) Handshake packets MUST be padded to InitialMaxLength, and DF SHOULD be set on them, and (ii) During the connection, larger packets MAY be sent occasionally/periodically to probe for a larger packet size, and the sender should take ICMP messages as well as loss indications as signals of PMTU exceeded.

I think (i) is needed, for QUIC to ensure passage of subsequent packets, and to ensure that the handshake fails if InitialMaxLength is too large for the path. Google-QUIC does this right now, and it is absolutely useful. I would argue that this part belongs somewhere in the handshake section (Section 5.2).

Recommendations on how to do PMTUD or PLPMTUD, as described in (ii), belong in this section, but I would make it the last subsection of the Packetization section, as a PMTU Considerations subsection.  Bear in mind that this part has very little to motivate implementing it -- it's really tricky to get right and the benefits don't seem terribly high. That said, I think it's useful for completeness.

> -endpoints MAY use PLPMTUD {{!RFC4821}} for detecting the path's MTU and setting
-the maximum packet size appropriately.
+The maximum packet size for QUIC is the maximum size of the entire UDP payload,
+including the public header, encrypted payload, and any authentication fields.
+All QUIC packets SHOULD be sized to fit within the path's MTU to avoid IP
+fragmentation. To optimize bandwidth efficiency, endpoints SHOULD use Path MTU
+Discovery ({{!RFC1191}}, {{!RFC1981}}) or Packetization Layer Path MTU
+Discovery ({{!RFC4821}}) for detecting the path's MTU, setting the maximum
+packet size appropriately, and storing the result of previous PMTU
+determinations. In the absence of these mechanisms,  the recommended default
+maximum packet size is 1350 bytes for IPv6 and 1370 bytes for IPv4. 
+
+(TODO: Should there be a high minimum MTU for QUIC to avoid ICMP attacks? If so,
+does the endpoint fail over to TCP or simply allow fragmentation?)
+
+QUIC endpoints MUST maintain a separate PMTU estimate for each IP address the

I don't think the MUST makes sense here. I'd suggest something like "If a QUIC endpoint employs PMTUD, then the endpoint SHOULD use a separate PMTU estimate for every pair of source and destination IP used by the connection."

> +packet size appropriately, and storing the result of previous PMTU
+determinations. In the absence of these mechanisms,  the recommended default
+maximum packet size is 1350 bytes for IPv6 and 1370 bytes for IPv4. 
+
+(TODO: Should there be a high minimum MTU for QUIC to avoid ICMP attacks? If so,
+does the endpoint fail over to TCP or simply allow fragmentation?)
+
+QUIC endpoints MUST maintain a separate PMTU estimate for each IP address the
+peer is using in the connection. Endpoints SHOULD maintain an estimate for each
+combination of local and remote IP addresses (as each pairing may have a different
+minimum MTU in the path).
+
+## Packet MTU Determination
+
+QUIC endpoints SHOULD set the Don't Fragment (DF) bit in the IP header of selected
+QUIC datagrams. These packets MUST use PADDING frames, as necessary, to raise the

Which datagrams?

> +packet size appropriately, and storing the result of previous PMTU
+determinations. In the absence of these mechanisms,  the recommended default
+maximum packet size is 1350 bytes for IPv6 and 1370 bytes for IPv4. 
+
+(TODO: Should there be a high minimum MTU for QUIC to avoid ICMP attacks? If so,
+does the endpoint fail over to TCP or simply allow fragmentation?)
+
+QUIC endpoints MUST maintain a separate PMTU estimate for each IP address the
+peer is using in the connection. Endpoints SHOULD maintain an estimate for each
+combination of local and remote IP addresses (as each pairing may have a different
+minimum MTU in the path).
+
+## Packet MTU Determination
+
+QUIC endpoints SHOULD set the Don't Fragment (DF) bit in the IP header of selected
+QUIC datagrams. These packets MUST use PADDING frames, as necessary, to raise the

I don't think MUST is needed here. Saying that "the sender MAY pad packets as necessary" is adequate.

> +minimum MTU in the path).
+
+## Packet MTU Determination
+
+QUIC endpoints SHOULD set the Don't Fragment (DF) bit in the IP header of selected
+QUIC datagrams. These packets MUST use PADDING frames, as necessary, to raise the
+overall packet size to the expected maximum.
+
+### DF Marking {#dfmarking}
+
+The first packet from the client MUST be maximum-size and SHOULD set the DF bit.
+
+The last server-generated packet in the Transport Handshake SHOULD be padded to
+maximum-size with the DF bit set. Earlier packets may be smaller to save server
+resources until various handshake mechanisms have validated the client.
+

I don't think that padding earlier packets is an issue. I think it's reasonable to set DF on all packets carrying handshake data.

+potentially vulnerable to off-path attacks that succesfully guess the IP/port 4-tuple
+and reduce the MTU to a bandwidth-inefficient value. TCP connections mitigate this
+risk by using the (at minimum) 8 bytes of transport header echoed in the ICMP message
+to validate the TCP sequence number as valid for the current connection. However, as
+QUIC operates over UDP, this echoed information may consist only of the IP and UDP
+headers, which hold no connection-specific context except the IP/port 4-tuple.
+
+Furthermore, PMTUD also performs poorly with misbehaving routers that do not send
+ICMP Packet Too Big messages {{!RFC2923}}.
+
+To mitigate this, QUIC endpoints MAY use PLPMTUD, {{!RFC4821}}, either in place of, or
+in addition to, PMTUD. This avoids the off-path attack and misbehaving router problems
+while potentially introducing multiple packet losses and long timeouts. 
+
+Due to the drawbacks of PLPMTUD, QUIC endpoints MAY use ICMP-based PTMU discovery.

I don't follow -- the text here seems to say first that QUIC MAY use PLPMTUD to mitigate issues with PMTUD, and now says that QUIC MAY use PMTUD to address drawbacks of PLPMTUD.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#pullrequestreview-15444596

---

From: martinduke <[email-masked]>
Date: Tue, 10 Jan 2017 15:34:19 -0800

@martinduke pushed 1 commit.

2021025  Update draft-ietf-quic-transport.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/106/files/159063288265d7a581e2170672194f29d672e984..2021025b266b9ab983c22e3b428ea878db45f58c

---

From: martinduke <[email-masked]>
Date: Tue, 10 Jan 2017 15:34:51 -0800

Changes reflected in patch-1.

---

From: martinduke <[email-masked]>
Date: Tue, 10 Jan 2017 15:37:40 -0800

I submitted patch-1, which is a major rewrite.

I realized that my PR was too long because it was extended digression on PMTU issues. So I shrunk it down to achieve a couple of things:
1) explicitly allow ICMP-based PMTUD
2) Codify the requirement to pad/DF-mark handshake packets.
3) List a few QUIC-specific considerations about ICMP PMTUD, given that many readers are coming from the TCP world where it is less fraught.

I believe this version either obviates or answers each comment on both #105 and #106.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#issuecomment-271733098

---

From: martinduke <[email-masked]>
Date: Tue, 10 Jan 2017 15:37:51 -0800

eliminated.

---

From: martinduke <[email-masked]>
Date: Tue, 10 Jan 2017 15:38:08 -0800

+

I made the edit.

---

From: martinduke <[email-masked]>
Date: Tue, 10 Jan 2017 15:38:14 -0800

text eliminated

---

From: ianswett <[email-masked]>
Date: Wed, 11 Jan 2017 13:22:17 -0800

ianswett commented on this pull request.

Thanks Martin, this looks very good to me.  A few editorial comments.

> @@ -1198,12 +1198,25 @@ The fields of a GOAWAY frame are as follows:
 
 # Packetization and Reliability {#packetization}
 
-The maximum packet size for QUIC is the maximum size of the encrypted payload of
-the resulting UDP datagram.  All QUIC packets SHOULD be sized to fit within the
-path's MTU to avoid IP fragmentation.  The recommended default maximum packet
-size is 1350 bytes for IPv6 and 1370 bytes for IPv4.  To optimize better,
-endpoints MAY use PLPMTUD {{!RFC4821}} for detecting the path's MTU and setting
-the maximum packet size appropriately.
+The maximum packet size for QUIC is the maximum size of the entire UDP payload,
+including the public header, encrypted payload, and any authentication fields.
+All QUIC packets SHOULD be sized to fit within the path's MTU to avoid IP
+fragmentation. To optimize bandwidth efficiency, endpoints MAY use Path MTU
+Discovery ({{!RFC1191}}, {{!RFC1981}}) or Packetization Layer Path MTU
+Discovery ({{!RFC4821}}) for detecting the path's MTU, setting the maximum
+packet size appropriately, and storing the result of previous PMTU
+determinations. In the absence of these mechanisms,  the recommended default

nit: extra space after "mechanisms,"

> -size is 1350 bytes for IPv6 and 1370 bytes for IPv4.  To optimize better,
-endpoints MAY use PLPMTUD {{!RFC4821}} for detecting the path's MTU and setting
-the maximum packet size appropriately.
+The maximum packet size for QUIC is the maximum size of the entire UDP payload,
+including the public header, encrypted payload, and any authentication fields.
+All QUIC packets SHOULD be sized to fit within the path's MTU to avoid IP
+fragmentation. To optimize bandwidth efficiency, endpoints MAY use Path MTU
+Discovery ({{!RFC1191}}, {{!RFC1981}}) or Packetization Layer Path MTU
+Discovery ({{!RFC4821}}) for detecting the path's MTU, setting the maximum
+packet size appropriately, and storing the result of previous PMTU
+determinations. In the absence of these mechanisms,  the recommended default
+maximum packet size is 1350 bytes for IPv6 and 1370 bytes for IPv4. 
+
+QUIC endpoints that implement any kind of MTU discovery SHOULD maintain a
+separate PMTU estimate for each IP address the peer is using in the
+connection. Endpoints SHOULD maintain an estimate for each combination of

Is this first sentence attempting to anticipate multipath or connection migration, where a connection may send to multiple IPs?

Is the second sentence implying that an endpoint should persist PMTU across connections?  If so, could you clarify slightly, possibly "Endpoints SHOULD maintain an estimate for each combination of local and remote IP addresses for use in future connections."?

+All QUIC packets SHOULD be sized to fit within the path's MTU to avoid IP
+fragmentation. To optimize bandwidth efficiency, endpoints MAY use Path MTU
+Discovery ({{!RFC1191}}, {{!RFC1981}}) or Packetization Layer Path MTU
+Discovery ({{!RFC4821}}) for detecting the path's MTU, setting the maximum
+packet size appropriately, and storing the result of previous PMTU
+determinations. In the absence of these mechanisms,  the recommended default
+maximum packet size is 1350 bytes for IPv6 and 1370 bytes for IPv4. 
+
+QUIC endpoints that implement any kind of MTU discovery SHOULD maintain a
+separate PMTU estimate for each IP address the peer is using in the
+connection. Endpoints SHOULD maintain an estimate for each combination of
+local and remote IP addresses (as each pairing may have a different minimum
+MTU in the path).
+
+All handshake packets MUST include enough PADDING frames to bring the packet

You can only have one padding frame per packet, so maybe "All handshake packets MUST include a PADDING frame to bring the packet ..."?

> +All QUIC packets SHOULD be sized to fit within the path's MTU to avoid IP
+fragmentation. To optimize bandwidth efficiency, endpoints MAY use Path MTU
+Discovery ({{!RFC1191}}, {{!RFC1981}}) or Packetization Layer Path MTU
+Discovery ({{!RFC4821}}) for detecting the path's MTU, setting the maximum
+packet size appropriately, and storing the result of previous PMTU
+determinations. In the absence of these mechanisms,  the recommended default
+maximum packet size is 1350 bytes for IPv6 and 1370 bytes for IPv4. 
+
+QUIC endpoints that implement any kind of MTU discovery SHOULD maintain a
+separate PMTU estimate for each IP address the peer is using in the
+connection. Endpoints SHOULD maintain an estimate for each combination of
+local and remote IP addresses (as each pairing may have a different minimum
+MTU in the path).
+
+All handshake packets MUST include enough PADDING frames to bring the packet
+to the maximum size the endpoint is enforcing. Furthermore, all IPv4 handshake

On a practical note, some OS's(ie: MacOSX) don't allow userspace apps to set the DF bit.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#pullrequestreview-16236829

---

From: martinduke <[email-masked]>
Date: Wed, 11 Jan 2017 16:27:22 -0800

Yes, this refers to multipath. It's not terribly clearly written. What it's trying to say that if you have N addresses and the peer has M addresses, you should keep at least M MTU estimates. It'd be even better, though, to keep M x N MTU estimates.

---

From: martinduke <[email-masked]>
Date: Wed, 11 Jan 2017 16:43:24 -0800

Isn't this a problem for the requirement to pad the handshake packets? Without this bit, routers could just fragment and we haven't achieved anything.

---

From: martinduke <[email-masked]>
Date: Wed, 11 Jan 2017 17:03:04 -0800

I'm not sure why it's not showing up in this thread, but the latest version in patch-1 reflects the following:
- Added the clarification that packet number ambiguity is only a problem in IPv4
- Addressed Ian's comments, except...
- I kept the DF requirement for handshake packets, because I don't see how the existing Google implementation works without it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#issuecomment-272045480

---

From: ianswett <[email-masked]>
Date: Wed, 11 Jan 2017 19:07:22 -0800

I completely agree.  In practice, most people are on networks with largish(ie: >1400 byte) MTUs, but it's a highly suboptimal situation.  I'm fine with your wording, but it means we should ensure Apple plans to add support for the DF bit to MacOSX and iOS.  If I've just overlooked an existing interface, then all the better.

---

From: ianswett <[email-masked]>
Date: Wed, 11 Jan 2017 19:08:38 -0800

This seems like a pragmatic change to the current docs.  We can iterate in the future, but it's much better than the status quo, so I'd like to see it merged before -01.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#issuecomment-272064102

---

From: janaiyengar <[email-masked]>
Date: Wed, 11 Jan 2017 19:35:17 -0800

janaiyengar commented on this pull request.

Looks great! A few comments.

I think the second sentence subsumes the first one (a change in local IP is a change in the IP pair). I think the sentences can be combined: "QUIC endpoints that implement any kind of MTU discovery SHOULD maintain a separate PMTU estimate for each combination of local and remote IP addresses (as each pairing may have a different maximum MTU in the path)."

nit: minimum --> maximum

+1, except that the transport draft defines a PADDING frame currently as simply the Frame Type 0, meaning that a PADDING frame is basically 1 byte long. I'll change this to be Type + 0-padding to the end of the packet.

I would make this a SHOULD. We can't ensure that existing implementations will allow this -- I'm not sure what Win10 does, but I believe older versions of Windows don't support setting IP header bits.

I would switch the order of this paragraph with the previous one, to put requirements (MUST) before others.

> @@ -1257,6 +1270,27 @@ frames), the receiving peer MAY send an ACK frame after a reasonable number
 Strategies and implications of the frequency of generating acknowledgments are
 discussed in more detail in {{QUIC-RECOVERY}}.
 
+## Special Considerations for Path MTU Discovery 
+
+Traditional ICMP-based path MTU discovery ({{!RFC1191}}, {{!RFC1981}}) is
+potentially vulnerable to off-path attacks that succesfully guess the IP/port
+4-tuple and reduce the MTU to a bandwidth-inefficient value. TCP connections
+mitigate this risk by using the (at minimum) 8 bytes of transport header
+echoed in the ICMP message to validate the TCP sequence number as valid for
+the current connection. However, as QUIC operates over UDP, the echoed
+information may consist only of the IP and UDP headers, which usually has
+insufficient entropy to mitigate off-path attacks.
+
+As a result, endpoints that implement PMTUD SHOULD take steps to mitigate this
+risk, which may include:

nit: "SHOULD take steps to mitigate this risk. For instance, an application may:"

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#pullrequestreview-16284257

---

From: martinduke <[email-masked]>
Date: Thu, 12 Jan 2017 09:16:09 -0800

9fbcca3  Revised PMTU section

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/106/files/6ef293130e793f8a5ec4459f4fa55a801cf99c66..9fbcca385dbf5a43916b4f217ec4d4b62407a458

---

From: Martin Thomson <[email-masked]>
Date: Thu, 12 Jan 2017 15:43:33 -0800

This is getting pretty close.

> @@ -1198,12 +1198,23 @@ The fields of a GOAWAY frame are as follows:

s/path's/estimated path/

> @@ -1198,12 +1198,23 @@ The fields of a GOAWAY frame are as follows:

s/path's/path/ - the usual form drops the possessive.

> @@ -1198,12 +1198,23 @@ The fields of a GOAWAY frame are as follows:

I don't remember any discussion concluding that those numbers are OK.  Are we assuming 1398 bytes is available to IP?

I find it a little strange that this is accounting for the IP and UDP headers.  I usually see MTU discussed in terms of the entire IP packet.  I would phrase this differently: "Prior to attempting PMTUD, or if PMTUD is not attempted, an endpoint SHOULD assume a path MTU of 1398 octets.  Assuming that there are no IPv6 extension headers or IPv4 options in use, this results in a UDP payload length of 1350 octets for IPv6 and 1370 octets for IPv4."

Or is the intent to actually say that either extension headers are forbidden or that elements that intend to add extension headers have to be able to pass larger packets?

+All handshake packets MUST include a PADDING frame if necessary to bring the

Is this ALL packets?  If the server needs to send multiple handshake packets (that happens often because certificates get bloated), do they really need to pad the 10th packet they send?  Surely the goal has been achieved after sending the first packet at the maximum size.  (The client should only ever send one handshake packet.)

+All handshake packets MUST include a PADDING frame if necessary to bring the
+packet to the maximum size the endpoint is enforcing. Furthermore, all IPv4

The phrasing here is odd.  Which endpoint is enforcing a size?  Endpoints don't really enforce limits, it's something on the path that is doing any enforcement.  I would say that to the estimated maximum packet size for the path.

Given that we don't have an estimate for the path at this point, why not use the defaults?  Or are we keeping the option open to remember path MTU from a previous use of that path?  FWIW, I would rather we *explicitly* enabled that as a feature if we decide that it's OK.  I've had bad experiences with smuggling features into protocols, even seemingly innocuous ones.

+All QUIC packets SHOULD be sized to fit within the path's MTU to avoid IP
+fragmentation. To optimize bandwidth efficiency, endpoints MAY use Path MTU
+Discovery ({{!RFC1191}}, {{!RFC1981}}) or Packetization Layer Path MTU
+Discovery ({{!RFC4821}}) for detecting the path's MTU, setting the maximum
+packet size appropriately, and storing the result of previous PMTU
+determinations. In the absence of these mechanisms, the recommended default
+maximum packet size is 1350 bytes for IPv6 and 1370 bytes for IPv4. 
+
+All handshake packets MUST include a PADDING frame if necessary to bring the
+packet to the maximum size the endpoint is enforcing. Furthermore, all IPv4
+handshake packets SHOULD have the DF bit set.
+
+QUIC endpoints that implement any kind of MTU discovery SHOULD maintain an
+estimate for each combination of local and remote IP addresses (as each pairing
+may have a different maximum MTU in the path).

s/may/could

> @@ -1257,6 +1268,27 @@ frames), the receiving peer MAY send an ACK frame after a reasonable number
 
+## Special Considerations for Path MTU Discovery 
+
+Traditional ICMP-based path MTU discovery in IPv4 ({{!RFC1191}} is potentially
+vulnerable to off-path attacks that succesfully guess the IP/port 4-tuple and

s/succesfully/successfully

> @@ -1257,6 +1268,27 @@ frames), the receiving peer MAY send an ACK frame after a reasonable number
 
+## Special Considerations for Path MTU Discovery 
+
+Traditional ICMP-based path MTU discovery in IPv4 ({{!RFC1191}} is potentially
+vulnerable to off-path attacks that succesfully guess the IP/port 4-tuple and
+reduce the MTU to a bandwidth-inefficient value. TCP connections mitigate this
+risk by using the (at minimum) 8 bytes of transport header echoed in the ICMP
+message to validate the TCP sequence number as valid for the current
+connection. However, as QUIC operates over UDP, in IPv4 the echoed information
+may consist only of the IP and UDP headers, which usually has insufficient

s/may consist/consists ?  I don't know if there are cases where you get more or less data, are there?

> @@ -1257,6 +1268,27 @@ frames), the receiving peer MAY send an ACK frame after a reasonable number
 
+## Special Considerations for Path MTU Discovery 
+
+Traditional ICMP-based path MTU discovery in IPv4 ({{!RFC1191}} is potentially
+vulnerable to off-path attacks that succesfully guess the IP/port 4-tuple and
+reduce the MTU to a bandwidth-inefficient value. TCP connections mitigate this
+risk by using the (at minimum) 8 bytes of transport header echoed in the ICMP
+message to validate the TCP sequence number as valid for the current
+connection. However, as QUIC operates over UDP, in IPv4 the echoed information
+may consist only of the IP and UDP headers, which usually has insufficient
+entropy to mitigate off-path attacks.
+
+As a result, endpoints that implement PMTUD in IPv4 SHOULD take steps to mitigate
+this risk. For instance, an application may:

Here you use "may" when talking about a SHOULD, which will confuse some people.

+Traditional ICMP-based path MTU discovery in IPv4 ({{!RFC1191}} is potentially
+vulnerable to off-path attacks that succesfully guess the IP/port 4-tuple and
+reduce the MTU to a bandwidth-inefficient value. TCP connections mitigate this
+risk by using the (at minimum) 8 bytes of transport header echoed in the ICMP
+message to validate the TCP sequence number as valid for the current
+connection. However, as QUIC operates over UDP, in IPv4 the echoed information
+may consist only of the IP and UDP headers, which usually has insufficient
+entropy to mitigate off-path attacks.
+
+As a result, endpoints that implement PMTUD in IPv4 SHOULD take steps to mitigate
+this risk. For instance, an application may:
+
+* Set the IPv4 Don't Fragment (DF) bit on a small number of packets per RTT, so
+that most invalid ICMP messages arrive when there are no DF packets
+outstanding.

Maybe just  "a small proportion of packets".  If you DF on every RTT as this implies, then you will pretty much always have a DF packet outstanding.

> +reduce the MTU to a bandwidth-inefficient value. TCP connections mitigate this
+risk by using the (at minimum) 8 bytes of transport header echoed in the ICMP
+message to validate the TCP sequence number as valid for the current
+connection. However, as QUIC operates over UDP, in IPv4 the echoed information
+may consist only of the IP and UDP headers, which usually has insufficient
+entropy to mitigate off-path attacks.
+
+As a result, endpoints that implement PMTUD in IPv4 SHOULD take steps to mitigate
+this risk. For instance, an application may:
+
+* Set the IPv4 Don't Fragment (DF) bit on a small number of packets per RTT, so
+that most invalid ICMP messages arrive when there are no DF packets
+outstanding.
+
+* Store additional information from the IP or UDP headers (for example, the IP
+ID or UDP checksum) to further authenticate incoming Packet Too Big messages.

Maybe just "store the first IP and UDP headers from DF packets" to further authenticate [...]"

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#pullrequestreview-16478135

---

From: Martin Thomson <[email-masked]>
Date: Thu, 12 Jan 2017 17:06:31 -0800

Separately, it might be wise to expand MTU on first use.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#pullrequestreview-16492273

---

From: martinduke <[email-masked]>
Date: Fri, 13 Jan 2017 10:11:49 -0800

I agree with this fix and will make the change.

---

From: martinduke <[email-masked]>
Date: Fri, 13 Jan 2017 10:15:37 -0800

The PMTU RFCs explictly discussing storing PMTU results from previous connections. I don't think it's "smuggling" to not enumerate the components of those RFCs, nor do I think the language above implies a choice of either active PMTUD or just using the defaults.

---

From: martinduke <[email-masked]>
Date: Fri, 13 Jan 2017 10:17:01 -0800

Routers have the option of echoing more than 28 bytes of IP packet, and I've seen pcaps where this is the case.

---

From: martinduke <[email-masked]>
Date: Fri, 13 Jan 2017 10:31:36 -0800

I followed a suggestion in this thread to make it all packets. It's also hard to specify individual packets in the handshake because that's not part of this document. But I'll make it "at least" one packet in the handshake.

---

From: ianswett <[email-masked]>
Date: Fri, 13 Jan 2017 10:35:09 -0800

One word of warning.  If you pad the first CHLO, but not the retransmission of that, bad things happen.  So I'd rather be safe and say all packets because it's simple and hard to get wrong.

---

From: ianswett <[email-masked]>
Date: Fri, 13 Jan 2017 10:35:29 -0800

One comment on the importance of padding more than 1 packet in the handshake.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#issuecomment-272512642

---

From: martinduke <[email-masked]>
Date: Fri, 13 Jan 2017 10:35:31 -0800

e7b455c  Update draft-ietf-quic-transport.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/106/files/9fbcca385dbf5a43916b4f217ec4d4b62407a458..e7b455c745f1c2da93b1b393cad030e818e986e1

---

From: Martin Thomson <[email-masked]>
Date: Sun, 15 Jan 2017 20:09:04 -0800

I'd consider retransmission of an initial packet to be equivalent to starting over.  I'm concerned that we're going to affect the multi-packet server flight unnecessary, that's all.

---

From: martinduke <[email-masked]>
Date: Mon, 16 Jan 2017 12:06:20 -0800

I believe this PR is close to ready. The main open issue, I believe, is what the specification says about padding handshake packets.

1) All handshake packets to be padded to maximum size. 
2) Pad at least one packet per handshake (in each direction) _that is successfully acknowledged_
3) More specifically, the first client packet (CHLO) and retransmissions, and last server packet [SHLO]  and retransmissions. In many cases this will be the same as #2, but is a bit more prescriptive.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#issuecomment-272951282

---

From: martinduke <[email-masked]>
Date: Mon, 16 Jan 2017 12:13:23 -0800

I'm having trouble finding this comment on the GitHub page, but I believe
that the padding requirement is the only outstanding issue, and laid out
the options in another comment.

On Sun, Jan 15, 2017 at 8:09 PM, Martin Thomson <[email-masked]>
wrote:

> *@martinthomson* commented on this pull request.
> ------------------------------
>
> In draft-ietf-quic-transport.md
> <https://github.com/quicwg/base-drafts/pull/106>:
>
> I'd consider retransmission of an initial packet to be equivalent to
> starting over. I'm concerned that we're going to affect the multi-packet
> server flight unnecessary, that's all.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/quicwg/base-drafts/pull/106>, or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AXRMEcQ84m15PlxO-GHxS0a863LToxrSks5rSu1ggaJpZM4LXJpx>
> .
>

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#issuecomment-272952497

---

From: ianswett <[email-masked]>
Date: Mon, 16 Jan 2017 12:13:30 -0800

(1) seems simpler, but (3) seems both correct and reasonable as well.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#issuecomment-272952525

---

From: Martin Thomson <[email-masked]>
Date: Mon, 16 Jan 2017 18:05:13 -0800

Can we split the discussion about padding handshake packets from this change?  That would let us merge this and allow discussion to continue.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#issuecomment-273003239

---

From: martinduke <[email-masked]>
Date: Tue, 17 Jan 2017 08:45:56 -0800

e768325  Update draft-ietf-quic-transport.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/106/files/e7b455c745f1c2da93b1b393cad030e818e986e1..e7683255809c372d2de7a4355a94036f384943b1

---

From: martinduke <[email-masked]>
Date: Tue, 17 Jan 2017 08:46:53 -0800

OK, the handshake padding issue is gone from this PR. If there are no further objections, I believe the editors can merge this into the base draft.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#issuecomment-273225617

---

From: martinduke <[email-masked]>
Date: Thu, 26 Jan 2017 12:43:22 -0800

5bf3168  Update draft-ietf-quic-transport.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/106/files/e7683255809c372d2de7a4355a94036f384943b1..5bf31689873343f75f8e7f7d854c360dc602a884

---

From: Martin Thomson <[email-masked]>
Date: Mon, 30 Jan 2017 22:00:20 -0800

-endpoints MAY use PLPMTUD {{!RFC4821}} for detecting the path's MTU and setting
-the maximum packet size appropriately.
+The Path Maximum Transmission Unit (PTMU) is the maximum size of the entire IP
+header, UDP header, and UDP payload. The UDP payload includes the QUIC public
+header, encrypted payload, and any authentication fields.
+
+All QUIC packets SHOULD be sized to fit within the estimated PMTU to avoid IP
+fragmentation or packet drops. To optimize bandwidth efficiency, endpoints
+SHOULD use Packetization Layer PMTU Discovery ({{!RFC4821}}) and MAY use PMTU
+Discovery ({{!RFC1191}}, {{!RFC1981}}) for detecting the PMTU, setting the PMTU
+appropriately, and storing the result of previous PMTU determinations.
+
+In the absence of these mechanisms, QUIC endpoints SHOULD use a default PMTU of
+1280 octets. Assuming the minimum IP header size, this results in a UDP payload
+length of 1232 octets for IPv6 and 1252 octets for IPv4.

Given that 1280 is the minimum, this really isn't saying that there is a default PMTU, it's more saying that QUIC implementations can assume that 1280 is OK.

> +Discovery ({{!RFC1191}}, {{!RFC1981}}) for detecting the PMTU, setting the PMTU
+appropriately, and storing the result of previous PMTU determinations.
+
+In the absence of these mechanisms, QUIC endpoints SHOULD use a default PMTU of
+1280 octets. Assuming the minimum IP header size, this results in a UDP payload
+length of 1232 octets for IPv6 and 1252 octets for IPv4.
+
+QUIC endpoints that implement any kind of PMTU discovery SHOULD maintain an
+estimate for each combination of local and remote IP addresses (as each pairing
+could have a different maximum MTU in the path).
+
+QUIC connnections MUST NOT operate over a network path with an estimated PMTU
+below 1280 octets. This is equal to the IPv6 minimum. Most modern IPv4 networks
+support at least this packet size. A large packet size allows the first client
+handshake packet to fit in a single datagram, which in turn enables stateless
+rejects by the server.

Nothing in the protocol will stop a QUIC endpoint from using a path with a smaller MTU, especially if they can't set DF.  So I would frame this differently:

> QUIC depends on the network path supporting a MTU of at least 1280 octets.  This is the IPv6 minimum and therefore also supported by most modern IPv4 networks.  An endpoint MUST NOT reduce their MTU below this number, even if it receives signals that indicate a smaller limit might exist.

Then I would follow with the ClientHello paragraph:

> A client MUST construct an initial packet that is at least 1280 octets in size, including IP and UDP headers.  It is RECOMMENDED that a packet be padded to exactly 1280 octets unless the client has a reasonable assurance that the PMTU is larger.  A server MUST reject initial packets that are less than 1280 octets in size (caveat warning: see below).  Sending a packet of this size ensures that the network path supports an MTU of this size and helps mitigate amplification attacks caused by server responses toward an unverified client address.

Then I would include the text on PMTU changes, which isn't as important.

+If a QUIC endpoint determines that the PMTU between any pair of local and remote
+IP addresses has fallen below 1280 octets, it MUST immediately cease sending
+QUIC packets between those IP addresses. This may result in abrupt termination
+of the connection if all pairs are affected. In this case, an endpoint SHOULD
+send a Public Reset packet to indicate the failure. The application SHOULD
+attempt to use TLS over TCP instead.
+
+Clients MUST ensure that the first packet in a connection, and any
+retransmissions of those octets, has a total size (including IP and UDP headers)
+of at least 1280 bytes. This might require inclusion of a PADDING frame. Clients
+MAY use a larger first packet size if it has reasonable assurance of a higher
+PMTU through out-of-band information.
+
+Servers MUST reject the first plaintext packet received from a client if it its
+total size is less than 1280 octets, to mitigate amplification attacks.

Policing this will be difficult, if not impossible, if there are extension headers.  Userspace implementations might not even learn about them.  From what I can tell, there is no limit to the size of extension headers in IPv6, so we can't really require that this enforcement happen.

> +## Special Considerations for PMTU Discovery 
+
+Traditional ICMP-based path MTU discovery in IPv4 ({{!RFC1191}} is potentially
+vulnerable to off-path attacks that successfully guess the IP/port 4-tuple and
+reduce the MTU to a bandwidth-inefficient value. TCP connections mitigate this
+risk by using the (at minimum) 8 bytes of transport header echoed in the ICMP
+message to validate the TCP sequence number as valid for the current
+connection. However, as QUIC operates over UDP, in IPv4 the echoed information
+could consist only of the IP and UDP headers, which usually has insufficient
+entropy to mitigate off-path attacks.
+
+As a result, endpoints that implement PMTUD in IPv4 SHOULD take steps to mitigate
+this risk. For instance, an application could:
+
+* Set the IPv4 Don't Fragment (DF) bit on a small proportion of packets, so that
+most invalid ICMP messages arrive when there are no DF packets outstanding.

This is missing the (obvious) and can therefore be easily identified as spurious.

+risk by using the (at minimum) 8 bytes of transport header echoed in the ICMP
+message to validate the TCP sequence number as valid for the current
+connection. However, as QUIC operates over UDP, in IPv4 the echoed information
+could consist only of the IP and UDP headers, which usually has insufficient
+entropy to mitigate off-path attacks.
+
+As a result, endpoints that implement PMTUD in IPv4 SHOULD take steps to mitigate
+this risk. For instance, an application could:
+
+* Set the IPv4 Don't Fragment (DF) bit on a small proportion of packets, so that
+most invalid ICMP messages arrive when there are no DF packets outstanding.
+
+* Store additional information from the IP or UDP headers from DF packets (for
+example, the IP ID or UDP checksum) to further authenticate incoming Packet Too
+Big messages.

RFC 1191 uses "Datagram Too Big" rather than "Packet Too Big".

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/106#pullrequestreview-19246448