Subject: [quicwg/base-drafts] Refactor the section on connection termination (#721)

From: Martin Thomson <[email-masked]>
Date: Fri, 11 Aug 2017 03:57:03 +0000 (UTC)

This makes progress toward #61, but doesn't actually commit to a particular design.  It still needs fleshing out with details of how long to wait around and so forth, but I hope that it's a reasonable start.
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/721

-- Commit Summary --

  * Refactor the section on connection termination

-- File Changes --

    M draft-ietf-quic-transport.md (99)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/721.patch
https://github.com/quicwg/base-drafts/pull/721.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721

---

From: janaiyengar <[email-masked]>
Date: Mon, 14 Aug 2017 22:47:48 +0000 (UTC)

janaiyengar commented on this pull request.

Thanks, Martin. A few comments, inline.

> -received before sending a response.  After this time, implementations
-SHOULD respond to unexpected packets with a Public Reset packet.
-
-
-## Stateless Reset {#stateless-reset}
+of four ways: negotiated shutdown, idle timeout, immediate close, and a
+connection reset.
+
+
+### Negotiated Shutdown
+
+An application protocol might arrange to abandon a connection after negotiating
+a graceful shutdown.  The application protocol exchanges whatever messages that
+are needed to cause both endpoints to agree to close the connection, after which
+the connection is closed.  A negotiated shutdown might not result in exchanging
+messages that are visible to the transport.

I don't understand what QUIC mechanism this is discussing -- specifically, why is this a QUIC connection termination mechanism? It seems to suggest that the app can use its own signaling to determine when to shutdown, but what QUIC mechanism is used for shutdown?

> +the connection is closed.  A negotiated shutdown might not result in exchanging
+messages that are visible to the transport.
+
+
+### Idle Timeout
+
+A connection that remains idle for longer than the idle timeout (see
+{{transport-parameter-definitions}} becomes closed.  Either peer removes
+connection state if they have neither sent nor received a packet for this time.
+
+The time at which an idle timeout takes effect won't be perfectly synchronized
+on peers.  Endpoints might allow for the possibility that the remote side might
+attempt to send packets before the timeout.  In this case, an endpoint might
+choose to retain enough information to generate a CONNECTION_CLOSE.  Endpoints
+MAY instead rely on sending Public Reset in response to packets that arrive
+after an idle timeout.

I don't think this advice is correct or adequate -- the time can be synchronized reasonably well. I would leave this subsection as is, if it needs to change in the future anyways.

> +an endpoint maintains in this case, they MAY send the exact same
+CONNECTION_CLOSE packet.
+
+Note:
+
+: This intentionally contradicts other advice in this document that recommends
+  the creation of new packet numbers for every packet.  Sending new packet
+  numbers is primarily of advantage to loss recovery and congestion control,
+  which are not expected to be relevant for a closed connection.  Retransmitting
+  the final packet requires less state at the server.
+
+Implementations SHOULD limit the number of CONNECTION_CLOSE messages they
+generate.  For instance, an implementation could exponentially increase the
+number of packets that it receives before sending the packet containing
+CONNECTION_CLOSE.  Once enough time has passed to allow a peer to receive the
+CONNECTION_CLOSE, an endpoint SHOULD discard per-connection state and MAY

The current text says idle_timeout period of time, since that's the amount of time after which we expect the peer to drop state anyways. Is there a reason to change that?

SImilarly, the current text recommends throttling responses, which is really important. I would argue for retaining that.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#pullrequestreview-56223350

---

From: Martin Thomson <[email-masked]>
Date: Tue, 15 Aug 2017 01:27:57 +0000 (UTC)

martinthomson commented on this pull request.

There isn't a mechanism, but the QUIC connection goes away.

Do I need to say that explicitly?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#discussion_r133100888

---

From: Martin Thomson <[email-masked]>
Date: Mon, 14 Aug 2017 18:32:40 -0700

@martinthomson pushed 1 commit.

58ab5f6  Stateless reset

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/721/files/60546fe7e8241b244a215e6ee91c90834054e2c4..58ab5f6d7d37da3825b63089999b4e578323d76b

---

From: Martin Thomson <[email-masked]>
Date: Mon, 14 Aug 2017 18:32:41 -0700

I disagree about synchronization.  Say that I set a timeout of 10s.  At 9.8s, one endpoint sends a packet.  That packet gets lost somehow.  Now we have a difference of opinion about when the connection started being "idle" that is almost as long as the timeout duration.

I agree that we might want to have a better or clearer description of how to synchronize these things, but I was aiming to just make it clear that there is a potential problem in synchronization and that a stateless reset is the fallback.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#discussion_r133101297

---

From: Martin Thomson <[email-masked]>
Date: Tue, 15 Aug 2017 01:35:22 +0000 (UTC)

If the idle timeout is 2 minutes, then that is far larger than the time you might wait to have the other side receive the close.  Unreasonably so in my view.  2xRTT is probably adequate.

As for retaining text on throttling, that's right at the start of this paragraph: 

> Implementations SHOULD limit the number of CONNECTION_CLOSE messages they generate. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#discussion_r133101553

---

From: Martin Thomson <[email-masked]>
Date: Tue, 15 Aug 2017 23:00:35 +0000 (UTC)

Notes from editors chat: make the negotiated shutdown an "application shutdown" and explain that the application tells QUIC to die and then QUIC dies.

When the application shutdown happens, then we need to talk about what happens to streams.

Tie the CONNECTION_CLOSE case to the application shutdown case: you need to hang around for TIME_WAIT during which you acknowledge packets and nothing else really.  3*max(MIN_RTO, RTT) should be OK as a conservative start.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#issuecomment-322612345

---

From: Martin Thomson <[email-masked]>
Date: Tue, 22 Aug 2017 21:11:28 -0700

c4a246a  Add the concept of a draining period

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/721/files/58ab5f6d7d37da3825b63089999b4e578323d76b..c4a246a6da28841480ea23d4669253d10cf8f075

---

From: Martin Thomson <[email-masked]>
Date: Wed, 23 Aug 2017 04:21:42 +0000 (UTC)

I've introduced the notion of a "draining period" after close.  This is like the TCP TIME_WAIT period, but what happens depends on how the connection was closed.  On Jana's advice, I've used 3*max(MIN_RTO, RTT), recognizing that this is probably just a starting point and we can refine the exact value later.

During the draining period, the baseline here is that the endpoint generates ACK frames, but nothing else.  That's all we need for an application close (the success case).  For an idle timeout, the text permits reviving of the connection if new packets are received.  In immediate close, the text requires that the CONNECTION_CLOSE be sent, and permits re-sending the last packet to save on state space.

@janaiyengar, can you check to see that I've captured the intent well-enough here?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#issuecomment-324217020

---

From: Martin Thomson <[email-masked]>
Date: Tue, 22 Aug 2017 21:22:38 -0700

28e7cb2  Editorial cleanup after self-review

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/721/files/c4a246a6da28841480ea23d4669253d10cf8f075..28e7cb2b9a5afafe3b8512121609477d02b097aa

---

From: Nick Banks <[email-masked]>
Date: Wed, 23 Aug 2017 06:36:00 -0700

nibanks commented on this pull request.

+
+The time at which an idle timeout takes effect won't be perfectly synchronized
+on peers.  A connection enters the draining period when the idle timeout
+expires.  During this time, an endpoint that receives new packets MAY choose to
+restore the connection.  Alternatively, an endpoint that receives packets MAY
+signal the timeout using an immediate close.
+
+
+### Immediate Close
+
+An endpoint sends a CONNECTION_CLOSE frame to terminate the connection
+immediately.  A CONNECTION_CLOSE causes all open streams to immediately become
+closed; open streams can be assumed to be implicitly reset.  After receiving a
+CONNECTION_CLOSE frame, endpoints MUST NOT send additional packets on that
+connection.

What would be the harm in allowing the endpoint to ACK a CONNECTION_CLOSE frame/packet? It would allow for the side that initiated the close to safely, immediately end its draining period. The ACK wouldn't be required, but a useful signal if used.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#pullrequestreview-58080446

---

From: Mike Bishop <[email-masked]>
Date: Wed, 23 Aug 2017 16:22:35 +0000 (UTC)

MikeBishop commented on this pull request.

> @@ -1415,31 +1415,59 @@ TODO: see issue #161
 Connections should remain open until they become idle for a pre-negotiated
 period of time.  A QUIC connection, once established, can be terminated in one
 of four ways: negotiated shutdown, idle timeout, immediate close, and a

This list still refers to "negotiated shutdown" while the subsections refer to "application close."  Might want to align them.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#pullrequestreview-58135543

---

From: Mike Bishop <[email-masked]>
Date: Wed, 23 Aug 2017 09:29:18 -0700

+

This probably falls under the above clause:

> In all cases, it is possible to acknowledge packets that are received as normal, but other reactions might be preferrable depending on how the connection was closed.

However, that's in conflict with the MUST NOT in this paragraph.

+After a connection is closed for any reason, an endpoint might receive packets
+from its peer.  These packets might have been sent prior to receiving any close
+signal, or they might be retransmissions of packets for which acknowledgments
+were lost.
+
+The draining period persists for three times the maximum of minimum RTO
+(kMinRTOTimeout) or the round trip time (smoothed_rtt), see {{QUIC-RECOVERY}}
+for descriptions of these values.  During this period, new packets can be
+attributed to the correct connection and acknowledged, but the connection is no
+longer considered active and usable.
+
+Different treatment is given to packets that are received while a connection is
+in the draining period depending on how the connection was closed.  In all
+cases, it is possible to acknowledge packets that are received as normal, but
+other reactions might be preferrable depending on how the connection was closed.

"preferable", not "preferrable"

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#pullrequestreview-58138369

---

From: Martin Thomson <[email-masked]>
Date: Wed, 23 Aug 2017 23:56:54 +0000 (UTC)

+

Yeah, that's an error.  I think that the right answer here is to enter the draining period and to separate prohibit frames other than ACK, PADDING, and CONNECTION_CLOSE while in that state.  I think that addresses the concerned @nibanks has raised about acknowledging a CONNECTION_CLOSE.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#discussion_r134899536

---

From: Martin Thomson <[email-masked]>
Date: Wed, 23 Aug 2017 16:57:49 -0700

7220765  Receipt of CONNECTION_CLOSE commences draining

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/721/files/28e7cb2b9a5afafe3b8512121609477d02b097aa..72207659068826e9be662e04044cb354600d4183

---

From: Martin Thomson <[email-masked]>
Date: Thu, 24 Aug 2017 00:43:04 +0000 (UTC)

This is what you get for trying to clean things up: they end up being substantial.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#issuecomment-324500580

---

From: MikkelFJ <[email-masked]>
Date: Thu, 24 Aug 2017 08:42:54 -0700

Is there a scenario where the draining period can be truncated by sending ACK in response to CONNECTION_CLOSE and where CONNECTION_CLOSE is retransmitted if ACK is not received?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#issuecomment-324674516

---

From: Martin Thomson <[email-masked]>
Date: Thu, 24 Aug 2017 15:30:22 -0700

If an endpoint receives an ACK for a CONNECTION_CLOSE, it still needs to handle more packets (it remains in its draining period) because of the chance that packets sent before the ACK are still in transit.  Thus, the shortcut isn't really that useful here.  Obviously the endpoint that acknowledges the CONNECTION_CLOSE won't send any more packets, but if its own packets are still in flight, then it might get a few more CONNECTION_CLOSE frames (as a result of packets sent before it acknowledged the CONNECTION_CLOSE). 

The only real trimming we might do is to have the endpoint that sends the CONNECTION_CLOSE stop sending more CONNECTION_CLOSE frames in response to receiving more packets.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#issuecomment-324774857

---

From: Martin Thomson <[email-masked]>
Date: Fri, 25 Aug 2017 00:16:44 +0000 (UTC)

6861f9d  Stop sending _CLOSE when you get an ACK

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/721/files/72207659068826e9be662e04044cb354600d4183..6861f9d728d2f0e760c0b5e5e61a2f67cbab1161

---

From: Martin Thomson <[email-masked]>
Date: Fri, 25 Aug 2017 04:59:04 +0000 (UTC)

9218dcc  Might as well stop on CONNECTION_CLOSE as well

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/721/files/6861f9d728d2f0e760c0b5e5e61a2f67cbab1161..9218dcc4d4aeef892583845170c41053a0461135

---

From: Mike Bishop <[email-masked]>
Date: Fri, 25 Aug 2017 16:49:48 +0000 (UTC)

MikeBishop approved this pull request.

Very much nitpicks, and if you merged without them I wouldn't scream.

> -After receiving either a CONNECTION_CLOSE frame or a Public Reset, an
-endpoint MUST NOT send additional packets on that connection. After
-sending either a CONNECTION_CLOSE frame or a Public Reset packet,
-implementations MUST NOT send any non-closing packets on that
-connection. If additional packets are received after this time and
-before idle_timeout seconds has passed, implementations SHOULD respond
-to them by sending a CONNECTION_CLOSE (which MAY just be a duplicate
-of the previous CONNECTION_CLOSE packet) but MAY also send a Public
-Reset packet.  Implementations SHOULD throttle these responses, for
-instance by exponentially backing off the number of packets which are
-received before sending a response.  After this time, implementations
-SHOULD respond to unexpected packets with a Public Reset packet.
-
-
-## Stateless Reset {#stateless-reset}
+of four ways: negotiated shutdown, idle timeout, immediate close, and a

I think this comment got lost somewhere amongst the commits, so I'll re-add it:  You have "negotiated shutdown" on this line, but "application close" on 1449.  I think consistency would help.

+signal, or they might be retransmissions of packets for which acknowledgments
+were lost.
+
+The draining period persists for three times the maximum of minimum RTO
+(kMinRTOTimeout) or the round trip time (smoothed_rtt), see {{QUIC-RECOVERY}}
+for descriptions of these values.  During this period, new packets can be
+attributed to the correct connection and acknowledged, but the connection is no
+longer considered active and usable.
+
+Different treatment is given to packets that are received while a connection is
+in the draining period depending on how the connection was closed.  In all
+cases, it is possible to acknowledge packets that are received as normal, but
+other reactions might be preferable depending on how the connection was closed.
+An endpoint that is in a draining period MUST NOT send packets other than ACK,
+PADDING, or CONNECTION_CLOSE.

Nit:  These aren't packets, they're frames.

+(kMinRTOTimeout) or the round trip time (smoothed_rtt), see {{QUIC-RECOVERY}}
+for descriptions of these values.  During this period, new packets can be
+attributed to the correct connection and acknowledged, but the connection is no
+longer considered active and usable.
+
+Different treatment is given to packets that are received while a connection is
+in the draining period depending on how the connection was closed.  In all
+cases, it is possible to acknowledge packets that are received as normal, but
+other reactions might be preferable depending on how the connection was closed.
+An endpoint that is in a draining period MUST NOT send packets other than ACK,
+PADDING, or CONNECTION_CLOSE.
+
+Once the draining period has ended, an endpoint SHOULD discard per-connection
+state.  This results in new packets on the connection being discarded.  An
+endpoint MAY use a stateless reset in response to any further incoming packets.

use => send

> +{{transport-parameter-definitions}}) becomes closed.  Either peer removes
+CONNECTION_CLOSE frame, endpoints immediately enter a draining period.

Endpoints presumably enter a draining period after sending a CONNECTION_CLOSE as well, no?

+restore the connection.  Alternatively, an endpoint that receives packets MAY
+signal the timeout using an immediate close.
+
+
+### Immediate Close
+
+An endpoint sends a CONNECTION_CLOSE frame to terminate the connection
+immediately.  A CONNECTION_CLOSE causes all open streams to immediately become
+closed; open streams can be assumed to be implicitly reset.  After receiving a
+CONNECTION_CLOSE frame, endpoints immediately enter a draining period.
+
+An endpoint that sends a CONNECTION_CLOSE frame SHOULD respond to any packet
+that it receives in the draining period with another packet containing a
+CONNECTION_CLOSE frame.  To reduce the state that an endpoint maintains in this
+case, they MAY send the exact same packet.  However, endpoints SHOULD limit the

they => it, since it's a single endpoint

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#pullrequestreview-58704654

---

From: Martin Thomson <[email-masked]>
Date: Sun, 27 Aug 2017 21:13:04 -0700

a215b53  Minor editorial things

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/721/files/9218dcc4d4aeef892583845170c41053a0461135..a215b5312f4a86c45caf13bed624179085196e46

---

From: Martin Thomson <[email-masked]>
Date: Wed, 30 Aug 2017 06:31:22 +0000 (UTC)

@janaiyengar, this really needs your review.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/721#issuecomment-325895490