Subject: Re: [quicwg/base-drafts] Loosen the MUST requirement to close if an unsent packet is acknowledged (#565)

From: ianswett <[email-masked]>
Date: Sun, 04 Jun 2017 14:18:48 -0700

ianswett commented on this pull request.

Agreed this needs to be fixed.

> @@ -1590,7 +1590,7 @@ frames. A receiver MAY send timestamps for non-retransmittable packets.
 A receiver MUST not send timestamps in unprotected packets.
 
 A sender MAY intentionally skip packet numbers to introduce entropy into the
-connection, to avoid opportunistic acknowledgement attacks.  The sender MUST
+connection, to avoid opportunistic acknowledgement attacks.  The sender SHOULD

It could also say "The sender MUST close the connection if a packet it knows is unsent is acknowledged."  Which keeps the must, but does not require unlimited state.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#pullrequestreview-41951351

---

From: Martin Thomson <[email-masked]>
Date: Mon, 05 Jun 2017 02:06:31 -0700

martinthomson commented on this pull request.

@mcmanus, @siyengar, does Ian's suggestion work better?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#discussion_r120069281

---

From: ekr <[email-masked]>
Date: Mon, 05 Jun 2017 03:50:20 -0700

Before we spend a lot of time bikeshedding this, maybe we should consider whether we want to adopt #509 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306159820

---

From: Martin Thomson <[email-masked]>
Date: Mon, 05 Jun 2017 03:57:43 -0700

The same general question applies even if we adopt #509.  Do we require that the sender track what it has sent in order to be able to correctly validate an ACK?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306161027

---

From: MikkelFJ <[email-masked]>
Date: Mon, 05 Jun 2017 04:05:00 -0700

More generally, I believe an earlier draft had a concept of the oldest known package number which got replaced by ACK'ing acks periodically.

I think it would be good if the protocol maintains a concept of what is the oldest packet number still of interest to the sender such that the receiver can promptly drop state older than this value for dupe rejection etc., and also such that received acks pertaining packets older than this value can promptly be dropped (or at least the older part of the ACK frame).

Any packet number newer than this lower bound could then be required to be remembered and invalid acks SHOULD/MUST be rejected.

An adversarial sender could fail to update the lower bound forcing more state at receiver. The current behavior is safe in the sense that there is no lower bound other than heuristics anyway, but if a lower bound was placed, this issue must be considered.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306162196

---

From: ianswett <[email-masked]>
Date: Mon, 05 Jun 2017 04:26:39 -0700

@mikkelfj I don't think the least unacked value that was in the stop waiting helps here, except as a convenient lower bound on the range the MUST applies to.  Even without the frame, there must be a lower bound on the range of tracked packets sender side.

For the record, I'm fine with my suggested wording or Martin's PR, I just was trying to provide an option if we want to keep the MUST.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306165684

---

From: MikkelFJ <[email-masked]>
Date: Mon, 05 Jun 2017 04:31:54 -0700

An explicit lower bound might significantly reduce the state the receiver needs to maintain and perhaps the frequency of both acks and ack acks - at least that is my intuition. But it is correct that some bound must be maintained regardless.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306166492

---

From: MikkelFJ <[email-masked]>
Date: Mon, 05 Jun 2017 04:33:35 -0700

But that discussion is perhaps more relevant to https://github.com/quicwg/base-drafts/issues/423

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306166739

---

From: Patrick McManus <[email-masked]>
Date: Mon, 05 Jun 2017 04:51:07 -0700

mcmanus commented on this pull request.

"it knows" isn't really verifiable in any normative sense - it pretty much means SHOULD. prefer SHOULD.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#discussion_r120092997

---

From: ekr <[email-masked]>
Date: Mon, 05 Jun 2017 04:52:27 -0700

Not exactly. You won't be able to validate the ACK at all without keeping exact track, so either you are implementing the defense or you're not, but the implications are clear either way. In any case, you won't need it for basic ACK functionality

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306169805

---

From: Patrick McManus <[email-masked]>
Date: Mon, 05 Jun 2017 04:55:42 -0700

if you get a duplicated ack from long ago, how do you know if it is (dup) acking a packet you sent or not? (assuming you skipped some numbers non deterministically)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306170314

---

From: MikkelFJ <[email-masked]>
Date: Mon, 05 Jun 2017 05:08:27 -0700

> if you get a duplicated ack from long ago, how do you know if it is (dup) acking a packet you sent or not? (assuming you skipped some numbers non deterministically)

The sender maintains an oldest relevant packet number regardless of whether it is transmitted or not (it currently isn't). If the ack refers to an older packet it is ignored - even if it could be by malicious intent, it cannot really do any harm anymore. If the ack refers to a more recent packet number, it could be used to confuse the bandwidth estimation algorithm and causing the sender to DoS the receiver, which harms the ack sender, but also any other connection to the server.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306172296

---

From: Subodh Iyengar <[email-masked]>
Date: Mon, 05 Jun 2017 06:19:57 -0700

siyengar commented on this pull request.

Generally I worry with a SHOULD no-one will end up implementing it because people will realize that it requires maintaining a lot of state. I like the spirit of Ian's wording but it's super confusing. SHOULD seems like the right thing here. I think if we acknowledge why it is a SHOULD it might convince more people to implement it along with a suggestion of how to implement it. 

For example a sender could implement this by keeping track of all the packets sent and by advancing a smallest ackable packet periodically.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#discussion_r120105701

---

From: MikkelFJ <[email-masked]>
Date: Mon, 05 Jun 2017 06:34:34 -0700

mikkelfj commented on this pull request.

The wording could also be "SHOULD close a connection when receiving an ACK for a packet that has not been sent unless the packet is older that oldest retransmittable packet in which case the ackowledgement of that packet should be ignored."

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#discussion_r120108448

---

From: Subodh Iyengar <[email-masked]>
Date: Mon, 05 Jun 2017 07:29:18 -0700

I was thinking that when you advance the ACK lower bound you would call getting an ACK for a packet below that bound would be an error. There are several triggers that could be used to advance the lower bound of ACKs. 

* If you keep track of the ACK of the ACK of a packet sequence number you could maintain a variable called
mininum_acceptable_packet = smallest packet that we're keeping state for, for which we have received the ACK of the ACK for the receipt of the packet. This puts a lower bound on the state while tolerating 1 RTT of reordering

* You could just use RTTs, i.e after getting an ACK for a packet and then a few RTTs have elapsed just advance the minimum_acceptable_packet

a reasonable receiver would not send ACks in such a delayed manner which could be assumed.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306201984

---

From: MikkelFJ <[email-masked]>
Date: Mon, 05 Jun 2017 07:58:51 -0700

> I was thinking that when you advance the ACK lower bound you would call getting an ACK for a packet below that bound would be an error. 

I believe @ianswett mentioned that GQUIC through live testing found it better to acknowledge latest packets over earliest acknowledge packets at the possible cost of retransmission. That is, when you have to make a choice due to ACK frame size constraints. But, it is not clear exactly how to handle ACKS that do not cover the full range of received packets, as discussed in  #423
The sender also cannot trust the ACK

Rant: I am personally not too happy with the current ACK concept that seems open to many interpretations and risk holding a lot of excessive state both for dupe detection and retransmission of ACK frames. I would much rather have a concept such as STOP_WAITING, or a similar simple range of fully handled packet number ranges without needing the details of which ACK frames have been sent or in need of transmission. Other retransmissions can be stored by reference to external data (mmap etc.).

> a reasonable receiver would not send ACks in such a delayed manner which could be assumed.

A major part of the excercise is dealing with unreasonable receivers.  Also, for low powered devices or for avoiding unnecessary network activity that triggers context switches every time a packet is received, slowing down busy systems, ACKs could quite well be rather delayed. Only, this generates more state that must be kept. As long as this state are only packet ranges acked or not, this is not a big problem.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-306209874

---

From: Mike Bishop <[email-masked]>
Date: Wed, 07 Jun 2017 07:16:32 -0700

MikeBishop commented on this pull request.

But the retransmission queue may be sparse -- you don't necessarily know whether a packet no longer there was never sent or was simply already acknowledged.

SHOULD is supposed to have an explanation of why you might not.  You SHOULD do this if you detect it, but you're not required to maintain enough state to detect it 100% of the time.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#discussion_r120637525

---

From: MikkelFJ <[email-masked]>
Date: Wed, 07 Jun 2017 07:21:12 -0700

@MikeBishop I think I agree but isn't 

> But the retransmission queue may be sparse -- you don't necessarily know whether a packet no longer there was never sent or was simply already acknowledged.

a good reason for SHOULD?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#discussion_r120638959

---

From: Martin Thomson <[email-masked]>
Date: Fri, 09 Jun 2017 06:27:57 -0700

Merged #565.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#event-1117244676

---

From: Martin Thomson <[email-masked]>
Date: Fri, 09 Jun 2017 06:31:32 -0700

Great discussion everyone.  I don't think that I can recognize this as a bike shed given the many layers of paint it now sits under.

We discussed this at the interim and concluded that we would remove the ability to have gaps (see https://github.com/quicwg/base-drafts/issues/161#issuecomment-307387364), but this uncertainty still exists to some extent due to the NEW_CONNECTION_ID frame and the use of packet number skipping in that context.  This will remove one source of gaps in the packet number sequence and make detecting ACKs for unsent packets easier, but not guaranteed.

Thus, I think that we have a SHOULD here and we can point people who ask at this wonderful thread if they want to understand all the gory details.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/565#issuecomment-307389200