Subject: [quicwg/base-drafts] Transport: Wrongly hinting of issues not existing for packet number size increase (#605)

From: Magnus Westerlund <[email-masked]>
Date: Wed, 07 Jun 2017 05:47:15 -0700

draft-ietf-quic-transport-03, section 5.8:

   The sender MUST use a packet number size able to represent more than
   twice as large a range than the difference between the largest
   acknowledged packet and packet number being sent.  A peer receiving
   the packet will then correctly decode the packet number, unless the
   packet is delayed in transit such that it arrives after many higher-
   numbered packets have been received.  An endpoint MAY use a larger
   packet number size to safeguard against such reordering.

I don't quite understand the second sentence here. If I understand the first sentence with the MUST, that forces the sender to increase the packet number least significant bit with additional byte(s) to fit the outstanding part of the sequence number space so that it doesn't wrap when having outstanding packets from the senders perspective. Thus, even the very old packet case should only force the sender to increase the size of the field while it is outstanding. There shouldn't be a uncertainty for the receiver. This as the higher bits added will match the receivers representation for the highest acked packet number without any outstanding packet prior to it. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605

---

From: MikkelFJ <[email-masked]>
Date: Wed, 07 Jun 2017 06:11:30 -0700

The sender cannot control network properties. There might be some old packet already retransmitted packets suddenly resurfacing, albeit unlikely.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-306790071

---

From: Martin Thomson <[email-masked]>
Date: Mon, 19 Jun 2017 23:05:20 -0700

There is a real issue here.  Say that a sender sends 100 packets with 8-bit packet numbers.  That's perfectly OK according to the rules in the draft.

If the sender then immediately send another 1000 packets while those 100 are outstanding, the new packets will have 16-bit packet numbers.  But the sender can't retroactively increase the size of the packet numbers of packets already sent.  And those packets might be reordered with respect to these newer packets.

If a packet from the set with 8-bit packet numbers is delayed and arrives after receiving a higher numbered packet from the 1000 packets, then the shorter packet number could be decoded incorrectly.  For example, assuming the recommended decoding algorithm, packet 16 encoded on 8 bits will be incorrectly decoded as packet 272 if packet 144 or higher has been received.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-309653680

---

From: Patrick McManus <[email-masked]>
Date: Tue, 20 Jun 2017 00:17:34 -0700

yes; you can be semi-blocked from sending by the range of the lowest unacked packet you have outstanding.

I say semi-blocked (as opposed to say, deadlocked) because you can just move to the bigger packet number space and essentially declare the older uacked ones presumed-lost. They'll fail decryption at the receiver when given the 'wrong' packet number.

8 bit numbers  aren't especially useful.. perhaps banning them as footguns would be a practical step to help quality of implementations.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-309666359

---

From: MikkelFJ <[email-masked]>
Date: Tue, 20 Jun 2017 01:19:55 -0700

> I say semi-blocked (as opposed to say, deadlocked) because you can just move to the bigger packet number space and essentially declare the older uacked ones presumed-lost. 

This will not work if it is decided to require gapless packet numbers except on path migration - something I'd argue for.

#595 suggests variable length integers which I think is a good idea.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-309680149

---

From: Patrick McManus <[email-masked]>
Date: Tue, 20 Jun 2017 05:24:39 -0700

I don't think gapless creates a problem with the existing spec (though I also support #595). Maybe I'm misunderstanding you.

Gap is a property of the sender, and there would be no gap just by virtue of moving to a bigger encoding - that's obvious to the receiver if the packets are delivered in order but its true whether the receiver realizes it or not.  So out of order delivery would basically just be a form of loss/corruption (the misinterpreted number would cause a decrypt fail) fixed in the usual way (retransmit of data). 

To me, the most important part of this issue is that the current design is prone to this self-induced loss and while not a critical fail, it should be improved.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-309736878

---

From: MikkelFJ <[email-masked]>
Date: Tue, 20 Jun 2017 05:45:32 -0700

>  Maybe I'm misunderstanding you.

Or may be the opposite - I thought you would introduce a gap in the packet number space while advancing the larger range, but that no longer makes sense to me.

Either way, I'm  not sure there is a problem:

If you send 100 packets in 8-bit encoding, then decide to send 100000 packets more, you encode 8 bit until packet 127, use 16 bit until packet 32767, and 32 bit after that.

Regardless of delivery order, there shouldn't be any ambiguity about the packer number.

But this isn't exactly how things are specified currently - it uses a diff against largest received or something like that.

If the spec would state that only half the representable range must be used for any given encoding, except for 64 bit, then I think it should be working.

Simpler yet, use varlen.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-309741686

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 20 Jun 2017 06:04:10 -0700

@mikkelfj I think you are correct that this can be resolved by changing the rules for how one handle or recover the full packet number. I think the current text actually requires one to not send any more packets when one reaching having half the current smallest packet field space as diff between next packet and oldest outstanding. Thus, one could actually deadlock, which is bad. 

If one statys with the current style of the packet numbers, I would agree with @mcmanus 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-309746385

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 20 Jun 2017 06:04:19 -0700

Reopened #605.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#event-1130942632

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 20 Jun 2017 06:04:12 -0700

Closed #605.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#event-1130942345

---

From: ianswett <[email-masked]>
Date: Tue, 20 Jun 2017 09:52:07 -0700

@martinthomson The concern you point out is real, though as @mcmanus points out, it just causes a few packets to be dropped when they can't be decrypted in the worst case.

Reordering is somewhat common, but reordering of >100 packets is pretty rare, so even 1 byte works very well in the real world.  

I remember there being some text on how to choose the packet number length, but I can't find it anymore.  The early draft(https://tools.ietf.org/html/draft-hamilton-quic-transport-protocol-00) had this text, which I think may have been lost when the packet header was changed?

"To enable unambiguous
      reconstruction of the packet number by the receiver, a QUIC
      endpoint MUST NOT transmit a packet whose packet number is larger
      by (2^(bitlength-2)) than the largest packet number for which an
      acknowledgement is known to have been transmitted by the receiver.
      Therefore, there must never be more than (2^46) packets in flight.
      Any truncated packet number received from a peer shall be inferred
      to have the value closest to the one more than the largest known
      packet number received from that peer.  The transmitted portion of
      the packet number matches the lowest bits of the inferred value."

To be clear, I think this text can be improved, but it's important to have a recommendation both based on number of packets currently outstanding and number that may be sent in the near future(likely limited by CWND).  

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-309820441

---

From: ianswett <[email-masked]>
Date: Tue, 20 Jun 2017 10:03:54 -0700

After re-reading the existing text, I think we need to add some text about the largest packet number likely to be sent before a larger packet is acknowledged.

Specifically, I'd change "An endpoint MAY use a larger packet number size to safeguard against such reordering." to "An endpoint SHOULD use a large enough packet number size to allow any packets sent in the near future be unambiguously decoded."

Also @gloinul , I can't find any text that would create a deadlock.  The only MUST is: "The sender MUST use a packet number size able to represent more than twice as large a range than the difference between the largest acknowledged packet and packet number being sent."

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-309823683

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 21 Jun 2017 01:28:55 -0700

@ianswett sorry for being unclear. I don't think the limitation do exist in the text to cause an actual deadlock. I meant if one would enforce to changes of packet number to happen so that no receiver confusion would be possible then a deadlock could occur. 

What I think needs to happen is that one clarify a couple of things:
1. When one increase the packet number field size, that can happen at any time because the number of outstanding packets becomes to large.
2. When one increase, then I think the receiver processing should be clarified, so that it would be clear how the shorter packet numbers relates to the larger ones to avoid the issue of wrongly reconstructing the full packet number. 
3. The conditions in which one can shrink the packet number field needs to be clear so that one minimizes the risk that packets gets wrongly reconstructed packet numbers. These recommendations will of course need to take 2. into account. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-310005793

---

From: ianswett <[email-masked]>
Date: Wed, 21 Jun 2017 07:22:13 -0700

The text for 1 and 3 should be the same, since it's just a matter of determining what the right length is at any given moment.  I don't think anything special needs to be noted about increases or decreases, but Martin's PR should make it clearer how to pick a length.

Re 2, I think the receiver processing is quite simple and works correctly without any special language about shorter vs longer numbers, so I'd prefer to avoid it.  Is there a specific case you think is confusing?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#issuecomment-310094191

---

From: Martin Thomson <[email-masked]>
Date: Wed, 21 Jun 2017 17:59:43 -0700

Closed #605 via #649.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/605#event-1133918982