Subject: Re: [quicwg/base-drafts] Public Flags to Aid Troubleshooting (#279)

From: Martin Thomson <[email-masked]>
Date: Wed, 08 Mar 2017 16:24:48 -0800

The flow control one might run afoul of attacks like [HEIST](https://www.blackhat.com/docs/us-16/materials/us-16-VanGoethem-HEIST-HTTP-Encrypted-Information-Can-Be-Stolen-Through-TCP-Windows-wp.pdf) which benefit from being able to observe flow control.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285214683

---

From: ianswett <[email-masked]>
Date: Wed, 08 Mar 2017 16:45:20 -0800

Re 2, Flow control issues are to easily monitor end-to-end, and in QUIC not modifiable by middleboxes, so QUIC removes responsibility for flow control issues from the network operators entirely.  As a person who's debugged my share of flow control issues, I'm trying to imagine a case where you or a network operator would have to debug flow control issues with QUIC, and I can't come up with one.  

Re 3, That's congestion control's domain, not flow control(which uses BLOCKED), and definitely feels like it's in the sender's domain.  In most applications, the sender is congestion control limited a large portion of the time, so I wouldn't expect this to be a useful signal, as most connections would see it.

I'm not excited about putting ECE/CWR bits in the unencrypted portion of the packet, as I can't imagine a use case.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285218253

---

From: martinduke <[email-masked]>
Date: Wed, 08 Mar 2017 16:52:24 -0800

I agree that there is no action a middlebox can take on flow control. As I said above, I think this is a "stop worrying" bit when service providers are trying to diagnose performance issues, which is logistically hard to do at endpoints.

ECE/CWR in the public flags would be sort of like the LOSSDETECT bit, if we wanted to separate those signals. But I don't think it would hurt much to encrypt them, as long as LOSSDETECT is there.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285219482

---

From: ianswett <[email-masked]>
Date: Wed, 08 Mar 2017 17:06:25 -0800

I understand the "stop worrying" bit, but I'm hoping flow control is rarely the limiting factor for QUIC.  TCP has suffered from a variety of flow control problems, but QUIC doesn't inherit any of those issues.

That's not to say all implementers of QUIC will implement flow control correctly, but it's more likely website A's upload performance will be awful than it will be a widespread problem.  As such, it's incentive compatible for both the client and server to implement flow control well, because otherwise performance will suffer.  

For example, I believe Chrome's flow control window is 8MB, which is vastly larger than the TCP defaults.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285221962

---

From: Martin Thomson <[email-masked]>
Date: Wed, 08 Mar 2017 17:12:58 -0800

To add to what @ianswett says, Firefox uses a different strategy for h2.  While you might see flow control limits being hit on individual streams (push in particular), that doesn't mean that there is a problem.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285223090

---

From: martinduke <[email-masked]>
Date: Wed, 08 Mar 2017 17:23:23 -0800

Obviously if BLOCKED frames are exceedingly rare, then the it's not worth the real estate in the public header. But then, then (different issue) perhaps BLOCKED doesn't make any sense either. 

But if they're not exceedingly rare...
> As such, it's incentive compatible for both the client and server to implement flow control well, because otherwise performance will suffer.

This is true of TCP, too. HOL blocking problems are much diminished in QUIC, but not quite eliminated in the case of large streams.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285224936

---

From: Martin Thomson <[email-masked]>
Date: Wed, 08 Mar 2017 17:31:45 -0800

@martinduke, Google also proposed BLOCKED for h2.  You can see for yourself the results of that.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285226387

---

From: ianswett <[email-masked]>
Date: Wed, 08 Mar 2017 17:45:04 -0800

BLOCKED frames are rare, but tend to be isolated to a small number of services, which either are slow at consuming data or serve clients with much higher than average bandwidth.  As such, they serve as an excellent monitoring and debugging tool if you understand the full context.

If this was available in QUIC, what use cases would you expect it to help with?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285228785

---

From: martinduke <[email-masked]>
Date: Thu, 09 Mar 2017 13:58:18 -0800

> If this was available in QUIC, what use cases would you expect it to help with?

Are you asking me to defend BLOCKED frames, or the header bit? In the use case where BLOCKED would be helpful, the bit would be helpful for monitoring performance of provider networks. The BLOCKED _frame_ makes window-limitation more obvious, but if you have the decrypted payload, you can also glean this from WINDOW_UPDATE frames.

I can see three possibilites:
- window-limited connections are a common occurrence in QUIC: in this case, the header bit is essential to have any idea what is a real network performance problem and what comes from client configuration. In some cases, it might lead to a mobile software update that fixes the config.
- window-limited connections are extremely rare: it is not worth it to take a bit in the header. In the extreme limit, there is no need for flow control at all because QUIC endpoints have ample memory resources and reduced HOL blocking.
- window-limited connections are "sort of" rare: in which case we ought to have flow control and BLOCKED, but it's not worth a valuable public header bit. I _think_ this is where you believe we are.

Obviously, I have no real-world QUIC deployment experience, so I can't state with authority where we are on this spectrum, and if current experience is relevant to a future with IoT devices, etc. All I can say is that in TCP the flow control information is very valuable to have, and it's not hard to see how certain applications and implementations in QUIC might have the same problems.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285496787

---

From: martinduke <[email-masked]>
Date: Thu, 09 Mar 2017 15:55:04 -0800

@martinthomson I have now taken a look at HEIST, and I won't pretend to fully understand it. But I'm having trouble seeing if a BLOCKED bit will generally introduce a larger problem.

To defeat the cwnd attack, I believe QUIC has to insert PADDING frames prior to the full delivery of the last STREAM data, to make the delivery of the resource consistent (in terms of RTTs) regardless of echoed requests. This is all orthogonal to BLOCKED, of course. Even this only works if PADDING counts against cwnd, which I don't believe is established.

If we can resolve this issue, there has to be access to the IFCW to have a problem. This is in the clear, but  I'm not sure it's visible to the javascript the attacker has installed on the browser. But if the attacker can overcome these issues, I can see a problem where a blocked bit, specifically tied to connection flow control, can indicate when the actual payload has exceeded the flow control. If we reach this point, I would probably abandon the idea of having a blocked bit specifically related to connection flow control, and instead make it deliberately ambiguous between stream and connection flow control. That probably makes things ambiguous enough to defeat this attack.

Anyway, I'd have to define exactly what the blocked bit is, to have a proposal that security guys could properly shoot at. The intent of this issue was to gauge general interest in a PR that would specify one or both concepts. The signal I'm getting is mixed, to say the least. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285523898

---

From: ianswett <[email-masked]>
Date: Thu, 09 Mar 2017 14:35:40 -0800

Yes, I believe we're in the third case, at least today.  I may be able to produce some stats on what percentage of connections see blockage with existing settings, but it's very close to 0 today from server to client.  It's low, but not that close to 0 from client to server, where the flow control windows are much smaller.

But yes, it will depend upon the application, and so I can't forsee every possible use case.

I'm also a bit concerned about privacy leakage, as Martin pointed out, even though I wouldn't anticipate that being an information rich channel.

I'm happy to defend BLOCKED frames.  They've been amazingly useful in debugging and monitoring specific services and detecting implementation bugs.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-285505854

---

From: martinduke <[email-masked]>
Date: Tue, 28 Mar 2017 17:08:19 -0700

See the PR in #418 so that there is a coherent proposal to shoot at.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/279#issuecomment-289942484