Subject: [quicwg/base-drafts] Split headers/data streams (#245)

From: Mike Bishop <[email-masked]>
Date: Sun, 29 Jan 2017 16:00:25 -0800

In Tokyo, we agreed that headers streams would no longer be exempt from connection-level flow control.  But the reason that headers and data streams are separate is so that one can be exempt and the other not.  So.... why are we keeping them separate again?

It might be simpler, particularly with the specter of #175 around, to bring back DATA frames and say everything's on one stream.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245

---

From: Martin Thomson <[email-masked]>
Date: Sun, 29 Jan 2017 16:48:30 -0800

The primary reason I have for maintaining separation is #176.  At least with separate streams, you can cancel a request without having to also cancel headers.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245#issuecomment-275960094

---

From: Marten Seemann <[email-masked]>
Date: Sun, 29 Jan 2017 17:28:21 -0800

If I remember correctly, during this discussion we discovered that having multiple HTTP parsers operating at the same time requires a huge memory commitment on the receiving side (issue #238). Furthermore, having a smaller number of HTTP parses running than there are concurrent requests potentially leads to a deadlock caused by connection-level flow control.

The problem with HTTP/2 DATA frames is that it requires a layer violation: Since the remaining size of the packet is not known to the H2 layer, it can't create the DATA frame. This would have to be done by the QUIC layer, which ideally should be completely independent from the protocol that it is used by.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245#issuecomment-275963177

---

From: Martin Thomson <[email-masked]>
Date: Sun, 29 Jan 2017 17:38:20 -0800

That deadlock exists as long as you have the ability to fill the connection window with non-header data, so it's more a problem that is created by having headers governed by flow control.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245#issuecomment-275963941

---

From: ianswett <[email-masked]>
Date: Sun, 29 Jan 2017 18:04:49 -0800

Re: 2 streams per HTTP stream, I agree with Martin that the reason is you can't cancel headers streams.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245#issuecomment-275966179

---

From: Mike Bishop <[email-masked]>
Date: Sun, 29 Jan 2017 18:20:30 -0800

@marten-seemann -- no layer violation.  DATA frames are an HTTP/2 concept, and would be on the same stream with everything else.

I should clarify my original statement of the issue -- could it be more sensible to fix this in HPACK instead of having two streams just so one of them is a cancel target?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245#issuecomment-275967457

---

From: Martin Thomson <[email-masked]>
Date: Sun, 29 Jan 2017 19:26:24 -0800

I think that @marten-seemann's point is that you have to drive frame-size decisions based on packetization.  You don't want to split a DATA frame across packets, but you don't know where a DATA frame will end up until packets are being constructed.  The HTTP layer can no longer just flush data into a buffer and leave it at that.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245#issuecomment-275973166

---

From: Mike Bishop <[email-masked]>
Date: Sun, 29 Jan 2017 19:38:25 -0800

Perhaps I'm being stupid because I'm stuck in an airport, but... why not?  There's no requirement that the edges of the data frames line up with the packetization.  When the HTTP layer reads it, it's a bytestream.  That byte-stream contains frames -- or call them regions, if you prefer.  Those frame boundaries are independent of STREAM frame or packet boundaries.

The ugly situations I can envision are that an implementation *could* shoot itself in the foot with trailers:  QPACK encode the headers, dump a giant file into the transport, QPACK encode the trailers.  If they're all on-stream, anything that depends on the trailers (potentially other requests) will be blocked behind the giant body.  You still don't deadlock, though, because nothing stops that stream from making progress.  But you can generate some very nasty head-of-line blocking.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245#issuecomment-275974129

---

From: Martin Thomson <[email-masked]>
Date: Sun, 29 Jan 2017 21:07:37 -0800

Ahh, you are right, as long as the implementation is willing to deliver partial DATA frames, there isn't a performance penalty.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245#issuecomment-275981151

---

From: Mike Bishop <[email-masked]>
Date: Sun, 29 Jan 2017 22:23:25 -0800

(Note that the nasty trailer situation is equally possible on two streams if the receiving implementation doesn't read the trailers until the body has arrived....)

If we went that way, I  would expect the DATA frames to be large -- either the size of the content, or the size of the chunks being handed off to the HTTP stack by the next layer.  However, there are comments on the list that being able to have framing "get out of the way" once you've hit body is attractive.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245#issuecomment-275988022

---

From: Patrick McManus <[email-masked]>
Date: Mon, 30 Jan 2017 05:53:51 -0800

I'm sympathetic to having framing get out of your way, and the 2 streams interact intuitively with cancel. So in general the split is something I support. 

maybe worth noting the link frame stuff from the unidirectional #175 concept is worth reading when thinking about where this might end up.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/245#issuecomment-276067683