Subject: [quicwg/base-drafts] Streams do not need to be implicitly opened (#662)

From: janaiyengar <[email-masked]>
Date: Fri, 23 Jun 2017 12:00:41 -0700

Opening a stream opens all streams with lower numbered stream IDs. IIRC, this was put in place when we had max open streams. Now that we have max stream IDs, this isn't required anymore. We can remove implicit open entirely.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662

---

From: Subodh Iyengar <[email-masked]>
Date: Fri, 23 Jun 2017 13:46:54 -0700

I think this still needed to make accounting of closed and open streams more efficient otherwise you would have to explicitly keep track of streams that are currently closed instead of only keeping track of streams that are currently open

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310769122

---

From: Igor Lubashev <[email-masked]>
Date: Fri, 23 Jun 2017 14:33:53 -0700

I think it can work.  Namely, implementation will typically have an internal idea about `max_open_streams` value they want to enforce.  They will want to keep a count of `max_explicitly_closed` streams (i.e. STREAM+FIN and RST_STREAM frames they've received for unique streams).  Then, they should send MAX_STREAM_ID = `max_open_streams + max_explicitly_closed`.

Internally, implementations should keep track of both `open` streams and 'still unopened' streams (streams up to MAX_STREAM_ID that are not `open` and have not been explicitly closed).  There would be at most `max_open_streams` streams there.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310778194

---

From: krasic <[email-masked]>
Date: Fri, 23 Jun 2017 14:41:13 -0700

+1

A closed stream is one that is not open or "still unopened".     The cardinality of those two sets is exactly max_open_streams, which seems like an not unreasonable amount of accounting state.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310779450

---

From: Igor Lubashev <[email-masked]>
Date: Fri, 23 Jun 2017 14:51:13 -0700

> A closed stream is one that is not open or "still unopened".

Just to be clear, `A closed stream is one that is not (open or "still unopened").`

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310781053

---

From: Subodh Iyengar <[email-masked]>
Date: Fri, 23 Jun 2017 14:58:20 -0700

If you have to keep track of max unopened anyway, what benefit does opening an out of order streams without opening lower streams have? I know it's possible to do it either way, however for a normal sender / receiver keeping track of only open streams can necessarily be more efficient than keeping track of unopened streams. Is this proposal more of a philosophical point or does it solve some concrete problem? Having lower streams open up has a bunch of desirable properties, so removing them should probably have some benefits.

We are also planning to maybe use the current behavior (opening a higher stream implicitly opens lower streams) in the http mapping to reorder the body and header streams so that the header stream implicitly opens the body stream.

Another aspect to this this proposal (of not opening lower streams), is that it makes max stream id advertisement less intuitive and more implicit, which might result in deadlock like bugs in the future. In the current version, it makes it more explicit that I need to close a stream to get more future streams, vs I would need to reset an idle stream to open up more streams.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310782162

---

From: Igor Lubashev <[email-masked]>
Date: Fri, 23 Jun 2017 15:08:45 -0700

> what benefit does opening an out of order streams without opening lower streams have

Doing so by the sender deliberately has no benefit. However, receiver may get streams opened out of order anyway due to packet reordering/loss, and it needs to deal with this.

> Is this proposal more of a philosophical point or does it solve some concrete problem?

This simplifies state tracking for @ianswett's #656 PR -- implicitly opened streams could no longer be considered just "open".  I do not see a significant benefit otherwise.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310783720

---

From: Subodh Iyengar <[email-masked]>
Date: Fri, 23 Jun 2017 15:12:43 -0700

ya i guess you wouldn't know whether an implicitly opened stream was unidirectional or not, unless you got the stream data for it which kind of makes sense, but that proposal is not done yet, so maybe we should wait on it being more fleshed out. 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310784335

---

From: krasic <[email-masked]>
Date: Fri, 23 Jun 2017 16:12:30 -0700

Maybe a minor point, but the memory size of the state for "still unopened" streams, i.e. an entry in a membership set, should be much smaller than for an opened stream, i.e. the per-stream object state.   

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310792322

---

From: krasic <[email-masked]>
Date: Fri, 23 Jun 2017 16:24:46 -0700

On the HTTP mapping and stream pairs:  you can't count on the order of arrival, a frame for the body stream could arrive before any frames of the header.   So implicit open didn't fully solve that anyway.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310793978

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 23 Jun 2017 17:07:41 -0700

Oh, this is very clever. I quite love it!

On Fri, Jun 23, 2017 at 4:24 PM, krasic <[email-masked]> wrote:

> On the HTTP mapping and stream pairs: you can't count on the order of
> arrival, a frame for the body stream could arrive before any frames of the
> header. So implicit open didn't fully solve that anyway.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/quicwg/base-drafts/issues/662#issuecomment-310793978>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ASp6ytCLSRfvB7-bmuH7c8T8BebnyzGhks5sHEk9gaJpZM4OD7iR>
> .
>

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310798332

---

From: ianswett <[email-masked]>
Date: Fri, 23 Jun 2017 17:09:48 -0700

The only benefit of having streams implicitly opened I can imagine is the ability to respond on a stream you believe has been implicitly opened(ie: ekr's IMAP example).  I can't think of many applications where that would be at all useful, and in the few cases it is, waiting for a few more packets in the case of reordering, or 1 RTT in the case of loss, seems like a much better solution than responding with no prior information from the peer.

@siyengar Besides the above edge case, I don't understand what "Having lower streams open up has a bunch of desirable properties..." are?  In the HTTP mapping, if there's a separate headers and body stream, then one will have to be unidirectional and the other bidirectional OR there will be 3 unidirectional streams paired together at the app layer.  Either way, I wouldn't expect implicit opening to help.

My main motivation is general simplification and making #656 simpler.  I don't see any downside of this, but possibly I'm missing something.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310798696

---

From: Subodh Iyengar <[email-masked]>
Date: Fri, 23 Jun 2017 17:59:50 -0700

@krasic even opened but unused streams can be lazily initialized, which is precisely what our implementation does right now. When implemented that way that necessarily will store less state than storing unopened streams.

@krasic the reordering in http i was referring to was if the http request headers were sent, in some cases the server can respond even before receiving the entire body. In those cases if the body streams get lost, then the server has to wait till the client retransmits them, which is sad. Reversing the stream order solves that issue and we exploit the property of opening lower streams for that.

@ianswett we should chat offline if that's helpful to clarify my points, but I believe the properties that I like about the implicit opening of lower streams, i.e. 

1. Being able to implicitly open lower streams for http as described above
2. Being able to keep less state in the common case for opened streams
3. Making stream limits more semantically meaningful for the application, i.e. I have to close streams even if I have never referred to them if I open an out of order stream.

For #656 i've left a few comments there which might simplify things, i.e. if we apply uni or bi streams to the entire connection and have the application choose one or the other and stick to it, doesn't that solve the problem in #656?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-310803175

---

From: ThomasSwindells <[email-masked]>
Date: Mon, 26 Jun 2017 04:49:02 -0700

Allowing an application to open stream ids (without opening everything underneath them) does have potential benefits that shouldn't just be dismissed out of hands. Conceptually some benefits follow, whether these benefits are better than implicit opening or if the other parts of QUIC even make them realisable I'll leave up to you to consider:

1. being able to align stream id's with application level conversation ids
2. Avoid contention in allocating stream ids in a multi-threaded environment (e.g. pre-assign batches to different threads/jobs) without knowing if/when the thread/job will actually use the stream
3. allow stream ID to influence CPU affinity or other hardware optimizations (e.g. future hardware optimizations may use stream id ranges to map request processing to different CPU to preserve data locality etc.

@siyengar 
Can you explain your reasoning in a bit more detai?. In particular I don't understand why in your http case there is any benefit of implicitly opening streams rather than http explicitly signalling the set of streams they have (through an OPEN_STREAM frame or whatever). Also isn't it (marginally) more complex with implicit streams that despite not explicitly opening, using or refering to that skipped stream number it still needs closing, compared to a simple rule along the lines of 'currently open streams' = number explicitly opened-number explicitly closed. (Though I'm sure in practice when dealing with packet loss it is a bit more complicated than that).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-311037076

---

From: Igor Lubashev <[email-masked]>
Date: Mon, 26 Jun 2017 07:38:15 -0700

@ThomasSwindells I think the main obstacle to a design allowing apps to assign random identifiers to streams is that no one has suggested a way how to build such a design.  The key requirement is that a rogue client is not able to cause the server to keep excessive amount of state just to track what Stream IDs are "idle" (i.e. have not been used before) and "closed" (have been used before and now closed).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-311078748

---

From: Subodh Iyengar <[email-masked]>
Date: Mon, 26 Jun 2017 08:32:41 -0700

@ThomasSwindells all of those can be accomplished even with implicitly opening lower streams as well. The key point here is that even with implicit open, you don't actually have to store state for streams that you don't send data on or don't have an updates for, like flow control, you can lazily allocate them. We currently implement it as such and it is feasible. I'm not sure I completely understand why the current approach precludes any of the CPU optimizations specified? You're always limited by max stream id of the peer so you can't open random streams.

It's also not accurate to specify that if an app only opens stream id 11 it can ignore stream id 5. Due to stream id limitations, the app would have to reset 5 in order for the peer to open up some stream id space (see @igorlord's example). If you can't ignore streams you don't refer to anyway, we should make the behavior of what happens to lower streams explicit. This makes it more intuitive semantic behavior for the app as well (only if i close open streams, I can get more streams).

I would argue that it's probably more complex overall for an implementation and an application to not implicitly open lower streams, the stream id accounting becomes more complex and has the potential to cause deadlocks. Closing streams that you have not referred to has to be done in both designs, the current approach just makes that more explicit that you had to do it to an app, because it was opened. 

I don't see how explicitly signaling http streams opens helps here, because open streams is a QUIC layer concept and what channel will signal that. I think this might be raising a completely different issue. If body stream is stream 7 and header is stream 9, if a header (stream 9) gets to the server but data from the body stream is lost (if stream 7 is sent in a different packet), the server can respond even without the body. The reordering of streams makes it explicit that the body stream must be open by the client if the header stream got to the server.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-311095068

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 26 Jun 2017 08:40:47 -0700

I support this idea. 

(Trying not to open the partial reliability can of worms) An unreliably delivered frame that is missed could lead to streams not opening. With implicit opening, a later frame that is delivered could create orphan/zombie streams - related to resources that are no longer relevant. The sending end has lost interest and potentially never closes the stream. That isn't the end of the world but introduces some burden of garbage collection onto the client.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-311097421

---

From: krasic <[email-masked]>
Date: Mon, 26 Jun 2017 10:20:46 -0700

I support the idea that mappings might need ways to explicitly reserve/allocate streams in potentially application specific ways.   I don't think that the implicit open semantic will be sufficient to cover those, or even if it does, it will be a subtle, fragile design.    I can see why reserving specific stream ids would be problematic, but I think the transport api supporting something along the lines of "allocate next outgoing stream" (which can be bidi or uni) is sensible.   

I'm also not entirely comfortable with the HTTP example here as it is yet to be resolved whether the HTTP mapping is going to revert back to having headers and data on the same transport stream.   

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/662#issuecomment-311124817