Subject: [quicwg/base-drafts] Allowing HolB on header compression and DoS (#1011)

From: Martin Thomson <[email-masked]>
Date: Tue, 12 Dec 2017 15:36:24 +0000 (UTC)

There's a memory exhaustion exposure that is generated by header compression if we allow head of line blocking.  Head of line blocking means buffering, which is currently not accounted for in the protocol.

We could decide that head of line blocking is not worth the effort, in which this becomes a non-event.  We could decide that this is not interesting.  Or we could decide to build a mechanism for accounting for the memory cost.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1011

---

From: ianswett <[email-masked]>
Date: Tue, 12 Dec 2017 15:59:01 +0000 (UTC)

I would think the memory issue would be dealt with by flow control.  I believe that does allow the remote possibility of a connection ending up flow control blocked on HolB data.  Is there any other issue?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1011#issuecomment-351095410

---

From: Martin Thomson <[email-masked]>
Date: Tue, 12 Dec 2017 11:20:27 -0800

Flow control can't help here.  You don't learn about the head of line blocking until the application consumes data and then it's out of the receive buffers.  You could imagine using peek-style APIs, but I don't think that it's reasonable to essentially force that on everyone.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1011#issuecomment-351162776

---

From: Mike Bishop <[email-masked]>
Date: Tue, 12 Dec 2017 19:32:01 +0000 (UTC)

The scenario in which it's not accounted for by flow control is if you block on something toward the end of a header block.  Consider the pathological case where you get a gigantic (e.g. 16KB+) Cookie, lots of other big headers, followed by a blocking reference -- or simply a gap in the stream.  You're left holding a mostly-decompressed header set in the HTTP stack, but you can't actually proceed to process the request until you get the last of the headers.  An attacker doing that on multiple streams simultaneously could force a server into holding many mostly-complete requests in a state not tracked by flow control (since the HTTP stack already read from the transport).

If the header compression scheme can't block, you can mitigate this by not beginning to process the header set until the whole of the HEADERS frame has arrived.  That probably means reading enough to find the length of the HEADERS frame, then reading no further until that much data was available to read.

In HTTP/2, we mitigated this by requiring header blocks to arrive as a single unit, even if they spanned multiple frames -- no multiplexing within header blocks, so you were never left with more than one partial block at a time.  Since the multiplexing is happening below us now, that path is closed.

I suspect the most effective mitigation for this is a request timer that requires that time from first byte to end of request headers be something reasonable, and reset the stream if it's exceeded.  This is already needed in HTTP/1.1 to protect against slow-loris-style attacks, so it's not new to HTTP stacks.  However, that does allow excess consumption of memory up to your maximum header set size per request in the meantime.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1011#issuecomment-351167530

---

From: krasic <[email-masked]>
Date: Tue, 12 Dec 2017 20:02:43 +0000 (UTC)

Am I correct that this can be further simplified:  gigantic headers not finished, followed by a gap in the stream.   If processing of headers isn't done in streaming fashion, the attacker can force the server into holding partial requests not tracked by flow control.  

Even if one does process headers in streaming fashion, it's arguable that higher level state will likely consume a proportionate amount memory while the stream/transaction remains alive.   

At the end of the day, a server might have to make it's own judgement that memory is too tight, and take action, e.g. refuse new streams.

There was some discussion way back about stream refusals and the mechanics of stream limits.  If I recall correctly, we dropped the capability of servers to refuse streams when the stream limits were not exceeded.    I think/thought it was a mistake, exactly because it's an important line of defense to control memory usage.   When I asked our server folks if dropping the right to refuse was reasonable,  their answer was a definite no.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1011#issuecomment-351177075

---

From: Dmitri Tikhonov <[email-masked]>
Date: Tue, 12 Dec 2017 20:13:01 +0000 (UTC)

> Even if one does process headers in streaming fashion, it's arguable that higher level state will likely consume a proportionate amount memory while the stream/transaction remains alive.

I think this is correct.

> At the end of the day, a server might have to make it's own judgement that memory is too tight, and take action, e.g. refuse new streams.

The problem is that the unfinished header exists somewhere between the transport and the application layers.  It is difficult to make judgment there.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1011#issuecomment-351179815

---

From: Martin Thomson <[email-masked]>
Date: Tue, 12 Dec 2017 15:23:39 -0800

The idea that was discussed was - in combination with the checkpointing scheme @dtikhonov proposes - is to advertise the checkpoint that a block depends on.  The decoder would advertise how many streams it would allow to be blocked on an unfulfilled dependency.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1011#issuecomment-351228596

---

From: Mike Bishop <[email-masked]>
Date: Sat, 16 Dec 2017 00:12:25 +0000 (UTC)

The first piece of this means that a decoder could read only up to that point (the first few bytes of a HEADERS frame), then decide whether to proceed.  If it chooses not to proceed yet, then the data sits at the QUIC layer and flow control provides the back-pressure to manage memory consumption.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/1011#issuecomment-352142821