Subject: Re: [quicwg/base-drafts] HTTP header compression: static table fit for QUIC (#904)

From: Dmitri Tikhonov <[email-masked]>
Date: Fri, 08 Dec 2017 13:17:37 +0000 (UTC)

I think the largest benefit to keeping the HPACK static table is interoperability between HPACK and the new compression mechanism.  If this property is deemed unnecessary, the static table can be updated to use the latest common headers.

In my mind, this is a good feature to have and QMIN is designed to be interoperable with HPACK given a few conditions.

As for telemetry, I think we have to guess -- unless some big companies are willing to enable stats gathering and sharing them -- as more and more HTTP connections are secure.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350260711

---

From: Mike Bishop <[email-masked]>
Date: Fri, 08 Dec 2017 15:39:09 -0800

That's how the static table and Huffman table for HPACK were generated -- statistics of common values and character frequency in Chrome.  Obviously, some things have shifted since then.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350399647

---

From: Lucas Pardue <[email-masked]>
Date: Sat, 09 Dec 2017 00:13:48 +0000 (UTC)

I was also wondering if something like a HTTP archive query might be able to bring up some useful data.

This issue is obviously not critical or blocking but if is a nice opportunity. I appreciate there is probably lead time in generating appropriate data and therefore wonder if it might help to loop the relevant people into this issue, if they already aren't. Not sure exactly the right people though.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350404453

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 11 Dec 2017 00:17:31 +0000 (UTC)

Wishful thinking hat on: I'd love to see a means for per-session static table ( i.e. an immutable table negotiated on start up). The flexibility could vary from completely user-defined, to pre-defined tables that exist in a spec (e.g. "legacy HPACK" and "future compressor").

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350594341

---

From: Dmitri Tikhonov <[email-masked]>
Date: Sun, 10 Dec 2017 18:08:52 -0800

> Wishful thinking hat on: I'd love to see a means for per-session static table ( i.e. an immutable table negotiated on start up).
@afrind has presented an almost exactly the same idea to the Compression Design Team.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350604998

---

From: afrind <[email-masked]>
Date: Mon, 11 Dec 2017 06:14:29 +0000 (UTC)

The basic idea was to allow the encoder to include compression instructions in the SETTINGS frame, and require the SETTINGS frame to be processed before any request streams (it already must be processed before responses are sent).  If the HoL blocking seems onerous,we could add a transport parameter to the handshake 'MUST_PROCESS_SETTINGS' telling the peer if HoL blocking is required.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350631710

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 11 Dec 2017 05:24:57 -0800

> I was also wondering if something like a HTTP archive query might be able to bring up some useful data.

So I had a look at the archive and hit a learning curve. A plea for help got me a great answer from @paulcavano - https://discuss.httparchive.org/t/is-hpack-static-table-fit-for-todays-web/1177/5

The results are calculated over a subset of the available data. I'd like to do some further analysis.

Interestingly, in spite of [Deprecating the "X-" Prefix and Similar Constructs in Application Protocols](https://tools.ietf.org/html/rfc6648), there are lots of instance of "X-" prefix.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350723138

---

From: MikkelFJ <[email-masked]>
Date: Mon, 11 Dec 2017 15:11:40 +0000 (UTC)

How about a static huffman table of complete words commonly seen in header values? Including Mon, Tue, gzip, ';', ...

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350752210

---

From: MikkelFJ <[email-masked]>
Date: Mon, 11 Dec 2017 15:18:38 +0000 (UTC)

It could be created with a DFA such that looking up the next char would either give a final value, or ask for one more making the encoding. Possibly using something like:
http://iis.ipipan.waw.pl/2008/proceedings/iis08-22.pdf
But the actual encoder/decoder would just simple lookups.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350754331

---

From: MikkelFJ <[email-masked]>
Date: Mon, 11 Dec 2017 08:49:16 -0800

Given a standard DFA algorithm it ought to be possible negotiate a list of words. Such a list could be identified by a SHA256 hash. This would allow total freedom of dictionary choice as long as both parties agree. The DFA encoding would use 1 bit for each step through the DFA resulting in a bit pattern similar to huffman.

This can be combined with basic huffman encoding. A special code, such as a 1-bit prefix, activates the word DFA. If the huffman table also has code for termination then the length need not be carried.

A static dictionary could also be negotiated via a SHA256 of the sorted list of strings.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350783786

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 11 Dec 2017 09:13:04 -0800

First off, this ticket risks exposing or deep diving into work-in-progress of the compression design team. I'd like to avoid doing that too early, so please someone weigh in if we think it might be better to hold off. 

@afrind I don't quite follow from your brief summary. What is an appropriate moniker for the table resulting from compression encoding negoition? Does your idea apply symmetrically or could there be different header tables contexts for client and server? 

The main use case I have is the unidirectional HTTP case, where SETTINGS cannot be exchanged. I've used Alt-Svc parameters to articulate most connection-level settings and wonder if such proposed compression instructions could also be articulated as an Alt-Svc parameter.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350791388

---

From: afrind <[email-masked]>
Date: Mon, 11 Dec 2017 17:25:18 +0000 (UTC)

@LPardue : Sorry for the confusion, I will write up the idea formally.  The goal I had in mind was seeding the dynamic table once at the beginning of the connection.  

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350794945

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 11 Dec 2017 10:05:34 -0800

ok cool, I've had some half-baked ideas on that topic. Look forward to something a bit more tangible. 

Is the resulting table locked, or open to modification through the course of a session?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350807186

---

From: Mike Bishop <[email-masked]>
Date: Mon, 11 Dec 2017 10:20:47 -0800

The QPACK editor's copy incorporates Alan's idea along with a few others.  Basically, it's just a series of updates that are guaranteed to have been processed prior to any of your requests, so you can reference them without risk of blocking.  If the decompressor supports blocking anyway, it may be negligible, but it's a simpler model to get good initial performance even if the decompressor won't support blocking in the general case.

As to the design team, my hope is that we bring a starting point for header compression back to the working group by Melbourne, and the working group can iterate further from there.  Since the concept of a static table is common to most of the proposals at this point, I'd prefer to leave refining the contents of that table to the "iteration" stage.  The discussion can happen in parallel if you want, but don't expect to see action on that until we have a single scheme that we're all moving forward with.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-350811497

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 12 Dec 2017 11:08:47 +0000 (UTC)

SGTM. I agree that HPACK static table is good enough for now, changes can be iterative later on. However, to go back to your earlier point, QPACK was suggested to offer the possibility of a much larger static table. If other proposals do not offer such a possibility it might be useful as a comparison point.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/904#issuecomment-351020064