Subject: [quicwg/base-drafts] Support for inter-frame padding (#158)

From: Lars Eggert <[email-masked]>
Date: Mon, 16 Jan 2017 00:31:59 -0800

The section on padding frames currently says:
```
   The PADDING frame (type=0x00) pads a packet with 0x00 bytes.  When
   this frame is encountered, the rest of the packet is expected to be
   padding bytes.  The frame contains 0x00 bytes and extends to the end
   of the QUIC packet.  A PADDING frame has no additional fields.
```

I'd like to discuss whether a padding frame that can be inserted anywhere, instead of only as the last frame in a packet, makes sense.

I'm currently doing a zero-copy implementation of QUIC on top of [netmap](http://info.iet.unipi.it/~luigi/netmap/), which due to static DMA mappings requires an app to place stream frame data directly into packet buffers at a (fixed) offset. The QUIC stack then inserts the common header and other frames before the stream frames, and needs to pad any unused space. Currently, this isn't possible.

In other words, I'd like something similar to the [No-Operation TCP option](https://tools.ietf.org/html/rfc793#page-18).

(And it's an open question whether going to great lengths to support zero-copy TX is actually all that useful, given that we're going to do crypto over the entire buffer anyway.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/158

---

From: Martin Thomson <[email-masked]>
Date: Thu, 19 Jan 2017 23:40:18 -0800

There is actually a very simple fix for this, though it increases the work required to handle a packet: you could say that a PADDING frame is empty.

I'm not convinced of the necessity though, is this because lengths occasionally need to grow?  Can you identify the cases where this happens?

FWIW, crypto shouldn't affect zero-copy because most AEAD implementations encrypt and decrypt in place.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/158#issuecomment-273997710

---

From: Lars Eggert <[email-masked]>
Date: Fri, 20 Jan 2017 00:38:58 -0800

Yes, and empty PADDING frame would work, but you need to parse each byte of the padding then, which is not so great. I was thinking more about about a frame type prefix like STREAM and ACK have, and using the remaining bits to encode a length.

I'm not sure I fully understand your comment about necessity. For an implementation where data is pre-placed into a packet buffer and the QUIC stack needs to prepend its headers. Because the buffers are statically DMA-mapped, the beginning of the buffer cannot be changed to account for the variable-length QUIC header from packet to packet. Therefore, the stack needs to leave some slack space for QUIC to use for its header, and pad the unused slack space until the beginning of the first STREAM frame (for which the app has placed the data already.)

Let me try and draw something:
    
                            also at fixed offset, but var. length
      fixed length headers  |        var. length
      v          v    v     v        v
    +----------+----+-----+--------+--------+--------+--------+
    |          |    |     | QUIC   | ACK &  | STREAM | app    |
    | Ethernet | IP | UDP | common | other  | hdr    | stream |
    |          |    |     | header | frames |        | data   |
    +----------+----+-----+--------+--------+--------+--------+
    ^                                         ^        ^
    |                                         |        placed at fixed offset before
    start of packet buffer (can't change)     |        calling into stack
                                              |
                                              placed at neg. offset from beginning
                                              of app data

This is roughly what it looks like. On TX, the app needs to place the data for a stream frame at a fixed offset into the packet buffer that is larger than the network-layer plus QUIC headers (at least in the common case). The app then calls into the QUIC stack, which calls into the network stack to get Ethernet/IP/UDP headers stamped onto the buffer. Those are fixed length, so the QUIC stack can then place the QUIC common header at a known fixed offset. The QUIC common header and most other QUIC headers are variable-length though. Therefore, there will usually be a gap between the end of the QUIC headers and the beginning of the app-placed data (or more correctly, the beginning of the STREAM frame header for this data, which is placed at a negative offset from the beginning of the app data.)

I need to pad that gap somehow, and the current PADDING frame isn't suitable.

Finally, regarding in-place crypto: My comment on whether zero-copy is even worthwhile was trying to express than because crypto touches all bytes in the buffer already, they are in the CPU cachlines and a memory copy is therefore super cheap.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/158#issuecomment-274010446

---

From: Lars Eggert <[email-masked]>
Date: Fri, 20 Jan 2017 00:45:55 -0800

(Note for people following along by email: I edited the drawing in the previous post, but I don't think GitHub emails out notifications when that happens.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/158#issuecomment-274011984

---

From: ianswett <[email-masked]>
Date: Fri, 20 Jan 2017 08:11:34 -0800

It's easy enough to make the QUIC header fixed length.  More challenging is to make other frames fixed length, and it's impossible if you don't know which ones they'll be.

In our experience, as long as buffers are in cache already, saving the copies is a really tiny savings, so I'd be biased towards saying it's not worth worrying about in this case.

However, if any of the issues you mentioned were relevant to hardware implementations of QUIC, I think we should discuss them here as well.  

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/158#issuecomment-274110912

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 25 Jan 2017 23:21:09 -0800

As discussed in Tokyo, remove the restriction that zero-byte padding can only occur at the end of the packet

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/issues/158#issuecomment-275322998