Subject: Re: [websec] Minor feedback on draft-ietf-websec-mime-sniff-03

From: Adam Barth <[email-masked]>
Date: Sun, 15 Jan 2012 11:52:38 -0800

The requirement in the spec is what we intend.  The rule applies only
to that exact octet sequence.

Adam

On Sun, Jan 15, 2012 at 11:51 AM, Willy Tarreau <[email-masked]> wrote:
> Hello Adam, Ian,
>
> Today I came across your draft "draft-ietf-websec-mime-sniff-03", and
> noticed the point below :
>
>   2.  If the octets were fetched via HTTP and there is an HTTP Content-
>       Type header field and the value of the last such header field has
>       octets that *exactly* match the octets contained in one of the
>       following lines:
>
>      +-------------------------------+--------------------------------+
>      | Bytes in Hexadecimal          | Textual Representation         |
>      +-------------------------------+--------------------------------+
>      | 74 65 78 74 2f 70 6c 61 69 6e | text/plain                     |
>      +-------------------------------+--------------------------------+
>      | 74 65 78 74 2f 70 6c 61 69 6e | text/plain; charset=ISO-8859-1 |
>      | 3b 20 63 68 61 72 73 65 74 3d |                                |
>      | 49 53 4f 2d 38 38 35 39 2d 31 |                                |
>     .../...
>
> I was having a doubt about spaces being optional around the semi-colon,
> so I just checked and indeed we have OWS before and after it :
>
>   http://www.ietf.org/id/draft-ietf-httpbis-p3-payload-18.txt
>
>   2.3.  Media Types
>
>   HTTP uses Internet Media Types [RFC2046] in the Content-Type
>   (Section 6.8) and Accept (Section 6.1) header fields in order to
>   provide open and extensible data typing and type negotiation.
>
>     media-type = type "/" subtype *( OWS ";" OWS parameter )
>     type       = token
>     subtype    = token
>
>   The type/subtype MAY be followed by parameters in the form of
>   attribute/value pairs.
>
>     parameter      = attribute "=" value
>     attribute      = token
>     value          = word
>
> Also, it is said here that quotes are allowed around the parameter
> value :
>
>   A parameter value that matches the token production can be
>   transmitted as either a token or within a quoted-string.  The quoted
>   and unquoted values are equivalent.
>
> So examples below are completely valid :
>
>   Content-type: text/plain;charset="ISO-8859-1"
>
>   Content-type: text/plain   ;  charset=ISO-8859-1
>
>   Content-type: text/plain ;
>         charset="ISO-8859-1"
>
> Thus the byte matching can only apply to the tokens and values. I think the
> safest thing to do would be to refer to the HTTP spec to define the header
> format then suggest byte matches for each fields, for instance :
>
>       If the octets were fetched via HTTP and there is an HTTP Content-
>       Type header field and the value of the last such header *exactly*
>       matches one of the media-types below, then the sniffed-type is
>       defined as the concatenation of the unquoted matching parts :
>
>       media-type = type "/" subtype *( OWS ";" OWS parameter )
>       sniffed-type = type "/" subtype 1*( "; " attribute "=" value )
>
>       All accepted media-types must *exactly* match :
>          - type    = "text" (hex 74 65 78 74)
>          - subtype = "plain" (hex 70 6c 61 69 6e)
>
>       If a parameter is present, its attribute must be "charset"
>       (hex 63 68 61 72 73 65 74) and the value must be one of :
>          - "ISO-8859-1" (hex 49 53 4f 2d 38 38 35 39 2d 31)
>          - "iso-8859-1" (hex 69 73 6f 2d 38 38 35 39 2d 31)
>          - "UTF-8"      (hex 55 54 46 2d 38)
>
> Please also note that HTTP indicates that some attributes accept a
> case-insensitive value. I have not yet found in the spec if "charset"
> accepts a case-insensitive value, but given that you identified two
> possible cases for "iso-8859-1", it is likely that "charset" falls into
> this case.
>
> Best regards,
> Willy
>

---

From: Adam Barth <[email-masked]>
Date: Sun, 15 Jan 2012 12:53:00 -0800

On Sun, Jan 15, 2012 at 12:41 PM, Willy Tarreau <[email-masked]> wrote:
> On Sun, Jan 15, 2012 at 11:52:38AM -0800, Adam Barth wrote:
> But then what are the impacts of not matching the correct content-type ?

I'm not sure I understand your question.  Can you explain a scenario
in which something happens that causes someone to be sad with the
current requirements?

Adam

---

From: Julian Reschke <[email-masked]>
Date: Sun, 15 Jan 2012 22:00:37 +0100

On 2012-01-15 21:53, Adam Barth wrote:
> On Sun, Jan 15, 2012 at 12:41 PM, Willy Tarreau<[email-masked]>  wrote:

Translating Adam: matching only some specific header field instances is 
intentional, as these are the ones we know misconfigured servers send.

(right?)

It wouldn't hurt if the spec would explain that choice, if it doesn't 
right now.

Best regards, Julian

---

From: Adam Barth <[email-masked]>
Date: Sun, 15 Jan 2012 13:06:20 -0800

On Sun, Jan 15, 2012 at 1:00 PM, Julian Reschke <[email-masked]> wrote:
> It wouldn't hurt if the spec would explain that choice, if it doesn't right
> now.

I believe there's a ticket about adding that description.  We've been
focusing more on the introduction/scope editing at the moment, but
we'll get to this point.

More specifically, this is a workaround for an old (still widely
deployed) version of Apache that used that exact octet sequence to
identify resources for which it didn't know the MIME type.  Apache has
since been changed to omit the Content-Type header in these cases, but
old Apache installs stay around for a very, very long time.  Matching
this exact octet sequence is the minimum injury way of dealing with
this legacy content.

Adam

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 15 Jan 2012 20:51:20 +0100

Hello Adam, Ian,

   2.3.  Media Types

Best regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 15 Jan 2012 21:41:54 +0100

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 15 Jan 2012 22:17:02 +0100

> in which something happens that causes someone to be sad with the

The draft presents an algorithm to determine a content type based on
available information, but ignores some important information such as
valid header values when presented in an unexpected form.

For instance, if I get a file advertised like this :

   Content-type: text/plain; charset=us-ascii

then it will not be interpreted as text/plain but rather based on what
is found in the contents. This can make it impossible to read some
documents purposely sent as plain text (eg: HTML or PS source code).

It could also also have security impacts such as causing some plugins
to be fed with the mis-identified data. For instance, imagine that I'm
browsing behind a filtering proxy which checks that PDF documents are
exempt of any exploit. This proxy will most likely consider the advertised
content-type and won't fire the PDF analyser on text/plain documents. But
the browser at the end of the chain ignores the text/plain and decides to
launch the plugin.

As you know, in general, having multiple components interprete different
things from similar contents is dangerous for interoperability and for
security.

Note that it is possible that I missed something as it's not trivial to
get right the way it's written :-/

Best regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 15 Jan 2012 22:25:51 +0100

> this exact octet sequence is the minimum injury way of dealing with

OK I think I'm getting it now.

I assumed that at step 2 of section 4, the matching text was constituting
the official-type, which apparently it's not. Getting it wrong at this
point causes a jump to the "unknown type" section.

Maybe some names should be changed (eg: "content-type" instead of
"official-type") to make it easier to read. Confusion is too easy this
way and immediately makes code derived from a wrong interpretation to
be buggy :-(

Regards,
Willy

---

From: Ian Hickson <[email-masked]>
Date: Tue, 17 Jan 2012 21:32:03 +0000 (UTC)

On Sun, 15 Jan 2012, Willy Tarreau wrote:
> then it will not be interpreted as text/plain

What makes you think that? As far as I can tell, the algorithm given in 
the spec requires that such a file be treated as text/plain.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 17 Jan 2012 22:39:48 +0100

Hi Ian,

> the spec requires that such a file be treated as text/plain.

You're right, as I said in the second mail following this one, I found
what I misread in the algorithm. In short, I thought we'd jump to the
content analysis if the header did not match these exact forms, which
is not the case, I conflated official-type and sniffed-type in the text
if my memory serves me right.

Regards,
Willy