Subject: Re: [websec] draft-ietf-websec-key-pinning

From: Trevor Perrin <[email-masked]>
Date: Thu, 4 Sep 2014 16:02:57 -0700

I agree with Eric and Joe that the current definition of PKP-RO is not
that useful and potentially surprising to deployers.

Ryan argues that storing PKP-RO is dangerous because a TLS MITM who
could set pins for a lot of clients for a lot of sites could flood a
victim with reports.

Couldn't the MITM also set PKP pins with report-uri, or set cached
redirects or other cached resources to generate traffic toward a
victim?

Ryan argues PKP-RO is worse because it's "conceptually and practically
silent to the user".  But once the browser has sent junk traffic to a
victim, I don't really see how seeing how displaying a pinning error
makes things better.

Trevor

On Thu, Aug 28, 2014 at 9:13 AM, Eric Lawrence <[email-masked]> wrote:
>> testing with PKP-RO may
>> be fine before first deploying PKP, but it doesn’t help you where it’s
>> dangerous -
>> when it’s time to change certificates.
>
> In addition to raising confidence before you first deploy, PKP-RO helps when
> you wish to retire a pinned SPKI.
>
> For instance:
>
>   0. Acme Corp has a secure website; they’re using PKP with
> includeSubdomains.
>   1. Acme Corp currently buys its certs from Verisign and uses PKP to pin
> the Verisign intermediate SPKI.
>   2. Next year, Acme Corp decides that they want to buy GoDaddy certs
> instead. They update their PKP header to add the GoDaddy intermediate’s SPKI
> to the list.
>   3. Because Acme Corp is a huge enterprise, they don’t know if some
> business unit (e.g. SpecialProjects.Acme.com) didn’t get the memo about the
> move to GoDaddy and may still be using a Verisign cert. So, in addition to
> the updated PKP header, they ALSO send a PKP-RO header that specifies *only*
> the GoDaddy SPKI and watch for any reporting failures.
>   4. After an appropriate period, Acme concludes that they are safe in
> replacing the PKP set of SPKIs with the set they prototyped in PKP-RO. They
> do so and remove the PKP-RO header.
>
> -Eric
>
> *An aside vis-à-vis the plausibility of #3, a site not knowing about its own
> subdomains: I recently spoke to a site (“example.com”) that wasn’t using
> includeSubdomains on their HSTS header. I asked why not, since all of the
> company’s public websites were HTTPS. It turns out that they can’t use
> includeSubdomains because their private Intranet uses internally-mapped
> domains subordinate to their public domain name suffix. So, while
> “hr.example.com” isn’t publicly reachable, browsers don’t know or care, and
> HSTS+includeSubdomains will fail any insecure connection for the employees
> inside the company.
>
> A similar challenge will face any such companies when they try to use PKP
> with includeSubdomains.
>
>
> From: Yoav Nir
> Sent: Thursday, August 28, 2014 9:59 AM
> To: Eric Lawrence
> Cc: Ryan Sleevi ; Tom Ritter ; draft-ietf-websec-key-pinning ;
> [email-masked]
> Subject: Re: [websec] draft-ietf-websec-key-pinning
>
> Hi Eric
>
> Part of the issue is that PKP (and PKP-RO) follow the lead of other headers
> such as CSP that also have report-only alternatives, but PKP is inherently
> different.
>
> CSP has directives that relate to the content delivered, so if CSP has a
> directive that says the resource cannot be displayed behind a transparent
> floating frame, the content will not be displayed. If it’s CSP-RO instead of
> CSP, the content is displayed, but a report is also sent. That makes sense,
> and if you see that the CSP-RO header causes reports, you can adjust it.
> Once you move to CSP, you’re fine as long as the content does not change,
> and even if it does change, you can test it first with a test server.
>
> For PKP it’s different. The thing that changes is the certificate presented
> to the client. You can’t test the new certificate on a side server, at least
> not in a way that will fill you with confidence. And because of the way
> certificates are used, there is no such thing as a static structure to the
> site. You have to change the certificate (and usually the key) every year,
> so testing with PKP-RO may be fine before first deploying PKP, but it
> doesn’t help you where it’s dangerous - when it’s time to change
> certificates.
>
> This leads some people around here to conclude that PKP-RO is not useful.
>
> Yoav
>
>
> On Aug 28, 2014, at 5:26 PM, Eric Lawrence <[email-masked]> wrote:
>
> I’ll take one last tilt at this, because I think this spec is quite
> important and folks aren’t actually very far apart on this.
>
> To start, I want to reiterate that Draft #20 was the first time I got
> involved in PKP, so my perspective comes from that draft and not any other
> conversations, tribal knowledge, meetings, etc, that others in the group may
> have been a part of.
>
> Here’s how I *thought* Draft #20 specified things to work:
>
>   1> PKP and PKP-RO are equivalent in every way except that PKP mismatch
> triggers a Report *and* fails the connection, while PKP-RO mismatch only
> triggers a Report. That’s why PKP-RO is named “Public Key Pins Report Only”
> and not “Sorta Like Public Key Pins But Does Not Break Connections And Does
> Not Persist (SLPKPBDNBCADNP)”
>
>   2> Site administrators should use PKP-RO to prototype a policy to be later
> deployed PKP. They use PKP-RO first to generate confidence that they will
> not be self-inflicting any broad denial-of-service when enabling PKP.
>
>   3> When PKP and PKP-RO headers are initially encountered (pins not
> previously stored), a failure to match the specified policy triggers a
> Report. The mismatched policy is NOT stored, which blocks the DOS bomb
> attack Ryan mentions below.
>
>   4> PKP and PKP-RO only exist as different headers because it allows a site
> to have one policy in force and also prototype proposed policy changes.
>
> This design made perfect sense to me, and my feedback on the draft was
> primarily intended to clarify the language around this design.
>
> Instead, it appears that PKP-RO works dramatically differently than PKP, in
> ways that I believe are entirely non-obvious to implementers who only read
> the draft. While I believe it would be possible to editorially change the
> language to make PKP-RO’s different behavior, to me, that approach only
> seems to be codifying a more confusing, less useful design, and would
> require more work on the part of the authors.
>
> I don’t believe there are any privacy or security implications in allowing
> PKP-RO to behave like PKP except that it’s “Report Only.” Any privacy or
> security implications in PKP-RO are shared by PKP.
>
> -Eric Lawrence
>
>
>
> _______________________________________________
> websec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/websec
>

---

From: Trevor Perrin <[email-masked]>
Date: Thu, 4 Sep 2014 16:53:48 -0700

On Thu, Sep 4, 2014 at 4:09 PM, Ryan Sleevi <[email-masked]> wrote:
>
>
>
> On Thu, Sep 4, 2014 at 4:02 PM, Trevor Perrin <[email-masked]> wrote:
>>
> Yes, and that's a user visible effect, rather than transparent.

Hmm, there's nothing clever that can be done with cached javascript or
html to also generate silent traffic toward a victim?

> It also
> requires that a connection be validated first, whereas the very definition
> of PKP-RO is that you send it when a connection is not validated.

I'd expect stored PKP-RO to follow the same rules as PKP and be stored
only if it meets the bullet points in 2.5 (error-free TLS connection,
backup pins, etc.).

It would make sense for any report-uri header (PKP or PKP-RO) that
fails the 2.5 checks, or has some other invalid syntax (eg bad
base64), to *also* generate a different sort of report and not be
stored.

Would it be reasonable to add a reporting message for 2.5 failure or
syntax failure?  Otherwise, when asserting a report-uri pin you can't
distinguish pin-not-stored errors from
pin-stored-and-validation-successful.

> First, the attacker can only mount the hostile attack after having first
> established a good connection.

Debateable per above.

> PKP-RO allows an attacker to perpetually
> mount a hostile attack.
> Second, the presence of a pinning error is an inherently rate-limiting
> factor to the user experience. You're not going to be able to trigger 100
> reports if you're causing the user 100 error pages.

Rate-limiting is already proposed:
"""
UAs SHOULD limit the rate at which they send reports.  For example,
   it is unnecessary to send the same report to the same report-uri more
   than once per distinct set of declared Pins.
"""

Wouldn't that take care of it?

> Third, it seems like you're arguing for removing report-uri altogether,
> given the privacy and DoS risks identified. That is, the more that  you
> establish RO is similar to PKP in level of what an attacker can do, the more
> it argues for removing report-URI entirely.

Dunno.  I'm still trying to understand these risks, whether they're
really different for PKP-RO vs PKP, and for HPKP versus other web
technologies.

Trevor

---

From: Trevor Perrin <[email-masked]>
Date: Thu, 4 Sep 2014 17:23:31 -0700

On Thu, Sep 4, 2014 at 5:06 PM, Ryan Sleevi <[email-masked]> wrote:
> I wish you would have raised this during WGLC, which would have benefited
> from the review and discussion now being paid.

I did raise this in response to the February Last Call, and multiple
times later.

There was some very light discussion which favored storing PKP-RO, the
draft was edited in the opposite direction, and it was sent on while
we were still discussing (with no additional WGLC).

But I agree that we're now having the discussion this issue deserves
(and which we never had earlier).

> That is, my understanding is that this is a bit like discovering a possible
> issue after you've shipped your gold master to 100K stores.

I don't see how we've shipped anything to anyone, so I don't see why
this is so painful to discuss.

But I'm just a random security nerd, the opinions of browsers and
websites are what matter here, so I'll defer to those.

Trevor

---

From: Ryan Sleevi <[email-masked]>
Date: Thu, 4 Sep 2014 16:09:05 -0700

> redirects or other cached resources to generate traffic toward a

Yes, and that's a user visible effect, rather than transparent. It also
requires that a connection be validated first, whereas the very definition
of PKP-RO is that you send it when a connection is not validated.

First, the attacker can only mount the hostile attack after having first
established a good connection. PKP-RO allows an attacker to perpetually
mount a hostile attack.
Second, the presence of a pinning error is an inherently rate-limiting
factor to the user experience. You're not going to be able to trigger 100
reports if you're causing the user 100 error pages.
Third, it seems like you're arguing for removing report-uri altogether,
given the privacy and DoS risks identified. That is, the more that  you
establish RO is similar to PKP in level of what an attacker can do, the
more it argues for removing report-URI entirely.

---

From: Ryan Sleevi <[email-masked]>
Date: Thu, 4 Sep 2014 17:06:10 -0700

I guess echoing Yoav's earlier remarks that:

"At this stage, we can make editorial changes, but we cannot make
> significant changes on our own. We can withdraw the request to publish, and
> take it back to the working group, but I think that would be inadvisable.
> I think we should proceed, making only editorial changes, and changes
> resulting from discussion with IESG members."

that it's probably not worth discussing much further.

If the WG feels strongly about this, we can certainly withdraw from
publication. If so, I think the Chris' and myself would need to recuse
ourselves from being able to edit this document any further, so the WG
would need to find a new set of editors to proceed with this draft.

Other than that, either we accept these as ERRATA, or we accept them as
being what they are.

That is, my understanding is that this is a bit like discovering a possible
issue after you've shipped your gold master to 100K stores. Yes, it's
awful, but either it's a recall or you fix it in post. My gut is that it's
for post. However, based on Yoav's remarks, I'm not sure that there's very
much we can do editorially to address these. It appears my proposed
suggestion does not address your concerns, so I will not be added it to the
next (and hopefully final) draft.

On Thu, Sep 4, 2014 at 4:53 PM, Trevor Perrin <[email-masked]> wrote:

> more

---

From: Tobias Gondrom <[email-masked]>
Date: Fri, 05 Sep 2014 13:08:23 +0100

Hi Ryan,

just to balance this:
yes, we are in IESG review and we should focus on giving the draft the 
last touches.

With one remark: even in IESG review, if we find a major security 
problem, we need to go back to WGLC.
But so far I am not convinced that this problem is that big in this case 
here and had the perception that we had a WG discussion about this 
before with rough WG consensus (albeit quite rough) as is. (And I admit, 
the long latency times in the WG discussions between emails and draft 
iterations did not make things easier.)

But if anyone thinks this problem will be major, please say so now and 
we could have a quick (5-day) discussion window in the WG whether this 
needs to go back to the WG or whether the chairs did misjudge WG consensus.

Just my personal view, but with hat of WG chair on.

Best wishes, Tobias

On 05/09/14 01:06, Ryan Sleevi wrote:
> I wish you would have raised this during WGLC, which would have 
> benefited from the review and discussion now being paid.
>
>     significant changes on our own. We can withdraw the request to
>     publish, and take it back to the working group, but I think that
>     would be inadvisable.
>     I think we should proceed, making only editorial changes, and
>     changes resulting from discussion with IESG members."
>
>
> Other than that, either we accept these as ERRATA, or we accept them 
> as being what they are.
>
> That is, my understanding is that this is a bit like discovering a 
> possible issue after you've shipped your gold master to 100K stores. 
> Yes, it's awful, but either it's a recall or you fix it in post. My 
> gut is that it's for post. However, based on Yoav's remarks, I'm not 
> sure that there's very much we can do editorially to address these. It 
> appears my proposed suggestion does not address your concerns, so I 
> will not be added it to the next (and hopefully final) draft.
>
>
>
> On Thu, Sep 4, 2014 at 4:53 PM, Trevor Perrin <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     On Thu, Sep 4, 2014 at 4:09 PM, Ryan Sleevi <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >
>     >
>     >
>     > On Thu, Sep 4, 2014 at 4:02 PM, Trevor Perrin <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >>

---

From: Yoav Nir <[email-masked]>
Date: Sat, 6 Sep 2014 12:39:05 +0300

Hi

I agree with Tobias.

The attack that Trevor describes is done by a TLS MitM. TLS MitM can be either trusted or not trusted. 

Non-trusted MitM is not a problem. Even if the user is allowed to click through the interstitial and get the page, the browser still considers the connection insecure (red slash in the UI of Chrome) and the draft says not to note a pin in that case (Section 2.5: "It received the PKP response header field over an error-free TLS connection.”)

Trusted MitM are a bigger issue. I know that Google’s implementation disables pinning for trusted MitM, but even UAs that won’t make a distinction between regular trusted CA and MitM CA won’t pin the data, because the pins (that come from the origin server) will not match the public keys (that come from the proxy). The only issue is if the MitM is malicious and stores its own pins via the PKP-RO headers, causing the UA to be tracked by sending reports to the attacker whenever they access the website.

I think it’s important in this case that the MitM is trusted, meaning that the user (or their IT department) installed that root trust anchor. In that case the user has to trust that this root CA won’t do harm. That’s part of the trust. This scenario may deserve a sentence or two in the security considerations, not a change in the protocol.

Yoav
(with chair and shepherd hat on)