Subject: [Webpush] Eric Rescorla's Discuss on draft-ietf-webpush-vapid-03: (with DISCUSS and COMMENT)

From: Eric Rescorla <[email-masked]>
Date: Tue, 15 Aug 2017 16:42:29 -0700

Eric Rescorla has entered the following ballot position for
draft-ietf-webpush-vapid-03: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-webpush-vapid/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

This design seems to have the unfortunate security property that the
JWT is really just a bearer token. The only reason it has to involve
public key cryptography at all is to allow the push cient to refer to
the public key when it makes a subscription. However, as the
Security Considerations acknowledge, this allows a cut-and-paste attack
(more than just replay) by an attacker who acquires any JWT,
because it does not include the message itself.

The primary motivation for this appears to be to minimize CPU cost
on the push service. However, there are designs which do this
without allowing replay. For instance:

- Have the push service have a static public key K_svc which
  is published to application servers (e.g., via well-known).
- In order to form the JWT, have the application server generate
  a fresh DH key K_app, which is embedded in the JWT.
- The message which the app server sends to the push service
  is then MACed with the DH shared secret Z.

This removes the cut-and-paste attack (though of course replay
attacks are still possible) unless the push service keeps a replay
cache. The replay service can trivially amortize the DH computation
(it has to amortize the signature verification in any case to
get any computational benefit) but it's soft state, so it can
just forget it at any time.

Ultimately, this is a WG decision, but given that there are designs
with much better security properties, I'd like to know that the WG
considered and rejected this kind of design alternative before
we advance the weaker design.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Abstract
   This identification information can be used to restrict the use of a
   push subscription a single application server.

to a single

S 1.
   Additionally, this design also relies on endpoint secrecy as any
   application server in possession of the endpoint is able to send
   messages, albeit without payloads.  In situations where usage of a
   subscription can be limited to a single application server, the
   ability to associate a subscription with the application server could
   reduce the impact of a data leak.

I don't understand this text.

S 1.2
   The words "MUST", "MUST NOT", "SHOULD", and "MAY" are used in this
   document.  It’s not shouting, when they are capitalized, they have
   the special meaning described in [RFC2119].

This seems over-cute. We now have a document that describes this
convention.

S 2.
   This JWT is included in an Authorization header field, using an auth-
   scheme of "vapid".  A push service MAY reject a request with a 403
   (Forbidden) status code [RFC7235] if the JWT signature or its claims
   are invalid.

Given that "vapid" is tied to "P-256" it seems like it would be better
to rename this "vapid-p256"

S 4.2.
   When a push subscription has been associated with an application
   server, the request for push message delivery MUST include proof of
   possession for the associated private key that was used when creating
   the push subscription.

This really isn't a proof of possession, as the Security Considerations
makes clear.

S 5.
You should call out that the email address is just a bare assertion.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 16 Aug 2017 11:56:39 +1000

> we advance the weaker design.

We discussed the implications of the bearer token design and concluded
that this was acceptable.  We didn't discuss additionally signing
messages in this way.  Another alternative is to describe how token
binding could be used to prevent token theft.

One complaint I get about webpush is the amount of overhead crypto
adds to sending messages.  Mostly from a complexity standpoint.  It's
a real implementation barrier.  We've forced people to jump through a
lot of hoops and that has made them pretty annoyed.  Deploying this as
a required extension could be additionally annoying, not to mention
incompatible with the existing deployed code.

What is nice about the specific design you describe (and my flippant
tokbind suggestion as well coincidentally) is that it can be
incrementally added to this scheme without a complete overhaul,
probably without even changing the scheme name, suggestions below
notwithstanding.

It could be an optional increment, if we could decide that the push
service doesn't require this additional protection and it was
primarily for the benefit of application servers.  Yes, there are
benefits that accrue to push services and user agents alike, but we
might decide that they don't gain any certainty about these benefits.
That suggests a second, though less satisfying, path: a new draft
describing the enhancement.

I did have a stab at fixing your nits here:
  https://github.com/webpush-wg/webpush-vapid/pull/43

> I don't understand this text.

I realize that this is not at all clear.  How about:

Additionally, the design of RFC 8030 relies considerably on the secrecy of push
subscription URIs.  Any application server in possession of this URI is able to
send messages to the user agent.  If use of a subscription could be limited to
a single application server, this would reduce the impact of the push
subscription URI being learned by an unauthorized party.

> to rename this "vapid-p256"

That's just a cosmetic change.  If we change the scheme in an
incompatible way based on the comment above, then a name change might
come with that, but your design might be able to use the same name.

> This really isn't a proof of possession, as the Security Considerations

Reworded in a couple of places to make the weaker claim.

> You should call out that the email address is just a bare assertion.

See PR.

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 15 Aug 2017 19:04:19 -0700

On Tue, Aug 15, 2017 at 6:56 PM, Martin Thomson <[email-masked]>
wrote:

> that this was acceptable.

That's a much easier discussion to have without an alternative.

> We didn't discuss additionally signing

The flippant thing to say would be that I-Ds come with a big disclaimer
for exactly this reason. More seriously, the future is bigger than the past
and given that (as you observe) the vapid token is used to identify
what you support, it seems like deploying the new thing would be
straightforward.

I'm not terribly sympathetic to the complexity point: this is a very modest
amount of crypto.

> describing the enhancement.

I agree that that's possible, but it's kind of unsatisfying to be publishing
a draft with this kind of known deficiency.

Maybe the chairs or ART ADs would like to weigh in here?

> come with that, but your design might be able to use the same name.

I agree that it is largely a cosmetic change, which is why it's in the
COMMENT section.
With that said, I don't think it's purely a cosmetic change: given that we
know that
there is diversity along this dimension, why are we baking one identifier
into the
name

---

From: Ben Campbell <[email-masked]>
Date: Tue, 15 Aug 2017 21:31:36 -0500

> On Aug 15, 2017, at 9:04 PM, Eric Rescorla <[email-masked]> wrote:
> 
> 
> 
>> On Tue, Aug 15, 2017 at 6:56 PM, Martin Thomson <[email-masked]> wrote:

Weighing in as an irresponsible (for this draft) ART AD:

My knee jerk response is to agree with Ekr. But it’s worth discussing how big of an incompatibility with existing code people are talking about. Are there a lot of deployed services that support vapid? Do people think that existing implementors will choose not to implement/deploy the additional protection?

Ben.

[…]

---

From: Adam Roach <[email-masked]>
Date: Tue, 15 Aug 2017 21:35:47 -0500

> My knee jerk response is to agree with Ekr. But it’s worth discussing how big of an incompatibility with existing code people are talking about. Are there a lot of deployed services that support va…

My inclination is to ask the WG to weigh in on this issue in particular. 
I've received some pretty confident responses indicating that VAPID has 
no issues with crypto-agility. Given that such is the case, it seems 
like the question of how many deployed implementations we have is pretty 
irrelevant: if we can't change crypto schemes at scale, then we 
shouldn't publish this document anyway.

/a

---

From: Martin Thomson <[email-masked]>
Date: Wed, 16 Aug 2017 15:09:42 +1000

On 16 August 2017 at 12:35, Adam Roach <[email-masked]> wrote:
>> My knee jerk response is to agree with Ekr. But it’s worth discussing how
>> big of an incompatibility with existing code people are talking about. Are
>> there a lot of deployed services that support vapid? Do people think that
>> existing implementors will choose not to implement/deploy the additional
>> protection?
>
>
>
> I've received some pretty confident responses indicating that VAPID has no
> issues with crypto-agility. Given that such is the case, it seems like the
> question of how many deployed implementations we have is pretty irrelevant:
> if we can't change crypto schemes at scale, then we shouldn't publish this
> document anyway.

No change is free.  We'd first have to establish whether the change is
a net gain.

The presumption here seems to be that the presented design is
unconditionally superior.  That's not true.  This is a trade-off, and
- in my opinion - not one that favours any change.

For me the important point here is that the proposed change (in any
form) is of fairly marginal value.  Yes, the IETF is investing in
anti-bearer-token-theft technology, but in this case what you get is
the ability to send a message to user agent and wake it up.

You get that after stealing two pieces of information: the push
subscription URI and this token.  (That's assuming that the first was
not already sufficient, which it is in some cases.)  You can't send
arbitrary data, for that you need two more pieces of information: a
public key and authorization code, and neither of these appear on the
HTTPS connection from which the other items were stolen.

So that's benefits.  Let's talk about costs.

If we take the suggested design the biggest challenge is key
management for K_p (the key that the push service would have to
maintain).  Maintaining a set of keys across a globally-distributed
service isn't free.  Given that key compromise would undermine the
effort we'd be putting in here, you also need a strategy for updating
those keys and revocation of old or compromised keys.  For that you
want multiple keys, which means key identifiers in messages.  You also
need some way of having application servers update their view of what
keys are current, maybe using expiration times.

Technically, we also have to work out what to cover with the MAC.  My
experience with HTTP and signing suggests that this isn't as trivial
as it might seem.  I think that we could canonicalize the push
subscription URI, Content-Encoding header field and payload and be
reasonably sure that the MAC couldn't be transplanted.  Probably also
the RFC 8030 TTL, Urgency and Topic header fields, which suggests that
there needs to be a way to sign any new and relevant header fields
that might be added later.  This would need some more analysis than
I've given it here.

Then we have to decide whether this is a mandatory part of the
mechanism.  It's marginally easier if it isn't, but the arguments ekr
has presented suggest that he thinks it should be.  That makes the
deployment story harder.  For that, we really would want to exercise
the cryptographic agility Adam refers to here.  It's possible that we
don't need to worry too much about removing the old stuff on the basis
that it's all voluntary anyway, but now there are two things that need
to be implemented and supported.  What incentive have we provided to
use the new thing as opposed to the old thing?

To answer Ben's question: there are a small number of push services
(one for each browser, effectively http://caniuse.com/#feat=push-api),
but a LOT of application servers.

And to briefly address the costs of the other alternative: Doing
something like token binding has a bunch of implementation costs.  It
cross layers in ways that make it difficult to both implement and
deploy.  For instance, we couldn't deploy it without
draft-campbell-tokbind-ttrp being supported by our vendors.  This is
the same reason we considered and rejected client certificates for
this - one of the many alternatives that were considered.

p.s., It's been a while since I've been on the receiving end of the
old "your code doesn't matter, you took a risk implementing an I-D"
argument.  As an argument it kinda stinks.  I'm embarrassed to confess
that I've used it in the past, despite reminding myself not to.  I'd
be interested in learning whether the IESG has an opinion on the
subject.  A statement of policy might be valuable.

---

From: Adam Roach <[email-masked]>
Date: Wed, 16 Aug 2017 00:41:33 -0500

> subject.  A statement of policy might be valuable.

Given that the alternative is effectively forbidding breaking changes 
after WG adoption -- which is clearly untenable -- I'm not sure what 
kind of policy you're looking for here. Can you clarify?

/a

---

From: Martin Thomson <[email-masked]>
Date: Wed, 16 Aug 2017 16:22:26 +1000

On 16 August 2017 at 15:41, Adam Roach <[email-masked]> wrote:
> Given that the alternative is effectively forbidding breaking changes after
> WG adoption -- which is clearly untenable -- I'm not sure what kind of
> policy you're looking for here. Can you clarify?

(Hmm, I got this message twice...)

The usual exchange goes a: "I have code", b: "you implemented an I-D,
that's your problem".

Some guidance about how to manage this situation would be appreciated.

I'm not looking for a prohibition on changes at any point in the
process.  That's ludicrous, after all, people do understand what it
means to implement an I-D.  But statements like "there is code" or
"there is deployed code" are valid and useful input to a discussion
and not just as proof that a given thing is possible.  That means
potentially affecting the outcome of a decision.

In this particular case, the arguments presented weren't problematic,
because the real arguments was actually "well, this change doesn't
seem that bad to me".  But it's not the first time someone has opened
with that argument and it annoyed me enough to comment.

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 16 Aug 2017 05:55:33 -0700

On Tue, Aug 15, 2017 at 10:09 PM, Martin Thomson <[email-masked]>
wrote:

> no

This is a rather odd argument, as it also applies to the mechanism in this
document as a whole.

> keys are current, maybe using expiration times.

You raise several issues here:

1. The cost of coordinating the key. I agree that this is a real cost to
the mechanism I proposed here, though it's possible to mitigate it in a
number of ways (e.g., by having a centralized oracle for doing the
verification, given that you don't need to do a lot of EC computation
here).

2. The need for revocation. You don't actually need that here, just a key
update mechanism and a way to tell the app server "use this new key". But
this latter mechanism is the same one you use for key updates and for
updates to the mechanism, which this document implies the WG thinks is easy.

3. Key identifiers in messages are just specification work, so I don't
think that that's a strong argument (see below).

4. Yes, it's a concern if these keys are compromised, but note that even if
that's so, the security of the system reduces back to a bearer token
(because you need the DH public key and the signed JWT), so that doesn't
seem like much of a cost.

> that might be added later.  This would need some more analysis than

I agree that there are complexities here, but they're complexities that are
motivated by actually trying to design something with stronger security
properties. More generally, they are costs to the WG, but not really costs
to the world at large.

Finally, I think this discussion has ratholed a bit on this particular
counter-proposal. My point isn't that you should do this particular thing
but rather that it seems like the WG decided on a bearer-token model
without giving sufficient consideration to alternatives that have better
security (of which this is merely one). The fact that I was able to produce
two such alternatives in a few hours (which I'll concede are not in every
way superior) and your response was not "yes, we considered those and here
is a pointer to the WG archives" seems to lend that point some plausibility.

As I said initially, this is ultimately a WG decision and I'm fine with
being told by the ADs and the Chairs that they believe sufficient
consideration has been given, in which case I'll withdraw my discuss. So
far I haven't heard that.

-Ekr

> that I've used it in the past, despite reminding myself not to.

I think I explicitly labelled this argument as flippant, but I also don't
think it's valueless, especially when put up against "Three large companies
implemented and are too lazy to change".

---

From: Ben Campbell <[email-masked]>
Date: Wed, 16 Aug 2017 09:47:57 -0500

> On Aug 16, 2017, at 1:22 AM, Martin Thomson <[email-masked]> wrote:
>

---

From: Ben Campbell <[email-masked]>
Date: Wed, 16 Aug 2017 10:03:37 -0500

(Oops ignore the empty message—itchy mousepad finger. Or maybe I want to make sure Martin gets everything twice.)

> with that argument and it annoyed me enough to comment.

From a general policy perspective, I think there are two principles:

1. The IETF maintains change control on IETF stream documents. That suggests we can change things at any time.

2. Working code matters.

How you balance the two depends on the context. We encourage early code. But that’s not necessarily the same as encouraging going production with early code. It’s reasonable to push back on gratuitous changes if code demonstrates things already work well. But I think the usual basis of such an argument is that the existing protocol has been proven to work as is, not that the weight of that code makes it too hard to change.

 It’s reasonable to argue that, while a new feature may be of value, implementors will not deploy it because <arguments>. That seems to happen more often with security features than other things, maybe because the IETF as a whole cares more about security than many deployment communities. But that’s not really an “existing code” argument per se (although existing code can contribute to it.)

There’s a difference between when we choose to standardize something already in wide deployment, vs when we invent something new and people deploy code based on drafts. But even that doesn’t mean we can’t change anything (see principle 1)

 But in any case, I agree that existing implementations/deployments and their scope are proper inputs to the discussion. But they are not trump cards.

---

From: jr conlin <[email-masked]>
Date: Wed, 16 Aug 2017 08:38:06 -0700

If I may lend a bit of prospective from the point of view of the Push
Service provider.

We enjoy VAPID because it provides us a way to associate contact
information with the data we receive. It's (in theory) voluntary, and we
can prefer feeds that provide the data. VAPID provides us a reasonably
low cost means to "validate" the information we get *if* the requested
URI was secured using the same key, as well as prove that there has been
no casual MITM alterations of data. As Martin notes, the bar is set
fairly high for an attacker sending malicious data to the remote customer.

>From our point of view, VAPID isn't really a security device, it's a
contact device. It's equivalent to someone stapling their business card
to their letter, and is most useful for our ops folk when things go
awry. That's why we accept feeds that use VAPID even if the requested
URI was not secured. In addition, our most pressing concern is with
sites that perform non-hostile abuse. (e.g. we currently get near 2:1
traffic from sites sending to URIs that are no longer active, and do not
provide good ways to contact their operations.)

That said, we still encourage frankly horrible security practices around
VAPID just so that the barrier to use is as low as possible. We find the
data to still be that useful. For example, we encourage application
servers to cache the VAPID header for as long as possible to reduce the
publication load.

Naturally, we look forward to the guidance provided by the working group
and will adopt their suggestions, but I did want to offer what the more
hands-on usage of the spec has been so far.

On 8/16/17 5:55 AM, Eric Rescorla wrote:
>
>
> On Tue, Aug 15, 2017 at 10:09 PM, Martin Thomson
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     On 16 August 2017 at 12:35, Adam Roach <[email-masked]
>     <mailto:[email-masked]>> wrote:
> This is a rather odd argument, as it also applies to the mechanism in
> this document as a whole.
>
>
> 1. The cost of coordinating the key. I agree that this is a real cost
> to the mechanism I proposed here, though it's possible to mitigate it
> in a number of ways (e.g., by having a centralized oracle for doing
> the verification, given that you don't need to do a lot of EC
> computation here).  
>
> 2. The need for revocation. You don't actually need that here, just a
> key update mechanism and a way to tell the app server "use this new
> key". But this latter mechanism is the same one you use for key
> updates and for updates to the mechanism, which this document implies
> the WG thinks is easy.
>
> 4. Yes, it's a concern if these keys are compromised, but note that
> even if that's so, the security of the system reduces back to a bearer
> token (because you need the DH public key and the signed JWT), so that
> doesn't seem like much of a cost.
>
>
> I agree that there are complexities here, but they're complexities
> that are motivated by actually trying to design something with
> stronger security properties. More generally, they are costs to the
> WG, but not really costs to the world at large.
>
>
> counter-proposal. My point isn't that you should do this particular
> thing but rather that it seems like the WG decided on a bearer-token
> model without giving sufficient consideration to alternatives that
> have better security (of which this is merely one). The fact that I
> was able to produce two such alternatives in a few hours (which I'll
> concede are not in every way superior) and your response was not "yes,
> we considered those and here is a pointer to the WG archives" seems to
> lend that point some plausibility.
>
> As I said initially, this is ultimately a WG decision and I'm fine
> with being told by the ADs and the Chairs that they believe sufficient
> consideration has been given, in which case I'll withdraw my discuss.
> So far I haven't heard that.
>
> -Ekr
>
> I think I explicitly labelled this argument as flippant, but I also
> don't think it's valueless, especially when put up against "Three
> large companies implemented and are too lazy to change".
>
>
>
>
> _______________________________________________
> Webpush mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/webpush

---

From: Ted Hardie <[email-masked]>
Date: Wed, 16 Aug 2017 09:10:01 -0700

On Wed, Aug 16, 2017 at 8:03 AM, Ben Campbell <[email-masked]> wrote:

> 1. The IETF maintains change control on IETF stream documents. That
> suggests we can change things at any time.
>
> 2. Working code matters.
>
> How you balance the two depends on the context. We encourage early code.
> But that’s not necessarily the same as encouraging going production with
> early code. It’s reasonable to push back on gratuitous changes if code
> demonstrates things already work well. But I think the usual basis of such
> an argument is that the existing protocol has been proven to work as is,
> not that the weight of that code makes it too hard to change.
>
>
I think one way of looking at the context is to look at the reason for the
code.

Code built explicitly to test a particular approach can change at any time
that the approach needs to change; we get to presume that sort of code
doesn't underpin any running systems and that the friction to change is
thus very low.

Code that is currently deployed in production systems can be very difficult
to change and deciding to change it may break interoperability. In some
cases it may permanently change the community of use for a protocol (some
changes to JSEP, for example, would break its ability to interoperate with
deployed non-browser systems, thus changing the community of use).

The reality is that there almost no code purely in category one and a lot
of IETF draft code hasn't been deployed at scale enough to be in category
two.   Lars' and Christian's implementation of QUIC demonstrate that one
isn't gone completely, but almost all code written for IETF standards is
intended for production systems eventually.  It lives in a gray area
between the two poles above; changing it requires resources and time but
does not change deployed systems.  Changes are costs, in other words, that
are incurred by the developer prior to getting to deploy.

 Pushing back on changes to "gray code" is entirely normal, and you can see
that play out inside working groups as different parties determine whose
code has to change to meet a requirement or solve a problem.  Inside a
working group, the weight of the code does matter.  A two-line changes to a
server may result in the same behavior as a new module for the client, and
the working group should take that into account.

Where the IESG analysis is a bit different is that it has to take into
account the value of the change not just to the current writers of code,
but for anyone implementing the spec later and to the users of the
protocol.  That change in context of interpretation tends to make the IESG
a bit more willing to require changes to gray code than a working group
would be; its collective thumb is lightly on the scale of the future
implementers and users.

I personally think that's a reasonable result and part of the division of
labor, but I also think the IESG has to do so knowingly and make that
transparent to the community.  That means that we probably should drop "you
paid your money and knew the risks" as messaging for late code change.
Perhaps "we recognize the costs here but believe they are balanced,
especially when considering new implementations which may arise" would be
better PR.

Take that advice with a large grain of salt, though, as no one has ever
suggested I was good at PR.

regards,

Ted

---

From: Martin Thomson <[email-masked]>
Date: Thu, 17 Aug 2017 10:53:36 +1000

On 16 August 2017 at 22:55, Eric Rescorla <[email-masked]> wrote:
> but rather that it seems like the WG decided on a bearer-token model without
> giving sufficient consideration to alternatives that have better security
> (of which this is merely one). The fact that I was able to produce two such
> alternatives in a few hours (which I'll concede are not in every way
> superior) and your response was not "yes, we considered those and here is a
> pointer to the WG archives" seems to lend that point some plausibility.

Maybe the specific suggestion was the problem.  We did discuss a
number of alternatives.  The history of this draft actually includes
those options.  Some of them did not have the unfortunate property you
don't like.  We considered client certificates, but rejected them on
operational grounds (you might think that unfortunate, because that
would be superior in many ways to all options discussed on this
thread).

Here are the options as originally presented:

https://tools.ietf.org/html/draft-thomson-webpush-vapid-00

       2.1.1.  Application Tokens  . . . . . . . . . . . . . . . . .   4
       2.1.2.  Contact Information Header Field  . . . . . . . . . .   4
       2.1.3.  Request Signing . . . . . . . . . . . . . . . . . . .   5
       2.1.4.  Token Binding . . . . . . . . . . . . . . . . . . . .   5

I apologize for my reaction.

> implemented and are too lazy to change".

That's a strawman.  The implementations I worry about are those in the
very many application servers.  The limited number of push service
operators have demonstrated an ability to ship changes fairly rapidly.