Subject: [Webpush] Require the TTL header

From: Benjamin Bangert <[email-masked]>
Date: Thu, 4 Feb 2016 18:12:32 -0800

Can we require a TTL value for web-push notifications?

The current behavior is not obvious to an implementer and likely to lead
them to think they've successfully sent messages when they're actually
dropped.

If you omit the TTL, then the value is assumed to be 0. This results in the
message being dropped, but the 201 status is still returned. An astute
implementer will notice in this case that no Location header is present,
since no resource was actually created.

Regarding the resource creation, the spec does not clearly state whether a
Location header should be returned for a TTL of 0 since its intended that
the message is only delivered to an online user. If so, it would be useful
to mention.

In the past 2 days, out of approximately 76k webpush notifications to our
service, 23k of them were dropped due to a missing TTL header, but a 201
status was returned.

Defaulting a missing TTL header seems ripe for implementer issues (I have a
hard time believing this many dropped notifications was intentional). If it
was a required header, we could return a 401 which would very quickly alert
developers about the problem.

Cheers,
Ben

---

From: Brian Raymor <[email-masked]>
Date: Fri, 5 Feb 2016 05:42:19 +0000

> Can we require a TTL value for web-push notifications?

I'd support this requirement if it helps to enforce correct behavior in implementations.

> Regarding the resource creation, the spec does not clearly state whether a Location header should be returned for a TTL of 0
> since its intended that the message is only delivered to an online user. If so, it would be useful to mention.

RFC7231 states:

http://tools.ietf.org/html/rfc7231#section-4.3.3

   If one or more resources has been created on the origin server as a
   result of successfully processing a POST request, the origin server
   SHOULD send a 201 (Created) response containing a Location header
   field that provides an identifier for the primary resource created
   (Section 7.1.2) and a representation that describes the status of the
   request while referring to the new resource(s).

> If it was a required header, we could return a 401 which would very quickly alert developers about the problem.

Did you intend a 400 (Bad Request) rather than a 401 (Unauthorized)?

...Brian

---

From: Martin Thomson <[email-masked]>
Date: Fri, 5 Feb 2016 22:26:13 +1100

> I'd support this requirement if it helps to enforce correct behavior in implementations.

I'm also in support of this.  Ben makes a good argument.  Removing
defaults and being up-front about potential mistakes was a pretty good
way to ensure solid interoperability for h2.

---

From: Miguel Garcia <[email-masked]>
Date: Fri, 5 Feb 2016 11:31:30 +0000

I agree with this as well. Especially since the default behavior in GCM
<https://developers.google.com/cloud-messaging/concept-options#ttl> is
different (not supplying it defaults to ttl of 4 weeks). Making it required
would avoid unnecessary confusion.

On Fri, Feb 5, 2016 at 11:26 AM, Martin Thomson <[email-masked]>
wrote:

> _______________________________________________
> Webpush mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/webpush
>

---

From: Brian Raymor <[email-masked]>
Date: Fri, 5 Feb 2016 16:26:34 +0000

I opened - https://github.com/webpush-wg/webpush-protocol/issues/76

From: [email-masked] [mailto:[email-masked]] On Behalf Of Miguel Garcia
Sent: Friday, February 5, 2016 3:32 AM
To: Martin Thomson <[email-masked]>
Cc: Brian Raymor <[email-masked]>; Benjamin Bangert <[email-masked]>; [email-masked]
Subject: Re: [Webpush] Require the TTL header

I agree with this as well. Especially since the default behavior in GCM<%20https:/developers.google.com/cloud-messaging/concept-options#ttl> is different (not supplying it defaults to ttl of 4 weeks). Making it required would avoid unnecessary confusion.

On Fri, Feb 5, 2016 at 11:26 AM, Martin Thomson <[email-masked]<mailto:[email-masked]>> wrote:

_______________________________________________
Webpush mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/webpush

---

From: Benjamin Bangert <[email-masked]>
Date: Fri, 5 Feb 2016 08:55:43 -0800

On Thu, Feb 4, 2016 at 9:42 PM, Brian Raymor <[email-masked]>
wrote:

> > Regarding the resource creation, the spec does not clearly state whether
> a Location header should be returned for a TTL of 0

I took that to mean that no Location is returned if the user is offline
since no notification was created.

> > If it was a required header, we could return a 401 which would very
> quickly alert developers about the problem.
>

Yes, sorry, a 400 would be returned if it's missing.

---

From: Benjamin Bangert <[email-masked]>
Date: Fri, 5 Feb 2016 09:09:28 -0800

On Fri, Feb 5, 2016 at 8:29 AM, Brian Raymor <[email-masked]>
wrote:

>
>
> If you don’t create a push message resource, how do you handle all the
> related dependencies?
>
>
>
> For example, Receiving Push Messages:
>
>
>
> https://tools.ietf.org/html/draft-ietf-webpush-protocol-03#section-7
>
>
>
>    Each push message is pushed as the response to a synthesized GET
>
>    request in a PUSH_PROMISE.  This GET request is made to the push
>
>    message resource that was created by the push service when the
>
>    application server requested message delivery.
>

In our system we determine if the user agent is available as we process the
notification from the application server and attempt delivery immediately
if so.

>  and in Acknowledging Push Messages:
>
>
>
> https://tools.ietf.org/html/draft-ietf-webpush-protocol-03#section-7.2
>
>
>
>    To ensure that a push message is properly delivered to the user agent
>
>    at least once, the user agent MUST acknowledge receipt of the message
>
>    by performing a HTTP DELETE on the push message resource.
>
>
>
> There is also a caveat in the TTL section that the push message resource
> may be deleted before the acknowledgement:
>
>
>
> https://tools.ietf.org/html/draft-ietf-webpush-protocol-03#section-6.2
>
>
>
>    A Push message with a zero TTL is immediately delivered
>
>    if the user agent is available to receive the message.  After
>
>    delivery, the push service is permitted to immediately remove a push
>
>    message with a zero TTL.  This might occur before the user agent
>
>    acknowledges receipt of the message by performing a HTTP DELETE on
>
>    the push message resource.  Consequently, an application server
>
>    cannot rely on receiving acknowledgement receipts for zero TTL push
>
>    messages.
>

 It looks like our current choice of not returning a Location if the user
is offline is not to spec, since returning a 201 implies it must contain a
Location header. We will be changing this to a 400 for a Bad Request if the
TTL header is missing in the future, and always include the Location field
for TTL: 0 messages.

Cheers,
Ben

---

From: Costin Manolache <[email-masked]>
Date: Sat, 6 Feb 2016 12:47:35 -0800

+1 on requiring ttl, and returning a Location even for TTL=0

However receipt behavior is a bit tricky. The service may 'delete
immediately' ( which is the same with
not storing it in the first place ), but the delivery receipts should still
be returned when the device
acks.

The push service may believe the device is connected, and call send - but
in many cases send
will succeed even if the other end is gone - TCP will do its retry and
eventually will timeout and
close the connection, but it takes some time and it's well after send.

I assume 202 (Accepted) would be better for TTL=0 if the service is not
actually storing the message.

GCM ( and I assume other services ) may also optimize small TTLs by not
persisting - so
it may make sense to indicate this with 202, and not require in the spec
that the service is persisting
all messages.

Finally, it is very impractical for us to determine if the user agent is
available at the time of the send
(replication and cross-DC latency, plus the TCP retries)

Costin

On Fri, Feb 5, 2016 at 9:09 AM, Benjamin Bangert <[email-masked]>
wrote:

> In our system we determine if the user agent is available as we process
> the notification from the application server and attempt delivery
> immediately if so.
>
>

---

From: Benjamin Bangert <[email-masked]>
Date: Sat, 6 Feb 2016 13:40:20 -0800

> +1 on requiring ttl, and returning a Location even for TTL=0

I realized on more careful reading that to return a 201, a Location must
accompany it, so our implementation was wrong in this regard.

> not storing it in the first place ), but the delivery receipts should
> still be returned when the device
> acks.
>

I'd be in favor of that, not sure if there's any relevant privacy
implications from immediately indicating that a user is offline. And as you
note, it still leaves open the possibility of returning a 201 with the
belief the client is connected before it times out on the TCP retries. So
all the 202 indicates in this regard is that the client is definitely not
connected, nor has been recently enough for a stale connection to be
present.

> that the service is persisting

Yep, we don't store TTL=0 messages at all.

Cheers,
Ben

---

From: Costin Manolache <[email-masked]>
Date: Sat, 6 Feb 2016 14:04:04 -0800

> connected, nor has been recently enough for a stale connection to be

What I meant is: if TTL=0, don't store the message, and return 202
(Accepted) regardless of user status, since
no resource is actually created. However in order to return a delivery
receipt, it needs to have an identifier.

Regarding privacy implication - the delivery receipt has the same
characteristics, if you send a message and later
get a delivery receipt, it implies the user recently connected, and you can
estimate how long he was offline.
It's not very precise - at least GCM batches and may delay receipts, I
don't think the draft requires immediate
delivery of a receipt.

In either case - a very useful feature would be to reject a message if user
has not been connected for a 'long' time -
I think this is covered implicitly by the expiration of the subscription.

> Yep, we don't store TTL=0 messages at all.

I suspect nobody will, one of the benefits of TTL=0 is that it's likely to
be cheaper and faster in most
implementations.

Costin