Subject: [dispatch] RTCWEB I-D with thoughts on the framework

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 08 Feb 2011 09:48:46 -0500

Some of my colleagues from Skype and I have put together a draft which 
gives our thoughts on the scope of the protocols and functionality for 
enabling browser RTC. I've just submitted the draft, which you can find 
here:

http://www.ietf.org/id/draft-rosenberg-rtcweb-framework-00.txt

Some of the key points:

* keep it minimal
* APIs for controlling behaviors of the various media components, with 
defaults for those that dont care
* no SIP or Jingle; leave that to proprietary over websockets/HTTP
* no ICE - just STUN. ICE can be a javascript library.

Thanks,
Jonathan R.
-- 
Jonathan D. Rosenberg, Ph.D.                   SkypeID: jdrosen
Skype Chief Technology Strategist
[email-masked]                              http://www.skype.com
[email-masked]                            http://www.jdrosen.net

---

From: Adam Roach <[email-masked]>
Date: Tue, 08 Feb 2011 10:35:31 -0600

I agree with the general architectural principles expressed in this 
document.

I think the STUN-based approach for voice hammer avoidance is 
problematic for the purpose of legacy interoperation (e.g, Figure 2 with 
the right-hand-side replaced with a normal SIP Proxy/Registrar and 
normal SIP user agent), as few (if any?) deployed SIP user agents are 
likely to support this exchange. I believe this specific approach may 
need some tweaking.

/a

On 2/8/11 8:48 AM, Jonathan Rosenberg wrote:
> enabling browser RTC. I've just submitted the draft, which you can 
> find here:
>

---

From: Matthew Kaufman <[email-masked]>
Date: Tue, 08 Feb 2011 09:15:28 -0800

On 2/8/2011 8:35 AM, Adam Roach wrote:
> problematic for the purpose of legacy interoperation (e.g, Figure 2 
> with the right-hand-side replaced with a normal SIP Proxy/Registrar 
> and normal SIP user agent), as few (if any?) deployed SIP user agents 
> are likely to support this exchange. I believe this specific approach 
> may need some tweaking.

I agree that it interferes with legacy interop, however I see no way 
around it that doesn't allow a browser to be used inappropriately as a 
UDP-based attack vector behind a firewall.

Existing SIP UAs can either be upgraded or sit behind a very simple 
SBC-like device that affirms the willingness to receive.

Matthew Kaufman

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Tue, 8 Feb 2011 11:42:21 -0600

Jonathan,

The diagram makes this look like a media gateway control protocol,
depending on where the feature set resides.

Also, how would the Web interface differ from the stimulus protocols of
the past?

How much of this is meta-packaging of blobs passed between browser and
its server?

Seems to be half-way between a completely proprietary interface, and one
where every message and parameter is defined.

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Jonathan Rosenberg
Sent: Tuesday, February 08, 2011 9:49 AM
To: 'DISPATCH list'
Subject: [dispatch] RTCWEB I-D with thoughts on the framework

Some of the key points:

* keep it minimal

Thanks,
Jonathan R.

_______________________________________________
dispatch mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dispatch

---

From: Henry Sinnreich <[email-masked]>
Date: Tue, 08 Feb 2011 12:01:20 -0600

+1

I fully agree with Matthew.

Thanks, Henry

---

From: Peter Musgrave <[email-masked]>
Date: Tue, 8 Feb 2011 13:39:07 -0500

I like the draft and think this is a useful framework.

Regards, 

Peter Musgrave

On 2011-02-08, at 9:48 AM, Jonathan Rosenberg wrote:

> Some of my colleagues from Skype and I have put together a draft which gives our thoughts on the scope of the protocols and functionality for enabling browser RTC. I've just submitted the draft, which you can find here:
> 
> * APIs for controlling behaviors of the various media components, with defaults for those that dont care

---

From: Adam Roach <[email-masked]>
Date: Tue, 08 Feb 2011 12:53:27 -0600

> SBC-like device that affirms the willingness to receive.

If a device can be upgraded to support this new behavior, I agree that 
it solves the issue. But such upgrades are often impractical or 
impossible. For example, I doubt I'm ever going to see another software 
update for the 6-year-old SIP phone on my desk; support for it reached 
EOL a while ago, and it has been superseded by newer hardware.

In any case, I think you can determine willingness to receive through 
other mechanisms. For example, receiving a media stream from a remote 
location is probably a pretty good indication that the remote location 
is willing to engage in a media exchange.

There may be other mechanisms to achieve the desired security properties 
without sacrificing interoperability with things that just do normal, 
stock RTP. I think it is premature to dismiss them out of hand.

/a

---

From: Adam Roach <[email-masked]>
Date: Tue, 08 Feb 2011 13:15:55 -0600

> Also, how would the Web interface differ from the stimulus protocols of

It could be as low-level as a stimulus protocol, or as high-level as:

{
   "action":"call",
   "phone_number":"+12145550107",
   "my_addresses":[{"ip":"192.0.2.7","port":53872}]
}

Or anything in between. That's kind of the point of relegating as much 
behavior as possible to Javascript. It lets people do whatever suits 
their application.

If we defined what this looked like, it would be far more constraining 
on the web-based applications.

> How much of this is meta-packaging of blobs passed between browser and

However much the application designer wants.

> where every message and parameter is defined.

I think you've missed the key behavior that this draft is attempting to 
promote. The browser will run javascript downloaded from the server, and 
use it to talk to the server. Exactly how these things communicate with 
each other is a private matter to be determined by the application 
developer.

The parallel to make is the instant-messaging and presence functionality 
you see in certain web pages (Facebook and GMail to name a couple of 
examples). The protocol that the GMail chat web widget uses to talk to 
the GMail chat servers is something that Google invented themselves out 
of whole cloth. And they can change it whenever they want. And 
everything still works. It interoperates with other XMPP systems just 
fine, too, since the GMail servers have an XMPP interface that talks to 
the rest of the world.

To put it into a diagram:

                 +-----------+             +-----------+
                 |   Web/    |             |           |
                 |   XMPP    |     XMPP    |   XMPP    |
                 |           |-------------|           |
                 |  Server   |             |  Server   |
                 |           |             |           |
                 +-----------+             +-----------+
                      /                           \
                     /                             \
                    /                               \  XMPP
                   /                                 \
                  /  Proprietary over                 \
                 /   HTTP/Websockets                   \
                /                                       \
          +-----------+                           +-----------+
          |JS/HTML/CSS|                           |           |
          +-----------+                           |   XMPP    |
          +-----------+                           |  Client   |
          |           |                           |           |
          |           |                           +-----------+
          |  Browser  |
          |           |
          |           |
          +-----------+

Does that look familiar? This has been successfully deployed at scale, 
and it works great. I can't imagine why it wouldn't work just fine for 
real-time media.

/a

---

From: "Mike Hammer (hmmr)" <[email-masked]>
Date: Tue, 8 Feb 2011 13:53:24 -0600

Adam,

So, this effectively devolves to a mechanism to push an application to
the browser and have it then run.

This seems to argue for most of how it works to be proprietary, and just
enough meta-packaging 
to let the browser know that it is a real-time blob and how to hook into
the platform hardware.

Not saying I disagree, just making some observations.

Just gauging the amount of standards definition here.

Mike

-----Original Message-----
From: Adam Roach [mailto:[email-masked]] 
Sent: Tuesday, February 08, 2011 2:16 PM
To: Mike Hammer (hmmr)
Cc: Jonathan Rosenberg; DISPATCH list
Subject: Re: [dispatch] RTCWEB I-D with thoughts on the framework

{
   "action":"call",

/a

---

From: Adam Roach <[email-masked]>
Date: Tue, 08 Feb 2011 14:02:21 -0600

On 2/8/11 1:53 PM, Mike Hammer (hmmr) wrote:
> This seems to argue for most of how it works to be proprietary...

Yes -- as proprietary as the internal function calls inside an application.

> ...and just enough meta-packaging to let the browser know that it is a real-time blob and how to hook into
> the platform hardware.

There are a few more tasks than that -- you've got to be able to send & 
receive UDP (or at least RTP over UDP); you need to be able to collect 
candidate IP addresses and ports; and you need to access more from the 
platform than just hardware (e.g., audio and/or video codecs, encryption 
support, etc).

But, yes, defined this way, it's a relatively small amount of work, with 
potentially huge payoffs.

/a

---

From: <[email-masked]>
Date: Wed, 9 Feb 2011 19:52:28 +0000

Hi,

I mostly agree with the model described in this draft

>* no ICE - just STUN. ICE can be a javascript library.

One point about this: I suppose here you mean that the browser just implement the STUN connectivity check between candidate address pairs for media authorization. But how about gathering the candidate addresses? I think that, i.e. STUN and TURN, should also be implemented by the browser. And also preferably a HTTP tunneled candidate. How those candidate addresses are used/passed (the "ICE logic") within the session setup would in itself be an application specific matter.

Markus

>Behalf Of ext Jonathan Rosenberg
>Sent: 08 February, 2011 16:49
>To: 'DISPATCH list'

---

From: Jonathan Rosenberg <[email-masked]>
Date: Wed, 09 Feb 2011 17:20:42 -0500

inline:

On 2/9/2011 2:52 PM, [email-masked] wrote:
> One point about this: I suppose here you mean that the browser just
> implement the STUN connectivity check between candidate address pairs
> for media authorization.

Yes.

> But how about gathering the candidate
> addresses?

This would be orchestrated by the Javascript (and server as needed) by 
telling the browser how to use STUN. I had in mind an API that might 
look like:

{IPaddr:port} = STUNCheck(destinationIP,
                           destinationPort,
                           username,
                           password,
                           AttributeValues);

To gather a server reflexive, the Javascript uses this API against a 
server IP/port which the application provider knows is a STUN server of 
its choosing.

To gather a relayed candidate, the Javascript once again uses this API 
against a server IP/port, but this STUN server acts like a TURN server 
of sorts. The STUN check is merely used to create the NAT binding to 
allow for relayed traffic to reach the client. The actual relayed 
candidate and other semantics of the relay operation can be handled by 
the server entirely. This is kind of like the old slushy proposal if 
anyone remembers that...

There are clearly some benefits to using TURN directly - primarily it 
would allow more easy reuse of existing servers. However, if we focus on 
a minimum browser capability, a basic STUN probe which returns a 
reflexive candidate is actually sufficient; TURN is not needed even for 
the purposes of gathering relayed candidates for interoperable ICE.

Thanks,
Jonathan R.

---

From: <[email-masked]>
Date: Thu, 10 Feb 2011 20:04:14 +0000

Hi,

>server IP/port which the application provider knows is a STUN server of

Looks good to me.

>the server entirely. This is kind of like the old slushy proposal if

OK, I can see that this would be the bare minimum needed to support UDP relaying.

>the purposes of gathering relayed candidates for interoperable ICE.

I think this is an issue that deserves more discussion. I don't have a strong opinion about it yet. I would say that if a lot of service providers already use or intend to use TURN servers, it should be included in the browser. Those providers who don't use it could still use something else and just utilize the STUN probe from the browser. 

Markus

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 11 Feb 2011 15:25:35 +0100

I agree to large parts of the document, and I think that the section on interoperability with existing VoIP gear brings up important aspects.

However, I am not completely comfortable with a couple of things:
* The application would be responsible for rate adaptation, but what is the incentive for the application developer to do something that is fair to other applications or users? I would prefer that this is part of the browser (and with the auto update feature most browsers use the update cycles could be short anyway)
* Are you suggesting that the <video> (and consequently also the <audio>) tag should not be used? I don't think this is a good idea. Since they are accepted and used by the web community we should use them also for RTC-Web rather than trying to invent something new.

--Stefan 

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Jonathan Rosenberg
> Sent: den 8 februari 2011 15:49
> To: 'DISPATCH list'
> Some of my colleagues from Skype and I have put together a 
> draft which gives our thoughts on the scope of the protocols 
> and functionality for enabling browser RTC. I've just 
> submitted the draft, which you can find
> here:
> 
> * APIs for controlling behaviors of the various media 
> components, with defaults for those that dont care

---

From: Adam Roach <[email-masked]>
Date: Fri, 11 Feb 2011 08:45:48 -0600

On 2/11/11 8:25 AM, Stefan Håkansson LK wrote:
> * Are you suggesting that the<video>  (and consequently also the<audio>) tag should not be used? I don't think this is a good idea. Since they are accepted and used by the web community we should use them also for RTC-Web rather than trying to invent something new.

I don't think any group in the IETF is in a position to standardize HTML 
semantics. We're defining what goes on the wire. The stuff that goes 
inside the browser has to come from W3C.

/a

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 11 Feb 2011 15:48:31 +0100

Exactly my point. //S 

> Sent: den 11 februari 2011 15:46
> To: Stefan Håkansson LK
> Cc: Jonathan Rosenberg; [email-masked]; 'DISPATCH list'
> > * Are you suggesting that the<video>  (and consequently 
> also the<audio>) tag should not be used? I don't think this 
> is a good idea. Since they are accepted and used by the web 
> community we should use them also for RTC-Web rather than 
> trying to invent something new.
> 
> I don't think any group in the IETF is in a position to 
> standardize HTML semantics. We're defining what goes on the 
> wire. The stuff that goes inside the browser has to come from W3C.
> 
> /a
>

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 11 Feb 2011 17:54:15 +0100

Hi Jonathan,

I pretty much share the interoperability concerns other have raised regarding mandatory usage of STUN.

I do understand the concern you describe, with downloading a JavaScript that triggers the sending of malicious media. However, assume you download a legitimate JavaScript SIP stack. The JavaScript itself is not going to trigger any media to be sent. The media will be sent once you've used SIP to establish a session, and then the media is sent based on the negotiated SDP. Of course, the remote SDP could also be malicious, but that is valid for any SIP implementation - browser based or not :)

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Jonathan Rosenberg
Sent: 8. helmikuuta 2011 16:49
To: 'DISPATCH list'
Subject: [dispatch] RTCWEB I-D with thoughts on the framework

Some of my colleagues from Skype and I have put together a draft which gives our thoughts on the scope of the protocols and functionality for enabling browser RTC. I've just submitted the draft, which you can find
here:

Some of the key points:

* keep it minimal

Thanks,
Jonathan R.

---

From: Matthew Kaufman <[email-masked]>
Date: Fri, 11 Feb 2011 09:01:25 -0800

> I do understand the concern you describe, with downloading a JavaScript that triggers the sending of malicious media. However, assume you download a legitimate JavaScript SIP stack. The JavaScript…

Once one is downloading a native application, it is possible for it to 
do all sorts of bad things, including sending streams of UDP data to 
arbitrary addresses behind your firewall.

However users have a much higher expectation of their web browser (and 
popular browser plug-ins), namely that simply visiting a web page won't 
be a vector for an attack.

So far there are no proposals which allow interoperability with 
completely unmodified RTP endpoints while also ensuring that the browser 
user remains reasonably well protected from the behavior of 
potentially-malicious sites they visit.

Matthew Kaufman

---

From: <[email-masked]>
Date: Fri, 11 Feb 2011 19:49:55 +0000

Hi Christer,

Christer Holmberg wrote:
>
>I do understand the concern you describe, with downloading a JavaScript
>that triggers the sending of malicious media. However, assume you
>download a legitimate JavaScript SIP stack. The JavaScript itself is not
>going to trigger any media to be sent. The media will be sent once
>you've used SIP to establish a session, and then the media is sent based
>on the negotiated SDP. Of course, the remote SDP could also be
>malicious, but that is valid for any SIP implementation - browser based
>or not :)
>

But from the browser point of view it *is* the JavaScript that is asking the media to be sent. And since the browser does not understand anything about SIP, it won't be able to distinguish a legitimate JavaSript from malicious one unless it is given some extra information. Having a successful STUN connectivity check done with the peer tells at least that the peer is willing to receive the media (and is not e.g. a printer or mail server on the browser's local network). Receiving RTP media from the peer first also tells something. Of course, one model would be for the user to give the permission, which also tells something.

Markus

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 11 Feb 2011 22:59:22 +0100

Hi Markus,

>But from the browser point of view it *is* the JavaScript that is asking the media to be sent. And since the browser does not understand anything about SIP, it won't be able to distinguish a legitimate JavaSript from malicious one unless it is given some extra information.

I agree. Technically the browser does not know what triggers the media (the JavaScript code itself or SDP that the code processes). My comment was more from a use-case perspective

>Having a successful STUN connectivity check done with the peer tells at least that the peer is willing to receive the media (and is not e.g. a printer or mail server on the browser's local network). Receiving RTP media from the peer first also tells something. 

Using RTP assumes that there will be any RTP received from that direction in the first place. Also, even if you have negotiated a bi-directional RTP session, it might still take time before RTP is actually received.

>Of course, one model would be for the user to give the permission, which also tells something.

Yeah, like browsers must ask whether they are allowed to send your GPS location...

I think that would be rather clumsy, and eventhough it would allow the user to give permission WHEN to send media, in cases where the user expects media to be sent the user most likely still wouldn't know whether the destination address is legitimate or not.

In addition, it's not only about the browser sending data to perform attacks. The JavaScript could also establish incoming incomming connections from malicious servers.

Regards,

Christer

---

From: Göran Eriksson AP <[email-masked]>
Date: Sat, 12 Feb 2011 00:32:49 +0100

Hi 

> I think that would be rather clumsy, and eventhough it would 
> allow the user to give permission WHEN to send media, in 
> cases where the user expects media to be sent the user most 
> likely still wouldn't know whether the destination address is 
> legitimate or not.

This is yet an area where close coordination with w3c&whatwg is key, since we can't
invent our own permission stuff (or similar)- whatever is done need to fit with
other models for accessing browser platform stuff.

The <device> element for instance is an experimental element in whatwg to grant access to device resources
such as camera and mic, which are needed to set up the outgoing media chain. The <device>
need some good permission handling, which the browser manufacturers surely is looking at, and we would
need to follow that.

> 
> In addition, it's not only about the browser sending data to 
> perform attacks. The JavaScript could also establish incoming 
> incomming connections from malicious servers.

---

From: Christer Holmberg <[email-masked]>
Date: Sat, 12 Feb 2011 17:04:30 +0100

Hi,

>other models for accessing browser platform stuff.

I agree. So, it is good to note is as something that needs to be looked into.

And, we can of course still have a general discussion about whether STUN should be in the browser :)

Regards,

Christer

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 14 Feb 2011 18:31:07 +0100

> I think that would be rather clumsy, and eventhough it would allow the user to give permission WHEN to send media, in cases where the user expects media to be sent the user most likely still wouldn…
I also suspect that "are you willing to send media to 123.45.67.89 port 
1234" is going to be even less comprehensive to users than "Do you 
accept the legitimacy of the certificate for ACME Inc. issued by 
Bozos-r-us?" so well known from certificate handling.

All experience shows that asking the user when he normally hits "yes" 
irritates the user, and makes him very much used to hitting "yes".

                 Harald

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 14 Feb 2011 21:17:50 +0100

Hi,

>>I think that would be rather clumsy, and eventhough it would allow the user to give permission WHEN to >>send media, in cases where the user expects media to be sent the user most likely still wouldn't know >>whether the destination address is legitimate or not.

Yes. 

In some cases you can even configure the browser to automatically hit "yes" for you. I've seen that when it comes to sending GPS location, for example.

Regards,

Christer

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Wed, 23 Feb 2011 00:03:28 -0800

>> * no SIP or Jingle; leave that to proprietary over websockets/HTTP

One of the issues that will have to be solved, even without SIP or 
Jingle in the browser, is codec negotiation. If the webserver is using 
SIP on the back-end, then it needs to know about the clients' codec 
support in enough detail to enable the SIP negotiation process. Your 
simple example sends no details at all about the clients' supported 
codecs, which I guess means the server can only assume support for the 
MTI codecs.

Maybe this is a question for the W3C, and not IETF, but I don't think 
the existing canPlayType API is sufficient. It doesn't give a guarantee 
more promising than "probably", and doesn't allow for negotiating 
specific codec parameters (e.g., bitrate, sampling rate). For example, 
even with an MTI codec like Opus, some devices with limited memory may 
not support stereo. Presumably these would not be devices capable of 
running a web browser, but they might be the one talking to the web 
browser, and the browser needs to know not to send them stereo frames.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Wed, 23 Feb 2011 10:24:59 +0100

I agree fully, the API(s) must provide enough info so that a proper negotiation can take place. This is one feedback I intend to give to the W3C charter. 

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Timothy 
> B. Terriberry
> Sent: den 23 februari 2011 09:03
> Subject: Re: [RTW] [dispatch] RTCWEB I-D with thoughts on the 
> framework
> 
> One of the issues that will have to be solved, even without 
> SIP or Jingle in the browser, is codec negotiation. If the 
> webserver is using SIP on the back-end, then it needs to know 
> about the clients' codec support in enough detail to enable 
> the SIP negotiation process. Your simple example sends no 
> details at all about the clients' supported codecs, which I 
> guess means the server can only assume support for the MTI codecs.
> 
> Maybe this is a question for the W3C, and not IETF, but I 
> don't think the existing canPlayType API is sufficient. It 
> doesn't give a guarantee more promising than "probably", and 
> doesn't allow for negotiating specific codec parameters 
> (e.g., bitrate, sampling rate). For example, even with an MTI 
> codec like Opus, some devices with limited memory may not 
> support stereo. Presumably these would not be devices capable 
> of running a web browser, but they might be the one talking 
> to the web browser, and the browser needs to know not to send 
> them stereo frames.
> RTC-Web mailing list
> [email-masked]
> http://www.alvestrand.no/mailman/listinfo/rtc-web
>