Subject: Re: [rtcweb] Media neg-eo-tiation and signalling archa-tecture

From: Bernard Aboba <[email-masked]>
Date: Tue, 5 Jul 2011 23:06:44 -0700

Stefan said:

A couple of quick comments:

* The last slide now says that WhatWG proposal is low path; this is not true. The WhatWG proposal as I read it is mute on how the SDPs are exchanged, it just hands those SDPs over to the 
application (though high path is indicated: "Communications are coordinated via a signaling channel provided by script in the page via the server, e.g. using XMLHttpRequest.")

* Notably the WhatWG PeerConnection API does allow both high path and low path. You can exchange all the SDPs over the server as the spec indicates, but if your application 
initially only sets up a PeerConnection with no media Streams, there will be a peer-to-peer datagram channel available. It can be used by the web app to exchange the SDPs 
for media negotiation when you add Streams (then we have a separate thread on that datagram channel - is it reliable or not - or would the app have to add reliability)

* Even if you use the low path approach you still need a high path available all the time. It is needed to exchange candidates so that ICE can be used to open the low path - 
and this would have to be done at the start of the session and every time you do a NIC change (a use case you added BTW!)

Stefan

[BA] I agree that the WhatWG proposal is not inherently low, medium or high path.   The signalling mechanism is totally up to the signalling callback function.

I did have a question about how the PeerConnection API manages the gathering of the ICE candidates, though.  It was my (naive?) assumption that beyond the host candidates (always included?),
the other gathered candidates would based based on the Type information passed (e.g. STUN/STUNS, TURN/TURNS).   If the SDP Blob is not considered "opaque" the signalling 
callback could manipulate/change it, to affect the  subsequent ICE processing.  The peer could do the same thing on its end.  If this were true (is it?) then that would allow quite a bit
of flexibility. 

I was also a bit uncertain about the precise timing of when ICE connectivity checks are kicked off within the API, and how things like a NIC change are handled.   For example, is it
envisaged that it would be possible to implement RTCWEB mobility (e.g. handoff of media between interfaces)?

---

From: Stefan Håkansson LK <[email-masked]>
Date: Wed, 6 Jul 2011 08:21:07 +0200

> callback could manipulate/change it, to affect the  subsequent ICE processing.  The peer could do the same thing on its end.  If this were true (is it?) then that would allow quite a bit
[SH] I have the same understanding.
> envisaged that it would be possible to implement RTCWEB mobility (e.g. handoff of media between interfaces)?
[SH] I think that is the assumption. Quote from the API spec "...any 
time the network topology changes and the ICE Agent performs an ICE 
Restart".

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 6 Jul 2011 21:43:25 -0600

> application (though high path is indicated: "Communications are coordinated via a signaling channel provided by script in the page via the server, e.g. using XMLHttpRequest.")

Hmm - my understanding was only the ICE related information was communicated that way and the SDP related information when across the channel set up by ICE but I could be totally wrong. It's pretty vague in the spec - I got that idea by talking to some of the people involved. Anyways, glad to retract that WhatWG is doing low path and just change it to not clear.

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 6 Jul 2011 21:46:17 -0700

My understanding of the API is that the signalling callback is handed the SDP blob which includes ICE candidates gathered as directed, to manipulate and send as it pleases.  Once an answer is received it is passed down.  The spec is indeed vague on some points, but hey, that's what sample code is for : (

> Subject: Re: [rtcweb] Media neg-eo-tiation and signalling archa-tecture
> From: [email-masked]
> Date: Wed, 6 Jul 2011 21:43:25 -0600
> CC: [email-masked]
> To: [email-masked]
> 
>

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 7 Jul 2011 11:52:32 -0600

I went and looked at some call flows to try and sort out what is going on in the current released chrome code and as far as I can tell, Bernard you are right, it is passing the data over the HTTP channel. An example of what I see is below 

As far as I can tell this is the JSON encoding of XEP-180 used in jingle and a separate JSON encoding of some of SDP. Not sure this matters much one way or the other but wanted to pass on what I had found and I will correct the slides at some point - my apologies for getting them wrong. 

Cullen

{
   "media" : [
      {
         "attributes" : {
            "candidate" : [
               {
                  "component" : 1,
                  "foundation" : 1,
                  "generation" : 0,
                  "ip" : "144.254.150.241",
                  "name" : "rtp",
                  "network_name" : "Intel(R) WiFi Link 1000 BGN",
                  "password" : "OpawIyXLYcJkwxFa",
                  "port" : "61880",
                  "priority" : 1.0,
                  "proto" : "udp",
                  "type" : "local",
                  "username" : "90B0CPvC8p8XPMFt"
               }
            ]
         },
         "label" : 1,
         "rtpmap" : [
            {
               "103" : {
                  "codec" : "audio/ISAC"
               }
            },
            {
               "104" : {
                  "codec" : "audio/ISAC"
               }
            },
            {
               "9" : {
                  "codec" : "audio/G722"
               }
            },
            {
               "102" : {
                  "codec" : "audio/iLBC"
               }
            },
            {
               "0" : {
                  "codec" : "audio/PCMU"
               }
            },
            {
               "8" : {
                  "codec" : "audio/PCMA"
               }
            },
            {
               "99" : {
                  "codec" : "audio/CN"
               }
            },
            {
               "98" : {
                  "codec" : "audio/CN"
               }
            },
            {
               "13" : {
                  "codec" : "audio/CN"
               }
            },
            {
               "127" : {
                  "codec" : "audio/red"
               }
            },
            {
               "106" : {
                  "codec" : "audio/telephone-event"
               }
            }
         ]
      },
      {
         "attributes" : {
            "candidate" : [
               {
                  "component" : 1,
                  "foundation" : 1,
                  "generation" : 0,
                  "ip" : "144.254.150.241",
                  "name" : "video_rtp",
                  "network_name" : "Intel(R) WiFi Link 1000 BGN",
                  "password" : "G4dq4e3f/G7FDJ5h",
                  "port" : "61879",
                  "priority" : 1.0,
                  "proto" : "udp",
                  "type" : "local",
                  "username" : "CBkZYp6/DTTMEFeM"
               }
            ]
         },
         "label" : 2,
         "rtpmap" : [
            {
               "120" : {
                  "codec" : "video/VP8"
               }
            }
         ]
      }
   ]
}

On Jul 6, 2011, at 9:43 PM, Cullen Jennings wrote:

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Niklas Enbom <[email-masked]>
Date: Fri, 8 Jul 2011 09:58:50 +0200

Not sure in which thread I should answer... In Chrome we're trying to
implement the whatwg proposal with one intentional discrepancy, and that's
the string format pointed out by Cullen. See details in this thread:
http://www.mail-archive.com/[email-masked]/msg26163.html. There's
also non-intentional discrepancies that we know of, which we're trying to
remove.

Just be careful when looking at Chrome code to interpret the whatwg spec.
We're trying to interpret that spec in the same way as anyone else, and the
whatwg author is not involved in the Chrome implementation.

Niklas

On Thu, Jul 7, 2011 at 7:52 PM, Cullen Jennings <[email-masked]> wrote:

>
> I went and looked at some call flows to try and sort out what is going on
> in the current released chrome code and as far as I can tell, Bernard you
> are right, it is passing the data over the HTTP channel. An example of what
> I see is below
>
> As far as I can tell this is the JSON encoding of XEP-180 used in jingle
> and a separate JSON encoding of some of SDP. Not sure this matters much one
> way or the other but wanted to pass on what I had found and I will correct
> the slides at some point - my apologies for getting them wrong.
>
> Cullen
>
>
>
> {
>   "media" : [
>      {
>         "attributes" : {
>            "candidate" : [
>               {
>                  "component" : 1,
>                  "foundation" : 1,
>                  "generation" : 0,
> >> * The last slide now says that WhatWG proposal is low path; this is not
> true. The WhatWG proposal as I read it is mute on how the SDPs are
> exchanged, it just hands those SDPs over to the
> >>
> >> application (though high path is indicated: "Communications are
> coordinated via a signaling channel provided by script in the page via the
> server, e.g. using XMLHttpRequest.")
> >
> > Hmm - my understanding was only the ICE related information was
> communicated that way and the SDP related information when across the
> channel set up by ICE but I could be totally wrong. It's pretty vague in the
> spec - I got that idea by talking to some of the people involved. Anyways,
> glad to retract that WhatWG is doing low path and just change it to not
> clear.
> >
> >

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 8 Jul 2011 08:11:23 -0600

Makes sense - thanks. I'm trying to interpret the spec by looking at the code. I'll stop doing that :-)

On Jul 8, 2011, at 1:58 AM, Niklas Enbom wrote:

> Not sure in which thread I should answer... In Chrome we're trying to implement the whatwg proposal with one intentional discrepancy, and that's the string format pointed out by Cullen. See details in this thread: http://www.mail-archive.com/[email-masked]/msg26163.html. There's also non-intentional discrepancies that we know of, which we're trying to remove.
> 
> Just be careful when looking at Chrome code to interpret the whatwg spec. We're trying to interpret that spec in the same way as anyone else, and the whatwg author is not involved in the Chrome implementation.
> 
> Niklas
>

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sun, 10 Jul 2011 14:05:23 +0200

We also followed the WhatWG proposal as well as possible in our implementation (that you can download and test here: https://labs.ericsson.com/apis/web-real-time-communication/downloads). We're using SDP offer/answers as proposed in the whatwg spec; however there is a separate SDP dialogue per stream (and not per session) to avoid glaring conditions.

The work also generated quite some feedback to whatwg: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-June/031950.html

Stefan

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Cullen Jennings [[email-masked]]
Sent: Friday, July 08, 2011 4:11 PM
To: Niklas Enbom
Cc: [email-masked]
Subject: Re: [rtcweb] Media neg-eo-tiation and signalling archa-tecture

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 11 Jul 2011 15:44:12 +0200

> Hmm - my understanding was only the ICE related information was communicated that way and the SDP related information when across the channel set up by ICE but I could be totally wrong. It's pretty…
Google's current implementation of the WhatWG spec is "high path". If 
you dump the requests that the example client programs pass to the 
mediation server (which is talking fairly normal HTTP), you will see the 
JSON objects that contain the codec negotiation information.