Subject: [rtcweb] What parts of SDP need to changeable by the JS application

From: Cullen Jennings <[email-masked]>
Date: Mon, 13 Feb 2012 08:19:11 -0700

I'd like to figure out what parts of the SDP that the JS App needs to be able to change. 

The one clear example I have heard is removing a codec that the App does not want to use.

Are there others?

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 13 Feb 2012 07:20:46 -0800

> The one clear example I have heard is removing a codec that the App does not want to use.

Hadriel suggested adding ICE Candidates. [I'm not taking a position on this.]
-Ekr

---

From: "Jim Barnett" <[email-masked]>
Date: Mon, 13 Feb 2012 07:23:43 -0800

If ICE is under browser control, how would the JS app be able to _add_
candidates?  Where would it get them?  Though I can imagine that it
might want to remove candidates.

- Jim

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Eric Rescorla
Sent: Monday, February 13, 2012 10:21 AM
To: Cullen Jennings
Cc: [email-masked]
Subject: Re: [rtcweb] What parts of SDP need to changeable by the
JSapplication

> The one clear example I have heard is removing a codec that the App
does not want to use.

Hadriel suggested adding ICE Candidates. [I'm not taking a position on
this.] -Ekr _______________________________________________
rtcweb mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 13 Feb 2012 07:27:51 -0800

> might want to remove candidates.

IIRC Hadriel's suggestion was to enable SBCs to steer media.

-Ekr

---

From: Justin Uberti <[email-masked]>
Date: Mon, 13 Feb 2012 22:29:59 -0500

We've had cases where we've needed to change the codec payload types, codec
parameters (e.g. bitrate), and SRTP keys.

For the rehydration use case, you really need to be able to supply just
about everything though.

On Mon, Feb 13, 2012 at 10:19 AM, Cullen Jennings <[email-masked]> wrote:

> _______________________________________________
> rtcweb mailing list
> [email-masked]

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 14 Feb 2012 10:15:39 +0100

> The one clear example I have heard is removing a codec that the App does not want to use.
Off the top of my head, stuff that has been mentioned that seems 
sensible to me:

1) If candidates become available after the CreateOffer / CreateAnswer 
call, but before the SDP is sent to the peer, the JS needs to add them 
to the SDP.

2) If the application wants to not use BUNDLE for some reason, the JS 
needs to disable BUNDLE

3) Reducing offered resolution for video to the needs of the application 
(if draft-ietf-mmusic-sdp-media-capabilities is supported)

4) Reducing the b= (bandwidth) value if the application knows of a 
bandwidth constraint that the browser can't know about

Apart from that, I think it's hard to say without a concise list of the 
SDP features that the browser has to support; we haven't gotten around 
to that one yet.
(Is there an existing document that this would fit into, or a clear 
vision that we need a separate document for this one? Should we ask for 
contributions, or pick an editor?)

              Harald

---

From: "Ravindran, Parthasarathi" <[email-masked]>
Date: Tue, 14 Feb 2012 09:44:49 +0000

JS application shall also change few other SDP attributes like: 

1) "direction" attribute by which control the mediaflow. For example: changing from "sendrecv" to "sendonly" or "inactive".
2) "ptime" attribute in case of audio codecs by which packetization time shall be changed. (Here, browser codec support is required).

Just my 2 cents.

Thanks
Partha
>Of Harald Alvestrand
>Sent: Tuesday, February 14, 2012 2:46 PM
>To: Cullen Jennings
>Cc: [email-masked]
>Subject: Re: [rtcweb] What parts of SDP need to changeable by the JS
>application
>

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 14 Feb 2012 13:27:07 +0100

Hi,

A related question is: if the JS App changes some information, does it need to inform the browser about it?

OR, should the JS App only be allowed to change information which the browser does not need to be informed about?

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Cullen Jennings
Sent: 13. helmikuuta 2012 17:19
To: [email-masked]
Subject: [rtcweb] What parts of SDP need to changeable by the JS application

Are there others?

---

From: <[email-masked]>
Date: Tue, 14 Feb 2012 12:33:17 +0000

Hi,

I suppose there is no-one to control what is "allowed". So, anything is allowed as long as it "works". And I guess it works as long as the service provider thinks it does. Right?

Markus

>Of ext Christer Holmberg
>Sent: 14 February, 2012 14:27
>To: Cullen Jennings; [email-masked]
>OR, should the JS App only be allowed to change information which the
>browser does not need to be informed about?
>
>Regards,
>
>Christer
>
>
>
>
>
>
>
>Of Cullen Jennings

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 14 Feb 2012 13:55:34 +0100

Hi,

Then the question is: should the API support informing the browser what has been changed, or do we assume that everything "will be taken care of" once the answer is provided?

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: 14. helmikuuta 2012 14:33
To: Christer Holmberg; [email-masked]; [email-masked]
Subject: RE: [rtcweb] What parts of SDP need to changeable by the JS application

Hi,

Markus

>From: [email-masked] [mailto:[email-masked]] On 
>Behalf Of ext Christer Holmberg

---

From: "Jim Barnett" <[email-masked]>
Date: Tue, 14 Feb 2012 05:41:40 -0800

In the rehydration case, much of what the JS is supplying are values that the browser initially generated (but lost on page reload), so it’s not really _changing_ those values.  The underlying question is: which parts of the SDP can the JS change to something other than what the browser initially provided? 

-          Jim

From: [email-masked] [mailto:[email-masked]] On Behalf Of Justin Uberti
Sent: Monday, February 13, 2012 10:30 PM
To: Cullen Jennings
Cc: [email-masked]
Subject: Re: [rtcweb] What parts of SDP need to changeable by the JSapplication

We've had cases where we've needed to change the codec payload types, codec parameters (e.g. bitrate), and SRTP keys.

For the rehydration use case, you really need to be able to supply just about everything though.

Are there others?

---

From: "Jim Barnett" <[email-masked]>
Date: Tue, 14 Feb 2012 05:43:12 -0800

My understanding based on earlier emails is that calling
setLocalDescription informs the browser of the changes.  In other words,
the JS must call setLocalDescription after making any changes.  

Somebody please let me know if I'm wrong...

- Jim

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Christer Holmberg
Sent: Tuesday, February 14, 2012 7:27 AM
To: Cullen Jennings; [email-masked]
Subject: Re: [rtcweb] What parts of SDP need to changeable by the JS
application

Hi,

A related question is: if the JS App changes some information, does it
need to inform the browser about it?

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Cullen Jennings
Sent: 13. helmikuuta 2012 17:19
To: [email-masked]
Subject: [rtcweb] What parts of SDP need to changeable by the JS
application

I'd like to figure out what parts of the SDP that the JS App needs to be
able to change. 

The one clear example I have heard is removing a codec that the App does
not want to use.

Are there others?

---

From: Randell Jesup <[email-masked]>
Date: Tue, 14 Feb 2012 11:26:21 -0500

On 2/13/2012 10:29 PM, Justin Uberti wrote:
> We've had cases where we've needed to change the codec payload types,
> codec parameters (e.g. bitrate), and SRTP keys.

So, swapping payload values for different ones (the most common case 
I've seen is devices that insist that the payloads in each direction are 
the same numeric value).  So long as generated answers follow that, it's 
generally not a problem in my experience; if that's the case, changing 
them might not be needed.  I'll check some old notes to see if I've seen 
this need elsewhere.

SRTP keys obviously only matters in the case of SDES.

Codec Parameters: Have you had to modify any other than b=*? (which 
really isn't a codec parameter (fmtp) per-se).

> For the rehydration use case, you really need to be able to supply just

I'd assume in that case you'd largely cache the entire most recent 
accepted offer or answer.  But rehydration could be very tricky; what if 
it occurs during a renegotiation?  Also, I'm concerned about taking an 
answer and sending it as an offer, especially if there are SBCs in the 
loop (think gateway to legacy).

*Reliable* rehydration is a tougher problem than I think people are 
waving their hands at.  How long does the other side keep the call 
'live' waiting for rehydration?  Does this mean all calls have a delay 
before they end?  Just ones where the two apps are the same?  I think 
rehydration will be a rats-nest of edge cases and race conditions - 
especially when the page/app has more state than just the SDP media 
description (though that's largely the problem of the app).  And I'm 
especially doubtful about rehydration in federation and legacy gateway 
cases.

I realize that a lot of JS programmers have dealt with bugs by "reload 
the page!" (from what Justin and others said), but that seems like a 
huge cop-out to me.  And yes, I know some other reasons were mentioned 
in the Interim - maybe those are valid, but a 
purposely-decided-on-reload is different than a user-accidental-reload - 
there can be communication that it's about to happen, for example, so 
the other side knows to wait, or to prepare states.  But again, in 
federation cases this may fall down.

-- 
Randell Jesup
[email-masked]

---

From: Justin Uberti <[email-masked]>
Date: Tue, 14 Feb 2012 08:40:01 -0800

On Tue, Feb 14, 2012 at 5:41 AM, Jim Barnett <[email-masked]>wrote:

> In the rehydration case, much of what the JS is supplying are values that
> the browser initially generated (but lost on page reload), so it’s not
> really _*changing*_ those values.  The underlying question is: which
> parts of the SDP can the JS change to something other than what the browser
> initially provided?
>

As a start, I think the JS should be able to:
- remove or reorder codecs (m=)
- change codec attributes, if it makes sense (a=fmtp; Partha also mentioned
ptime)
- enable/disable BUNDLE (a=group)
- enable/disable RTCP mux (a=rtcp-mux)
- reorder SRTP crypto-suites (a=crypto)
- change SRTP parameters or keys (a=crypto)
- change send resolution or framerate (TBD)
- change desired recv resolution or framerate (TBD)
 - change total bandwidth (b=)
- remove desired AVPF mechanisms (a=rtcp-fb)
- remove RTP header extensions (a=rtphdr-ext)
- add/change SSRC grouping (e.g. FID, RTX, etc) (a=ssrc-group)
- add SSRC attributes (a=ssrc)
- change ICE ufrag/password (a=ice-ufrag/pwd)
- change media send/recv state (a=sendonly/recvonly/inactive)

The JS must inform the browser of most of these changes in order for them
to work correctly. (Reordering codecs is one that could be done by JS
without the browser knowing.)

****
>
> ** **
>
> **-          **Jim****
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On
> Behalf Of *Justin Uberti
> *Sent:* Monday, February 13, 2012 10:30 PM
> *To:* Cullen Jennings
> *Cc:* [email-masked]
>
> *Subject:* Re: [rtcweb] What parts of SDP need to changeable by the
> JSapplication****
>
> ** **
>
> codec parameters (e.g. bitrate), and SRTP keys.****
>
> ** **
>
> about everything though.****
>
> On Mon, Feb 13, 2012 at 10:19 AM, Cullen Jennings <[email-masked]> wrote:**
> **
>
>
> https://www.ietf.org/mailman/listinfo/rtcweb****
>
> ** **
>

---

From: Justin Uberti <[email-masked]>
Date: Tue, 14 Feb 2012 08:49:26 -0800

On Tue, Feb 14, 2012 at 8:26 AM, Randell Jesup <[email-masked]>wrote:

> So, swapping payload values for different ones (the most common case I've
> seen is devices that insist that the payloads in each direction are the
> same numeric value).  So long as generated answers follow that, it's
> generally not a problem in my experience; if that's the case, changing them
> might not be needed.  I'll check some old notes to see if I've seen this
> need elsewhere.
>
> Codec Parameters: Have you had to modify any other than b=*? (which really
> isn't a codec parameter (fmtp) per-se).
>
>
> accepted offer or answer.  But rehydration could be very tricky; what if it
> occurs during a renegotiation?  Also, I'm concerned about taking an answer
> and sending it as an offer, especially if there are SBCs in the loop (think
> gateway to legacy).
>
> *Reliable* rehydration is a tougher problem than I think people are waving
> their hands at.  How long does the other side keep the call 'live' waiting
> for rehydration?

The remote side needs to tolerate some network burps; walking into a
location with bad wifi signal, a reset within the network; this would look
exactly like one of those.

>  Does this mean all calls have a delay before they end?

No; calls that end normally signal this via some sort of BYE message.

> Just ones where the two apps are the same?  I think rehydration will be a
> rats-nest of edge cases and race conditions - especially when the page/app
> has more state than just the SDP media description (though that's largely
> the problem of the app).  And I'm especially doubtful about rehydration in
> federation and legacy gateway cases.
>

It might not be trivial for all applications, but I don't think that's a
reason not to try. Once the APIs are there, I think it should be fairly
easy to put together a demo and see if we are overlooking anything.

>
> I realize that a lot of JS programmers have dealt with bugs by "reload the
> page!" (from what Justin and others said), but that seems like a huge
> cop-out to me.  And yes, I know some other reasons were mentioned in the
> Interim - maybe those are valid, but a purposely-decided-on-reload is
> different than a user-accidental-reload - there can be communication that
> it's about to happen, for example, so the other side knows to wait, or to
> prepare states.  But again, in federation cases this may fall down.
>
> --
> Randell Jesup
> [email-masked]
>
> ______________________________**_________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 14 Feb 2012 18:24:07 +0100

On 02/14/2012 05:40 PM, Justin Uberti wrote:
>
>
> On Tue, Feb 14, 2012 at 5:41 AM, Jim Barnett 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     In the rehydration case, much of what the JS is supplying are
>     values that the browser initially generated (but lost on page
>     reload), so it’s not really _/changing/_ those values.  The
>     underlying question is: which parts of the SDP can the JS change
>     to something other than what the browser initially provided?
>
>
> - change codec attributes, if it makes sense (a=fmtp; Partha also 
> mentioned ptime)
> The JS must inform the browser of most of these changes in order for 
> them to work correctly. (Reordering codecs is one that could be done 
> by JS without the browser knowing.)
reordering m= lines has to be messaged to the browser, since it depends 
on the order to correlate m= lines in an answer with the offer.

reordering the list of codecs in the end of the m= line should also be 
signalled to the browser, I think; it might affect which codecs it 
starts using when it sends, I think (for the case where the respondent 
supports multiple codecs).

>
>     -Jim
>
>     *From:[email-masked] <mailto:[email-masked]>
>     [mailto:[email-masked] <mailto:[email-masked]>]
>     *On Behalf Of *Justin Uberti
>     *Cc:* [email-masked] <mailto:[email-masked]>
>
>
>     *Subject:* Re: [rtcweb] What parts of SDP need to changeable by
>     the JSapplication
>
>     We've had cases where we've needed to change the codec payload
>     types, codec parameters (e.g. bitrate), and SRTP keys.
>
>     On Mon, Feb 13, 2012 at 10:19 AM, Cullen Jennings <[email-masked]
>     <mailto:[email-masked]>> wrote:
>
>
>     [email-masked] <mailto:[email-masked]>