Subject: [rtcweb] JSEP - Modifying the SDP

From: Cullen Jennings <[email-masked]>
Date: Thu, 26 Jan 2012 13:41:46 -0700

I've like to understand what parts of the SDP the JS app can change between the createOffer and the setLocal. And similarly the createAnswer and setLocal?  

So for some specific examples - say the SDP had a v4 and v6 address - could the JS remove one? can it change the SSRC? ports ? Can it remove a codec? what can it do to things like bandwidth limits? Can it change the video resolution to a smaller resolution? any limits on this? can it change the aspect ratio? can it change to square pixels?  frame rate? Can it change the profile information for a video codec?  

What parts of SDP is the createOffer allowed to use - can it use any valid SDP extension? If the browser wants to modify the SDP to use some SDP extension, what extensions does the browser need to be able to support in setLocal?

If due to some prior knowledge such as a previous session on the same browser, the application knows that the browser supports what it needs, can the JS just create it own offer and never call createOffer but just directly call setLocalSDP ? This might seem sort of far fetched but if the JS was going to talk to a conference bridge for a game that only supported G.711, it might be reasonable for the JS to just create an offer that was only 711 and if you were on a browser that did not support that, it was never going to work anyways. Anyways, is this sort of use of the API allowed ?

Related to this is the issues of what sort of errors will be reported on a setLocal. I don't really care how they are reported, exceptions, error codes, whatever - but I would like to see a list of all the different error. This seems pretty important to decide how complex the JS code needs to be to decide to recover from these error and also will impact how flexible the mechanism ends up being. What are you thinking of as the list of errors?

---

From: Justin Uberti <[email-masked]>
Date: Fri, 27 Jan 2012 14:10:43 -0500

Good questions. Inline.

On Thu, Jan 26, 2012 at 3:41 PM, Cullen Jennings <[email-masked]> wrote:

>
> I've like to understand what parts of the SDP the JS app can change
> between the createOffer and the setLocal. And similarly the createAnswer
> and setLocal?
>

Regarding what the application can change in the offer/answer it generates
and supplies as its own local description, I think most things should be
fair game, provided the offer or answer still properly adheres to the
relevant RFCs (e.g. answer must have a single a=crypto: line, etc)

>
> So for some specific examples - say the SDP had a v4 and v6 address -
> could the JS remove one?

Candidates aren't part of the session descriptions; the ICE mechanism runs
separately. Even so, the JS could eat generated or received v4/v6
candidates.

> can it change the SSRC?

yes.

> ports ?

no, since ICE is controlled separately.

> Can it remove a codec?

yes.

> what can it do to things like bandwidth limits?

If you mean "b:", it could add its own, or change the generated value.

> Can it change the video resolution to a smaller resolution? any limits on
> this? can it change the aspect ratio? can it change to square pixels?
>  frame rate? Can it change the profile information for a video codec?
>

I don't think we've defined exactly how we intend resolution negotiation to
work in WebRTC (RFC 6236?) so I'm going to defer on these questions for
now. That said, I think it should be possible to control the requested
resolution.

>
> What parts of SDP is the createOffer allowed to use - can it use any valid
> SDP extension? If the browser wants to modify the SDP to use some SDP
> extension, what extensions does the browser need to be able to support in
> setLocal?
>

There is a general question of what SDP/RTP features must be supported for
a conformant WebRTC implementation.
http://tools.ietf.org/html/draft-ietf-rtcweb-rtp-usage-01 covers some of
this.

createOffer will generate SDP that is compliant with the baseline
requirements, and may include optional features that the implementation
knows it supports. If the application wants to modify the SDP, it cannot
add optional features that are not part of the baseline (unless it has some
caps or similar mechanism to determine browser support).

>
> If due to some prior knowledge such as a previous session on the same
> browser, the application knows that the browser supports what it needs, can
> the JS just create it own offer and never call createOffer but just
> directly call setLocalSDP ?

yes.

> This might seem sort of far fetched but if the JS was going to talk to a
> conference bridge for a game that only supported G.711, it might be
> reasonable for the JS to just create an offer that was only 711 and if you
> were on a browser that did not support that, it was never going to work
> anyways. Anyways, is this sort of use of the API allowed ?
>

sure. If you can generate a compatible session description through some
other mechanism, that is fine. There are a number of scenarios where I
think this makes sense (including Matthew's example of pushing session
state back down to resurrect a call).

>
> Related to this is the issues of what sort of errors will be reported on a
> setLocal. I don't really care how they are reported, exceptions, error
> codes, whatever - but I would like to see a list of all the different
> error. This seems pretty important to decide how complex the JS code needs
> to be to decide to recover from these error and also will impact how
> flexible the mechanism ends up being. What are you thinking of as the list
> of errors?
>

Agree this is important, but sort of beyond the scope of an -00. Anyway, I
think there are a few classes of errors we could return:
- invalid-syntax (malformed SDP)
- invalid-state (offer or answer at wrong time)
- invalid-content (SDP was invalid, e.g. conflicting payload types)
- unsupported-codecs (desc contains unknown codec)
- unsupported-crypto (desc contains unknown crypto parameters)
- insufficient-resources (too many decoders, or decoders already in use)
- invalid-format (bad audio sampling rate, or unsupported video resolution)
- unsupported-option (app tried to use a SDP option the impl doesn't
support)

This is just an initial list, and while I'm sure there are more, I'm not
sure how many are recoverable. I imagine that most of these represent
programming errors, and only things like insufficient-resources or
invalid-format would actually involve retry behavior. We also need to keep
in mind that exposing very granular error info could allow this to be used
for fingerprinting.
(The same goes for setRemoteDescription, of course, and affects probably
any API we could choose.)

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 27 Jan 2012 11:58:46 -0800

On Fri, Jan 27, 2012 at 11:10 AM, Justin Uberti <[email-masked]> wrote:
> other mechanism, that is fine. There are a number of scenarios where I think
> this makes sense (including Matthew's example of pushing session state back
> down to resurrect a call).

I wonder if this has API implications. What I mean here is:
Does the API (capabilities, I guess) need to provide enough information that
one can independently write myCreateOffer() in JS? Or do I generally need to
use createOffer(), but under some limited circumstances I can get away
with taking an extant offer, modifying it, and pushing it down?

-Ekr

---

From: Justin Uberti <[email-masked]>
Date: Fri, 27 Jan 2012 15:03:36 -0500

On Fri, Jan 27, 2012 at 2:58 PM, Eric Rescorla <[email-masked]> wrote:

> think
> one can independently write myCreateOffer() in JS?

No. We might decide we want to support that in the future, but for now,
that's not a requirement.

> Or do I generally need to
More or less. In addition to rehydrating calls, you can use a previously
generated offer to rollback to a previous state (perhaps if the remote side
rejected a session modification).

Of course, if you really know what you're doing, you could generate your
own offer based on what you think you know about the browser and the
mandatory-to-implement WebRTC SDP features (e.g. Cullen's G.711 example).

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 27 Jan 2012 12:04:40 -0800

> that's not a requirement.

WFM.

> rejected a session modification).

Rehydrate... I like that.

-Ekr

---

From: Bernard Aboba <[email-masked]>
Date: Fri, 27 Jan 2012 19:59:09 -0800

I agree that these are great questions, but we need to keep them in context. 

Most of these questions arise, not just for JSEP, but also with respect to the existing API. 

In fact, I would argue that most of them will probably arise with respect to *any* API that utilizes SDP. 

I'd also argue that many of these questions arise outside of the kind of basic scenarios that we are targeting for these "high level" APIs.  

And one question that comes to mind is whether any SDP-based API can reasonably be expected to provide all the flexibility that an advanced developer (e.g. someone looking to develop an interoperable telepresence app, for example) would want. 

My answer to that is NO, because such an application would be highly likely to demand a level of SDP functionality that goes beyond what a browser might be expected to provide natively.  Think about all the things that such an application might need in terms of SDP:  security (ICE, SDES, SRTP), RTP profiles (SAVP/SAVPF), layering, bundling, capability negotiation, etc.   

Do we *really* think that all of this will be implemented natively and that (as importantly) it is likely to interoperate if it were? 

Experience tells us that the answer is no.  

If you compare what SDP functionality is needed for some of the most advanced use cases versus what is widely implemented today, it's hard to escape the conclusion that an SDP-based API will be very painful to advanced developers and that such an API is highly likely to deliver interoperability in complex scenarios, even if lots of other issues are worked out (e.g. codecs). 

Don't get me wrong -- IMHO JSEP is a *major* step forward, because it gets the SDP state machine out of the browser, and will potentially make basic functionality more resilient.  

However, JSEP (or ROAP for that matter) is not the answer to every scenario in the use case document. 

For some of those use cases a lower level API will be required, and IMHO that API will have no SDP dependency at all.   JSEP can be built on top of that.