Subject: [rtcweb] codec and connection negotiation

From: Matthew Kaufman <[email-masked]>
Date: Fri, 05 Aug 2011 11:33:30 -0700

I put this together to try to help myself, and perhaps others, 
understand the various ways in which codec and connection negotiation 
might be decomposed and then put together for RTCWEB applications. It is 
a bit of a stream of consciousness, but hopefully we can get some good 
discussion provoked.

A. How to encode codec capabilities or choices

A1 - Don't provide a canonical way of encoding codec settings. The 
choice of which codec and what parameters is encoded in an ad-hoc way 
that requires no standardization.

A2 - Encode codec settings using SDP. The choice of which codec and what 
parameters is encoded using the syntax and semantics of SDP and reuses 
the SDP standardization work.

A3 - Encode codec settings using a JSON encoding of SDP. The choice of 
which codec and what parameters is using the semantics of SDP but a JSON 
syntax. Reuses the SDP standardization plus new standards around how to 
encode SDP in JSON.

B. How to expose codec capabilities and controls

B1. Expose codec capabilities and settings via individual Javascript 
APIs. For example one might say "camera.encodeMode = "H.264"; 
camera.encodeWidth = 640;" or "if (camera.canEncode("H.264")) ..."

B2. Expose codec capabilities and settings via a combined Javascript API 
that concatenates all the capabilities and settings into a single 
object. Calling "camera.encodeCapabilities()" would return a large 
object that is a full enumeration of what it can do.

B3. Expose codec capabilities and settings via a combined Javascript API 
that produces and consumes SDP strings.

B4. Don't expose codec capbilities and settings via Javascript, rather 
handle this outside of Javascript.

C. How to agree on which codec and settings to use

C1. The server receives information about the capabilities, decides what 
settings should be used, and communicates them to the endpoints

C2. The endpoints agree using offer-answer, using the server as a 
communication channel

C3. The endpoints agree using offer-answer, directly between the two 
endpoints

(There's obviously a few additional nuances, like whether to use 
something like RFC 5939 SDP capability negotiation, that aren't 
well-captured above.)

Now, with the above, we find that some of the models that have been 
discussed are combinations of the above choices.

X1 - Use A2, B3, C2 - The endpoints generate SDP for offer-answer by 
firing an SDP event, having it sent via a server to the far end, where 
it is injected as SDP into the Javascript.

X2 - Use A1, B1, C1 - The endpoints run Javascript that inspects their 
capabilities, sends that to the server which decides what is best, and 
sends back information that the Javascript uses to set up the encoders.

X3 - Use A2, B4, C3 - The endpoints establish a communication channel 
using ICE and an out-of-band channel, they then use SDP inside of SIP to 
directly negotiate using offer-answer, the server has no information 
about what was chosen and does not participate.

We also find that almost all the models can map, though with varying 
degrees of pain, to others.

For instance:

Y1 - Use A2, B1, C2 - The endpoints run Javascript that inspects their 
capabilities and encodes it into an SDP string. This is sent via the 
server to the far end, where the SDP is parsed in Javascript and the 
individual APIs called to implement SDP offer-answer.

Y2 - Use A1, B3, C1 - The endpoints generate SDP for offer-answer by 
firing an SDP event. The endpoint then runs Javascript that extracts 
from the SDP all the individual parameters and re-encodes them using an 
ad-hoc scheme. The server determines what each end should do and sends 
ad-hoc messages to the endpoints which then generate the SDP answer that 
causes the desired outcome.

It should be noted that extracting a full set of capabilities when only 
offer-answer is available via the API is particularly painful, as it 
might be necessary to explore a very wide space of fake offers to get 
answers that clarify things like "can do G.722 audio but only if not 
doing H.264 video".

We might be able to learn from some previous experiences:

1. Web-based email. Web browsers run Javascript that uses ad-hoc 
signaling to/from the web site in order to send and receive email. The 
browser does not have SMTP, POP, or IMAP implementations, or even know 
how to parse RFC822 headers on its own. This argues for A1, B1 (or maybe 
B2), C1.

2. Web-based image display. Web browsers send an "Accept" header via 
HTTP indicating which MIME types they can handle. Knowing whether a 
browser can do JPEG is as simple as looking for "image/jpeg" in the 
header. This is probably at the wrong layer for sophisticated web 
applications (as this isn't about what comes back over HTTP, but what 
can be supported over a separate channel), but might be appropriate if 
we could agree on a small number of supported codecs and simply have the 
browser tell the server in a similar manner whether or not it can do RTCWEB.

3. SIP. SIP endpoints use SDP directly between the endpoints (though 
possibly with intermediaries that rarely change the SDP) to do 
offer-answer. This is most like A2, B3 (or B4), C3 (or C2).

4. MGCP. MGCP uses SDP for both capabilities and settings, but the 
server is in control. This is most like A2, B3, C1.

5. H.323 (H.245) uses a protocol other than SDP for both capabilities 
and settings, but the server is in control. This is like A3 (or some 
other encoding), B2, C1

As for connection negotiation, most of the separation previously also 
applies... again whether to put the ICE candidates into SDP or not, 
whether to have APIs that take blobs or individual calls, etc.

But the other issue that arises when we start talking about connection 
negotiation is that if you choose SDP offer-answer for codec 
negotiation, and SDP for ICE candidates, there is the temptation to 
combine the SDP for the two, thus conflating the process of establishing 
a logical channel over which various types of media might flow, and 
establishing just which media should be sent over that channel.

Matthew Kaufman

---

From: Matthew Kaufman <[email-masked]>
Date: Fri, 05 Aug 2011 11:53:32 -0700

Now, to follow up on my previous message with a bit of opinion...

I think we should not use SDP as the API, and we should not have the 
browser implementing SDP offer-answer.

1. If SDP offer-answer is all that is available to the Javascript 
programmer (and server programmer), it becomes very difficult to know 
what the full capability set is without complex probing. This 
significantly complicates the building of future innovative applications 
on top of the browser as platform. Many of the current applications that 
show off browser capabilities do so by using capabilities that were 
already present for other reasons, and we can expect the same innovation 
to occur here, if we provide the tools.

2. Using SDP offer-answer has the advantage of reusing all the IETF 
standards work that occurs to define SDP when a new codec comes along. 
But it also has the *disadvantage* of having to wait for the IETF to 
produce these standards, which may be incomplete, or unable to control 
parameters which are necessary for web applications.

3. Anything that is missing in SDP (example: forcing the Opus codec to 
"music" mode for encoding) will still need to be exposed as a Javascript 
API on the encoder. Thus we end up with a mix of possibly-conflicting 
settings that are adjusted via the explicit API and the opaque SDP API.

4. Other work in browsers (like recording camera and microphone to disk) 
will require the ability to directly control the encoders. If these APIs 
exist then there will definitely be conflicting settings. What happens 
if you feed in an SDP answer that requires VP8 encoding and then set the 
encoder to H.264 mode via the Javascript?

Matthew Kaufman

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Sat, 6 Aug 2011 15:13:10 +0530

Matthew,

I'm seeing two aspects here

1) API: Level of control has to be given to the Javascript programmer.
Here, the discussion has to focus on the abstraction by which
application shall be developed easily without much understanding the
protocol intricacies. Even API shall be provided to the extent that
username & destination host ([email-masked]) as a input shall trigger the
session creation with default codec.

2) Protocol: The protocol selection has to be based on better interop
with other browsers and other VoIP entity also. I'm not favoring for
creating different island of realtime protocol which has to interop with
"n" of gateways.

Please read inline for more comments.

Thanks
Partha

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Matthew Kaufman
Sent: Saturday, August 06, 2011 12:24 AM
To: [email-masked]
Subject: Re: [rtcweb] codec and connection negotiation

I think we should not use SDP as the API,

<partha> It is fine to discuss which level of control has to be
provided. </partha>

 and we should not have the browser implementing SDP offer-answer.
<partha> I could not see the strong reason for not doing so. </partha>

2. Using SDP offer-answer has the advantage of reusing all the IETF
standards work that occurs to define SDP when a new codec comes along. 
But it also has the *disadvantage* of having to wait for the IETF to
produce these standards, which may be incomplete, or unable to control
parameters which are necessary for web applications.
<partha> I prefer standard over proprietary mechanism because standard
may delay but proprietary is normal uncontrolled and impossible to
interop with other webservers </partha>

3. Anything that is missing in SDP (example: forcing the Opus codec to
"music" mode for encoding) will still need to be exposed as a Javascript
API on the encoder. Thus we end up with a mix of possibly-conflicting
settings that are adjusted via the explicit API and the opaque SDP API.
<partha> It is possible to integrate unknown SDP attribute & known SDP
attribute using the proper API design. It is the basic requirement in
any SDP stack API today. </partha>

4. Other work in browsers (like recording camera and microphone to disk)
will require the ability to directly control the encoders. If these APIs
exist then there will definitely be conflicting settings. What happens
if you feed in an SDP answer that requires VP8 encoding and then set the
encoder to H.264 mode via the Javascript?
<partha> I'm reading your query as how to priortize the value in case
extra environment change the value. It has to be worked out </partha>

Matthew Kaufman
_______________________________________________
rtcweb mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Sohel Khan <[email-masked]>
Date: Sat, 6 Aug 2011 09:58:22 -0700 (PDT)

Thanks Matt and Partha for valuable comments and proposals.
The main point here is to create standards specifying methods to exchange desired session capabilities. From various discussions, it appears to me that there are two key options:
1. No SDP in browser. Object oriented and popular script language (e.g., JavaScript) implementation transfers desired session capabilities in between browser and server. The server performs SDP offer/answer in the VoIP core network.
2. SDP in browser. Use IETF SDP offer/answer standards to transfer desired session capabilities from browser to browser with/without core VoIP network.

In my opinion, we will need to create RTCWeb standards for both the options. We can either create one document containing both standards or two separate documents each containing an option. The architecture and use case documents should include both options.

Regards,
Sohel Khan

________________________________
From: Parthasarathi R (partr) <[email-masked]>
To: Matthew Kaufman <[email-masked]>; [email-masked]
Sent: Saturday, August 6, 2011 5:43 AM
Subject: Re: [rtcweb] codec and connection negotiation

Matthew,

Thanks
Partha

-----Original Message-----

Matthew Kaufman

---

From: Henry Sinnreich <[email-masked]>
Date: Sat, 06 Aug 2011 14:46:56 -0500

+1

Thanks, Henry

On 8/6/11 11:58 AM, "Sohel Khan" <[email-masked]> wrote:

> The main point here is to create standards specifying methods to exchange
> desired session capabilities. From various discussions, it appears to me that
> there are two key options:
> 1. No SDP in browser. Object oriented and popular script language (e.g.,
> JavaScript) implementation transfers desired session capabilities in between
> browser and server. The server performs SDP offer/answer in the VoIP core
> network.
> 2. SDP in browser. Use IETF SDP offer/answer standards to transfer desired
> session capabilities from browser to browser with/without core VoIP network.
> 
> In my opinion, we will need to create RTCWeb standards for both the options.
> We can either create one document containing both standards or two separate
> documents each containing an option. The architecture and use case documents
> should include both options.
> 
>  
> Regards,
> Sohel Khan
>

---

From: Sohel Khan <[email-masked]>
Date: Sat, 6 Aug 2011 14:41:43 -0700 (PDT)

Matt, Thank you for nicely presenting various sets of session capability transfer methods.
I like X1 and X2, and also Y1 and Y2.  Although I do not have any opinion on X3, it is okay to add this use case in the document. Diverse methods create innovation opportunities although create interoperability challenges. 

Although in a utopian network, it is tempting to consider that no-middle box (e.g., network transcoder) should exist, unfortunately, not all end system support the same sets of codec in the today's network. Therefore, a network transcoding system may be needed; thus, the need of servers in the network.

Apart from above method selections, in details level, we will need some important parameters. I am writing here although this thread may not be the right forum.
One important point is to ensure we have a field or parameter, or arrangement order to depict media capability preference. For example, in SDP, the first codec in the list is preferred. This is helpful for both network (e.g., transcoding mediation) and end systems, in which those support multiple codecs but would like to select the default codec. In addition, we have to consider a parameter for "force: no <parameter>" to explicitly mention unwanted media capability. For example, an end user or network may like to preclude a "bad-codec". In this case, the end system or network should be able to negotiate "force: no bad-codec".  

In DTMF codec negotiation, we will need to ensure that a parameter clearly indicates which DTMF transfer method is supported. Currently, RFC 2833  telephone-event expresses intention of out-of-band RTP DTMF digit transfer. In my opinion, we do not have a tool to deny SIP INFO-based DTMF digit transfer.  Some transcoder in the network or end user codec may prefer not to use INFO-based DTMF digit transfer. In this case, a parameter such as "force: no info-dtmf-event" will be helpful.

Regards,
Sohel Khan

________________________________
From: Matthew Kaufman <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: Friday, August 5, 2011 2:33 PM
Subject: [rtcweb] codec and connection negotiation

I put this together to try to help myself, and perhaps others, understand the various ways in which codec and connection negotiation might be decomposed and then put together for RTCWEB applications. It is a bit of a stream of consciousness, but hopefully we can get some good discussion provoked.

A1 - Don't provide a canonical way of encoding codec settings. The choice of which codec and what parameters is encoded in an ad-hoc way that requires no standardization.

A2 - Encode codec settings using SDP. The choice of which codec and what parameters is encoded using the syntax and semantics of SDP and reuses the SDP standardization work.

A3 - Encode codec settings using a JSON encoding of SDP. The choice of which codec and what parameters is using the semantics of SDP but a JSON syntax. Reuses the SDP standardization plus new standards around how to encode SDP in JSON.

B1. Expose codec capabilities and settings via individual Javascript APIs. For example one might say "camera.encodeMode = "H.264"; camera.encodeWidth = 640;" or "if (camera.canEncode("H.264")) ..."

B2. Expose codec capabilities and settings via a combined Javascript API that concatenates all the capabilities and settings into a single object. Calling "camera.encodeCapabilities()" would return a large object that is a full enumeration of what it can do.

B3. Expose codec capabilities and settings via a combined Javascript API that produces and consumes SDP strings.

B4. Don't expose codec capbilities and settings via Javascript, rather handle this outside of Javascript.

C1. The server receives information about the capabilities, decides what settings should be used, and communicates them to the endpoints

C2. The endpoints agree using offer-answer, using the server as a communication channel

C3. The endpoints agree using offer-answer, directly between the two endpoints

(There's obviously a few additional nuances, like whether to use something like RFC 5939 SDP capability negotiation, that aren't well-captured above.)

Now, with the above, we find that some of the models that have been discussed are combinations of the above choices.

X1 - Use A2, B3, C2 - The endpoints generate SDP for offer-answer by firing an SDP event, having it sent via a server to the far end, where it is injected as SDP into the Javascript.

X2 - Use A1, B1, C1 - The endpoints run Javascript that inspects their capabilities, sends that to the server which decides what is best, and sends back information that the Javascript uses to set up the encoders.

X3 - Use A2, B4, C3 - The endpoints establish a communication channel using ICE and an out-of-band channel, they then use SDP inside of SIP to directly negotiate using offer-answer, the server has no information about what was chosen and does not participate.

We also find that almost all the models can map, though with varying degrees of pain, to others.

For instance:

Y1 - Use A2, B1, C2 - The endpoints run Javascript that inspects their capabilities and encodes it into an SDP string. This is sent via the server to the far end, where the SDP is parsed in Javascript and the individual APIs called to implement SDP offer-answer.

Y2 - Use A1, B3, C1 - The endpoints generate SDP for offer-answer by firing an SDP event. The endpoint then runs Javascript that extracts from the SDP all the individual parameters and re-encodes them using an ad-hoc scheme. The server determines what each end should do and sends ad-hoc messages to the endpoints which then generate the SDP answer that causes the desired outcome.

It should be noted that extracting a full set of capabilities when only offer-answer is available via the API is particularly painful, as it might be necessary to explore a very wide space of fake offers to get answers that clarify things like "can do G.722 audio but only if not doing H.264 video".

1. Web-based email. Web browsers run Javascript that uses ad-hoc signaling to/from the web site in order to send and receive email. The browser does not have SMTP, POP, or IMAP implementations, or even know how to parse RFC822 headers on its own. This argues for A1, B1 (or maybe B2), C1.

2. Web-based image display. Web browsers send an "Accept" header via HTTP indicating which MIME types they can handle. Knowing whether a browser can do JPEG is as simple as looking for "image/jpeg" in the header. This is probably at the wrong layer for sophisticated web applications (as this isn't about what comes back over HTTP, but what can be supported over a separate channel), but might be appropriate if we could agree on a small number of supported codecs and simply have the browser tell the server in a similar manner whether or not it can do RTCWEB.

3. SIP. SIP endpoints use SDP directly between the endpoints (though possibly with intermediaries that rarely change the SDP) to do offer-answer. This is most like A2, B3 (or B4), C3 (or C2).

4. MGCP. MGCP uses SDP for both capabilities and settings, but the server is in control. This is most like A2, B3, C1.

5. H.323 (H.245) uses a protocol other than SDP for both capabilities and settings, but the server is in control. This is like A3 (or some other encoding), B2, C1

As for connection negotiation, most of the separation previously also applies... again whether to put the ICE candidates into SDP or not, whether to have APIs that take blobs or individual calls, etc.

But the other issue that arises when we start talking about connection negotiation is that if you choose SDP offer-answer for codec negotiation, and SDP for ICE candidates, there is the temptation to combine the SDP for the two, thus conflating the process of establishing a logical channel over which various types of media might flow, and establishing just which media should be sent over that channel.

Matthew Kaufman

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 08 Aug 2011 10:58:32 +0200

On 08/06/11 11:43, Parthasarathi R (partr) wrote:
> username&  destination host ([email-masked]) as a input shall trigger the
Partha,

two comments:
- I do not see a requirement that all scenarios use usernames.
- in "[email-masked]", "xyz" is not necessarily an username, and "abc.com" 
is not necessarily a hostname.
Even when identity is mediated via the Domain Name System, there may be 
no host with that name.

> provided.</partha>
> <partha>  I could not see the strong reason for not doing so.</partha>
>
> interop with other webservers</partha>
>
> <partha>  It is possible to integrate unknown SDP attribute&  known SDP
> any SDP stack API today.</partha>
>
> extra environment change the value. It has to be worked out</partha>
>
>
> Matthew Kaufman

---

From: Colin Perkins <[email-masked]>
Date: Mon, 8 Aug 2011 11:29:11 +0100

[inline]

> A3 - Encode codec settings using a JSON encoding of SDP. The choice of which codec and what parameters is using the semantics of SDP but a JSON syntax. Reuses the SDP standardization plus new stand…

The codec name and parameters are not defined in terms of SDP. Rather, SDP uses the MIME type of the codec, and the registered parameters of that MIME type. I don't much care whether the syntax used to convey codec MIME types and parameters is SDP, JSON, XML, or whatever, but I strongly suggest reusing that existing database of codec MIME types (registered following RFC 3555), rather than trying to re-invent it. 

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: "Parthasarathi R (partr)" <[email-masked]>
Date: Mon, 8 Aug 2011 17:17:40 +0530

Harald,

<snip>
> Even when identity is mediated via the Domain Name System, 
> there may be no host with that name.
</snip>

I provided username just as an example and the identity discussion is
not related to this mail thread. Intention of my example in the mail
thread is to highlight high level API possible from Javascript. 

I'm fine with any identity mechanism as long as multiple RTCWeb servers
are able to interop with the given identity.

Thanks
Partha

> From: Harald Alvestrand [mailto:[email-masked]] 
> Sent: Monday, August 08, 2011 2:29 PM
> To: Parthasarathi R (partr)
> Cc: Matthew Kaufman; [email-masked]
> Even when identity is mediated via the Domain Name System, 
> there may be no host with that name.
> 
> > creating different island of realtime protocol which has to interop 
> > with "n" of gateways.
> >
> > From: [email-masked] [mailto:[email-masked]] On 
> > Behalf Of Matthew Kaufman
> > significantly complicates the building of future innovative 
> > applications on top of the browser as platform. Many of the current 
> > applications that show off browser capabilities do so by using 
> > capabilities that were already present for other reasons, 
> and we can 
> > expect the same innovation to occur here, if we provide the tools.
> >
> > <partha>  I prefer standard over proprietary mechanism because 
> > standard may delay but proprietary is normal uncontrolled and 
> > impossible to interop with other webservers</partha>
> >
> > "music" mode for encoding) will still need to be exposed as a 
> > Javascript API on the encoder. Thus we end up with a mix of 
> > possibly-conflicting settings that are adjusted via the 
> explicit API and the opaque SDP API.
> > <partha>  It is possible to integrate unknown SDP attribute&  known 
> > SDP attribute using the proper API design. It is the basic 
> requirement 
> > in any SDP stack API today.</partha>
> >
> > 4. Other work in browsers (like recording camera and microphone to 
> > disk) will require the ability to directly control the encoders. If 
> > these APIs exist then there will definitely be conflicting 
> settings. 
> > What happens if you feed in an SDP answer that requires VP8 
> encoding 
> > and then set the encoder to H.264 mode via the Javascript?

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 08 Aug 2011 15:09:22 +0200

Trying to doodle on the opinions (the alternative lists were quite 
useful to clear the mind - I liked them a lot, and will return to them!)

On 08/05/11 20:53, Matthew Kaufman wrote:
> capabilities that were already present for other reasons, and we can 
This may be heretical ... but do we actually need to enumerate all 
capabilities?
It seems to me that we'll have exactly 2 groups of capabilities:
- Routine ones, which are generated automagically when saying "I want 
something working"
- Ones needed to "show off" specific new capabilities

The platforms, which are likely reusing code from outside WebRTC, may 
support many capabilities that are never required for any scenario 
(especially those that have been added over the years to support 
backwards compatibility in very baroque scenarios). Exposing those will 
just add clutter, not add quality.

The "show-offs" will be capable of probing.
> parameters which are necessary for web applications.
Hmmm.... since when have people *actually* waited for the IETF process 
to finish before starting to use newly defined capabilities?
This may argue for looking again at the IANA procedures for SDP, and 
institute something like the "provisional registries" that were 
introduced for email/web.
> possibly-conflicting settings that are adjusted via the explicit API 
> and the opaque SDP API.
Won't these be things people want in SDP too? See above.
> these APIs exist then there will definitely be conflicting settings. 
> What happens if you feed in an SDP answer that requires VP8 encoding 
Transcoding? :-)
The same problem exists today when trying to use cameras for multiple 
things concurrently.

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 08 Aug 2011 15:04:58 -0400

I also have some observations, without conclusions, on this decision:

SDP o/a has some well known limitations. It would be nice if rtcweb was 
not saddled with them. For instance, the offer can list multiple codecs, 
but the answerer is free to accept more than one. In that case any of 
those in both offer and answer can be used. But there are UAs that 
support multiple codecs, but only one at a time. O/A doesn't support 
that well.

Unfortunately, relaxing the limitations may break interoperation with 
non-rtcweb devices. For instance if you have an rtcweb client initiating 
a call to a pstn destination, the gateway will likely have the limitation.

Querying for capabilities before initiating a call could, in *some* 
cases transcend that incompatibility. But not always. In general it 
isn't easy to guarantee that the UA you reach with a query will be the 
same one you reach when you try to establish a call. If not, the 
capabilities may be wrong.

SDP has needed modernizing for ages. But SDPng went down in flames. 
Attempting to do something like that again would probably not be a good 
plan for rtcweb.

It might be easier to go with SDP and o/a in order to ease interop, and 
at the same time provide a capability query mechanism, also built on SDP 
that could be used optionally, with the understanding that it might not 
be available in all cases.

	Thanks,
	paul

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 9 Aug 2011 09:19:21 -0700

I think we have to distinguish between the use of SDP in the API versus use in signaling.  

When you say "attempting to do something like that again", I presume that you are referring to an attempt to develop a replacement for SDP that would be used on the wire between an RTCWEB implementation and a legacy implementation.   Given that legacy implementations won't support a replacement, it's hard to argue against use of SDP in that context. 

However, that doesn't necessarily argue for use of SDP (or something semantically equivalent) in the API.   The danger we face in adopting SDP within the API is we will be saddled with the limitations of SDP while not making use of all of its capabilities.  We saw hints of this already in the IETF 81 discussion -- where questions arose as to whether the JSON SDP blob was opaque or editable, how faithful the API is to existing SDP usage (including offer/answer, ICE), etc.   As a data point, I'd note that few if any of the existing Web-based realtime APIs have chosen to utilize the SDP format, and that attempts to "profile" SDP usage to avoid corner cases have encountered issues, particularly when dealing with PSTN interop.  

> Date: Mon, 8 Aug 2011 15:04:58 -0400
> From: [email-masked]
> To: [email-masked]

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 09 Aug 2011 12:41:47 -0400

On 8/9/11 12:19 PM, Bernard Aboba wrote:
> When you say "attempting to do something like that again", I presume
> that you are referring to an attempt to develop a replacement for SDP
> that would be used on the wire between an RTCWEB implementation and a
> legacy implementation. Given that legacy implementations won't support a
> replacement, it's hard to argue against use of SDP in that context.

Yes, I was talking about attempting to revive SDPng or something comparable.

> However, that doesn't necessarily argue for use of SDP (or something
> semantically equivalent) in the API. The danger we face in adopting SDP
> within the API is we will be saddled with the limitations of SDP while
> not making use of all of its capabilities. We saw hints of this already
> in the IETF 81 discussion -- where questions arose as to whether the
> JSON SDP blob was opaque or editable, how faithful the API is to
> existing SDP usage (including offer/answer, ICE), etc. As a data point,
> I'd note that few if any of the existing Web-based realtime APIs have
> chosen to utilize the SDP format, and that attempts to "profile" SDP
> usage to avoid corner cases have encountered issues, particularly when
> dealing with PSTN interop.

Certainly it would be possible to provide an API that had something 
semantically equivalent to SDP while using a different syntax. But thata 
comes at the price of having to then track future extensions to SDP.

An alternative is to profile/subset SDP and then perhaps develop a new 
representation semantically equivalent to that. The trouble with that is 
interoperation with things that don't fit the subset. Going that way 
will probably require making it pretty easy to extend the subset when 
things that had been excluded are found to be needed.

(It gets worse if you decide to "add a few things" to the API that 
aren't part of standard SDP.)

	Thanks,
	Paul

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 9 Aug 2011 10:18:21 -0700

> aren't part of standard SDP.)

[BA]  Given the discussion of IETF 81, my concern is that the API is based on  an SDP and ICE profile that is implicit and not explicit.    In a number of cases, it would appear that this implicit profile may ignore normative requirements (including MUSTs) in RFC 3264, 5245 and other relevant RFCs.  
The danger in continuing down that road is  that different implementations will make different assumptions about the profile, and the result will be implementations that are not only incompatible with each other, but also with existing implementations of the wire protocols.

---

From: Henry Sinnreich <[email-masked]>
Date: Tue, 09 Aug 2011 19:59:38 -0500

Leaving all the problems behind mentioned here for SDP (in full agreement),
the bigger picture is that apps may have so many other types of data to
exchange, besides codecs.
For example numbers, sizes and types of screens, keyboard/touch devices to
optimize user experience, data for mashup parts of the screen, say surround
sound(?), etc.
Folks not wedded to SDP use metadata to exchange app info. Why should RTC
continue to inhabit a world of its own, even in the browser?

Naïve question:
Does this separate use of SDP to convey RTC metadata not contradict the
target universality of apps in the browser?

Thanks, Henry

On 8/9/11 12:18 PM, "Bernard Aboba" <[email-masked]> wrote:

> [BA]  Given the discussion of IETF 81, my concern is that the API is based on
> an SDP and ICE profile that is implicit and not explicit.    In a number of
> cases, it would appear that this implicit profile may ignore normative
> requirements (including MUSTs) in RFC 3264, 5245 and other relevant RFCs.
> 
> The danger in continuing down that road is  that different implementations
> will make different assumptions about the profile, and the result will be
> implementations that are not only incompatible with each other, but also with
> existing implementations of the wire protocols.
>        
> 
>

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 10 Aug 2011 08:23:43 +0200

On 08/10/2011 02:59 AM, Henry Sinnreich wrote:
> Leaving all the problems behind mentioned here for SDP (in full 
> agreement), the bigger picture is that apps may have so many other 
> types of data to exchange, besides codecs.
> For example numbers, sizes and types of screens, keyboard/touch 
> devices to optimize user experience, data for mashup parts of the 
> screen, say surround sound(?), etc.
> Folks not wedded to SDP use metadata to exchange app info. Why should 
> RTC continue to inhabit a world of its own, even in the browser?

> Naïve question:
> Does this separate use of SDP to convey RTC metadata not contradict 
> the target universality of apps in the browser?

Sorry, I can't parse that sentence.

On one possible parsing, my response would be "no more than the use of 
UTF-8 to convey human names constrain the naming conventions of humanity".
(Prince, in his "symbol" phase, might not like it, though. No way to 
please them all.)

Seriously, if codec/stream-related metadata needs to be conveyed in one 
standard format, it seems that we're either using SDP or reinventing it. 
And so far, the reinventors seem to be unable to gather a consensus to 
go forward.

Other metadata doesn't seem (yet) to require one standard format.

>     [BA]  Given the discussion of IETF 81, my concern is that the API
>     is based on  an SDP and ICE profile that is implicit and not
>     explicit.    In a number of cases, it would appear that this
>     implicit profile may ignore normative requirements (including
>     MUSTs) in RFC 3264, 5245 and other relevant RFCs.
>
>     The danger in continuing down that road is  that different
>     implementations will make different assumptions about the profile,
>     and the result will be implementations that are not only
>     incompatible with each other, but also with existing
>     implementations of the wire protocols.
>
>
>     ------------------------------------------------------------------------

---

From: <[email-masked]>
Date: Wed, 10 Aug 2011 07:27:48 +0000

Hi,

Yes, it is important to distinguish between the API and the protocol in this discussion. Standardizing the Javascript API is necessary, but standardizing the "on the wire" protocol is not. At least one of the requirements should be that we leave enough flexibility about the protocol. A service provider should be able to implement/use any protocol that does the job required and works with the standard API offered by the browsers. SDP offer/answer could be one of those protocols.

The API itself is the critical piece. I'm not sure if there is any benefit in tying it to SDP, but certainly it should be possible to implement SDP offer/answer protocol in Javascript on top of it. That could be one requirement for it, and something that could be even proved by a real implementation.

So, in summary: Avoid strict dependency to SDP, but it must be possible to implement SDP offer/answer protocol on top of the standardized API.

Markus

From: [email-masked] [mailto:[email-masked]] On Behalf Of ext Bernard Aboba
Sent: 09 August, 2011 19:19
To: [email-masked]; [email-masked]
Subject: Re: [rtcweb] codec and connection negotiation

> From: [email-masked]<mailto:[email-masked]>
> To: [email-masked]<mailto:[email-masked]>
> >> [email-masked]<mailto:[email-masked]>
> > [email-masked]<mailto:[email-masked]>
> [email-masked]<mailto:[email-masked]>

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 10 Aug 2011 08:33:48 -0400

On 8/10/11 3:27 AM, [email-masked] wrote:
> Hi,
>
> Yes, it is important to distinguish between the API and the protocol in
> this discussion. Standardizing the Javascript API is necessary, but
> standardizing the “on the wire” protocol is not. At least one of the
> requirements should be that we leave enough flexibility about the
> protocol.  A service provider should be able to implement/use any
> protocol that does the job required and works with the standard API
> offered by the browsers.  SDP offer/answer could be one of those protocols.

SDP is *not* a protocol - it is a data format.
Offer/answer is, more or less, a protocol. It can, in principle, be used 
with other formats.

However, as things have evolved, there is a strong coupling between the 
SDP data format and the o/a protocol. For instance, IIRC, there are 
various extensions to SDP that come with their own rules for how o/a is 
applied to those pieces.

> The API itself is the critical piece. I’m not sure if there is any
> benefit in tying it to SDP,

Well, at the end of the day, an API must convey the critical data. It 
doesn't necessarily need to encode the data the same way it is encoded 
in SDP, and it doesn't need to convey that data for o/a in the same way 
that SIP does, but it does need to be able to represent the full range 
of SDP and the full semantics of o/a. And it needs to be flexible enough 
to do that as SDP evolves.

> but certainly it should be possible to
> implement SDP offer/answer protocol in Javascript on top of it. That
> could be one requirement for it, and something that could be even proved
> by a real implementation.

Certainly a real implementation is a useful existence proof. But it is 
not a proof of completeness. Completeness will be accomplished by 
defining a full mapping between the API and SDP + o/a.

*One* way to accomplish a complete mapping between the API and the SDP 
representation is to pass complete SDP in the API. Other ways are 
possible, but will be more work to define.

	Thanks,
	Paul

> So, in summary: Avoid strict dependency to SDP, but it must be possible
> to implement SDP offer/answer protocol on top of the standardized API.
>
> Markus
>
> *From:[email-masked] [mailto:[email-masked]] *On
> Behalf Of *ext Bernard Aboba
> *Sent:* 09 August, 2011 19:19
> *To:* [email-masked]; [email-masked]
> *Subject:* Re: [rtcweb] codec and connection negotiation
>
>>  From: [email-masked] <mailto:[email-masked]>
>>  To: [email-masked] <mailto:[email-masked]>
>>  >> [email-masked] <mailto:[email-masked]>
>>  > [email-masked] <mailto:[email-masked]>
>>  [email-masked] <mailto:[email-masked]>

---

From: Randell Jesup <[email-masked]>
Date: Wed, 10 Aug 2011 10:12:33 -0400

> to do that as SDP evolves.

So: this theoretical format we're considering must:

a) be a superset of the functionality of SDP + o/a
b) be able to provide equivalent ability to future extensions of SDP + o/a
c) be reasonably straightforwardly convertible to/from SDP+o/a in order to
    make gatewaying feasible (this could involve only allowing a subset to
    be easily converted, at the cost of poorer gateway abilities and experience)
d) be completely vetted in time for the timeline for this spec

I can't see how anything other than either SDP+o/a or
alternate-serialization-of-SDP + o/a could meet those goals.  An alternate
serialization of SDP *might* allow for extended capabilities outside of the
ability for SDP to express (an escape valve, if you will).  I expect a
alternate serialization could be speced out in the timeframe.

Even if we decide to relax the "superset" and allow a subset of current SDP
functionality, defining that and making sure it handles the cases we want
would be tough even in a longer timeframe, unless we devise something very
simple but extensible, and live with reduced functionality in gateway
situations (and more complex gateways probably to cover) until such extensions
are done.

If someone has a real non-SDP proposal, get it on the the floor *NOW*.
Otherwise you're just waving your hands.

> possible, but will be more work to define.

Exactly.

-- 
Randell Jesup
[email-masked]