Subject: [rtcweb] Fwd: New Version Notification for draft-guduru-rtcweb-codec-preferences-01.txt

From: Kiran Kumar Guduru <[email-masked]>
Date: Sat, 05 Jul 2014 01:26:36 +0000 (GMT)

Hi,

A new version of codec preferences draft has been published, by modifying as per the review comments received.

Please review and let me know your comments.

Thanks & Regards,

Kiran.

------- Original Message -------

Sender : [email-masked]<[email-masked]>

Date : Jul 02, 2014 18:28 (GMT+09:00)

Title : New Version Notification for draft-guduru-rtcweb-codec-preferences-01.txt

A new version of I-D, draft-guduru-rtcweb-codec-preferences-01.txt
has been successfully submitted by Kiran Kumar Guduru and posted to the
IETF repository.

Name: draft-guduru-rtcweb-codec-preferences
Revision: 01
Title: WebRTC Codec Preferences
Document date: 2014-07-02
Group: Individual Submission
Pages: 7
URL:            http://www.ietf.org/internet-drafts/draft-guduru-rtcweb-codec-preferences-01.txt
Status:         https://datatracker.ietf.org/doc/draft-guduru-rtcweb-codec-preferences/
Htmlized:       http://tools.ietf.org/html/draft-guduru-rtcweb-codec-preferences-01
Diff:           http://www.ietf.org/rfcdiff?url2=draft-guduru-rtcweb-codec-preferences-01

Abstract:
WebRTC specifies to implement a minimum set of required codecs to
ensure guaranteed interoperability between WebRTC peers.  WebRTC
allows browser implementers to support vendor specific codecs apart
from mandatory codecs.  This document specifies the way for
JavaScript applications to give preferences for media codecs in
WebRTC context out of the available codecs in browser for creating
the offer / answer.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Justin Uberti <[email-masked]>
Date: Mon, 21 Jul 2014 14:48:28 -0400

On Mon, Jul 21, 2014 at 12:14 AM, Kiran Kumar Guduru <
[email-masked]> wrote:

>  Hi Justin,
>
> I don't feel the number of API addition, is a big problem :).
>
You are proposing expanding the API surface of WebRTC 1.0 by over 10%. This
is a significant addition (the current PeerConnection object itself only
has 21 methods), and doesn't count the codec object that would have to be
added to the spec to complete this design.

> I feel this is very useful to add in 1.0. I don't know what made you feel
> the existing usecases to be contrived. AFAIK, apps has that intellegence to
> analyze on the bandwidth utilization.
>
I think the examples are contrived, because they don't show how the apps
would use this API to accomplish the desired behavior. Apps certainly don't
know the power characteristics of the various codecs (i.e. whether they are
hardware-backed or not), and only have indirect knowledge of the bandwidth
situation; they don't have all the details on congestion that the browser
has access to, and could end up incorrectly dropping to a low-bandwidth
codec, or oscillating between codecs, because of the lack of detailed
information.

> One use case I missed in the draft and I would like to add in the next
> version is, the problem arising at gateways, which can only be solved by
> apps, but never by browser as explained below.
>
>
>
> "Considering the usecase off an application is serving the needs between
> webrtc client and non-webrtc client (IMS for example), that does not
> support the mandatory codecs specified for WebRTC, then gateways will solve
> the problem by transcoding.
>
> If a browser is supporting codecs which don't need this kind of
> transcoding, but giving low priority to them, then according to existing
> specifications, the gateway has to accept the offer with priorities
> specified by webrtc (considering the case for webrtc client as offerer)
> client through answer, then again gateway has to send the offer with its
> order of priority to change to codec preferences, resulting in a second
> offer-answer negotiation."  In this kind of usecases API are known about
> the priority of codecs required and we can avoid this kind of unnecessary
> signaling if APIs are provided to apps to prioritize the codecs.
>
>
>
> I hope this example will add some more value to this idea.
>

This is a useful example, thanks for explaining this. But I think this is
the wrong solution. Quoth RFC3264, S. 7:

*   When the offerer receives the answer, it MAY send media on the*
*   accepted stream(s) (assuming it is listed as sendrecv or recvonly in*
*   the answer).  It MUST send using a media format listed in the answer,*
*   and it SHOULD use the first media format listed in the answer when it*
*   does send.*

Note the use of SHOULD vs MUST. Ergo, even if the offer has m=audio 1111
RTP/SAVPF X Y Z, it is completely legal for the callee to apply its own
priorities and send with codec Y or Z instead of X if there are compelling
reasons to do so. This makes more sense to me than doing an immediate
re-offer, or adding APIs to reorder the browser's codec list.

>
>
>
>
> ------- *Original Message* -------
>
> *Sender* : Justin Uberti<[email-masked]>
>
> *Date* : Jul 21, 2014 12:30 (GMT+09:00)
>
> *Title* : Re: New Version Notification for
> draft-guduru-rtcweb-codec-preferences-01.txt
>
>
> This seems like a pretty big hammer to solve a fairly small problem. This
> proposal adds 6 new API points for the purpose of changing the order of
> codecs in createOffer, which seems like a high cost-benefit ratio. And
> while the use cases listed here are helpful, they seem somewhat contrived
> to me, since it seems unlikely that the application can make better choices
> about bandwidth or power consumption than the browser.
>
> Without a specific, concrete example, I remain unconvinced that this is
> worth doing in the 1.0 timeframe. Post-1.0, we can probably find a way to
> provide this sort of lower-level control.
>
>
> On Sat, Jul 12, 2014 at 9:41 PM, franklin blek <[email-masked]>
> wrote:
>
>> Hi,
>> The draft seems to explain codec preferences in a good way.
>> I think this has a good potenitial to be a part of WebRTC1.0.
>>
>>
>>
>> On Sat, Jul 5, 2014 at 10:26 AM, Kiran Kumar Guduru <
>> [email-masked]> wrote:
>>
>>>  Hi,
>>>
>>> A new version of codec preferences draft has been published, by
>>> modifying as per the review comments received.
>>>
>>> ------- *Original Message* -------
>>>
>>> *Sender* : [email-masked]<[email-masked]>
>>>
>>> *Date* : Jul 02, 2014 18:28 (GMT+09:00)
>>>
>>> *Title* : New Version Notification for
>>> draft-guduru-rtcweb-codec-preferences-01.txt
>>>
>>>
>>>
>>> URL:
>>> http://www.ietf.org/internet-drafts/draft-guduru-rtcweb-codec-preferences-01.txt
>>> Status:
>>> https://datatracker.ietf.org/doc/draft-guduru-rtcweb-codec-preferences/
>>> Htmlized:
>>> http://tools.ietf.org/html/draft-guduru-rtcweb-codec-preferences-01
>>> Diff:
>>> http://www.ietf.org/rfcdiff?url2=draft-guduru-rtcweb-codec-preferences-01
>>>
>>> Abstract:

---

From: Kiran Kumar Guduru <[email-masked]>
Date: Tue, 22 Jul 2014 02:59:21 +0000 (GMT)

Hi Justin,

Please find my comments inline.

------- Original Message -------

Sender : Justin Uberti<[email-masked]>

Date : Jul 22, 2014 03:48 (GMT+09:00)

Title : Re: Re: New Version Notification for draft-guduru-rtcweb-codec-preferences-01.txt

On Mon, Jul 21, 2014 at 12:14 AM, Kiran Kumar Guduru <[email-masked]> wrote:

Hi Justin,

You are proposing expanding the API surface of WebRTC 1.0 by over 10%. This is a significant addition (the current PeerConnection object itself only has 21 methods), and doesn't count the codec object that would have to be added to the spec to complete this design.

[KIRAN] All the API's are not new here. createOffer and createAnswer are anyhow old APIs only, we are just extending the options for the sake of future streams. (First I thought of to remove modifying this place. After checking the Harald's point that giving the preferences for future streams I felt it is reasonable to add it here).

We can reduce the number of API's for gettingCodecs to 1 instead of 2. just by modifying

getSupportedCodecs(string type);

This will return the type of codecs based on the type "audio" or "video"

The other place which we added this is RTCRTPSender/Receiver. I added this API as per your suggestion that the right place to add these is RTCRTPSender/Receiver but not createOffer/Answer.

I feel the new API addition will now become 3 + constraint addition in createOffer/Answer but not 6.

I feel this is very useful to add in 1.0. I don't know what made you feel the existing usecases to be contrived. AFAIK, apps has that intellegence to analyze on the bandwidth utilization.

I think the examples are contrived, because they don't show how the apps would use this API to accomplish the desired behavior.

[KIRAN] Agreed, I think you are expecting some more explenation here. I will add it.

Apps certainly don't know the power characteristics of the various codecs (i.e. whether they are hardware-backed or not), and only have indirect knowledge of the bandwidth situation; they don't have all the details on congestion that the browser has access to, and could end up incorrectly dropping to a low-bandwidth codec, or oscillating between codecs, because of the lack of detailed information.

[KIRAN] I don't object this explenation regarding power consumption but certainly some codecs in marked are already proved about thier processing speeds and bandwidth utilizations.

Regarding knowledge of bandwidth, App can check bandwidhth availability through other APIs. Also in case of using applications using features like video on demand etc, where the APP has the control on bandwidth to increase or decrease the bandwidth, applications can get that knowledge prior to browser through signalling.

One use case I missed in the draft and I would like to add in the next version is, the problem arising at gateways, which can only be solved by apps, but never by browser as explained below.

"Considering the usecase off an application is serving the needs between webrtc client and non-webrtc client (IMS for example), that does not support the mandatory codecs specified for WebRTC, then gateways will solve the problem by transcoding.

If a browser is supporting codecs which don't need this kind of transcoding, but giving low priority to them, then according to existing specifications, the gateway has to accept the offer with priorities specified by webrtc (considering the case for webrtc client as offerer) client through answer, then again gateway has to send the offer with its order of priority to change to codec preferences, resulting in a second offer-answer negotiation."  In this kind of usecases API are known about the priority of codecs required and we can avoid this kind of unnecessary signaling if APIs are provided to apps to prioritize the codecs.

This is a useful example, thanks for explaining this. But I think this is the wrong solution. Quoth RFC3264, S. 7:

When the offerer receives the answer, it MAY send media on the

accepted stream(s) (assuming it is listed as sendrecv or recvonly in

the answer).  It MUST send using a media format listed in the answer,

and it SHOULD use the first media format listed in the answer when it

does send.

Note the use of SHOULD vs MUST.

[KIRAN] section 7 explains about "offerer processing of the Answer". But I think the correct section for my explanation is section 6, " Generating the Answer", which recommonds to use the same order of preference as that in the offer.

"Although the answerer MAY list the formats in their desired order of
preference, it is RECOMMENDED that unless there is a specific reason,
the answerer list formats in the same relative order they were
present in the offer."

Ergo, even if the offer has m=audio 1111 RTP/SAVPF X Y Z, it is completely legal for the callee to apply its own priorities and send with codec Y or Z instead of X if there are compelling reasons to do so. This makes more sense to me than doing an immediate re-offer, or adding APIs to reorder the browser's codec list.

[KIRAN] I agree changing the order is leagel but not RECOMMENDED as explained above. So generally implementors may not do this practice.

Chrome also seems to accept this https://code.google.com/p/webrtc/issues/detail?id=2868

Regards,

Kiran.

------- Original Message -------

Sender : Justin Uberti<[email-masked]>

Date : Jul 21, 2014 12:30 (GMT+09:00)

Title : Re: New Version Notification for draft-guduru-rtcweb-codec-preferences-01.txt

This seems like a pretty big hammer to solve a fairly small problem. This proposal adds 6 new API points for the purpose of changing the order of codecs in createOffer, which seems like a high cost-benefit ratio. And while the use cases listed here are helpful, they seem somewhat contrived to me, since it seems unlikely that the application can make better choices about bandwidth or power consumption than the browser.

Without a specific, concrete example, I remain unconvinced that this is worth doing in the 1.0 timeframe. Post-1.0, we can probably find a way to provide this sort of lower-level control.

On Sat, Jul 12, 2014 at 9:41 PM, franklin blek <[email-masked]> wrote:

Hi,

On Sat, Jul 5, 2014 at 10:26 AM, Kiran Kumar Guduru <[email-masked]> wrote:

Hi,

Thanks & Regards,

Kiran.

------- Original Message -------

Abstract:

The IETF Secretariat

---

From: Kiran Kumar Guduru <[email-masked]>
Date: Tue, 22 Jul 2014 03:19:35 +0000 (GMT)

Dear Raju,

Thanks for your comments.

Please see my comments inline.

------- Original Message -------

Sender : Makaraju, Maridi Raju (Raju)<[email-masked]>

Date : Jul 22, 2014 03:56 (GMT+09:00)

Title : RE: [rtcweb] New Version Notification for draft-guduru-rtcweb-codec-preferences-01.txt

>This seems like a pretty big hammer to solve a fairly small problem. This proposal adds 6 new API points for the purpose of >changing the order of codecs in createOffer, which seems like a high cost-benefit ratio. And while the use cases listed here are >helpful, they seem somewhat contrived to me, since it seems unlikely that the application can make better choices about >bandwidth >or power consumption than the browser.

[Raju] Per my understanding, the main object of this draft can be achieved with no additional APIs and with just the proposed introduction of preferredAudioCodecs and preferredVideoCodecs options to RTCOfferAnswerOptions constraint of createOffer()/createAnswer().

So, IMO I don’t think getCodecPreferences()/setCodecPreferences() add much value, so can be delayed or dropped.

The need for getSupportedAudioCodecs()/getSupportedVideoCodecs() in 1.0 can be questionable as the application can specify codecs order per known codecs (or get the list via a dummy createOffer() call).

[KIRAN] New API is added for this only because, it can be called on Peerconnection irrespective of its state and the order of parsing is also easy when compared to parsing of whole SDP returned through createOffer/Answer.

The draft talks about fulfilling A5 in [I-D.ietf-rtcweb-use-cases-and-requirements], but I do not see any explicit mention of how codecs can be removed? preferredAudio/VideoCodecs constraint only specifies the order of preference. Don’t you need another constraint to exclude specified codecs? It’s probably a bad design to have codecs not in the preferred list be removed automatically.

[KIRAN] The removal of codecs is explained in section 6.

"The offer / answer
SHOULD NOT contain audio codecs other than those specified by
JavaScript application and the order of preference SHOULD be with
high priority for the codecs first in the sequence."

Perhaps this is in indirect way. I will modify the statement to directly point this. I don't want to increase the constraints just for removal, which can be achieved with this constraint. So I followed this design.

BR

Raju

---

From: Justin Uberti <[email-masked]>
Date: Tue, 22 Jul 2014 23:42:32 -0400

>  [KIRAN] section 7 explains about "offerer processing of the Answer". But
> I think the correct section for my explanation is section 6, " Generating
> the Answer", which recommonds to use the same order of preference as that
> in the offer.
>
>
The text says "unless there is a specific reason". You have a specific
reason. Just do it.

It is far more sensible to change the order of codecs in the answer, than
to send a reoffer with changed codecs immediately afterwards.