Subject: [rtcweb] JSEP: codecs in answer: MUST vs. MAY

From: Jonathan Lennox <[email-masked]>
Date: Tue, 24 Oct 2017 15:58:55 -0400

JSEP (section 5.3.1) seems to be inconsistent about how whether locally-supported codecs that weren’t listed in a remote offer MUST, or MAY, be included in an answer:

   o  If codec preferences have been set for the associated transceiver,
      media formats MUST be generated in the corresponding order,
      regardless of what was offered, and MUST exclude any codecs not
      present in the codec preferences.

   o  Otherwise, the media formats on the m= line MUST be generated in
      the same order as those offered in the current remote description,
      excluding any currently unsupported formats.  Any currently
      available media formats that are not present in the current remote
      description MUST be added after all existing formats.

   o  In either case, the media formats in the answer MUST include at
      least one format that is present in the offer, but MAY include
      formats that are locally supported but not present in the offer,
      as mentioned in [RFC3264], Section 6.1 <https://tools.ietf.org/html/rfc3264#section-6.1>.  If no common format
      exists, the m= section is rejected as described above.

The first two paragraphs certainly seem to indicate that these codecs MUST be included in the answer; however, the third paragraph suddenly weakens this to a MAY.

Is the intent of the third paragraph simply to restate RFC 3264’s loose requirements, whereas the first two are JSEP’s more binding ones?  If so, I think this should be stated more clearly, otherwise the required behavior is unclear.

---

From: Ted Hardie <[email-masked]>
Date: Tue, 24 Oct 2017 13:36:54 -0700

I read this slightly differently, see below.

On Tue, Oct 24, 2017 at 12:58 PM, Jonathan Lennox <[email-masked]>
wrote:

> JSEP (section 5.3.1) seems to be inconsistent about how whether
> locally-supported codecs that weren’t listed in a remote offer MUST, or
> MAY, be included in an answer:
>
This is what you do if codec preferences are set (keep the order, and don't
add new ones)

>       description MUST be added after all existing formats.
This is what you do if codec preferences are not set (keep the order, add
any new ones to the end)

>       exists, the m= section is rejected as described above.
In either case, if you don't have at least one codec that matches, you
reject the m= section (don't add any new ones).

I think the MAY there from RFC 3264 permits the behavior in the  "if not
codec preferences are set" behavior branch, but doesn't require it, which
is why there can be a branch like the one where codec preferences are set
and they are not sent.

> The first two paragraphs certainly seem to indicate that these codecs MUST
> be included in the answer; however, the third paragraph suddenly weakens
> this to a MAY.
>
> Is the intent of the third paragraph simply to restate RFC 3264’s loose
> requirements, whereas the first two are JSEP’s more binding ones?  If so, I
> think this should be stated more clearly, otherwise the required behavior
> is unclear.
>
> _

So I don't personally see an inconsistency in this, but if you have other
language, please suggest it.  We're already in IESG evaluation, but it
might be a friendly amendment by Adam or someone else.

regards,

Ted

> ______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>
>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 24 Oct 2017 23:15:11 +0200

I thought that one of the reasons for adopting SDP in WebRTC was that
it allows rapid implementation.
Are we still defining how codecs are negotiated? Is not even that
properly defined in the SDP O/A spec?

On 24 October 2017 at 21:58, Jonathan Lennox <[email-masked]> wrote:
> locally-supported codecs that weren’t listed in a remote offer MUST, or MAY,
> be included in an answer:
>
>       as mentioned in [RFC3264], Section 6.1.  If no common format
> think this should be stated more clearly, otherwise the required behavior is
> unclear.
>
> _______________________________________________
> rtcweb mailing list
> [email-masked]

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Justin Uberti <[email-masked]>
Date: Tue, 24 Oct 2017 21:47:52 -0700

Ted's interpretation is correct - this text is intentionally reiterating
3264's point that it is permissible to respond with codecs that weren't
offered, as this has been a frequent misunderstanding.

One option to clarify this could be to replace the "MAY" with "is permitted
to".

On Tue, Oct 24, 2017 at 1:36 PM, Ted Hardie <[email-masked]> wrote:

> This is what you do if codec preferences are set (keep the order, and
> don't add new ones)
>
>
>> The first two paragraphs certainly seem to indicate that these codecs
>> MUST be included in the answer; however, the third paragraph suddenly
>> weakens this to a MAY.
>>

---

From: Jonathan Lennox <[email-masked]>
Date: Fri, 27 Oct 2017 13:14:33 -0400

> On Oct 25, 2017, at 12:47 AM, Justin Uberti <[email-masked]> wrote:
> 
> Ted's interpretation is correct - this text is intentionally reiterating 3264's point that it is permissible to respond with codecs that weren't offered, as this has been a frequent misunderstanding.

Okay, good, that’s what I hoped; but this is mixing up normative text (MUST reject if no common codec) with informative, which is confusing.

> One option to clarify this could be to replace the "MAY" with "is permitted to”.

I would go further than this, clearly distinguishing the normative text from the informative. Perhaps:

In either case, the media formats in the answer MUST include at least one format that is present in the offer.  If no common format exists, the m= section is rejected as described above.
(Note that in SDP offer/answer, an answer is permitted to include formats that are locally supported but not present in the offer, as mentioned in [RFC3264], Section 6.1.  As described in the preceding paragraphs, in JSEP including these formats is mandatory.)

> On Tue, Oct 24, 2017 at 1:36 PM, Ted Hardie <[email-masked] <mailto:[email-masked]>> wrote:
> On Tue, Oct 24, 2017 at 12:58 PM, Jonathan Lennox <[email-masked] <mailto:[email-masked]>> wrote:
> This is what you do if codec preferences are set (keep the order, and don't add new ones)
>  
> This is what you do if codec preferences are not set (keep the order, add any new ones to the end)
> In either case, if you don't have at least one codec that matches, you reject the m= section (don't add any new ones). 
> 
> I think the MAY there from RFC 3264 permits the behavior in the  "if not codec preferences are set" behavior branch, but doesn't require it, which is why there can be a branch like the one where codec preferences are set and they are not sent.
>  
> So I don't personally see an inconsistency in this, but if you have other language, please suggest it.  We're already in IESG evaluation, but it might be a friendly amendment by Adam or someone else.
> 
> regards,
> 
> Ted
>  
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/rtcweb <https://www.ietf.org/mailman/listinfo/rtcweb>
> 
> 
> 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/rtcweb <https://www.ietf.org/mailman/listinfo/rtcweb>
> 
>

---

From: Jonathan Lennox <[email-masked]>
Date: Fri, 27 Oct 2017 16:05:13 -0400

Another question about JSEP’s codec answer rules.

>> This is what you do if codec preferences are not set (keep the order, add any new ones to the end)

Why are remotely-unsupported codecs added at the end of the list, rather than just listing all locally-supported codecs in the browser’s preference order?

My concern is in an SFU conferencing scenario.  Consider the following:

* Participant 1 joins a conference.  It supports only codec A.
* Participant 2 joins the conference.  It supports codecs A and B, but prefers B.  Since the conference can only do A, if the offer/answer exchange goes SFU->Participant, the participant’s answer must list its codecs as A, B.
* Participant 3 joins the conference.  It is the same as Participant 2, preferring B to A.

If participant 1 leaves, the SFU can re-offer to participants 2 and 3; but as far as it knows, everyone prefers A to B, so the conference can end up in a scenario where everyone’s using a less-preferred codec.

I would suggest that instead, the browser’s codec preference order always be sent as-is in an answer (unless overridden by the app with setCodecPreferences).  This also has the benefit of aligning the behavior with and without app codec preferences, rather than having two separate behaviors (where the order is fixed if the app has set codec preferences, and mirrored otherwise).

---

From: Ted Hardie <[email-masked]>
Date: Fri, 27 Oct 2017 14:14:50 -0700

Hi Jonathan,

On Fri, Oct 27, 2017 at 1:05 PM, Jonathan Lennox <[email-masked]> wrote:

> Why are remotely-unsupported codecs added at the end of the list, rather
> than just listing all locally-supported codecs in the browser’s preference
> order?
>
> * Participant 2 joins the conference.  It supports codecs A and B, but
> prefers B.  Since the conference can only do A, if the offer/answer
> exchange goes SFU->Participant, the participant’s answer must list its
> codecs as A, B.
> If participant 1 leaves, the SFU can re-offer to participants 2 and 3; but
> as far as it knows, everyone prefers A to B, so the conference can end up
> in a scenario where everyone’s using a less-preferred codec.
>
>
>
I would suggest that instead, the browser’s codec preference order always
> be sent as-is in an answer (unless overridden by the app with
> setCodecPreferences).  This also has the benefit of aligning the behavior
> with and without app codec preferences, rather than having two separate
> behaviors (where the order is fixed if the app has set codec preferences,
> and mirrored otherwise).
>

So, we're currently past WG last call and IETF last call, with the draft in
IESG evaluation.  This proposal appears to change a method that has been
agreed through all of those process and which produces a correct answer.
While it may bee mildly suboptimal for the duration of the conference in
the case above, it works and simplifies the conference behavior by
presuming no re-offer occurs until a participant 4 arrives who does only B
(If participant 1 has left, the conference can then re-offer codec B to 2
and 3).

While I appreciate the care you're going through with this, I don't believe
that this trade-off rises to the level of concern that would warrant
re-opening the question.

regards,

Ted

---

From: Taylor Brandstetter <[email-masked]>
Date: Mon, 30 Oct 2017 15:06:23 -0700

Hi Jonathan,

This is why we have setCodecPreferences. There are certainly use cases
where you *do* want to list codecs in browser preference order. And there
are cases where you don't. setCodecPreferences lets the application
developer decide which is appropriate.

Regards,
-Taylor

On Fri, Oct 27, 2017 at 2:14 PM, Ted Hardie <[email-masked]> wrote:

>> If participant 1 leaves, the SFU can re-offer to participants 2 and 3;
>> but as far as it knows, everyone prefers A to B, so the conference can end
>> up in a scenario where everyone’s using a less-preferred codec.
>>
>>
>>
> So, we're currently past WG last call and IETF last call, with the draft
> in IESG evaluation.  This proposal appears to change a method that has been
> While I appreciate the care you're going through with this, I don't
> believe that this trade-off rises to the level of concern that would
> warrant re-opening the question.
>
> regards,
>
> Ted
>

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 31 Oct 2017 08:16:26 +0100

Den 27. okt. 2017 22:05, skrev Jonathan Lennox:
> than just listing all locally-supported codecs in the browser’s
> preference order?

I don't read it like that.

The answer consists of:

- The offer's listed codecs that are also supported by the answerer,
possibly rearranged according to codec preference
- The answerer's supported codecs that were not listed in the offer.

These aren't "remotely-unsupported", they are unlisted by the offerer.

If they're really not supported by the offerer, adding them will have no
effect.

If they're supported by the offerer but the offerer chose not to list
them (might happen for instance if one is an inferior mode of a
supported codec, which the offerer did not think was worth mentioning),
listing them before listed codecs would be overriding a strongly stated
preference by the offerer.

I like the current logic.

> but as far as it knows, everyone prefers A to B, so the conference can
> end up in a scenario where everyone’s using a less-preferred codec.

If the SFU offers A, B to participants 2 and 3, and both 2 and 3 prefer
B, they will answer B, A based on local codec preference.

It will have to ask, but is that a big deal?

> 
> I would suggest that instead, the browser’s codec preference order
> always be sent as-is in an answer (unless overridden by the app with
> setCodecPreferences).  This also has the benefit of aligning the
> behavior with and without app codec preferences, rather than having two
> separate behaviors (where the order is fixed if the app has set codec
> preferences, and mirrored otherwise).

I don't have a strong preference here, but do have a strong preference
against holding up the spec if this is the biggest issue left.

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 31 Oct 2017 09:37:51 +0000

Hi,

>>Another question about JSEP¹s codec answer rules.
>> 
>> The spec currently says:
>>>>
>> than just listing all locally-supported codecs in the browser¹s
>> preference order?
>

A few things to keep in mind, if adding codec X in the answer, when X was
not in the offer:

1) The answerer needs to be prepared to receive media using codec X, but
the answerer cannot send media using codec X.
2) If the offerer does support X (even if it did not include it in the
offer), the offerer cannot use it if using it would require providing
codec related SEND parameters.

Regards,

Christer