Subject: [rtcweb] Followup to discussion of O/A and codecs from Friday session

From: Paul Kyzivat <[email-masked]>
Date: Fri, 8 Apr 2016 17:41:03 -0400

During the rtcweb session today, while discussing (I think) JSEP issue 
#247 (Document what should happen when there are no matching codecs in 
answer), there was a question about happens in a particular case. I 
think it was:

- X offers codecs A and B (sendrecv)
- Y answers codec B (sendrecv)

The question was whether Y may use codec A when sending to X. Jonathan 
gave an answer I disagreed with and we had a chat about it. He got an 
action item to follow up afterwords, then he asked if I would do it.

As I read RFC3264, the answer is:

Yes, in the above case Y may send A to X.

Then a subsequent scenario came up: what would happen if Y has a need to 
send a (re)offer? Wouldn't this result in an inability to send using A? 
(Let's assume that Y is unable/unwilling to *receive* A but can send it, 
even though this is a weird case.)

- Y offers codec B (sendrecv)
- X answers codec B and A (sendrecv)

This *is* permitted by 3264. And in this case Y will be allowed to 
continue sending using codec A.

Relevant pieces of 3264 supporting this interpretation:

- section 5.1 (Generating the Initial Offer/Unicast Streams):

    ... If multiple formats are listed, it
    means that the offerer is capable of making use of any of those
    formats during the session.  In other words, the answerer MAY change
    formats in the middle of the session, making use of any of the
    formats listed, without sending a new offer.

- section 6.1 (Generating the Answer/Unicast Streams):

    ... For streams marked as sendrecv in the answer,
    the "m=" line MUST contain at least one codec the answerer is willing
    to both send and receive, from amongst those listed in the offer.
    The stream MAY indicate additional media formats, not listed in the
    corresponding stream in the offer, that the answerer is willing to
    send or receive (of course, it [the answerer] will not be able to
    send them at this time, since it was not listed in the offer).

- section 8.3.2 (Changing the Set of Media Formats):

    The list of media formats used in the session MAY be changed.  To do
    this, the offerer creates a new media description, with the list of
    media formats in the "m=" line different from the corresponding media
    stream in the previous SDP.  This list MAY include new formats, and
    MAY remove formats present from the previous SDP.  ...

    The corresponding media stream in the answer is formulated as
    described in Section 6, and may result in a change in media formats
    as well.  Similarly, as described in Section 6, as soon as it sends
    its answer, the answerer MUST begin sending media using any formats
    in the offer that were also present in the answer, and SHOULD use the
    most preferred format in the offer that was also listed in the answer
    (assuming the stream allows for sending), and MUST NOT send using any
    formats that are not in the offer, even if they were present in a
    previous SDP from the peer.  Similarly, when the offerer receives the
    answer, it MUST begin sending media using any formats in the answer,
    and SHOULD use the most preferred one (assuming the stream allows for
    sending), and MUST NOT send using any formats that are not in the
    answer, even if they were present in a previous SDP from the peer.

I'm not too clear on how this relates to issue #247, but at least I hope 
we can agree on what 3264 says.

	Thanks,
	Paul

---

From: Harald Alvestrand <[email-masked]>
Date: Sat, 9 Apr 2016 00:01:05 +0200

This indicates that the answer's omission of a codec does not allow the
offerer to deallocate resources that it has reserved for decoding that
type of stream (such as hardware).

Under this interpretation, if I reserve hardware for decoding VP8 and
H.264, and I get back an answer that indicates VP8 only, I cannot
deallocate my H.264 resources, since the answerer may choose to send me
H.264 anyway.

This contradicts something I think I've heard Cullen claim multiple
times, but I may have misunderstood Cullen.

On 04/08/2016 11:41 PM, Paul Kyzivat wrote:
> Then a subsequent scenario came up: what would happen if Y has a need
> to send a (re)offer? Wouldn't this result in an inability to send
> using A? (Let's assume that Y is unable/unwilling to *receive* A but
> can send it, even though this is a weird case.)
>
> I'm not too clear on how this relates to issue #247, but at least I
> hope we can agree on what 3264 says.
>
>     Thanks,
>     Paul
>
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

-- 
Surveillance is pervasive. Go Dark.

---

From: Taylor Brandstetter <[email-masked]>
Date: Fri, 8 Apr 2016 15:18:00 -0700

In Section 6 of RFC 3264:

> The answerer MUST send using a media format in the offer that is also listed in the answer
>
> So, if X offers A and B, and Y answers B, how would Y be able to send A?
It's not in both the offer and the answer.

---

From: Roman Shpount <[email-masked]>
Date: Fri, 8 Apr 2016 18:21:23 -0400

> deallocate my H.264 resources, since the answerer may choose to send me
Based on what I heard from multiple people (including Jonathan Rosenberg
when this document was originally discussed), Offer/Answer was always
intended to be asymmetrical. Each side can declare all the codecs it is
willing to receive and can send any codec remote party supports. I saw
multiple examples of this being implemented and used in exactly this manner.

In your example, if you need to free resources for H.264, you need to do
another Offer/Answer exchange in which you will send an offer with VP8 only.

Regards,
_____________
Roman Shpount

---

From: Roman Shpount <[email-masked]>
Date: Fri, 8 Apr 2016 18:27:30 -0400

> It's not in both the offer and the answer.
A does not have to be in both offer and answer. Offer/Answer is declarative
capability exchange with each side declaring what it is willing to receive.
It is not intended to pick a single codec. Negotiation is considered
successful if there is at least one common codec, but there are no other
requirements. Each party can send any codec remote party is willing to
receive and can switch between the codecs remote party supports at any
time. So in this example, Y can not only send A but can switch between A
and B at any time.

Also, answer can include codecs which were not present in the offer, so the
whole situation can be reversed.
_____________
Roman Shpount

---

From: Taylor Brandstetter <[email-masked]>
Date: Fri, 8 Apr 2016 15:36:10 -0700

I understand if that was the intent, but that's not what RFC 3264 seems to
say. Both Section 6 and Section 8.3.2 state that the answer MUST send a
codec listed in both the offer and the answer.

So is RFC 3264 wrong? Or is my interpretation of the rule wrong?

On Fri, Apr 8, 2016 at 3:27 PM, Roman Shpount <[email-masked]> wrote:

> A does not have to be in both offer and answer. Offer/Answer is
> declarative capability exchange with each side declaring what it is willing
> to receive. It is not intended to pick a single codec. Negotiation is
> considered successful if there is at least one common codec, but there are
> no other requirements. Each party can send any codec remote party is
> willing to receive and can switch between the codecs remote party supports
> at any time. So in this example, Y can not only send A but can switch
> between A and B at any time.
>
> Also, answer can include codecs which were not present in the offer, so
> the whole situation can be reversed.
> _____________
> Roman Shpount
>
>

---

From: Taylor Brandstetter <[email-masked]>
Date: Fri, 8 Apr 2016 15:47:00 -0700

Actually, I see that Section 5.1 states:
>
> [T]he answerer MAY change formats in the middle of the session, making use of any of the formats listed [in the offer], without sending a new offer.
>
> So, does the combination of these statements imply that the answerer MUST
start by sending a format common to both the offer and the answer, but
later MAY switch to a format specific to the offer? That seems odd, but I
can't think of any other conclusion.

On Fri, Apr 8, 2016 at 3:36 PM, Taylor Brandstetter <[email-masked]>
wrote:

>>>> So, if X offers A and B, and Y answers B, how would Y be able to send
>>> A? It's not in both the offer and the answer.
>>>
>>>

---

From: Roman Shpount <[email-masked]>
Date: Fri, 8 Apr 2016 18:47:03 -0400

Taylor,

After rereading RFC3264, I can see that you are correct: If the codec is
not present in the answer, offerer MAY stop listening for this payload type
(https://tools.ietf.org/html/rfc3264#section-7).

Sorry for confusion,

_____________
Roman Shpount

---

From: Paul Kyzivat <[email-masked]>
Date: Sat, 9 Apr 2016 16:19:47 -0400

On 4/8/16 6:47 PM, Roman Shpount wrote:
> not present in the answer, offerer MAY stop listening for this payload
> type (https://tools.ietf.org/html/rfc3264#section-7).
>
> Sorry for confusion,

Hmm.

Also, the end of section 6.1 says:

    The answerer MUST send using a media format in the offer
    that is also listed in the answer,

Together, those contradict what I said above.

So I guess Harald gets his wish to be able to discard resources for 
codecs that weren't mentioned in the answer.

Unfortunately this also means it isn't possible to have asymmetric 
behavior. For instance, it means you can't have telephone-events work in 
one direction and not the other.

	Sorry for missing the above,
	Paul