Subject: Re: [rtcweb] RTT (was Re: No Plan)

From: Gunnar Hellstrom <[email-masked]>
Date: Sat, 01 Jun 2013 09:34:10 +0200

On 2013-05-31 23:15, Paul Kyzivat wrote:
> I've been through this conversation before.
> There are no winners. Different strokes for different folks.
Agreed, or different strokes for different situations.

There is voice calls, video calls and real-time text for the active 
intensive conversational situations.
There is voice mail, video mail and text messaging for more loosely 
coupled exchange of information.

> IMO the texting UI should be as independent as possible of this 
> stylistic difference, and the actual protocol. The session 
> establishment should sort out the "best" compromise between the 
> desires and capabilities of the two ends.
>
>     Thanks,
>     Paul
>
Agreed.

> On 5/31/13 4:27 PM, Matthew Kaufman wrote:
>> On 5/30/2013 10:32 PM, Gunnar Hellstrom wrote:
>>>  I do not understand why modern communication users accept to see a
>>> chat state indication of "composing" instead of really seeing what
>>> text is composed.
>>
>> Perhaps because you haven't done user studies of SMS-style
>> compose-and-send vs. real-time text.
>>
>> I suggest you do that, and then you'll understand the several reasons
>> why most users (perhaps interestingly, excluding those users who are
>> hearing-impaired) prefer the former.
Tests are done and indicate the opposite.
>>
>>> With real-time text you get rid of the frustration that "composing"
>>> creates.
>>
>> And you add the sender's frustration of not being able to edit and
>> rethink their message before sending it, and the expectation on both the
>> sender and the receiver that they remain present for the duration of the
>> conversation rather than using it as a completely asynchronous messaging
>> modality, to reply when convenient. [this is just a subset of what the
>> user studies show, but touches a couple of the most common points]
Yes, that are commonly used arguments.  There are benefits of both. 
Different strokes and usability in different situations.

So I agree with Paul's conclusion above.

/Gunnar

>>
>> Matthew Kaufman
>> _______________________________________________
>> rtcweb mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/rtcweb
>>
>
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Emil Ivov <[email-masked]>
Date: Sat, 01 Jun 2013 10:50:58 +0300

Hey Christer,

Apologies for the lag.

On 30.05.13, 10:44, Christer Holmberg wrote:
> Hi Emil,
>
> The draft says:
>
> 	"For the sake of interoperability this specification strongly advises
>     	against the use of multiple m= lines for a single media type."

This should probably be clarified. The above referred mostly to a 
browser's expectations and default offers. Multiple m= lines can confuse 
a number of existing legacy endpoints which is why they should be 
avoided when initiating a session that could reach a similar device (and 
by default this should be assumed for any session).

If applications *know* that they need to have multiple m= lines of a 
given type they can request this the same way they could do it with Plan B:

    If the application wishes, it can request that a given
    media source be placed onto a separate m= line, by setting a new
    .content property on the desired MediaStreamTrack; the values for the
    .content property are those defined for the a=content attribute in
    [RFC4796].

I'll make sure this is part of the next version.

Does this make sense?

Emil

> My understanding is that the usage of multiple m= lines for a single media type would not affect the mechanism as such, but I just want to verify that :)
>
> Also, there ARE "legacy" implementations that use multiple m= lines for a single media type (e.g. video enabled devices using two video m= lines: one for camera content, and one for slides).
>
> So, while I definitely think that legacy interoperability shall be taken into consideration, I would not like to make such strong statements. In my opinion (the draft also talks about it), the usage of multiple simultaneous SSRCs per m- line is a much bigger issue when it comes to legacy interoperability.
>
> Also, in CLUE we have been working on signaling scenarios with multiple m= lines per media type.
 >
>
> Regards,
>
> Christer
>
>
>
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Emil Ivov
> Sent: 29. toukokuuta 2013 22:00
> To: [email-masked]
> Subject: [rtcweb] No Plan
>
> Hey all,
>
> Based on many of the discussions that we've had here, as well as many others that we've had offlist, it seemed like a good idea to investigate a negotiation alternative that relies on SDP and Offer/Answer just a little bit less.
>
> The following "no plan" draft attempts to present one such approach:
>
> http://tools.ietf.org/html/draft-ivov-rtcweb-noplan
>
> The draft relies on conventional use of SDP O/A but leaves the intricacies of multi-source scenarios to application-specific signalling, with potentially a little help from RTP.
>
> Hopefully, proponents of Plans A and B would find that the interoperability requirements that concerned them can still be met with "no plan". Of course they would have to be addressed by application-specific signalling and/or signalling gateways.
>
> Comments are welcome!
>
> Cheers,
> Emil
>
> --
> https://jitsi.org
> .

-- 
https://jitsi.org

---

From: Christer Holmberg <[email-masked]>
Date: Sat, 1 Jun 2013 11:05:39 +0000

Hi,

> I'll make sure this is part of the next version.

I have nothing against a general recommendation to, for a given media type, have as few m- lines as possible. 

But, I do think the draft need to point out that it is not always possible, e.g. because:

1) m- lines have different characteristics (normally indicated using SDP attributes) that does not "fit" all content for the given media type;
2) different protocols are used for different m- lines, even if the media type is the same; or
3) the remote endpoint only supports a single (or, another given number) of sources per m- line.

Etc.

Regards,

Christer

--
https://jitsi.org

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 3 Jun 2013 12:09:01 +0200

2013/5/31 Mark Rejhon <[email-masked]>:
>> Facebook, Google+, Twitter and any other website use their own custom
>> mechanisms for providing chat capabilities to *their own* users
>> *between them*. I don't expect that they will be interested in a
>> peer-to-peer (user-to-user) realtime text because they want to log the
>> messages, inspect them, offer you advertisements based on the content
>> of your "private" chats, etc.
>
>
> Objection, objection!
> They are NOT mutually exclusive.
>
> I have been in contact with Facebook as part of my XEP-0301 authorship (XMPP
> real-time text).
> See this animated GIF of Facebook real-time text concept:
> http://www.realjabber.org/anim/facebook_chat_concept.gif

Hi, I said "user-to-user realtime text", and not
"char-by-char-user-server-user chat" ;)

--
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 3 Jun 2013 12:28:09 +0200

2013/5/31 Paul Kyzivat <[email-masked]>:
>> That can be implemented on top of DataChannel anyway, I see no need
>> for mandating it as a new RTP media stream negotiated via SDP.
>
>
> IIUC, Gunnar is expecting that virtually every app that provides real time
> voice communication should also provide RTT. (And this has a habit of
> showing up as a legal mandate.)
>
> Leaving the decision about how to transport RTT to be reinvented by each
> application doesn't seem like a good way to achieve that.

Hi Paul. Standarizing the "app layer" in WWW does not work, nobody
wants it. What about if somebody (a website) wants to send custom
emoticons within the realtime text? should the WG define a "text
document format" for RTT? Each website does it in the custom way it
likes.

Tons of webs offer chat since years, why do we need a standard for it
now? A user in website-A will never chat with a user in website-B
anyway.

> Leaving the decision about how to transport RTT to be reinvented by each
> application doesn't seem like a good way to achieve that.

The Web has succeeded because each website has the capability of
reinventing the application by adding its own innovation and features.
Otherwise there would not be so many different social networks or
travel websites offereing "the same".

> If that is good enough for RTT, then why not voice and video too? Just send
> it over a data channel in any format you like. We don't need to standardize
> how to do it.

JS can not retrieve the audio/video bits received from the peer and
convert them into audio/video to be sent to the local multimedia
devices. That's why the WG defines a media format (I mean RTP +
security) and an API for providing such a multimedia flow to the JS
app.

The same is not true for text chat which can be achieved in tons of
ways in the WWW, without the need of a standard.

Regards.

--
Iñaki Baz Castillo
<[email-masked]>

---

From: Gunnar Hellstrom <[email-masked]>
Date: Mon, 03 Jun 2013 13:26:07 +0200

> now? A user in website-A will never chat with a user in website-B
As I see it, RTCWEB has a goal to enable audio, video and data 
communication  between a user in Website-A and a user in Website-B.

Sometimes I have seen in the discussion that we shall not build silos 
again.

To me, that means create and use standards for the features of common 
interest.

Video, real-time text and audio are the three real-time media of common 
interest.
All three are possible to transport as you like, but the RTCWEB activity 
is about making it in a similar way to enable interoperability and ease 
inclusion in web applications.

Even the isolated web designer who makes the dedicated site only 
offering communication between users of that site and the customer 
service will be glad to find common ways to include the three real-time 
communication modalities as components rather than being forced to 
invent everything again.

And users will be happy to find that it works in similar ways in 
different sites. ( just as you are happy to find the gas pedal located 
in the same place in every car )

There are more reasons why standards are good for this purpose, but I 
stop here. We had a discussion in November - January. It just popped up 
again because of how interop plans with SIP were expressed in the "No 
Plan". And it was not the "No Plan" itself that was limiting, it was 
rather the way the need for legacy interop was described in the No Plan 
draft.

My conclusion is again: whatever transport we select for Real-Time Text 
in the WebRTC environment, there is a need to standardize it.

/Gunnar

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 3 Jun 2013 13:38:17 +0200

2013/6/3 Gunnar Hellstrom <[email-masked]>:
> As I see it, RTCWEB has a goal to enable audio, video and data communication
> between a user in Website-A and a user in Website-B.

I strongly disagree. A WG won't change what the WWW wants, ant the WWW
has shown us, during years, a way of communication without real
interaction between different websites. Specifically users logged into
site-A don't directly interact with users logged into site-B, instead
site-A embed a JS code of site-B into the web, so users of site-B can
use site-B features (*same* JS app client in both users).

> My conclusion is again: whatever transport we select for Real-Time Text in
> the WebRTC environment, there is a need to standardize it.

My opinion is that we don't need a Real-Time Text specification in
WebRTC at all. We have DataChannel which means "realtime data", and
such a data can be text, images, binary content or whatever the
website wants to offer.

Regards.

--
Iñaki Baz Castillo
<[email-masked]>

---

From: Barry Dingle <[email-masked]>
Date: Mon, 3 Jun 2013 21:51:43 +1000

Real-time voice and real-time video need to agree on encode/decode rules
and transport.

Real-time text requires the same.

Using your rationale Inaki, then we don't need to standardise RT voice and
RT video - so WebRTC/rtcWeb is a waste of time!!

Or do you see RT Text as something different to RT Voice and RT Video?

Cheers,
/Barry Dingle

On Mon, Jun 3, 2013 at 9:38 PM, Iñaki Baz Castillo <[email-masked]> wrote:

> communication
> in

---

From: Gunnar Hellstrom <[email-masked]>
Date: Mon, 03 Jun 2013 14:00:07 +0200

On 2013-06-03 13:38, Iñaki Baz Castillo wrote:
> 2013/6/3 Gunnar Hellstrom<[email-masked]>:
Iñaki,
I disagree.

The use cases draft includes a case titled:

        4.2.10
        <http://tools.ietf.org/html/draft-ietf-rtcweb-use-cases-and-requirements-10#section-4.2.10>.
        Simple video communication service with inter-operator calling

See: 
http://tools.ietf.org/html/draft-ietf-rtcweb-use-cases-and-requirements-10#page-9

With RTCWEB, the web takes the step towards 
inter-provider-communication. I think that is one big reason why so much 
interest is shown to RTCWEB and WebRTC.

/Gunnar

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 3 Jun 2013 14:10:12 +0200

2013/6/3 Barry Dingle <[email-masked]>:
> Real-time voice and real-time video need to agree on encode/decode rules and
> transport.
>

I can tell you 100 ways for sending/receiving Real-time Text using
existing technology in browsers. Can you tell me how a browser can
send and receive RT audio/video without proprietary plugins? If there
is no way then we agree that WebRTC spec is needed for providing those
capabilities to browsers, but the same is not true for text.

> Or do you see RT Text as something different to RT Voice and RT Video?

Regardles RT Text is useful or not (it can be for some scenarios) it
can *already* be implemented over the standarized DataChannel (in
which the custom JS app can send/receive text, images, binary content
or whatever in realtime). No need at all to create a new RTP media
type.

--
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 3 Jun 2013 14:14:11 +0200

2013/6/3 Gunnar Hellstrom <[email-masked]>:
> 4.2.10.  Simple video communication service with inter-operator calling
>
> See:

That will just work if both websites want to offer such a
inter-website-communication. And that's already feasible. But we don't
need a new standard (RTT-over-RTP) for allowing two users in different
websites to intercommunicate via RTT. That can be achieved in multiple
ways with existing technology (WebSocket, DataChannel) but, still,
both websites must allow it.

--
Iñaki Baz Castillo
<[email-masked]>

---

From: Gunnar Hellstrom <[email-masked]>
Date: Mon, 03 Jun 2013 15:36:31 +0200

> both websites must allow it.
The inter - operator communication is also described in the overview:
http://tools.ietf.org/html/draft-ietf-rtcweb-overview-06#section-3

It is because it can be done in multiple ways that we need a standard 
for it. And a standard that combines audio, video and real-time text in 
the same session.

Outside of that it is of course valuable to have 1000 other ways to do 
it. But that does not reduce the need for a standard.

/Gunnar

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 3 Jun 2013 17:41:10 +0200

2013/6/3 Gunnar Hellstrom <[email-masked]>:
> It is because it can be done in multiple ways that we need a standard for
> it. And a standard that combines audio, video and real-time text in the same
> session.
>
> Outside of that it is of course valuable to have 1000 other ways to do it.
> But that does not reduce the need for a standard.

There are tons of ways of implementing chat or RTT in current
websites, but no one of them implements RT chat with users of a
different website. A WG standard will not change this reality and the
lack of interest. Said that, "SDP m=text" would just become an uneeded
and unrequested complexity.

--
Iñaki Baz Castillo
<[email-masked]>

---

From: Emil Ivov <[email-masked]>
Date: Mon, 03 Jun 2013 20:28:04 +0300

Hey Cullen, Paul, all

On 31.05.13, 22:19, Cullen Jennings (fluffy) wrote:
> On 31.05.2013, at 12:23 PM, Emil Ivov <[email-masked]> wrote:
>
>> Certainly. Could you please post the SDP that you would like to see
>> translated in a way that's compatible with "No Plan"?
>>
>> Emil
>
> We can start with the SDP in plan A

The example in "7.3. Many Videos" looks like a good start:

http://tools.ietf.org/html/draft-roach-rtcweb-plan-a-00#section-7.3

Here it is:

    v=0
    o=- 20518 0 IN IP4 198.51.100.1
    s=
    t=0 0
    c=IN IP4 203.0.113.1
    a=ice-ufrag:F7gI
    a=ice-pwd:x9cml/YzichV2+XlhiMu8g
    a=fingerprint:sha-1 42:89:c5:c6:55:9d:39:f9:b6:eb:e7
    a=group:BUNDLE m0 m1 m2 m3

    m=audio 56600 RTP/SAVPF 0 96
    a=mid:m0
    a=rtpmap:0 PCMU/8000
    a=rtpmap:96 opus/48000
    a=ptime:20
    a=sendrecv
    a=rtcp-mux
    [ICE Candidates]

    m=video 0 RTP/SAVPF 97 98
    a=mid:m1
    a=rtpmap:97 H264/90000
    a=fmtp:97 profile-level-id=4d0028;packetization-mode=1
    a=rtpmap:98 VP8/90000
    a=sendrecv
    a=rtcp-mux
    a=bundle-only
    a=ssrc:11111 cname:45:5f:fe:cb:81:e9

    m=video 0 RTP/SAVPF 97 98
    a=mid:m2
    a=rtpmap:97 H264/90000
    a=fmtp:97 profile-level-id=4d0028;packetization-mode=1
    a=rtpmap:98 VP8/90000
    a=sendrecv
    a=rtcp-mux
    a=bundle-only
    a=ssrc:22222 cname:45:5f:fe:cb:81:e9

    m=video 0 RTP/SAVPF 97 98
    a=mid:m3
    a=rtpmap:97 H264/90000
    a=fmtp:97 profile-level-id=4d0028;packetization-mode=1
    a=rtpmap:98 VP8/90000
    a=sendrecv
    a=rtcp-mux
    a=bundle-only
    a=ssrc:333333 cname:45:5f:fe:cb:81:e9

An offer generated by a "No Plan" browser in this case would look 
something like this:

    v=0
    o=- 20518 0 IN IP4 198.51.100.1
    s=
    t=0 0
    c=IN IP4 203.0.113.1
    a=ice-ufrag:F7gI
    a=ice-pwd:x9cml/YzichV2+XlhiMu8g
    a=fingerprint:sha-1 42:89:c5:c6:55:9d:39:f9:b6:eb:e7
    a=group:BUNDLE m0 m1

    m=audio 56600 RTP/SAVPF 96 0
    a=mid:m0
    a=rtpmap:96 opus/48000
    a=rtpmap:0 PCMU/8000
    a=ptime:20
    a=sendrecv
    a=rtcp-mux
    [ICE candidates]

    m=video 56602 RTP/SAVPF 97 98
    a=mid:m1
    a=rtpmap:97 H264/90000
    a=fmtp:97 profile-level-id=4d0028;packetization-mode=1
    a=rtpmap:98 VP8/90000
    a=sendrecv
    a=rtcp-mux
    [ICE candidates]

I. Talking to legacy

In case you need to talk to an actual legacy (as in widely deployed SIP) 
endpoint, the above would translate into a regular two-stream call. Both 
Plan A and No Plan would lead to essentially the same result (if we 
accept that older endpoints won't throw an exception at the sight of 4 
m= lines) so not much to discuss here.

II. Talking to Plan A style endpoints

If you need to talk to a Plan A endpoint you basically have the 
following options:

1. You use JavaScript to prettify the "No Plan" SDP and turn it into 
something that looks like "Plan A". Not my favourite option, but I am 
sure some would like to use it. Maybe vendors of Plan A equipment would 
even distribute JS libs that do this. It would basically all come down 
to generating one ssrc attribute and two additional m=lines and 
appending this to the existing SDP string.

2. The application retrieves SSRCs from the browser, adds additional 
application-specific signalling to it and then sends the whole thing to 
a signalling gateway. The gateway (which you would also have with Plan 
A) would convert the SDP into what it needs it to be.

The application specific signalling can look like this:
	
{
     "firstStream":
     {
         "SSRC": "11111",
         "CNAME": "45:5f:fe:cb:81:e9"
     },
     "secondStream":
     {
         "SSRC": "22222",
         "CNAME": "45:5f:fe:cb:81:e9"
     },
     "thirdStream":
     {
         "SSRC": "333333",
         "CNAME": "45:5f:fe:cb:81:e9"
     },
}

3. In Plan B, section 3.1 talks about generating "Plan A" style SDP with 
the help of .content properties. If browser vendors are willing to 
implement support for this then I suppose it would be a third option.

III. Talking to other WebRTC applications

This is the fun case and the one we should be most concerned with. Let's 
imagine that the answerer needs to add a fourth video stream. To make 
this work endpoints would need to do the following:

a) with Plan A and draft-roach-rtcweb-glareless-add:
    - send application specific signalling to the offerer
    - have a new O/A exchange

b) with Plan A:
    - have a new O/A exchange
    - potentially risk glare with some impact on user experience

c) with No Plan:
    ... nothing

I am intentionally not going into how all plans would require additional 
metadata that would place SSRC 1 left and 2 right as I don't think this 
conveys any meaningful differences.

Comments on the above are welcome. We could also move to another 
scenario from the Plan A draft, if you believe that 7.3 is not 
representative enough.

Emil

-- 
https://jitsi.org

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 3 Jun 2013 20:26:13 +0200

2013/6/3 Emil Ivov <[email-masked]>:
> If you need to talk to a Plan A endpoint you basically have the following
> options:
>
> something that looks like "Plan A". Not my favourite option, but I am sure
> some would like to use it. Maybe vendors of Plan A equipment would even
> distribute JS libs that do this. It would basically all come down to
> generating one ssrc attribute and two additional m=lines and appending this
> to the existing SDP string.

HI, I would really prefer something that does not require SDP mangling
via JS. That is dangerous. SDP is too much flexible and we never know
how a SDP will look, so artificial mangling at JS level seems
unfeasible IMHO.

--
Iñaki Baz Castillo
<[email-masked]>

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 3 Jun 2013 11:33:44 -0700

On Mon, Jun 3, 2013 at 10:28 AM, Emil Ivov <[email-masked]> wrote:
[...]

> imagine that the answerer needs to add a fourth video stream. To make this
> work endpoints would need to do the following:
>
> a) with Plan A and draft-roach-rtcweb-glareless-**add:
I'm really not following how this works in "No Plan". Ignoring Plan A and
Plan B, can you just
walk through a complete example of what happens in your plan? I.e., how
does browser
B learn learn that Browser A wants to send 3 streams instead of one?

-Ekr

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 3 Jun 2013 13:42:43 -0600

I just wanted to quickly clear up I was not treating any of your ideas as a joke at all. Inline ….

On May 31, 2013, at 12:27 PM, Emil Ivov <[email-masked]> wrote:

>> 
>> The idea of exposing a low level API to the media stack and having
>> the all proposing to do FEC, RTX, SDP processing ect has been
>> discussed many times across the various working groups. It' jokingly
>> refereed to as comment 22 at this point.
> 
> I appreciate you are trying to turn this into a joke (which I think is a pity provide your chairing role in this working group), but abandoning SDP really isn't what this is about. You might have noticed text about this as early as the abstract:

I am not treating any of this as a joke at all - I am trying to understand what your roposal is. The "Comment 22" jokingly refers to some comments made by Mathew at WebRTC and nothing really to do this given that, as you say, this still uses Offer / Answer 

> 
>  This document does not question the use of SDP and the Offer/Answer
>  model or the value they have in terms of interoperability with legacy
>  or other non-WebRTC devices.

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 3 Jun 2013 13:44:15 -0600

On May 31, 2013, at 12:27 PM, Emil Ivov <[email-masked]> wrote:

>> 
>> Can you explain what you are actually proposing we do do instead of
>> just saying what not to do ?
> 
> We create an offer that would describe the media types and the bundle. We use that offer to also describe capabilities in terms of maximum resolutions, supported header extensions, potentially max-ssrc-s, etc.
> 
> It is up to the application how to handle the rest. If it needs to transmit additional signalling: let it. If it wants to encode that in SDP, great. If it wants to attach it in JSON next to the browser generated SDP, that also works.

Great - I have super news for you. The WG agree to do that a year or more ago.

---

From: Emil Ivov <[email-masked]>
Date: Mon, 03 Jun 2013 23:16:24 +0300

> Great - I have super news for you. The WG agree to do that a year or more ago.

So I've heard indeed.

Unfortunately however, it seems that we might have forgotten this 
decision. We are now trying hard to come up with a signalling mechanism 
that will do everything with Offer/Answer.

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Jun 2013 16:37:30 -0400

On 6/1/13 1:40 AM, Emil Ivov wrote:
>
>
> On 31.05.13, 22:11, Paul Kyzivat wrote:
>> On 5/31/13 6:51 AM, Emil Ivov wrote:
>>> Hey Paul,
>>>
>>> On 29.05.13, 22:57, Paul Kyzivat wrote:
>>>> Emil,
>>>>
>>>> I'm going to reserve judgment on this pending further discussion.
>>>> I think this *might* work for CLUE, but I want to be certain.
>>>
>>> Sure!
>>>
>>>> I'm concerned with decomposed endpoints that can't manage all the
>>>> streams on the same address/port. They will need multiple independent
>>>> m-lines and/or bundle groups.
>>>
>>> This is obviously open for debate but the general idea of No Plan is
>>> that: Offer/Answer is used for configuring media and RTP stacks and
>>> additional signalling is not the browser's concern.
>>>
>>> Having extra m= lines, particularly when using BUNDLE, is in many ways
>>> just extra signalling.
>>
>> You may be able to argue that adding extra m-lines into an existing
>> bundle is "just extra signaling". But introducing an additional 5-tuple
>> is more substantial. It requires configuring a new RTP stack and media.
>
> Indeed, this would require BUNDLE to work.

I don't understand what that means - what point you are trying to make.

You seem to have some criterion for what requires sdp and what does not. 
And AFAIK you are already assuming bundle works - you are simply using 
it less. As best I can tell, you acknowledge that you need SDP to:
- negotiate a 5-tuple over which media will flow
- specify which types of media can be carried and the codec info
   they may use.

I'm just pointing out that for decomposed endpoints one 5-tuple may not 
be enough. And the need for this might not be known a priori. One side 
may be ready to add another stream for some purpose, and be willing/able 
to use private signaling to negotiate that. But the other side may not 
be able to handle that stream on the same 5-tuple. That in turn may 
result in a need to negotiate SDP changes to establish an additional 
5-tuple that can then be used for that new stream.

	Thanks,
	Paul

> Emil
>
>
>>
>>     Thanks,
>>     Paul
>>
>>> If you'd like for that signalling to be in SDP, I
>>> don't see any problem with it. However it would be best for this extra
>>> layer of SDP signalling to appear at either the application layer or in
>>> a signalling gateway (that is going to be there anyway).
>>>
>>> Does this make sense?
>>>
>>> Emil
>>>
>>>>
>>>> Further questions:
>>>>
>>>> I presume that you expect bandwidth in the SDP to be an aggregate
>>>> per-m-line, with application specific signaling for bandwidth at the
>>>> per-RTP-stream level?
>>>>
>>>>      Thanks,
>>>>      Paul
>>>>
>>>> On 5/29/13 2:59 PM, Emil Ivov wrote:
>>>>> Hey all,
>>>>>
>>>>> Based on many of the discussions that we've had here, as well as many
>>>>> others that we've had offlist, it seemed like a good idea to
>>>>> investigate
>>>>> a negotiation alternative that relies on SDP and Offer/Answer just a
>>>>> little bit less.
>>>>>
>>>>> The draft relies on conventional use of SDP O/A but leaves the
>>>>> intricacies of multi-source scenarios to application-specific
>>>>> signalling, with potentially a little help from RTP.
>>>>>
>>>>> Hopefully, proponents of Plans A and B would find that the
>>>>> interoperability requirements that concerned them can still be met
>>>>> with
>>>>> "no plan". Of course they would have to be addressed by
>>>>> application-specific signalling and/or signalling gateways.
>>>>>
>>>>> Comments are welcome!
>>>>>
>>>>> Cheers,
>>>>> Emil
>>>>>
>>>>

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Jun 2013 16:40:17 -0400

+1

The more we dig into this the more it looks like Plan B.

	Thanks,
	Paul

---

From: Martin Thomson <[email-masked]>
Date: Mon, 3 Jun 2013 13:49:38 -0700

On 3 June 2013 11:33, Eric Rescorla <[email-masked]> wrote:
> walk through a complete example of what happens in your plan? I.e., how does
> browser

I think that I've reached the same conclusion.  At some point, the
browser needs to be told what to do.  At browser A, this is fairly
obvious: streams are added to the RTCPeerConnection instance.  But
what happens at browser B?

I think that there are a couple of alternatives, and we might need to
enable more than one of those, but I don't think that Section 7 has
enough information yet.

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Jun 2013 17:02:42 -0400

> now? A user in website-A will never chat with a user in website-B

I think this comes down to whether you think text/chat is "application" 
or "media".

Contrast it to video. My web app can send an image as part of a web 
page. It is "just a part of the application". And video is "just" a 
matter of sending a bunch if images in succession. So why isn't that 
still just "application" stuff? Why should we need RTP for it?

One answer may be that the difference is that the real-time requirements 
for text aren't as demanding as they are for voice or video. But there 
are other good reasons for identifying interactive text as media. One 
reason is what we are arguing right how: the desire to negotiate 
properties of the transmission. (Should it be byte-by-byte or 
line-by-line.) Another is that, like voice and video there is likely to 
be a demand to record it for later analysis and playback, or to 
broadcast it to a collection of people.

When text is treated as media, then treating it in parallel with other 
media makes more sense. (For instance ensuring the that the text is 
synchronized with the other media, both initially and when recorded and 
played back later.)

> travel websites offereing "the same".

There is the good and the bad of this.

There are lots of web sites that off something they call "chat".
And they are all different. So I can't be sure if I will have a 
transcript of the chat at the end or not. Or any record of it after the 
fact.

It is evident to me that we can't prevent chat being done this way. But 
it doesn't make it best practice.

> devices.

Maybe true for audio, but not video. If the video bits were sent over a 
data channel the JS could retrieve them and put them into a window. (It 
would be a pig, but possible.)

> That's why the WG defines a media format (I mean RTP +

I agree there are some differences. But there are also a lot similarities.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Jun 2013 17:03:32 -0400

On 6/3/13 8:14 AM, Iñaki Baz Castillo wrote:
> websites to intercommunicate via RTT.

It isn't new.

> That can be achieved in multiple

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 3 Jun 2013 23:07:05 +0200

2013/6/3 Emil Ivov <[email-masked]>:
>>> We create an offer that would describe the media types and the bundle. We
>>> use that offer to also describe capabilities in terms of maximum
>>> resolutions, supported header extensions, potentially max-ssrc-s, etc.
>>>
>>> It is up to the application how to handle the rest. If it needs to
>>> transmit additional signalling: let it. If it wants to encode that in SDP,
>>> great. If it wants to attach it in JSON next to the browser generated SDP,
>>> that also works.
>>
>>
> Unfortunately however, it seems that we might have forgotten this decision.
> We are now trying hard to come up with a signalling mechanism that will do
> everything with Offer/Answer.

I think there is a misunderstanding here. I understand from Emil's
mail that media re-negotation or streams addition after the first SDP
O/A is not carried as a new full SDP O/A. Am I right?

--
Iñaki Baz Castillo
<[email-masked]>

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 00:09:42 +0300

Hey Paul,

> I don't understand what that means - what point you are trying to make.

Sorry about this. What I was trying to say was: yes, you are right, 
introducing a new 5-tuple is more substantial. Therefore adding a new m= 
line to an SDP is a lot simpler if it didn't also imply adding a new 
5-tuple. This would only be possible when m= lines are bundled. This is 
why I said: "Indeed, this would require BUNDLE to work."

Is this clearer?

> it less.

No, not really. The assumption is only necessary when trying to talk to 
Plan A devices and you want to have new m= lines added en route.

No such assumption is necessary, for example, when talking to legacy SIP 
phones that will only show one video and one audio stream.

> As best I can tell, you acknowledge that you need SDP to:

Yes.

> 5-tuple that can then be used for that new stream.

I think I see ... but not completely. Could you please post an SDP offer 
with BUNDLE and the corresponding answer?

(This might be a good point to move this to mmusic)

Thanks,
Emil

-- 
https://jitsi.org

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Jun 2013 17:16:36 -0400

I don't understand why we would be talking about translating from one of 
these plans to another. AFAIK they are *alternatives*, and that we will 
at some point agree on one, and that will be the only one implemented.

	Thanks,
	Paul

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 3 Jun 2013 23:20:12 +0200

2013/6/3 Paul Kyzivat <[email-masked]>:

> One answer may be that the difference is that the real-time requirements for
> text aren't as demanding as they are for voice or video. But there are other
> good reasons for identifying interactive text as media. One reason is what
> we are arguing right how: the desire to negotiate properties of the
> transmission. (Should it be byte-by-byte or line-by-line.) Another is that,
> like voice and video there is likely to be a demand to record it for later
> analysis and playback, or to broadcast it to a collection of people.
>
> When text is treated as media, then treating it in parallel with other media
> makes more sense. (For instance ensuring the that the text is synchronized
> with the other media, both initially and when recorded and played back
> later.)

Well, I must recognize that now I see a real usecase: realtime
subtitles synchronized with audio/video streams. That would be much
better than the typicall subtitles embed in the video stream (because
it could be rendered as real text, so the user could choose the text
size, and so on).

Problems of standarizing text over RTP:

- Encoding

- ¿just plain text? ¿could it be HTML code instead? ¿how to send
hyperlinks? ¿how to use custom markup?

- Integrity: RTP can loose packets, do we accept that a chat can loose
"chars" or "lines"?

--
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 3 Jun 2013 23:23:36 +0200

2013/6/3 Paul Kyzivat <[email-masked]>:
> these plans to another. AFAIK they are *alternatives*, and that we will at
> some point agree on one, and that will be the only one implemented.

+1

Just one please, just one...

--
Iñaki Baz Castillo
<[email-masked]>

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 00:36:27 +0300

> at some point agree on one, and that will be the only one implemented.

It has been pointed out that some existing non-WebRTC endpoints expect 
to get SDP "the Plan A way". Cullen asked how such devices could be made 
to communicate with "No Plan" browsers. This was my answer.

Personally I haven't come across such devices so I can only speculate 
about the reasons why they work the way they work. Still I don't see why 
they shouldn't be able to talk to WebRTC browsers ... hence my answer.

Emil

-- 
https://jitsi.org

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Jun 2013 17:40:30 -0400

RFC4103

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 00:41:44 +0300

> The more we dig into this the more it looks like Plan B.

I am not sure exactly what you mean by this. I did try to make it clear 
that "No Plan" has a lot in common with "Plan B". The main differences 
are that there is no expectation for SSRCs to be pre-announced and there 
are requirements for WebRTC APIs to provide the tools necessary for apps 
to control individual streams themselves.

Emil

-- 
https://jitsi.org

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 00:44:00 +0300

> O/A is not carried as a new full SDP O/A. Am I right?

Correct. Unless of course these streams require new payload types to be 
introduced, 5-tuples to be changed or other m= line parameters modified.

Emil

-- 
https://jitsi.org

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Jun 2013 17:45:28 -0400

> they shouldn't be able to talk to WebRTC browsers ... hence my answer.

They clearly *don'* expect to get it the Plan A way (bundled).

I accept that there must be a variety of things that employ multiple 
(unbundled) m-lines, each on its own 5-tuple with one RTP stream.

But that is not Plan A. You can view it as a special case of what I just 
responded to you on another thread (now on mmusic) where one end 
requires that a new media stream be on a different 5-tuple than those 
existing.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Jun 2013 17:48:08 -0400

> to control individual streams themselves.

I now think that it is enough like plan B that the two should be 
collapsed together. Make the signaling of the explicit SSRCs optional 
when there is some other mechanism to agree upon them.

	Thanks,
	Paul

>>> 1) m- lines have different characteristics (normally indicated using
>>> SDP attributes) that does not "fit" all content for the given media
>>> type;
>>>> My understanding is that the usage of multiple m= lines for a single
>>>> media type would not affect the mechanism as such, but I just want
>>>> to verify that :)
>>>>
>>>> Also, there ARE "legacy" implementations that use multiple m= lines
>>>> for a single media type (e.g. video enabled devices using two video
>>>> m= lines: one for camera content, and one for slides).
>>>>
>>>> So, while I definitely think that legacy interoperability shall be
>>>> taken into consideration, I would not like to make such strong
>>>> statements. In my opinion (the draft also talks about it), the usage
>>>> of multiple simultaneous SSRCs per m- line is a much bigger issue
>>>> when it comes to legacy interoperability.
>>>>
>>>> Based on many of the discussions that we've had here, as well as
>>>> many others that we've had offlist, it seemed like a good idea to
>>>> investigate a negotiation alternative that relies on SDP and
>>>> Offer/Answer just a little bit less.
>>>>
>>>> with "no plan". Of course they would have to be addressed by

---

From: Jonathan Lennox <[email-masked]>
Date: Mon, 3 Jun 2013 18:03:03 -0400

> The more we dig into this the more it looks like Plan B.

I think No Plan and Plan B are largely isomorphic in a WebRTC context.  The primary difference is that rather than using a=ssrc and a=receive-ssrc (and whatever other Plan B attributes would be needed) to control sources within an m-line, instead direct Javascript APIs are used for the same semantics. It's up to the application to communicate the relevant information end-to-end, in parallel to however it communicates the SDP blob.

As I see it, the primary advantages of No Plan, then, are that:

a) It allows "implicit" signaling of sources -- where that's sufficient -- by just sending RTP traffic.  This prevents problems related to the timing and synchronization between signaling and media.  (Note that Plan B probably actually needs this as well, for legacy interop.)

b) It carves a Comment 22 corner out of the SDP-based APIs, thus decoupling source signaling from SDP offer/answer.  This is for a scenario where there aren't any issues of legacy compatibility, and there isn't any (or at least much) existing IETF standardization -- as best I can recall, legacy compatibility and existing standardization were the primary motivations for using SDP as the WebRTC API.

Other than that, I think the two proposals are pretty similar for WebRTC, and it would probably be a Simple Matter of Programming to translate between the two (though you have to worry about the O/A state on the Plan B side).

In particular, if you have disaggregated media, you need multiple m-lines of the same type.  The No Plan document's suggestion that you offer only one m-line of a given media type is guidance on how to avoid interop failures on initial offers to legacy devices, I think, not an essential aspect of the proposal.

In a SIP context, No Plan requires defining a separate signaling channel for this source information, in scenarios where something beyond implicit signaling is needed.  As Emil has stated, I think that something like the SIP/XCon conference package, or something like the CLUE channel, would be the obvious places to start.

I think SIP, even more than RTCWeb, is where you'd want something lighter-weight and better-designed than SDP offer/answer to signal source changes.

--
Jonathan Lennox
[email-masked]

---

From: Jim Barnett <[email-masked]>
Date: Mon, 3 Jun 2013 22:16:19 +0000

Just checking if I understand No Plan:  since the bundling is added via JS APIs, and not SDP, then if the app looks at 
The localDescription and remoteDescription attributes of the PeerConnection, it won't be able to tell if sources are bundled or not (since localDescription and remoteDescription are defined to contain SDP)

- Jim
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Jonathan Lennox
Sent: Monday, June 03, 2013 6:03 PM
To: Paul Kyzivat
Cc: [email-masked]
Subject: Re: [rtcweb] No Plan

>>> browser's expectations and default offers. Multiple m= lines can 
>>> confuse a number of existing legacy endpoints which is why they 
>>> should be avoided when initiating a session that could reach a 
>>> similar device (and by default this should be assumed for any session).
>>> 

--
Jonathan Lennox
[email-masked]

---

From: "Roni Even" <[email-masked]>
Date: Tue, 4 Jun 2013 01:27:44 +0300

Emil,
I assume that in the example of no-plan the m-video section should have
a=max-send-ssrc:{*:3} and a=max-recv-ssrc:{*:3}

I had the other thread about the de-mux but this also shows my question, the
document says " This specification uses demuxing based on RTP payload
types." But in this example they are not unique, you may have three RTP
streams with the same pt number to de-mux using only pt-number.  

If you use max-send-ssrc to indicate the number of send streams, if you add
a stream you need to update the max-send-ssrc

Roni

> Sent: 03 June, 2013 8:28 PM
> To: Cullen Jennings (fluffy)
> Cc: [email-masked]
> Subject: [rtcweb] Translating Plan A into No Plan (Was: No Plan)
> 
> Hey Cullen, Paul, all
> 
something
> like this:
Plan
> A and No Plan would lead to essentially the same result (if we accept that
> older endpoints won't throw an exception at the sight of 4 m= lines) so
not
> much to discuss here.
something
> that looks like "Plan A". Not my favourite option, but I am sure some
would
> like to use it. Maybe vendors of Plan A equipment would even distribute JS
> libs that do this. It would basically all come down to generating one ssrc
> attribute and two additional m=lines and appending this to the existing
SDP
> string.
> application-specific signalling to it and then sends the whole thing to a
> signalling gateway. The gateway (which you would also have with Plan
the
> help of .content properties. If browser vendors are willing to implement
> support for this then I suppose it would be a third option.
> 
conveys
> any meaningful differences.
> 
representative
> enough.

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 3 Jun 2013 15:50:16 -0700

Jonathan said:
 
"I think No Plan and Plan B are largely isomorphic in a WebRTC context.  The primary difference is that rather than using a=ssrc and a=receive-ssrc (and whatever other Plan B attributes would be needed) to control sources within an m-line, instead direct Javascript APIs are used for the same semantics. It's up to the application to communicate the relevant information end-to-end, in parallel to however it communicates the SDP blob."
 
[BA] The use of O/A to turn sources on/off is definitely one thing that "No Plan" avoids, and that's something in its favor.  However, I don't think that's the only difference. 
 
As I see it, No Plan, when added to Plan B, results in simplifications along several dimensions.  I don't think that No Plan prohibits use of a=ssrc lines either in the API or over the wire, but it does makes it clear that they cannot be required for de-multiplexing (which No Plan handles via PTs) or rendering of incoming streams (which can be handled via APIs without MSID declarations).  Basically, No Plan says "Don't try to Bundle more things than you can de-multiplex using PTs".  It's a bit like telling a snake "don't eat an elephant whole, chop it up into bit sized pieces".  Yes, you could try it the other way,, but is the added complexity (and indigestion) really worth it? 
 
Personally I wouldn't say it is a "No Plan" no-no if createOffer were to produce SDP with a=ssrc lines, or if setRemoteDescription were to process a=ssrc lines included by the remote peer.  What I think No Plan would have a problem with is if undeclared sources couldn't be rendered, or if O/A were required to turn already declared sources on/off, which can happen so rapidly that O/A couldn't possibly handle it.  Also, requiring O/A for dropping streams breaks congestion control because in a highly congested situation, the signaling packets would probably get dropped along with the media, so that the signaling to drop streams couldn't get through.  A sender MUST be able to drop streams in that circumstance. 
 
In answer to questions about "whether you can declare separate streams for screen sharing versus other video" I would think that No Plan could allow a=ssrc lines for that, but if you just wanted to send them and have the application figure out which was which (e.g. via app-specific signaling) you could do that too.  
 
One area where I think all the  Plans are weak is handling simulcast/layered streams.  I don't think max*sssrc makes sense because it groups together sources, and streams (including simulcast, layered streams and FEC).  A receiver really needs to be able to declare the maximum number of sources it can receive and the maximum number of streams it can send, as well as declaring the simulcast and layering schemes it can handle.  This is NOT the same as actually stating what it will be sending or receiving at any given moment.  In general, it will be up to the sender to make the decision based on the feedback it is getting.  
 
Another comment on "No Plan" relates to the need for an RTP extension to declare things like simulcast and layered stream dependencies.  These dependencies can be expressed in SDP as alluded to in  Plan B, and they also can be described within some codecs (e.g. H.264/SVC allows this).  IMHO, SDP O/A does have a role here, in describing the receiver capabilities (what simulcast/layered streams it can support) as well as the sender capabilities (what the sender is capable of sending).  Since neither an RTP extension nor individual payloads can provide reliability, having this done in O/A via reliable signaling has an advantage.  But as long as the streams sent by the codec is self-describing, I don't think an RTP extension is needed.  
 
However, the case of FEC is a bit trickier for No Plan.   If you don't declare the FEC SSRCs and dependencies in SDP, then you do need to encode this in RTP, and the codec-specific route may not be sufficient (opinions?).  Also, doing this purely in RTP could create some problems if the descriptions were lost because the receiver could end up trying to render a FEC stream, with bizarre results.  So this is one area where it seems to me that "No Plan" may need some additional thought.

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 13:42:20 +0300

On 04.06.13, 01:03, Jonathan Lennox wrote:
> In particular, if you have disaggregated media, you need multiple
> m-lines of the same type.  The No Plan document's suggestion that you
> offer only one m-line of a given media type is guidance on how to
> avoid interop failures on initial offers to legacy devices, I think,
> not an essential aspect of the proposal.

Yes, this is correct.

Emil

-- 
https://jitsi.org

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 4 Jun 2013 12:35:10 +0000

Hi,

>> I am not sure exactly what you mean by this. I did try to make it 
>> clear that "No Plan" has a lot in common with "Plan B". The main 
>> differences are that there is no expectation for SSRCs to be 
>> pre-announced and there are requirements for WebRTC APIs to provide 
>> the tools necessary for apps to control individual streams themselves.
>
> I now think that it is enough like plan B that the two should be collapsed together. Make the signaling of the explicit SSRCs optional when there is some other mechanism to agree upon them.

When you say "signaling", do you refer to what is sent on the wire?

As others have said, we still need API support to control things - no matter whether it's done using SDP or some other mechanism.

Regards,

Christer

>>>> If applications *know* that they need to have multiple m= lines of 
>>>> a given type they can request this the same way they could do it 
>>>> with Plan B:
>>>>
>>> I have nothing against a general recommendation to, for a given 
>>> media type, have as few m- lines as possible.
>>>
>>>> My understanding is that the usage of multiple m= lines for a 
>>>> single media type would not affect the mechanism as such, but I 
>>>> just want to verify that :)
>>>>
>>>> statements. In my opinion (the draft also talks about it), the 
>>>> usage of multiple simultaneous SSRCs per m- line is a much bigger 
>>>> issue when it comes to legacy interoperability.
>>>>

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 14:59:26 +0300

Hey Bernard,

I obviously agree with what you are saying. Just one clarification 
regarding your layering and FEC comments:

I don't think we are likely to come up with one generic mechanism that 
would work and should be used with all possible kinds of repair flows 
and scenarios. As you have pointed on occasion: these mechanisms vary 
largely. Some, like SVC, handle it within codec. Others, such as 1-D 
Interleaved Parity [RFC6015] need a payload type of their own. Others 
yet require that NACKs be sent over signalling and of course there are 
those whose relation can be specified in Plan B style SDP [RFC5956].

No Plan is not explicitly incompatible with any of them. I don't see a 
reason why someone shouldn't be able to specify a simulcast relation 
between SSRCs in SDP. This is is totally fine.

What No Plan argues against is that this could be considered as enough 
of a reason to mandate that SSRCs should always be present in SDP. This 
argument has already been made and I think many here wouldn't agree with 
it because of the diversity in repair mechanisms that I mentioned above.

In that sense the RTP header extension in the draft is just one  example 
where such information could be delivered without any signalling. I 
personally believe such a mechanism could be quite useful in a number of 
cases. It can also be designed in such a way that browsers would use it 
transparently from applications, without bothering them with the 
exchange of O/A blobs.

Still this is only a possibility and not really a No Plan requirement.

Now, there is indeed something that we need to specify here and that is: 
how exactly does an application know whether it would need to worry 
about repair semantics and if so, would they work transparently or 
require the exchange of O/A blobs.

Not only can this change from one browser to the next but it can even 
vary across peer connections. Handling this could require some 
adaptations in the WebRTC API.

I might be wrong but I don't think any of the WebRTC supporting browsers 
currently send repair flows which is why all this is still hypothetical 
and now is probably the right time to think about it.

Cheers,
Emil

On 04.06.13, 01:50, Bernard Aboba wrote:
>
> Jonathan said:
>
> "I think No Plan and Plan B are largely isomorphic in a WebRTC context.
> The primary difference is that rather than using a=ssrc and
> a=receive-ssrc (and whatever other Plan B attributes would be needed) to
> control sources within an m-line, instead direct Javascript APIs are
> used for the same semantics. It's up to the application to communicate
> the relevant information end-to-end, in parallel to however it
> communicates the SDP blob."
>
> [BA] The use of O/A to turn sources on/off is definitely one thing that
> "No Plan" avoids, and that's something in its favor.  However, I don't
> think that's the only difference.
>
> As I see it, No Plan, when added to Plan B, results in simplifications
> along several dimensions.  I don't think that No Plan prohibits use of
> a=ssrc lines either in the API or over the wire, but it does makes it
> clear that they cannot be required for de-multiplexing (which No Plan
> handles via PTs) or rendering of incoming streams (which can be handled
> via APIs without MSID declarations).  Basically, No Plan says "Don't try
> to Bundle more things than you can de-multiplex using PTs".  It's a bit
> like telling a snake "don't eat an elephant whole, chop it up into bit
> sized pieces".  Yes, you could try it the other way,, but is the added
> complexity (and indigestion) really worth it?
>
> Personally I wouldn't say it is a "No Plan" no-no if createOffer were to
> produce SDP with a=ssrc lines, or if setRemoteDescription were to
> process a=ssrc lines included by the remote peer.  What I think No Plan
> would have a problem with is if undeclared sources couldn't be rendered,
> or if O/A were required to turn already declared sources on/off, which
> can happen so rapidly that O/A couldn't possibly handle it.  Also,
> requiring O/A for dropping streams breaks congestion control because in
> a highly congested situation, the signaling packets would probably get
> dropped along with the media, so that the signaling to drop streams
> couldn't get through.  A sender MUST be able to drop streams in that
> circumstance.
>
> In answer to questions about "whether you can declare separate streams
> for screen sharing versus other video" I would think that No Plan could
> allow a=ssrc lines for that, but if you just wanted to send them and
> have the application figure out which was which (e.g. via app-specific
> signaling) you could do that too.
>
> One area where I think all the  Plans are weak is handling
> simulcast/layered streams.  I don't think max*sssrc makes sense because
> it groups together sources, and streams (including simulcast, layered
> streams and FEC).  A receiver really needs to be able to declare the
> maximum number of sources it can receive and the maximum number of
> streams it can send, as well as declaring the simulcast and layering
> schemes it can handle.  This is NOT the same as actually stating what it
> will be sending or receiving at any given moment.  In general, it will
> be up to the sender to make the decision based on the feedback it is
> getting.
>
> Another comment on "No Plan" relates to the need for an RTP extension to
> declare things like simulcast and layered stream dependencies.  These
> dependencies can be expressed in SDP as alluded to in  Plan B, and they
> also can be described within some codecs (e.g. H.264/SVC allows this).
> IMHO, SDP O/A does have a role here, in describing the receiver
> capabilities (what simulcast/layered streams it can support) as well as
> the sender capabilities (what the sender is capable of sending).  Since
> neither an RTP extension nor individual payloads can provide
> reliability, having this done in O/A via reliable signaling has an
> advantage.  But as long as the streams sent by the codec is
> self-describing, I don't think an RTP extension is needed.
>
> However, the case of FEC is a bit trickier for No Plan.   If you don't
> declare the FEC SSRCs and dependencies in SDP, then you do need to
> encode this in RTP, and the codec-specific route may not be sufficient
> (opinions?).  Also, doing this purely in RTP could create some problems
> if the descriptions were lost because the receiver could end up trying
> to render a FEC stream, with bizarre results.  So this is one area where
> it seems to me that "No Plan" may need some additional thought.
>
>

-- 
https://jitsi.org

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 15:08:49 +0300

Hey Jim,

On 04.06.13, 01:16, Jim Barnett wrote:
> Just checking if I understand No Plan:  since the bundling is added via JS APIs, and not SDP,

I am not sure where you got this. No Plan does depend on the browser to 
provide support for BUNDLE and to generate the SDP accordingly.

Emil

> then if the app looks at

-- 
https://jitsi.org

---

From: <[email-masked]>
Date: Tue, 4 Jun 2013 17:09:39 +0200

> websites to intercommunicate via RTT.

In my experience many people have real difficulties to understand that
http://rtc.example.com/user/homer is as universal as is the E.164 number +1 555 3223

In the latter case, I need classical, PSTN-style interconnection between my telephony provider
and the telephony provider owning +1 555 3223.

With proper WebRTC, I simply open a link. Maybe I have to identify myself using
OpenID or a similar 3rd party auth. mechanism. This is up to rtc.example.com.
There's no need to agree on a RTT format, as both parties use the same client.
Works today.

The alternative is:
1. discuss whether RTT should be included into the browsers
   or agree on a standardized format for data channel messages which a gateway transcodes into RTP
2. discuss how to put this in SDP

Works in 2014 (2015?)

We will go through similar cycles for MSRP and every other datachannel content. And of course, for all options, like preference for 'is typing' over sending characters directly.

Regards,
Wolfgang Beck

--
Deutsche Telekom Netzproduktion GmbH
Aufsichtsrat: Timotheus Höttges (Vorsitzender)
Geschäftsführung: Dr. Bruno Jacobfeuerborn (Vorsitzender), Albert Matheis, Klaus Peren
Handelsregister: Amtsgericht Bonn HRB 14190
Sitz der Gesellschaft: Bonn
USt-IdNr.: DE 814645262

---

From: Gunnar Hellstrom <[email-masked]>
Date: Wed, 05 Jun 2013 00:11:48 +0200

>> websites to intercommunicate via RTT.
The standard exists with sdp and all. Ready to be included.
Since this way is described for audio and video, it is an easy option to 
use the same structure for RTT as well.
> 2. discuss how to put this in SDP
I see benefits and uses of both models.
The model of directly using the client of the destination's web server 
looks natural for customer services, and similar applications initiated 
from a web site with information about the organization.
For this model, it is very good to have standards for all media 
communication ( including RTT), so that the behavior in the text 
communication gadgets perform in a similar way from service to service. 
That makes it possible to link other apps conveniently to the browser 
and do specific actions on the text communication.

The model with inter-operator communication allows users to find out the 
client and service they like the best, that has the user interface 
features the user likes or needs, and look the same for every call.  And 
then use that for regular calling. The operator can have implemented 
good ways to display the text, or have your phone directory linked to 
the client, or have an implementation of a client that works well on you 
small device.  There are other benefits also, e.g. that adaptations for 
accessibility reasons will work best if you have a chance to adjust it 
to the client.

So, if we do it for the inter-operator model, it is easy to also use it 
for the single server model.

Gunnar

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Wed, 05 Jun 2013 00:13:05 +0200

El 04/06/2013 13:59, Emil Ivov escribió:
>
> I might be wrong but I don't think any of the WebRTC supporting 
> browsers currently send repair flows which is why all this is still 
> hypothetical and now is probably the right time to think about it.

IIRC, Chrome  encapsulates VP8 and FEC packets inside RED packets and 
then send them using the same SSRC.

Best regards
Sergio

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 5 Jun 2013 07:40:42 +0700

> O/A is not carried as a new full SDP O/A. Am I right?

I'm just trying to understand what the "No Plan" proposal is. I have not even started to think about what parts of if I like. So far, the more we talk about this, the more confused I get. I think some simple examples that showed what O/A got passed across the API from the JS into the browser for applications with multiple video steams would really help.

---

From: Emil Ivov <[email-masked]>
Date: Wed, 05 Jun 2013 17:21:40 +0200

On 05.06.13, 02:40, Cullen Jennings wrote:
>>>>> We create an offer that would describe the media types and
>>>>> the bundle. We use that offer to also describe capabilities
>>>>> in terms of maximum resolutions, supported header extensions,
>>>>> potentially max-ssrc-s, etc.
>>>>>
>>>>> It is up to the application how to handle the rest. If it
>>>>> needs to transmit additional signalling: let it. If it wants
>>>>> to encode that in SDP, great. If it wants to attach it in
>>>>> JSON next to the browser generated SDP, that also works.
>>>>
>>>>
>>> decision. We are now trying hard to come up with a signalling
>>> mechanism that will do everything with Offer/Answer.
>>
>>
>> mail that media re-negotation or streams addition after the first
>> SDP O/A is not carried as a new full SDP O/A. Am I right?
>>
>>
>
>
> I'm just trying to understand what the "No Plan" proposal is. I have
> not even started to think about what parts of if I like. So far, the
> more we talk about this, the more confused I get. I think some simple
> examples that showed what O/A got passed across the API from the JS
> into the browser for applications with multiple video steams would
> really help.

Then, in case you've missed any of these:

You asked how Plan A endpoints would work with No Plan browsers here:

http://www.ietf.org/mail-archive/web/rtcweb/current/msg07696.html

The reply, with sample SDP went here:

http://www.ietf.org/mail-archive/web/rtcweb/current/msg07724.html

You then moved the thread to MMUSIC and asked to see the answer here:

http://www.ietf.org/mail-archive/web/mmusic/current/msg11347.html

In the above you also asked how the answerer could refuse one video 
stream. I replied with the SDP answer and additional sample signalling here:

http://www.ietf.org/mail-archive/web/mmusic/current/msg11354.html

I don't think I've seen any additional questions from you since then.You 
are now saying that the situation is still unclear, so please let me 
know if I can help in any way. Otherwise we could also switch to 
whatever problems you have with the approach itself.

In the mean time, again on MMUSIC, Eric has asked for more detailed 
examples including sample JS code here:

http://www.ietf.org/mail-archive/web/mmusic/current/msg11360.html

Enrico has put up a very helpful and very complete example to this here:

http://www.ietf.org/mail-archive/web/mmusic/current/msg11382.html

I added a nit here:

http://www.ietf.org/mail-archive/web/mmusic/current/msg11386.html

Cheers,
Emil

-- 
https://jitsi.org

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 6 Jun 2013 12:41:17 +0700

I've trying to help you explain and I keep asking for an example but I don't think we have had a complete example yet. I know you think there is a complete example so let me try and be more specific so perhaps we can get to the bottom of the disconnect. 

Let me try to be specific. 

Say an application running on Alice's browser want to generate an offer that says the browser is ready to send and receive 2 streams of video and 1 stream of audio look like? Imagine that one of the video streams is a document camera running at high res but low frame rate while the other is video of the speaker running at a higher frame rate.  What exactly does the SDP passed from the browser to the JS applications look like.  I agree the applications can do whatever it wants to communicate the relevant data to the far end so I don't care about  the signaling protocol or JSON  that JS app can send across the wire. But next question, can the far end start sending media immediately to the browser? Finally the far end does something that causes the applications to generate an SDP answer from the JS applications to Alice's browser. I want and example of that that looks like in the cases where the far end a) accepted all the video streams and b) rejected some but not all of the video streams. 

If we can get an simple example like this sorted out, then perhaps it will be easy to extrapolate to the ones in the say the use case document and start looking at things like number of round trips and audio clipping. 

Cullen

PS - my apologies but a bunch of the time I have to work on this ends up on planes so there is some time warp sometimes from what data I had to read on the list vs what is on the list by the time my emails arrive. I also tend to try and process email sent to me before dealing with all list email which I tend to deal with in batches. So I'm sending this without having read the all the list traffic. 

On Jun 5, 2013, at 10:21 PM, Emil Ivov <[email-masked]> wrote:

> In the above you also asked how the answerer could refuse one video stream. I replied with the SDP answer and additional sample signalling here:
> 
> I don't think I've seen any additional questions from you since then.You are now saying that the situation is still unclear, so please let me know if I can help in any way. Otherwise we could also switch to whatever problems you have with the approach itself.
> 
> In the mean time, again on MMUSIC, Eric has asked for more detailed examples including sample JS code here:
>

---

From: Silvia Pfeiffer <[email-masked]>
Date: Thu, 6 Jun 2013 17:14:06 +1000

> If we can get an simple example like this sorted out, then perhaps it will be easy to extrapolate to the ones in the say the use case document and start looking at things like number of round trips…

Wouldn't that simply be multiplexed over one PeerConnection in the
browser using addStream()? I have an application like that working. I
can dig out the SDP packets that the browser sends in this case, if
you are interested.

Regards,
Silvia.

---

From: Emil Ivov <[email-masked]>
Date: Thu, 06 Jun 2013 11:16:58 +0200

Hey Cullen,

On 06.06.13, 07:41, Cullen Jennings wrote:
>
> I've trying to help you explain and I keep asking for an example but
> I don't think we have had a complete example yet. I know you think
> there is a complete example so let me try and be more specific so
> perhaps we can get to the bottom of the disconnect.
>
> Say an application running on Alice's browser want to generate an
> offer that says the browser is ready to send and receive 2 streams of
> video and 1 stream of audio look like? Imagine that one of the video
> streams is a document camera running at high res but low frame rate
> while the other is video of the speaker running at a higher frame
> rate.  What exactly does the SDP passed from the browser to the JS
> applications look like.

It looks exactly like the one I posted here:

For convenience:

    v=0

There is no information about the specific streams because such 
information isn't necessary for a remote WebRTC browser or a generic UA. 
They will both just decode incoming streams and at that point 
resolutions and frame rates will become clear.

If for some application specific reasons the remote party does need to 
know in advance what resolutions and frame rates each individual stream 
will have this will go in application-specific signalling. Such 
signalling may very well accompany the above SDP. An SDP gateway or a JS 
app may even choose to merge them both together, but this is out of 
scope.

> I agree the applications can do whatever it
> wants to communicate the relevant data to the far end so I don't care
> about  the signaling protocol or JSON  that JS app can send across
> the wire. But next question, can the far end start sending media
> immediately to the browser?

Of course! Why wouldn't it be able to?

> Finally the far end does something that
> causes the applications to generate an SDP answer from the JS
> applications to Alice's browser. I want and example of that that
> looks like in the cases where the far end a) accepted all the video
> streams and b) rejected some but not all of the video streams.

Both cases look the same way and they both look pretty much like the 
offer (except for the transport negotiation). This was explained here:

for convenience:

    v=0
    o=- 20518 0 IN IP4 198.51.100.2
    s=
    t=0 0
    c=IN IP4 203.0.113.1
    a=ice-ufrag:F7gI
    a=ice-pwd:x9cml/YzichV2+XlhiMu8g
    a=fingerprint:sha-1 42:89:c5:c6:55:9d:39:f9:b6:eb:e7
    a=group:BUNDLE m0 m1

    m=audio 5000 RTP/SAVPF 96 0
    a=mid:m0
    a=rtpmap:96 opus/48000
    a=rtpmap:0 PCMU/8000
    a=ptime:20
    a=sendrecv
    a=rtcp-mux
    [ICE candidates]

    m=video 5002 RTP/SAVPF 97 98
    a=mid:m1
    a=rtpmap:97 H264/90000
    a=fmtp:97 profile-level-id=4d0028;packetization-mode=1
    a=rtpmap:98 VP8/90000
    a=sendrecv
    a=rtcp-mux
    [ICE candidates]

As pointed out in the link above, the difference between a), and b) is 
going to be in application specific signalling that will turn off some 
specific SSRCs. That signalling can also accompany the SDP.

> If we can get an simple example like this sorted out, then perhaps it
> will be easy to extrapolate to the ones in the say the use case
> document and start looking at things like number of round trips and
> audio clipping.

Looking forward to that. I am especially interested as to why you think 
there could be any audio clipping or other negative effects with this.

Keep in mind that you can get exactly the same information to the remote 
end as with Plans A and B, the only difference is what part of that is 
in the SDP O/A generated by the browser. The advantage of No Plan is 
that it does not mandate for such information to be in SDP which 
lightens the signalling load in a number of cases.

Cheers,
Emil

P.S. I am confused as to where this discussion should be happening. 
There have been contradictory comments from chairs that seem to say how 
this should be in MMUSIC and how it actually doesn't belong to MMUSIC 
and should happen on RTCWEB. I'd be grateful if chairs of both WGs could 
sort this out and provide guidance.

> Cullen
>
>
> PS - my apologies but a bunch of the time I have to work on this ends
> up on planes so there is some time warp sometimes from what data I
> had to read on the list vs what is on the list by the time my emails
> arrive. I also tend to try and process email sent to me before
> dealing with all list email which I tend to deal with in batches. So
> I'm sending this without having read the all the list traffic.
>
>
>>>>>>> We create an offer that would describe the media types
>>>>>>> and the bundle. We use that offer to also describe
>>>>>>> capabilities in terms of maximum resolutions, supported
>>>>>>> header extensions, potentially max-ssrc-s, etc.
>>>>>>>
>>>>>>> It is up to the application how to handle the rest. If
>>>>>>> it needs to transmit additional signalling: let it. If it
>>>>>>> wants to encode that in SDP, great. If it wants to attach
>>>>>>> it in JSON next to the browser generated SDP, that also
>>>>>>> works.
>>>>>>
>>>>>>
>>>>> Unfortunately however, it seems that we might have forgotten
>>>>> this decision. We are now trying hard to come up with a
>>>>> signalling mechanism that will do everything with
>>>>> Offer/Answer.
>>>>
>>>>
>>>> I think there is a misunderstanding here. I understand from
>>>> Emil's mail that media re-negotation or streams addition after
>>>> the first SDP O/A is not carried as a new full SDP O/A. Am I
>>>> right?
>>>>
>>>>
>>>
>>>
>>> I'm just trying to understand what the "No Plan" proposal is. I
>>> have not even started to think about what parts of if I like. So
>>> far, the more we talk about this, the more confused I get. I
>>> think some simple examples that showed what O/A got passed across
>>> the API from the JS into the browser for applications with
>>> multiple video steams would really help.
>>
>> I don't think I've seen any additional questions from you since
>> then.You are now saying that the situation is still unclear, so
>> please let me know if I can help in any way. Otherwise we could
>> also switch to whatever problems you have with the approach
>> itself.
>>
>> Cheers, Emil
>> -- https://jitsi.org

-- 
https://jitsi.org

---

From: Flemming Andreasen <[email-masked]>
Date: Thu, 06 Jun 2013 11:36:22 -0400

On 6/6/13 5:16 AM, Emil Ivov wrote:
> There have been contradictory comments from chairs that seem to say 
> how this should be in MMUSIC and how it actually doesn't belong to 
> MMUSIC and should happen on RTCWEB. I'd be grateful if chairs of both 
> WGs could sort this out and provide guidance.
>

Very fair request. This (MMUSIC) chair is confused too at this point as 
you can see on the MMUSIC list. The challenge I have is teasing apart 
the RTCWeb specifics (APIs, browser, RTCWeb specific scenarios, etc.) 
from the more general MMUSIC (SDP and O/A) considerations. The former 
should (IMO) be handled in RTCWeb and the latter should be done in 
MMUSIC, but the "Plan xyz" discussions don't always have a clean 
separation. I'll see if the chairs can come up with some guidelines here 
(but would welcome input from others of course)

Thanks

-- Flemming

---

From: Cullen Jennings <[email-masked]>
Date: Sat, 8 Jun 2013 11:55:25 +0700

On Jun 6, 2013, at 4:16 PM, Emil Ivov <[email-masked]> wrote:

> P.S. I am confused as to where this discussion should be happening. There have been contradictory comments from chairs that seem to say how this should be in MMUSIC and how it actually doesn't belong to MMUSIC and should happen on RTCWEB. I'd be grateful if chairs of both WGs could sort this out and provide guidance.

Given my participation in this conversation as an individual contributor, I'm a little hesitant to try and answer this from a chair role but let me try and help…

I think (or at least hope) the chairs have been pretty clear that the Plan A vs Plan B conversation about how SDP is used to represent multiple things should be in MMusic. It's not clear to me if this is part of that discussion or not. A question like "Should WebRTC use SDP" is clearly not in MMusic WG while a question like "how to we extend SDP to do X" is typically in MMUSIC. 

Hope that helps and I CC'd the other chairs in case they want to add anything.

Cullen <with my rtcweb chair hat on>

---

From: Cullen Jennings <[email-masked]>
Date: Sat, 8 Jun 2013 12:43:26 +0700

On Jun 6, 2013, at 2:14 PM, Silvia Pfeiffer <[email-masked]> wrote:

> browser using addStream()?

Yes, that makes sense. I was assuming all the streams would be on the same PeerConnection

> I have an application like that working. I

Uh, not sure that would help much. The issues is that current standards don't say what the browsers should do and we have two proposal on how the browsers could implement this. One proposal from Mozilla and one proposal from Google. I don't think either browser implements exactly the corresponding proposal yet but they are trying to figure out what to do. We need to agree on roughly what is passed across the JS API such that the browser know how to set up the media stack to send and receive the appropriate RTP that is desired by the JS application.

---

From: Peter Thatcher <[email-masked]>
Date: Mon, 17 Jun 2013 05:58:46 -0700

>From Cullen: "We need to agree on roughly what is passed across the JS API
such that the browser know how to set up the media stack to send and
receive the appropriate RTP that is desired by the JS application"

I think this hits the nail on the head.  Generally, we need to decide what
the JS API is to allow the JS app to control what is sent and received.
 Specifically for NoPlan, we need a JS API that does not rely on encoding
multiple sources in SDP and doesn't require extra SDP exchanges to
add/remove sources.

I'm glad you asked for such an API, because I was interested in working on
one.  So, I just sent an email with a proposed JS API, and Emil will be
updating the draft RFC to incorporate it.

To be clear, Google is in full support of Plan B, and wants to see the Plan
A vs. Plan B decision resolved.  But if Plan A vs. Plan B cannot be
decided, then we support NoPlan with this JS API as a backup plan, since
Plan B could be implemented on top of NoPlan within JS (and so could Plan
A).

On Fri, Jun 7, 2013 at 10:43 PM, Cullen Jennings <[email-masked]> wrote:

> >> I've trying to help you explain and I keep asking for an example but I
> don't think we have had a complete example yet. I know you think there is a
> complete example so let me try and be more specific so perhaps we can get
> to the bottom of the disconnect.
> >>
> >> Say an application running on Alice's browser want to generate an offer
> that says the browser is ready to send and receive 2 streams of video and 1
> stream of audio look like? Imagine that one of the video streams is a
> document camera running at high res but low frame rate while the other is
> video of the speaker running at a higher frame rate.  What exactly does the
> SDP passed from the browser to the JS applications look like.  I agree the
> applications can do whatever it wants to communicate the relevant data to
> the far end so I don't care about  the signaling protocol or JSON  that JS
> app can send across the wire. But next question, can the far end start
> sending media immediately to the browser? Finally the far end does
> something that causes the applications to generate an SDP answer from the
> JS applications to Alice's browser. I want and example of that that looks
> like in the cases where the far end a) accepted all the video streams and
> b) rejected some but not all of the
>   video streams.
> >>
> will be easy to extrapolate to the ones in the say the use case document
> and start looking at things like number of round trips and audio clipping.
> >
> >
> Uh, not sure that would help much. The issues is that current standards
> don't say what the browsers should do and we have two proposal on how the
> browsers could implement this. One proposal from Mozilla and one proposal
> from Google. I don't think either browser implements exactly the
> corresponding proposal yet but they are trying to figure out what to do. We
> need to agree on roughly what is passed across the JS API such that the
> browser know how to set up the media stack to send and receive the
> appropriate RTP that is desired by the JS application.
>
>
> >
> > Regards,
> > Silvia.
>