Subject: [rtcweb] How to multiplex between peers

From: Magnus Westerlund <[email-masked]>
Date: Tue, 19 Jul 2011 17:11:41 +0200

WG,

As individual contributor.

I would recommend that people read both
https://datatracker.ietf.org/doc/draft-rosenberg-rtcweb-rtpmux/?include_text=1
and
https://datatracker.ietf.org/doc/draft-perkins-rtcweb-rtp-usage/?include_text=1
before getting into this debate.

Assuming that the consensus of "To multiplex or not!" thread is either
of the options requiring multiplexing we get to the question of how to
do the multiplexing.

Lets start with a very fundamental question:

Explicit vs Implicit multiplexing

So the explicit is when something in the received packets has the
explicit purpose to identifies a context that resolves the protocol, the
session context etc.

Implicit is when we have no such explicit field, instead we have to rely
on that investigating values of particular bits in the payload of the
packet. For example trying to identify RTP based on the first byte of
the RTP header where the version two bits shall be 10b.

I am strongly arguing against implicit identifying the protocol for the
following reasons.

1. We have 4 or more protocols (The possible set of protocols include:
STUN, RTP/RTCP, Datagram, and DTLS-SRTP. There is likely future
extensions, like reliable protocol)  that needs to be identified and
demultiplexed. A single extension in one of these protocols can cause
the whole house of cards to come tumbling down.

2. It will require one to find an RTP internal session multiplexing
mechanism. This is likely not easy or causes a fork of RTP into two
non-compatible versions. In addition such a mechanism needs defining
which takes time, likely substantial time as there is quite a lot of RTP
mechanisms to consider.

3. We likely need a multiplexing layer anyway if we want a fall back
over WebSockets or HTTP.

Thus we see this a very fragile mechanism that creates additional work
overhead and large uncertainties in when it would be available.

We have in our draft:
https://datatracker.ietf.org/doc/draft-perkins-rtcweb-rtp-usage/ looked
at two possible suggestions for explicit multiplexing. Either a thin
shim layer primarily consisting of a identifier that through negotiation
is bound to a context, like an particular RTP session or a datagram
channel.

The other alternative we considered was using DCCP over UDP for RTP and
datagram. That way we got a complete port space for multiplexing
different purposes for flows. We also got a transport framework with its
own negotiation, congestion control, a several other features.

We invite people to consider these proposals or suggest better
alternatives of explicit multiplexing. But I hope we can avoid a serious
mistake and select implicit multiplexing.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: Matthew Kaufman <[email-masked]>
Date: Tue, 19 Jul 2011 08:39:39 -0700

> the whole house of cards to come tumbling down.

Multiplexing RTP, RTCP, and STUN on the same port already needs to keep 
working for lots of other reasons, we can leverage those reasons.

draft-kaufman-rtp-compatible-data suggests that the data service can be 
multiplexed with these as well, in a way that is protected by the same 
things that protect the above.

> which takes time, likely substantial time as there is quite a lot of RTP

There's one, SSRC. Since we already allow two or more audio streams to 
be multiplexed using SSRC, and two or more video streams to be 
multiplexed using SSRC, it really isn't a stretch to allow one audio and 
one video to be multiplexed using SSRC when the two ends agree that this 
works for them.

> 3. We likely need a multiplexing layer anyway if we want a fall back

I agree that we currently don't have good agreement of how a TCP 
fallback works for RTCWEB, but it is clearly necessary.

> overhead and large uncertainties in when it would be available.

Not any more fragile than RTP+RTCP+STUN on the same port already is.

> is bound to a context, like an particular RTP session or a datagram

Adding a shim layer causes two problems:
1. You need to be able to run the system in a way that has the shim 
layer removed when you want to do legacy interoperability.
2. Existing protocol stacks and middleboxes cannot be reused as easily.

> own negotiation, congestion control, a several other features.

Same problem, no matter what the shim layer is.

I've already considered explicit multiplexing. It has lots of 
advantages. But it also fails several of the tests laid out in the 
requirements, and delays implementation and adoption.

So I think we're down to implicit multiplexing, or no multiplexing at 
all. And when we run out of IPv4 NAT mappings, you'll understand why 
multiplexing should be in there.

Matthew Kaufman

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 19 Jul 2011 17:43:36 -0700

inline ...

> the whole house of cards to come tumbling down.

Uh, we already demux STUN, RTP/RTC, DTLS-SRTP based on first byte and this is no big deal and is sort of water under the bridge at this point. But this first byte multiplex issues is a red herring to this topic, what happens with the demux of these protocols does not really have much to do with how to demux the different RTP sessions. 

> which takes time, likely substantial time as there is quite a lot of RTP

A bunch of us our proposing SSRC. We not seeing any problems with it.  I rather having this discussion be concrete around the specific proposal on the table and not just some abstract it might be hard to find something. 

> 3. We likely need a multiplexing layer anyway if we want a fall back

Not really - I think this is best done by making another TURN like protocol that is compatible uses WebSockets or HTTP as the underling transport. An architecture like this means that  that protocol is selected unilateral by the endpoint and used to get through that endpoints  firewalls. Both clients don't have to agree to do the same thing - just like If I am using a TURN and you are not, we can still  have a call. Avoid a bilateral solution where possible make it much easier to roll out. 

> overhead and large uncertainties in when it would be available.

This all sounds sort of doom and gloom but lets talk about real problems. 

> is bound to a context, like an particular RTP session or a datagram

The problem with this is that if you are talking to a device that does not support this you forever have to go through some intermediary gateway that strips the byte. Just like turns servers, this reduces the user experience by adding latency and jitter and is expensive to run due to the bandwidth required. 

> own negotiation, congestion control, a several other features.

This has the same problem as above with the added problem of I am not aware of a good (meaning widely tested) DCCP implementation in user land and to be deployable this pretty much needs to be all user land. 

> mistake and select implicit multiplexing.

I'd like people to start talking about what the actual serious mistakes of implicit multiplexing as specified in the SSRC proposal are. Ok, Have I thrown the gauntlet down enough :-) Really, I'm just not getting what the problems are - if I understood the problems, I might have a different view point on this. 

Cullen <with my individual contributor hat on>

---

From: Emil Ivov <[email-masked]>
Date: Wed, 20 Jul 2011 13:08:55 +0200

На 20.07.11 02:43, Cullen Jennings написа:
>>> We have in our draft: 
>>> https://datatracker.ietf.org/doc/draft-perkins-rtcweb-rtp-usage/
>>> looked at two possible suggestions for explicit multiplexing.
>>> Either a thin shim layer primarily consisting of a identifier
>>> that through negotiation is bound to a context, like an
>>> particular RTP session or a datagram channel.
>
> The problem with this is that if you are talking to a device that
> does not support this you forever have to go through some
> intermediary gateway that strips the byte. Just like turns servers,
> this reduces the user experience by adding latency and jitter and is
> expensive to run due to the bandwidth required.

How is this different from implicit multiplexing and talking to devices
that don't support it?

Emil

-- 
http://jitsi.org

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 20 Jul 2011 14:10:55 +0200

As an Individual

Follow up inline

On 2011-07-20 02:43, Cullen Jennings wrote:
>> 1. We have 4 or more protocols (The possible set of protocols
>> include: STUN, RTP/RTCP, Datagram, and DTLS-SRTP. There is likely
>> future extensions, like reliable protocol)  that needs to be
>> identified and demultiplexed. A single extension in one of these
>> protocols can cause the whole house of cards to come tumbling
>> down.
> 
> Uh, we already demux STUN, RTP/RTC, DTLS-SRTP based on first byte and
> this is no big deal and is sort of water under the bridge at this
> point. But this first byte multiplex issues is a red herring to this
> topic, what happens with the demux of these protocols does not really
> have much to do with how to demux the different RTP sessions.

I agree that we currently have a design that multiplex the three above
protocols onto the same lower layer when one uses ICE and DTSL-SRTP.
However, the proposal in the WG is pilling on to this, but suggesting to
add at least one protocol to the mix, maybe more if we do reliable in
the future. The more independent protocols there is in the mix the
bigger the risk is that we get a failure in the demultiplexing.

>From my perspective this is not a red-herring at all. It is about good
and future proof design.

Both this issue and the RTP session multiplexing needs to be considered.

> 
>> 
>> 2. It will require one to find an RTP internal session
>> multiplexing mechanism. This is likely not easy or causes a fork of
>> RTP into two non-compatible versions. In addition such a mechanism
>> needs defining which takes time, likely substantial time as there
>> is quite a lot of RTP mechanisms to consider.
> 
> A bunch of us our proposing SSRC. We not seeing any problems with it.
> I rather having this discussion be concrete around the specific
> proposal on the table and not just some abstract it might be hard to
> find something.

The proposal on the table is to change the semantics of the SSRC field.
So that it contains one part that is session identifier and another that
is stream identifier. This have substantial impact on the RTP/RTCP
specification and the implementation. To the degree where what you have
is not any longer compatible with RFC 3550. It looks like RTP/RTCP but
it isn't any more.

We have documented the impacts of your proposal in Sections 2.2.1 and
3.3.1 of
https://datatracker.ietf.org/doc/draft-perkins-rtcweb-rtp-usage/?include_text=1

So from my perspective the proposal on the table is a complete no-go.
And I have been thinking trying to find a proposal that wouldn't result
in all these issues and still be internal to RTP. And I haven't found
one yet. That is why I flag this.

If there is a solution that doesn't have as serious impact I do want
AVTCORE to standardize it.

> 
>> 
>> 3. We likely need a multiplexing layer anyway if we want a fall
>> back over WebSockets or HTTP.
> 
> Not really - I think this is best done by making another TURN like
> protocol that is compatible uses WebSockets or HTTP as the underling
> transport. An architecture like this means that  that protocol is
> selected unilateral by the endpoint and used to get through that
> endpoints  firewalls. Both clients don't have to agree to do the same
> thing - just like If I am using a TURN and you are not, we can still
> have a call. Avoid a bilateral solution where possible make it much
> easier to roll out.

Well, it still needs to put out compatible flows. And if we have a
multiplexing layer that becomes equally easy.

> 
>> 
>> Thus we see this a very fragile mechanism that creates additional
>> work overhead and large uncertainties in when it would be
>> available.
> 

Timing is a real problem. If we want to do multiplexing in RTCWEB 1.0 we
need to pick what is ready and available. We can't go around proposing
new things that are complicated. Changing the SSRC field semantics in
RTP/RTCP is a complicated thing.

We will have tons of discussion of what the requirements really are for
a more generally applicable solution. Not one that barely meets RTCWEBs
needs, when we know the minute this goes out the door a lot of other RTP
using services and applications would like to use it. The teleprecense
work in CLUE is just one example. There will be a bunch. Thus AVTCORE
needs to have a serious discussion of what the requirements are. Then we
will discuss what the architectural implications are of any proposal.
Review what existing functionality we are ready to break and likely need
to replace. What the issues are with gatewaying between old and new
semantics.

Cullen, you are an experienced IETF, you have been an AD for RAI. What
do think the probability is that this discussion has concluded and
resulted in an IESG approved specification before December 2012?

>> Either a thin shim layer primarily consisting of a identifier that
>> through negotiation is bound to a context, like an particular RTP
>> session or a datagram channel.
> 

Isn't that gateway going to be present anyway in most case. I think the
ICE connectivity checks are one thing that will force a gateway in many
cases. In the cases it isn't needed, then you can skip the multiplexing
in that case to reach the legacy.

Would it not be better to have a clean explicit design for how to
multiplex protocols between peers that can be used in RTCWEB and other
future applications that doesn't drag our past mistakes into the design
and forever prevent simple expansion with new functionality.

The cost, a low one from my perspective is to say that legacy don't get
the benefits of multiplexing. A feature they anyway are not going to get
unless they use a gateway or are updated to be RTCWEB compliant.

> 
>> 
>> The other alternative we considered was using DCCP over UDP for RTP
>> and datagram. That way we got a complete port space for
>> multiplexing different purposes for flows. We also got a transport
>> framework with its own negotiation, congestion control, a several
>> other features.
> 
> This has the same problem as above with the added problem of I am not
> aware of a good (meaning widely tested) DCCP implementation in user
> land and to be deployable this pretty much needs to be all user land.

Yes, I do agree that it needs to be a user-land implementation in the
browser until DCCP becomes available in the OSes. But that is true also
for all the alternatives that so far been discussed. TFRC in RTP will be
as much user-land as DCCP over UDP with TFRC.

The big difference is that we either have a well reviewed and by the
community and IESG approved specification in DCCP. Or we create
something our selves that needs to go through all that review and will
be even less tested than DCCP is.

>> alternatives of explicit multiplexing. But I hope we can avoid a
>> serious mistake and select implicit multiplexing.
> 
> I'd like people to start talking about what the actual serious
> mistakes of implicit multiplexing as specified in the SSRC proposal
> are. Ok, Have I thrown the gauntlet down enough :-) Really, I'm just
> not getting what the problems are - if I understood the problems, I
> might have a different view point on this.
> 

Please do read the latest version of
https://datatracker.ietf.org/doc/draft-perkins-rtcweb-rtp-usage/?include_text=1

We have expanded on the discussion, both about the particular proposal
and the general issues with multiplexing is RTP.

If you can give some hint what in that argumentation you don't
understand we can try to explain in some other way.

Cheers

Magnus Westerlund

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 20 Jul 2011 07:11:55 -0700

> How is this different from implicit multiplexing and talking to devices

Sorry, I should have been clear that I think the multiplexing has to be negotiated with SDP

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 20 Jul 2011 07:18:29 -0700

> RTP/RTCP is a complicated thing.

[BA] We need to ask ourselves "Does this really need to be in the critical path for RTCWEB 1.0"?  
As I understand it, this would be a negotiated feature, and presumably all implementations
would be required to support RTP/RTCP as defined in RFC 3550 and suitable profiles. 
So from my perspective, I'd prefer to see this handled as an extension on its own path, not a 
mandatory-to-implement feature in 1.0.   

> in that case to reach the legacy.

[BA] Just because we're likely to require a gateway doesn't mean we should
heap more and more things on the camel's back.  We're already talking about
ICE, shims, transcoding, changes to RFC 3550....  that's a lot of functionality
for the "gateway".  So I agree with Jonathan about the disadvantages of a shim
(or DCCP for that matter). 

> be even less tested than DCCP is.

[BA] Deployment counts, too.  How widely deployed is DCCP for realtime uses?

---

From: Justin Uberti <[email-masked]>
Date: Wed, 20 Jul 2011 15:46:26 -0400

> mandatory-to-implement feature in 1.0.

This is where I've ended up on this topic. We can easily multiplex multiple
RTP sources (of the same type) over a single RTP session using SSRC. We can
also mux RTCP over the RTP session using RTCP mux. So, for an arbitrary
video call, we have just 2 RTP sessions/NAT bindings.

Is it worth going the extra mile to get down to 1 in v1.0, given the lack of
consensus that exists right now? Is there even a compelling argument to do
so?

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>
>

---

From: Matthew Kaufman <[email-masked]>
Date: Wed, 20 Jul 2011 12:51:12 -0700

On Jul 20, 2011, at 12:46 PM, Justin Uberti wrote:

> 
> This is where I've ended up on this topic. We can easily multiplex multiple RTP sources (of the same type) over a single RTP session using SSRC. We can also mux RTCP over the RTP session using RTCP mux. So, for an arbitrary video call, we have just 2 RTP sessions/NAT bindings. 
> 
> Is it worth going the extra mile to get down to 1 in v1.0, given the lack of consensus that exists right now? Is there even a compelling argument to do so?

Yes and yes.

I really can't understand why, if we can multiplex 3 totally different types of video streams over the same RTP session using SSRC (with wildly different bit-rates, inter-packet times, etc.) we can't also multiplex audio and video. Not a single argument that has been presented has convinced me, and getting from 2 to 1 is a *50%* reduction in NAT port utilization. (And a significant reduction in the number of "strange" failure modes, where one traversal worked and the other didn't.)

Matthew Kaufman

---

From: Justin Uberti <[email-masked]>
Date: Wed, 20 Jul 2011 15:59:18 -0400

On Wed, Jul 20, 2011 at 3:51 PM, Matthew Kaufman
<[email-masked]>wrote:

> > This is where I've ended up on this topic. We can easily multiplex
> multiple RTP sources (of the same type) over a single RTP session using
> SSRC. We can also mux RTCP over the RTP session using RTCP mux. So, for an
> arbitrary video call, we have just 2 RTP sessions/NAT bindings.
> >
> > Is it worth going the extra mile to get down to 1 in v1.0, given the lack
> of consensus that exists right now? Is there even a compelling argument to
> do so?
>
> Yes and yes.
>
> I really can't understand why, if we can multiplex 3 totally different
> types of video streams over the same RTP session using SSRC (with wildly
> different bit-rates, inter-packet times, etc.) we can't also multiplex audio
> and video. Not a single argument that has been presented has convinced me,
> and getting from 2 to 1 is a *50%* reduction in NAT port utilization. (And a
> significant reduction in the number of "strange" failure modes, where one
> traversal worked and the other didn't.)
>

Clearly we can, but we need to do additional work to get there. This work
will take time and delay v1.0, so we need a pretty convincing rationale to
do so.

I concur that solving strange failure modes is significant. But with regard
to NAT port utilization, this ignores the utilization by the browser for
HTTP; 50% reduction seems possible in theory only. (Here I am essentially
restating 2.2.2 from
https://datatracker.ietf.org/doc/draft-perkins-rtcweb-rtp-usage/?include_text=1,
which I haven't yet seen a clear response to.)

---

From: Bala Pitchandi <[email-masked]>
Date: Wed, 20 Jul 2011 16:57:27 -0400

+1 for sending audio & video (and any other type of media, for that matter) on a single RTP session. With RTCP mux, that means all media can be sent on a single UDP port. 

-- Bala

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Matthew Kaufman
Sent: Wednesday, July 20, 2011 3:51 PM
To: Justin Uberti
Cc: [email-masked]
Subject: Re: [rtcweb] How to multiplex between peers

Yes and yes.

Matthew Kaufman

---

From: Justin Uberti <[email-masked]>
Date: Wed, 20 Jul 2011 17:15:31 -0400

On Wed, Jul 20, 2011 at 3:59 PM, Justin Uberti <[email-masked]> wrote:

>
>
> On Wed, Jul 20, 2011 at 3:51 PM, Matthew Kaufman <
> [email-masked]> wrote:
>
>>
>> > Is it worth going the extra mile to get down to 1 in v1.0, given the
>> lack of consensus that exists right now? Is there even a compelling argument
>> to do so?
>>
>> Yes and yes.
>>

Also: when falling back to send media over TCP/WebSockets, a lost packet may
cause a momentary audio dropout as the transport stalls while the packet is
retransmitted. I expect that this will occur more frequently if video and
audio are combined on the same TCP connection, if for no other reason than
the much higher packet rate.

---

From: Aron Rosenberg <[email-masked]>
Date: Wed, 20 Jul 2011 14:15:57 -0700

Another +1 for multiplexing everything together. My only worry is that the
base spec for RTCP doesn't mandate that SSRC is included in every RTCP
message type, and thus you could end up with the multiplexing layer needing
to understand every  RTCP messaged used for video (TMMBR, TMMBN, etc)

---

From: Colin Perkins <[email-masked]>
Date: Wed, 20 Jul 2011 23:31:12 +0100

Except that it doesn't work in all cases: please read the discussion in draft-perkins-rtpweb-rtp-usage-02. You may argue that you don't think the breakage is important, but there *is* breakage in doing this, and it is not backwards compatible.

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Wed, 20 Jul 2011 23:31:56 +0100

On 20 Jul 2011, at 22:15, Aron Rosenberg wrote:
> Another +1 for multiplexing everything together. My only worry is that the base spec for RTCP doesn't mandate that SSRC is included in every RTCP message type, and thus you could end up with the multiplexing layer needing to understand every  RTCP messaged used for video (TMMBR, TMMBN, etc)

...which is another reason why this doesn't work in all cases, in addition to the reasons we document in draft-perkins-rtcweb-rtp-usage-02

-- 
Colin Perkins
http://csperkins.org/

---

From: Matthew Kaufman <[email-masked]>
Date: Wed, 20 Jul 2011 15:38:56 -0700

On 7/20/2011 3:31 PM, Colin Perkins wrote:
>> Another +1 for multiplexing everything together. My only worry is 
>> that the base spec for RTCP doesn't mandate that SSRC is included in 
>> every RTCP message type, and thus you could end up with the 
>> multiplexing layer needing to understand every  RTCP messaged used 
>> for video (TMMBR, TMMBN, etc)
>
> ...which is another reason why this doesn't work in all cases, in 
> addition to the reasons we document in draft-perkins-rtcweb-rtp-usage-02
>

So let me get this straight... RTP and RTCP is broken and inflexible and 
therefore we must use it, and use it in the broken and inflexible way?

If anything this is an argument for ignoring RTP and RTCP and doing 
something entirely new that is actually appropriate for what we're 
trying to build, not living with crap just because there's an RFC for it.

Matthew Kaufman

---

From: Colin Perkins <[email-masked]>
Date: Wed, 20 Jul 2011 23:48:16 +0100

On 20 Jul 2011, at 23:38, Matthew Kaufman wrote:
> So let me get this straight... RTP and RTCP is broken and inflexible and therefore we must use it, and use it in the broken and inflexible way?
> 
> If anything this is an argument for ignoring RTP and RTCP and doing something entirely new that is actually appropriate for what we're trying to build, not living with crap just because there's an RFC for it.

You're free to re-invent that wheel, if you wish. However, if you want to use RTP for RTCWeb, then you need to accept that doing so has both some advantages and some disadvantages. One of those disadvantages, for this application domain, is that it requires an explicit demultiplexing layer for distinct RTP sessions. Whether that demultiplexing layer is UDP, or some yet-to-be defined shim layer is a question for this working group, but something is needed if RTP is used.

-- 
Colin Perkins
http://csperkins.org/

---

From: Matthew Kaufman <[email-masked]>
Date: Wed, 20 Jul 2011 15:50:06 -0700

On Jul 20, 2011, at 3:48 PM, Colin Perkins wrote:

> 
> 
> You're free to re-invent that wheel, if you wish. However, if you want to use RTP for RTCWeb, then you need to accept that doing so has both some advantages and some disadvantages.

I believe we've managed to demonstrate that the disadvantages are sufficient to argue against using RTP for RTCWeb. Too bad, because it seems otherwise like such a good choice.

Matthew Kaufman

---

From: Matthew Kaufman <[email-masked]>
Date: Wed, 20 Jul 2011 16:15:01 -0700

> Except that it doesn't work in all cases: please read the discussion in draft-perkins-rtpweb-rtp-usage-02. You may argue that you don't think the breakage is important, but there *is* breakage in d…

It doesn't need to be "backwards compatible" in order for two RTCWEB endpoints to communicate. And the muxing can be disabled in order to talk to a legacy endpoint.

Could we confine the claims about breakage to only ones that are NOT backwards compatibility issues?

Matthew Kaufman

---

From: Jonathan Lennox <[email-masked]>
Date: Wed, 20 Jul 2011 19:24:10 -0400

> Except that it doesn't work in all cases: please read the discussion in draft-perkins-rtpweb-rtp-usage-02. You may argue that you don't think the breakage is important, but there *is* breakage in d…

Hi, Colin,

I've read draft-perkins-rtcweb-rtp-usage-02, and I still don't understand what you're claiming SSRC-muxing would break.

Running audio and video (and FEC, and RTX, and simulcast video, and Real-Time Text, and etc. etc.) in a single RTP session does, not, as far as I've been able to tell (after a lot of thought), break anything in RTP.

To be sure, it's not backward compatible with RTP endpoints that assume a single source per session.  It would also be hard to retrofit SDP to describe and negotiate it.  And for some use cases it requires some source-level signaling, as in RFC 5576.  But the RTP model, per se, is perfectly amenable to it, as far as I can tell.

I do, very much, dislike the proposal in draft-rosenberg-rtcweb-rtpmux of encoding a magic number into the RTP SSRC -- I think that *does* break a fair amount of the RTP model.  But this isn't an essential feature of that proposal, as far as I can tell; it's just a convenience for deep packet inspectors.  (Deep packet inspectors already have to use heuristics to distinguish audio vs. video RTP flows today, if they can't see signaling; and those that *can* see signaling can just use payload types to distinguish media types in the mux'ed case.)

--
Jonathan Lennox
[email-masked]

---

From: Henry Sinnreich <[email-masked]>
Date: Wed, 20 Jul 2011 19:41:38 -0500

+1

> trying to build, not living with crap just because there's an RFC for it.

Not to forget disposing of ancient SDP as well.
Use standard metadata instead, since it is equally usable for all apps, not
only for RTC apps.

Thanks, Henry

---

From: Dzonatas Sol <[email-masked]>
Date: Wed, 20 Jul 2011 19:21:30 -0700

> I believe we've managed to demonstrate that the disadvantages are sufficient to argue against using RTP for RTCWeb. Too bad, because it seems otherwise like such a good choice.

If we digress to rhinophytophilia use-cases, which the step-by-step 
process exploits good flows in breath analyzers, in RTP sense, then I 
think this would be the focus to weigh advantages and disadvantages. Not 
everybody breaths syllables into the net online, imagine some terrified 
already of such analytics. Maybe someday we can predict this kind of 
discussion and prepare for the colossal mix of schemes, yet for now 
there is the coined bit of "no a/s/l". Sane? Certain safety.

-- 
--- http://twitter.com/Dzonatas_Sol ---
Web Development, Software Engineering
Ag-Biotech, Virtual Reality, Consultant

---

From: Igor Faynberg <[email-masked]>
Date: Wed, 20 Jul 2011 23:34:08 -0400

Maybe, at some point.

At the moment though I am not convinced that this is the right answer.  
It always looks easier to re-design everything, but this approach more 
often than not ends up in a standard (and implementations) full of 
bugs.  Looking at what has transpired in Transport (now RAI) in the past 
15 years, one would easily find examples.

I think the decision on whether to ignore backward compatibility ought 
to be made when there is an agreement on core use cases and it is 
understood that none of them would be broken by lack of backward 
compatibility.

(It is not at all that I am against re-design on principle.  Quite the 
opposite!  But I believe that re-design is the research issue; here, we 
write a standard, and a standard needs to take into account a plethora 
of issues that the research phase may and should ignore.)

Igor

On 7/20/2011 7:15 PM, Matthew Kaufman wrote:
> ...
> ...

---

From: Matthew Kaufman <[email-masked]>
Date: Wed, 20 Jul 2011 21:06:07 -0700

> understood that none of them would be broken by lack of backward 

It is trivial to demonstrate that an RTCWEB endpoint talking to another 
RTCWEB endpoint has no issues with backward compatibility.

The deceptive conflation of (possible) real issues with using SSRC as a 
multiplexing point for A+V over RTP for RTCWEB to RTCWEB and the class 
of issues that are ONLY relevant when talking RTCWEB to something that 
requires backwards compatibility is taking us farther, rather than 
closer, to a resolution on this issue. Once we agree to RTCWEB to Legacy 
will not use SSRC for multiplexing, there is no need, whatsoever, to 
bring up the potential backwards compatibility issues again.

Matthew Kaufman

---

From: "Roni Even" <[email-masked]>
Date: Thu, 21 Jul 2011 08:09:54 +0300

Hi,

I am not sure about your claim. The SSRC of the media sender is always there
, otherwise how can RTCP report or request action on a media stream. It is
true that for some of the RTCP messages like TMMBR it is inside the FCI part
of the message but it is there. As for understanding all RTCP messages, it
is not required today and the support is negotiated in SDP. 

For video streams I assume that RTCweb will mandate the support of controls
like FIR and TMBBR.

Regards

Roni Even

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Aron Rosenberg
Sent: Thursday, July 21, 2011 12:16 AM
To: [email-masked]
Subject: Re: [rtcweb] How to multiplex between peers

---

From: Colin Perkins <[email-masked]>
Date: Thu, 21 Jul 2011 07:51:55 +0100

On 21 Jul 2011, at 00:24, Jonathan Lennox wrote:
...
> To be sure, it's not backward compatible with RTP endpoints that assume a single source per session.  It would also be hard to retrofit SDP to describe and negotiate it.  And for some use cases it…

Section 2.2.1 of draft-perkins-rtcweb-rtp-usage-02 lists several problems that occur when multiplexing RTP sessions onto a single lower-level flow. 

There are clearly some limitations caused: separate endpoints, quality of service, limitations with mixers and translators, retransmission, FEC, sampling group membership, and so on. There is also some breakage when several sessions are multiplexed: RTCP is problematic with several clock rates per session, SSRC allocation is broken, and the RTCP reporting interval and timing rules are broken.

You may not think some of these are important. Some can be avoided by the use of appropriate signalling. Some can be mitigated by allowing multiple different media types in an RTP session, or by splitting the SSRC space. I'm not arguing that there aren't ways to fix this problem, or ignore it in some cases, but they cause various degrees of change to RTP, and need to be carefully evaluated by AVTCORE if they're to be done. 

-- 
Colin Perkins
http://csperkins.org/

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 21 Jul 2011 00:43:10 -0700

Hi Colin,

On 7.20.2011 23:51 , "Colin Perkins" <[email-masked]> wrote:

>> I've read draft-perkins-rtcweb-rtp-usage-02, and I still don't
>>understand what you're claiming SSRC-muxing would break.
>> 
>> Running audio and video (and FEC, and RTX, and simulcast video, and
>>Real-Time Text, and etc. etc.) in a single RTP session does, not, as far
>>as I've been able to tell (after a lot of thought), break anything in
>>RTP.
>> 
>> To be sure, it's not backward compatible with RTP endpoints that assume
>>a single source per session.  It would also be hard to retrofit SDP to
>>describe and negotiate it.  And for some use cases it requires some
>>source-level signaling, as in RFC 5576.  But the RTP model, per se, is
>>perfectly amenable to it, as far as I can tell.
>
>
>Section 2.2.1 of draft-perkins-rtcweb-rtp-usage-02 lists several problems
>that occur when multiplexing RTP sessions onto a single lower-level flow.
>
>There are clearly some limitations caused: separate endpoints, quality of
>service, limitations with mixers and translators, retransmission, FEC,
>sampling group membership, and so on. There is also some breakage when
>several sessions are multiplexed: RTCP is problematic with several clock
>rates per session, SSRC allocation is broken, and the RTCP reporting
>interval and timing rules are broken.
>
>You may not think some of these are important. Some can be avoided by the
>use of appropriate signalling. Some can be mitigated by allowing multiple
>different media types in an RTP session, or by splitting the SSRC space.
>I'm not arguing that there aren't ways to fix this problem, or ignore it
>in some cases, but they cause various degrees of change to RTP, and need
>to be carefully evaluated by AVTCORE if they're to be done.

Rtcweb is working towards what I would call a "system"
specification--something, the IETF (or at least RAI) is not usually
involved in.  Outside the IETF, it's not uncommon that "system"
specifications effectively modify an underlying base specification--RTP in
this case.  Such modifications can as easily remove functionality deemed
unnecessary as add new one.

Speaking about session multiplexing as an example, I would guess that a
lot of people here are implicitly arguing removal of functionality of RTP
not needed for the application--namely multicast support.  SSRC
(re-)allocation) may be broken, but since there should not be any need for
it (no multicast), who cares?  RTCP reporting intervals and timing rules
are broken: again, who cares if the RTP/RTCP communication relationship is
either endpoint-endpoint or endpoint-MCU (and RTCP is not being forwarded
to all endpoints)?  RTCP-Terminating MCU and not Mixer in RFC 5117
namespace?  We will not see network flooding or something with RTCP RRs in
point-to-point RTP communication relationships... I have not thought about
RTCP's issues with multiple clock rates, but those difficulties are
probably not insurmountable, either, because, while there may be multiple
clock rates, those ought to be a finite, and small, set.

Now to my key point: you argue these things need to be "carefully
evaluated" in AVTCORE.
IMO, while it may be desirable for AVTCORE experts to chip in (as they do
on this very thread already:-), it is by no means necessary.  Neither
formally, not practically.  And certainly AVTCORE should not assume a
"blocking" position.  This puts IETF system level specification at a
disadvantage to system level specification work elsewhere (where it
happens on a daily basis, see ITU, see 3GPP), and where no one asks for
AVTCORE's permission to re-use RTP data structures without re-using all of
RTP.  Tinkering with the multicast functionally of RTP is not new, there
is a lot of implementation experience in the field, and for once we can
indeed rely on "running code" answers.

I advocate pragmatism.
(And, heretic as I am, in the long term we should think about RTPv3,
dumping multicast.  Have I said this before? :-)

Regards,
Stephan

P.s.: I haven't though much about shim layers or other techniques that
multiplex RTP sessions on a single low level flow.  If any of these ideas
were implemented only to preserve RTP functionalities not needed for the
application in question, then I would argue that being a silly idea.

---

From: Colin Perkins <[email-masked]>
Date: Thu, 21 Jul 2011 09:25:07 +0100

On 21 Jul 2011, at 08:43, Stephan Wenger wrote:
> Rtcweb is working towards what I would call a "system" specification--something, the IETF (or at least RAI) is not usually involved in.  Outside the IETF, it's not uncommon that "system" specifications effectively modify an underlying base specification--RTP in this case.  Such modifications can as easily remove functionality deemed unnecessary as add new one.
> 
> Speaking about session multiplexing as an example, I would guess that a lot of people here are implicitly arguing removal of functionality of RTP not needed for the application--namely multicast support.  SSRC (re-)allocation) may be broken, but since there should not be any need for it (no multicast), who cares?  RTCP reporting intervals and timing rules are broken: again, who cares if the RTP/RTCP communication relationship is either endpoint-endpoint or endpoint-MCU (and RTCP is not being forwarded to all endpoints)?  RTCP-Terminating MCU and not Mixer in RFC 5117 namespace?  We will not see network flooding or something with RTCP RRs in point-to-point RTP communication relationships... I have not thought about RTCP's issues with multiple clock rates, but those difficulties are probably not insurmountable, either, because, while there may be multiple clock rates, those ought to be a finite, and small, set.

I don't think anyone is arguing that there difficulties cannot be solved. Of course they can. What's being argued is that the result is different to RTP, is not backwards compatible, and is difficult to gateway into a standard RTP environment. 

Some of us believe that there's benefit to being able to reuse the RTP infrastructure, and to easily interoperate with existing systems. I do not believe the benefit of saving a small number of UDP ports, in a browser environment that regularly uses dozens of TCP ports, outweighs the loss of compatibility with other systems.

> Now to my key point: you argue these things need to be "carefully evaluated" in AVTCORE. IMO, while it may be desirable for AVTCORE experts to chip in (as they do on this very thread already:-), it is by no means necessary.  Neither formally, not practically.  And certainly AVTCORE should not assume a "blocking" position.  This puts IETF system level specification at a disadvantage to system level specification work elsewhere (where it happens on a daily basis, see ITU, see 3GPP), and where no one asks for AVTCORE's permission to re-use RTP data structures without re-using all of RTP.  Tinkering with the multicast functionally of RTP is not new, there is a lot of implementation experience in the field, and for once we can indeed rely on "running code" answers.
> 
> I advocate pragmatism. (And, heretic as I am, in the long term we should think about RTPv3, dumping multicast.  Have I said this before? :-)

You have, of course. Many people have. What's not clear to be is how you would remove multicast support from RTP without also removing multiparty support? I see lots of RTP features that are there to support group communication, very few that are specific to multicast.

And, besides, I can certainly envisage scenarios where one might want a browser-based RTP implementation to watch the multicast IPTV flows that ISPs are now offering. 

Colin

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 21 Jul 2011 11:16:30 +0200

> Use standard metadata instead, since it is equally usable for all apps, not
Henry, which standard?
http://www.xkcd.com/927/ applies to metadata too, I think.

> Thanks, Henry
>
>
> On 7/20/11 5:38 PM, "Matthew Kaufman"<[email-masked]>  wrote:
>

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 21 Jul 2011 02:01:31 -0700

Hi Colin,

On 7.21.2011 01:25 , "Colin Perkins" <[email-masked]> wrote:

>[...]
>>evaluated" in AVTCORE. IMO, while it may be desirable for AVTCORE
>>experts to chip in (as they do on this very thread already:-), it is by
>>no means necessary.  Neither formally, not practically.  And certainly
>>AVTCORE should not assume a "blocking" position.  This puts IETF system
>>level specification at a disadvantage to system level specification work
>>elsewhere (where it happens on a daily basis, see ITU, see 3GPP), and
>>where no one asks for AVTCORE's permission to re-use RTP data structures
>>without re-using all of RTP.  Tinkering with the multicast functionally
>>of RTP is not new, there is a lot of implementation experience in the
>>field, and for once we can indeed rely on "running code" answers.
>> 
>> I advocate pragmatism. (And, heretic as I am, in the long term we
>>should think about RTPv3, dumping multicast.  Have I said this before?
>>:-)
>
>You have, of course. Many people have. What's not clear to be is how you
>would remove multicast support from RTP without also removing multiparty
>support? I see lots of RTP features that are there to support group
>communication, very few that are specific to multicast.

Remove multiparty support, then.  Terminology glitch on my side.

>
>And, besides, I can certainly envisage scenarios where one might want a
>browser-based RTP implementation to watch the multicast IPTV flows that
>ISPs are now offering.

There is no reason why RTPv2 and RTPv3 couldn't co-exist, in a browser or
elsewhere.  They do, today, if you have a videoconferencing software and
an IPTV software on the same box.

It would be stupid to deprecate RTPv2 including multiparty support; no one
I know is arguing in favor of that.

Problem is, AVT (and, admittedly, the industry at large) did not act on
the RTPv3 idea.  Instead, application spec profiling continued with many
(dozens?  I lost count a long time ago) of non-interoperabile solutions
for the optimized-for-point-to-point RTP space.  (To a certain extent that
even happens in the multiparty case with different IPTV standards.) We are
about to create one more P2P RTP flavor here.  That's sad, but (I fear)
now unavoidable.  From what I gather, people don't think we can wait half
a decade for an RTPv3 proposal to make it through an AVT-to-be-named WG.

Stephan

---

From: "DRAGE, Keith (Keith)" <[email-masked]>
Date: Thu, 21 Jul 2011 12:44:38 +0200

This discussion seems to be hovering in some sort of middle ground.

Either 1)

It has not decided what transport to use for real time data, in which case it should document the requirements for such a transport protocol, and then evaluate whether existing solutions meet those requirements, versus developing an entirely new protocol.

Or 2)

I has decided that RTP should be used, in which case it should identify the shortcomings (i.e. a set of requirements on how it needs to be further developed) and send these to the AVTCORE working group.

Neither of those seem to be happening at the moment.

Regards

Keith

> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Stephan Wenger
> Sent: 21 July 2011 08:43
> To: Colin Perkins; Jonathan Lennox
> Cc: [email-masked]

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 21 Jul 2011 13:05:30 +0200

> I has decided that RTP should be used, in which case it should identify the shortcomings (i.e. a set of requirements on how it needs to be further developed) and send these to the AVTCORE working g…
Don't forget

3)

It has decided that RTP should be used, with no changes and no new 
extensions, and is going through the ritual of swearing at the resulting 
shortcomings while figuring out which parts of RTP it can say "we don't 
need them, so it's OK to do things that mean we can't use them" to.

If this group gets lined up behind an 1-year dependency on another group 
before it can deliver its initial work products, this group is going to 
be irrelevant.

               Harald

---

From: "DRAGE, Keith (Keith)" <[email-masked]>
Date: Thu, 21 Jul 2011 13:56:00 +0200

Agreed. 3 options. I'd rephrase the third as package only what is available amongst existing protocol, and document what it doesn't do as a result.

But the current discussion is still in middle ground between all of them with no clear direction forward as to which it wants to do.

Regards

Keith

> Of Harald Alvestrand
> Sent: 21 July 2011 12:06
> > It has not decided what transport to use for real time data, in which
> case it should document the requirements for such a transport protocol,
> and then evaluate whether existing solutions meet those requirements,
> versus developing an entirely new protocol.
> >
> > Or 2)
> >
> > I has decided that RTP should be used, in which case it should identify
> the shortcomings (i.e. a set of requirements on how it needs to be further
> developed) and send these to the AVTCORE working group.
> Don't forget
> 
> 3)
>

---

From: Dzonatas Sol <[email-masked]>
Date: Thu, 21 Jul 2011 07:28:05 -0700

Yes, and note that the issue experienced is more wide-spread than this 
WG. This is clearly evident when protocols attempt to sustain stateful 
connections and virtualization later proves false security. Keep this in 
mind before thinking to toss RTP and UDP.

I can't even uphold SSL as sustainable anymore.

--

---

From: Henry Sinnreich <[email-masked]>
Date: Thu, 21 Jul 2011 10:36:22 -0500

> Henry, which standard?

Thanks for the good question Harald!

A good start for metadata standards to replace SDP and thus applicable
across any web apps is

http://dublincore.org/metadata-basics/

Such metadata is actually implemented in successful RAI products, but I
would rather not go there on this post.

Henry

---

From: Jonathan Lennox <[email-masked]>
Date: Thu, 21 Jul 2011 11:59:29 -0400

> You may not think some of these are important. Some can be avoided by the use of appropriate signalling. Some can be mitigated by allowing multiple different media types in an RTP session, or by sp…

I absolutely agree that multiplexing several RTP sessions into a single lower-level flow could break things, but that's not what I'm advocating.  Instead, I'm only advocating relaxing the rule that every source in a *single* RTP session -- a session which follows all the RFC 3550 rules -- must have the same top-level media type.

Since RTP was defined to allow up to 2^32-1 distinct sources in a session, all the rules of RFC 3550 seem to follow naturally, as do other RTP-family specs except those whose designers forgot to take RTP's group-communication nature into account.

It may be useful to relax *more* rules, for the topological cases that Stephan mentioned (endpoint-endpoint or RTCP-terminating-MCU) -- particularly since a lot of RTP's (or specifically, RTCP's) existing timing rules don't make a lot of sense in scenarios where you have widely asymmetric bandwidths.  This might need to formally be a new RTP profile, though I'd rather avoid that if possible.  But the starting point should be, I think, assuming a single RTP session with arbitrarily many distinct sources, and see if the RFC 3550 rules lead us somewhere bad *in that scenario*.

--
Jonathan Lennox
[email-masked]

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 22 Jul 2011 15:17:51 +0200

> Remove multiparty support, then.  Terminology glitch on my side.

So you argue for removing multi-party support the fact that RTCWEB have
clear multi-party use cases in our use case document? Even you make
arguments in your layered coding pitch using functions which is based on
the multi-party functions of RTP. So I don't see how that match up.

Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 22 Jul 2011 15:25:26 +0200

> therefore we must use it, and use it in the broken and inflexible way?

And Matthew we haven't argued against attempting to fix the big issue
you see with it, namely that it requires a lower layer to separate RTP
sessions. We just don't see it happing quickly or in the way you
proposed. But I think we are going to very seriously consider attempting
to do something about the short coming in AVTCORE.

> trying to build, not living with crap just because there's an RFC for it.

I don't think we have an realistic alternative of doing something new. I
would love to develop a new real-time media transport protocol. I just
think that it will take 3 years before we are close to having something
published. I also fear that it might suffer from second system syndrome
and be hellishly complex.

I also don't think it helps the people that want to gateway to their
legacy systems.

Do you have any other well defined media transport alternative for us to
use?

Cheers

Magnus Westerlund

---

From: Stephan Wenger <[email-masked]>
Date: Sat, 23 Jul 2011 14:12:55 -0700

Hi Magnus,

Not sure you got the context right here.  The "removal of multiparty
support" was related to my suggestion for a feature list of what I called
RTPv3, which would need to co-exist with RTPv2 (that includes multiparty
support).

That said, if I'm not completely mistaken, you can implement the
multiparty use cases using the "RTCP terminating MCU" topology of RFC
5117.  This is common implementation practice in video conferencing.  If
you do so, the multiparty support of RTPv2 is neither exercised nor needed
in a multiparty video conference.

Stephan

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 25 Jul 2011 06:30:40 -0400

> RTPv3, which would need to co-exist with RTPv2 (that includes multiparty

Ok, I misunderstood that.

> in a multiparty video conference.

First of all if you do that you loose functionality as discussed in RFC
5117. For example the indication of who actually sent a packet
originally or contribute to the mix can't be provided unless you let the
SSRC or CSRC plus CNAMES be forwarded across the Mixer/MCU.

In addition it forces the central unit into one particular
implementation, instead of allowing to use what it most appropriate.

An RTP mixer when you want something that has functionality and
application enhancing behavior and the cost is more complexity.

Or you use an relay which simply copies a packet from one entity to all
the others. That is the simplest central node you can build and it will
have no media processing at all.

Also, in any gateway case to existing legacy you don't want to have to
implement additional gatewaying functions just because one want to use
non standard RTP.

For me it is clear that ensuring that RTCWEBs RTP usage is vanilla will
help us avoid a lot of problems down the road.

If we are going to multiplex this mechanism needs to have a behavior
that doesn't change the core behaviors of RTP. If we can do that the
multiplexing becomes much less a hot topic.

Cheers

Magnus Westerlund

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 25 Jul 2011 14:03:49 -0700

Hi Magnus,

>SSRC or CSRC plus CNAMES be forwarded across the Mixer/MCU.

Correct.  So what?  There is such a thing as a signaling plane.  In all
practical commercial systems I'm aware of, the signaling plane information
is the ONLY information that is used for the identification of
participants (including mixed participants).  Unfortunately, this
information is most often conveyed using nonstandard means.  No one in
video conferencing that I'm aware of relies on the SSRC/CSRC + CNAME
mechanism.  Let me know if I'm wrong.

I'm not saying that this design choice is necessarily the best possible
for rtcweb.  But I'm claiming that the practical implementation experience
speaks in favor of it.  So the burden of proof of requiring its use (which
may break other aspects that have been identified by many here as
critical, such as minimizing the number of ports), should be on you, not
on me.  

>have no media processing at all.

Sorry, I wasn't able to follow this train of thought.  If this was related
to implementation complexity of media plane signaling of mixed media
streams vs. signaling plane signaling of similar information, I would
argue that the majority of the implementers, so far, appear to have sided
with signaling plane rather than media planeŠ  I'm sure that they had a
reason.  

>implement additional gatewaying functions just because one want to use

That's also my understanding.  Is it strong enough an argument, given the
(many, I believe) other issues a gateway may need to worry about?

>help us avoid a lot of problems down the road.

I'm arguing the same thing as you, but we define "vanilla" differently.
You seem to define it as "use of RTP as specified in both language and
intent of the standard, and following the 1990 generation of understanding
of the standard".  I define it "use RTP in the way it has been implemented
in the industry relevant here (video conferencing industry) over the last
decade or so."

>multiplexing becomes much less a hot topic.

Again, we agree, perhaps except for the interpretation of "core behavior".
 In my view, "core behavior" does not include mechanisms such as CC, CSRC,
as they are AFAIK  not practiced in most real-world systems.  It may
include, though, include mechanisms such as SSRC multiplexing, as this
mechanism IS deployed in millions of systems, with no known adverse
effects on the operation of the Internet as a whole, or the RTPish things
that are part of the Internet.

Cheers,
Stephan