Subject: [dispatch] RTC-Web: More thoughts on what needs to be standardized

From: Matthew Kaufman <[email-masked]>
Date: Mon, 14 Feb 2011 09:41:31 +0100

Note that while the signaling mechanisms (including capability negotiation) can be entirely ad-hoc, assuming the JavaScript APIs are standardized such that anyone can write JavaScript to check capabilities and open new sessions, there are a few areas where we do need standard protocols in order for browsers to communicate directly with one another. (And while we must also support relaying, including TCP relays of media to support all NAT/firewall cases, we hope that the majority of the media can flow directly between a pair of browsers.)

The first is relatively obvious: We must choose a transport protocol for browser-to-browser media that is implemented by all. DTLS-SRTP seems to fit the bill here, though in order to reduce the number of ports required and the NAT traversal complexities we should probably multiplex the various media channels in a session, and an obvious way to do that is to use the SSRC field. (Note that this suggestion explicitly removes the possibility that a single UDP port at one point will be talking to more than one other endpoint at a time. While that can be done, it would require the addition of a way to demultiplex the incoming packets (by something other than UDP port number) and also has API implications, in that a "session" is no longer the holder of the UDP port, but rather an additional object must be created to hold the local port binding that is then shared by one or more session objects.)

Related to the first: We must choose a way to ensure that the above protocol operates in a way that is not potentially damaging. This requires two things, an affirmative handshake (for which the STUN connectivity check from ICE seems appropriate) and some sort of rate/bandwidth control. SRTCP is the obvious choice as a feedback mechanism, but how that information would be used is still TBD.

Also related to the first: We must find a way for this transport protocol to operate through common NAT scenarios. ICE or a variant thereof is probably a solution. This then raises the next area of standardization concern...

The second area where standardization is likely necessary is the format of the "low-bandwidth" connection setup channel that must be provided between a pair of endpoints in order to complete the negotiation process required by ICE or the equivalent. While this *could* be simply defined as parameters which must be passed to the JavaScript API, it seems likely that for interoperability it will be helpful for the format of the "blob" of data which contains things like candidate address pairs to be standardized. It also potentially changes the API from "set a bunch of individual things" to "pass blob to connection object". One obvious choice for the blob format is SDP, as used by ICE already, but it isn't at all clear that we want to go down this road, especially since there would then be the temptation to use SDP for other things, like codec negotiation, where it is clearly not necessary in this environment. Good choices for the browser environment might be an XML-based, or even better, JSON format.

Matthew Kaufman

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 21 Feb 2011 18:27:05 +0100

Hi,

I want to respond to this in my role as AVTCORE chair and care-taker of
RTP.

Matthew Kaufman skrev 2011-02-14 09:41:
> 
> The first is relatively obvious: We must choose a transport protocol
> for browser-to-browser media that is implemented by all. DTLS-SRTP
> seems to fit the bill here, though in order to reduce the number of
> ports required and the NAT traversal complexities we should probably
> multiplex the various media channels in a session, and an obvious way
> to do that is to use the SSRC field. (Note that this suggestion
> explicitly removes the possibility that a single UDP port at one
> point will be talking to more than one other endpoint at a time.
> While that can be done, it would require the addition of a way to
> demultiplex the incoming packets (by something other than UDP port
> number) and also has API implications, in that a "session" is no
> longer the holder of the UDP port, but rather an additional object
> must be created to hold the local port binding that is then shared by
> one or more session objects.)

There has been a lot of proposal popping up to do away with one of RTP's
multiplexing points, namely the one for separating different RTP
sessions. This is as you know primarily done using Transport ports. This
is not the only way, but the most common method.

What I want to say here is that I do object in removing this
multiplexing point, If one don't have a clear separation of RTP sessions
this have impact on some RTP functions and extensions. The basis for
these separations are the thinking documented in Section 5.2 of RFC
3550. Bullet 4 and 5 are still applicable for RTC-WEB. Where I think 4
is the more likely to have real effect.

However, the real issue is that this separation has been used in a
number of extensions and are ingrained in the whole RTP model. This can
been find in how reporting and monitoring is done, in how configuration
normally is done, the handling of new SSRCs due to the separation of RTP
and RTCP. I do hope to be able to write an I-D on this that explores the
issues in details.

I am not saying that it is impossible, only that it will need new
mechanisms for assigining SSRCs to different sessions, and that these
needs to be used prior to actual transmissions. In other words some SSRC
reservation and assignment protocol would be needed. Even that can still
break certain mechanisms in certain usages. I also forsee that
implementations of RTP would need to be seriously changed.

I would also like to point out that if defined such usage could have
serious impact on any interoperability usage. As it is not obvious that
this can easily be split into multiple "normal" rtp sessions without
additional "washing" of RTCP etc. It could easily lead to SSRc collision
situations also.

There exist at least two alternatives to run all RTP sessions in one so
to say. One if to actually open several UDP flows between the peers.
With an established flow, using the candidate pair that worked, this
should be possible to open in 1-2 RTTs.

The other alternative is an explicit multiplexing shim on top of a
single UDP flow. This has the potential for no extra delay, but also for
creating interesting multiplexing considerations, with a bit of queue
handling and things like that.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: Colin Perkins <[email-masked]>
Date: Tue, 22 Feb 2011 17:21:00 +0000

I'd like to echo, and expand on, Magnus' comments. The RTP SSRC is not an appropriate demultiplexing point for different sessions, and trying to use it as such will cause problems. 

If there is a need to keep the number of transport connections small, and multiplex several RTP flows on a single transport connection, then a shim layer could be added to provide demultiplexing functionality similar to UDP ports. Unmodified RTP can then be run over this layer. This would avoid the need to modify the RTP SSRC allocation algorithm, or add complex signalling for the SSRC ranges allocated to particular sessions. It would also make it easier to gateway to standard RTP-over-UDP.

Colin

On 21 Feb 2011, at 17:27, Magnus Westerlund wrote:
> _______________________________________________
> dispatch mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dispatch

-- 
Colin Perkins
http://csperkins.org/

---

From: Matthew Kaufman <[email-masked]>
Date: Tue, 22 Feb 2011 19:37:02 +0100

> I'd like to echo, and expand on, Magnus' comments. The RTP SSRC is not an appropriate demultiplexing point for different sessions, and trying to use it as such will cause problems. 

Ok. So we're finally at an interesting point of controversy.

> If there is a need to keep the number of transport connections small, and multiplex several RTP flows on a single transport connection, then a shim layer could be added to provide demultiplexing fu…

We have the following three choices, none of which is compatible with another:

1. Use RTP, but multiplex on SSRC. This allows us to save on NAT traversal (and cuts down on the risk of running out of NAT bindings in some environments), but violates the previous reasoning for not using SSRC for this.

2. Use RTP, but inside of another wrapper. This makes the RTC-Web transport incompatible with legacy endpoints, even if they have something that can do the handshake. But if we go this way, there's lots more we could do, including multiplexing multiple endpoints against a single UDP port, as Adobe's RTMFP does. (And reviewing that reveals a whole raft of other features we could add if we're not constrained to look like RTP on the outside)

3. Use RTP as it is traditionally used, one UDP port per flow. This requires three UDP ports and NAT bindings at each end for the common (audio+video+data) cases, and even more in other cases. Makes us more compatible with legacy endpoints (though a handshake is still required, and we now need to do the handshake once per UDP port... so a minimum of twice for an audio+video call)

None of the above is particularly good, given the constraints. If we could relax the constraints, I'd relax the ones that make #2 possible first.

Matthew Kaufman

---

From: Matthew Kaufman <[email-masked]>
Date: Tue, 22 Feb 2011 10:43:45 -0800

> should be possible to open in 1-2 RTTs.

The "candidate pair that worked" is of course only marginally useful at best, as the UDP port mappings won't carry over. We'll also need to repeat the handshake (fortunately, part of the NAT traversal process if we use ICE) to ensure that the far end of each pairing is willing to receive. And hope that NAT devices don't run out of ports 3 times as fast with this approach.

> handling and things like that.

There's actually two benefits here. One is that we can reduce it to a single UDP port to talk to *multiple* far-end participants. The second is that it forces the sender to think about the relative prioritization of the transmissions across flows and across far-ends, which is especially useful if we have congestion control across the entire session.

Matthew Kaufman

---

From: Colin Perkins <[email-masked]>
Date: Thu, 24 Feb 2011 17:57:01 +0000

> 2. Use RTP, but inside of another wrapper. This makes the RTC-Web transport incompatible with legacy endpoints, even if they have something that can do the handshake. But if we go this way, there's…

A user-space DCCP protocol tunnelled over UDP would be an interesting wrapper if we have to go down this route.

> 3. Use RTP as it is traditionally used, one UDP port per flow. This requires three UDP ports and NAT bindings at each end for the common (audio+video+data) cases, and even more in other cases. Make…

RFC 5761 helps here, since it avoids the need for separate RTCP ports.

> None of the above is particularly good, given the constraints. If we could relax the constraints, I'd relax the ones that make #2 possible first.

I agree that there are limitations of all three approaches. Option 3 wins on compatibility with other endpoints, without the need to gateway media, and would seem to be preferred if its feasible. I guess the question is whether there's some constraint on a browser-based endpoint that stops us re-using this model?

-- 
Colin Perkins
http://csperkins.org/

---

From: Alex Eleftheriadis <[email-masked]>
Date: Sat, 26 Feb 2011 10:09:13 -0500

Hi everyone. 

The assumption that video communication requires one video, one audio, and one data connection is, for better or for worse,  outdated. While it was true in the days of the traditional MCU, or for point-to-point communication, there are now much better ways to do video using scalable video coding. Although Vidyo was the one to introduce it, you can find information about scalable video coding in the websites of nearly all videoconferencing companies. 

Architecturally, the implication is that the client may receive multiple streams, possibly as many as there are participants. Similarly to the web browser, where the client does the composition, in a system that uses scalable video coding one can have the client do the composition of the multiple video streams.

Additionally, we also have telepresence systems (CLUE wg) as yet another example of the use of multiple video streams. There are similar issues re. multiplexing there as well. (Need I mention TIP?)

While I do appreciate the need to be careful about design choices, we should also make sure we do not inadvertendly hamper important and much-needed innovations that greatly improve user experience. 

In this spirit, 3 would be the least preferred choice imho. 

--Alex

---

From: Adam Roach <[email-masked]>
Date: Sat, 26 Feb 2011 17:31:28 -0600

The problem is that #1 and #2 are point solutions for RTC-WEB and 
RTC-WEB alone.

If there's a problem with using different ports for different media 
streams, then there's a problem. It's not an RTC-WEB problem; it's a 
multimedia problem. And, if it needs a solution, then we need to start 
working on a solution for all multimedia, not just RTC-WEB.

The other side of that coin is: if it's not a general problem, then it's 
not an RTC-WEB problem either. In which case the additional work to 
design a non-compatible special-purpose media transport for RTC-WEB is 
just an exercise in recreational protocol design with no real need to fill.

Being able to send media to and from non-RTC-WEB terminals without a 
media gateway (and without the latency such gateways necessarily 
introduce) is a huge benefit for #3. I'm concerned by serious 
suggestions that we casually disregard this benefit.

/a