Subject: [rtcweb] To multiplex or not!

From: Magnus Westerlund <[email-masked]>
Date: Tue, 19 Jul 2011 16:28:20 +0200

Hi,

This email is as an individual contributor.

I want to get started on the discussion of the Multiplexing of the
various protocols over single lower layer transport flow, such as a UDP
flow. I will attempt to split up the questions into different emails.

The first question I think is reasonably easy to get answered, but I
think it is time we determine if my belief in the answer is correct or not.

The traffic between two RTCWEB peers from the various components, such
as RTP sessions, datagram service:

a) MUST be sent as Individual flows for each component.

b) MUST be multiplexed into a single transport flow.

c) SHOULD be multiplexed into a single transport flow, but the RTCWEB
peer MUST be able to send them as individual flows.

I would love if people can indicate their choice or preferences.

I personally prefer A as it it is simplest in all aspect except the NAT
traversal.
- It allows for flow based QoS.
- It is the what the implementation that exist mostly do
- Signaling protocols that exist support it, no extra functionality
- People are used to the concept
- It minimizes the difference to legacy.

Thus it is the quickest road to define something with the least formal
push back and concern over maturity of any solution.

The downside with B and C is that we do have to solve the multiplexing
and get an agreement that gets through all the hurdles.

Of these two opens I do prefer C.  Although it results in the extra
complexities of having both alternatives, it will give us both a
fallback, flow based QoS and better legacy support.

Now it is your time to make your opinion heard!

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: Justin Uberti <[email-masked]>
Date: Tue, 19 Jul 2011 08:20:56 -0700

> as RTP sessions, datagram service:

To be clear, we are talking about different RTP sessions, as opposed to
different RTP sources within a single RTP session, correct?

> - It allows for flow based QoS.

For video applications, it's not clear that flow-based QoS is sufficient, as
some packets are of higher importance than others. Diffserv markings allow
per-packet QoS to be applied, but this could be done even in a multiplexing
scenario.

> - It minimizes the difference to legacy.

I agree that existing implementations will need to change to support this.
However, is there a large enough base of deployed endpoints that support a)
multiple RTP sessions and b) the other RTCWEB constrains (ICE, etc) to make
this a significant concern?

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 19 Jul 2011 17:23:51 +0200

On 2011-07-19 17:20, Justin Uberti wrote:
> 
> On Tue, Jul 19, 2011 at 7:28 AM, Magnus Westerlund
> <[email-masked] <mailto:[email-masked]>>
> wrote:
> 

Exactly!

Magnus Westerlund

---

From: Dzonatas Sol <[email-masked]>
Date: Tue, 19 Jul 2011 08:30:34 -0700

> peer MUST be able to send them as individual flows.

One clarification please, if there are known ranges of A as one 
spectrum, does this WG still consider that multiplexed? IPv6, for 
example, allows us to multiplex in a TCP-less way, simply by fulfillment 
of the flows to more than one individual address. Those addresses could 
constitute one spectrum in a stateful manner, or not.

-- 
--- http://twitter.com/Dzonatas_Sol ---
Web Development, Software Engineering
Ag-Biotech, Virtual Reality, Consultant

---

From: Matthew Kaufman <[email-masked]>
Date: Tue, 19 Jul 2011 09:40:29 -0700

> I would love if people can indicate their choice or preferences.

C. But I'm not sure that the "must" needs to be anything more than "you 
just make two peer connection objects, one for each". (In other words, 
any API that allows for multiplexing almost certainly allows for 
non-multiplexing)... this assumes that the multiplexing is implicit, of 
course. With explicit multiplexing it is even harder... the API needs to 
support turning on and off the multiplexing shim as well, something I'm 
not in favor of.

Matthew Kaufman

---

From: Emil Ivov <[email-masked]>
Date: Tue, 19 Jul 2011 18:47:27 +0200

На 19.07.11 16:28, Magnus Westerlund написа:
<snip>

> a) MUST be sent as Individual flows for each component.

+1

> I personally prefer A as it it is simplest in all aspect except the NAT

It definitely simplifies it on paper - it would be far easier to draw an
ICE negotiation diagram that only deals with a single stream component.

I am not sure however to what extent this would actually simplify
implementations. They would still need to handle multiple candidates and
pacing and such ... and of course they'd also need to implement  a
de-multiplexing layer.

<snip>
> - It minimizes the difference to legacy.

Personally I consider this as the most important part.

Cheers,
Emil

-- 
Emil Ivov, Ph.D.                       67000 Strasbourg,
Project Lead                           France
Jitsi
[email-masked]                        PHONE: +33.1.77.62.43.30
http://jitsi.org                       FAX:   +33.1.77.62.47.31

---

From: Emil Ivov <[email-masked]>
Date: Tue, 19 Jul 2011 18:55:14 +0200

На 19.07.11 17:20, Justin Uberti написа:
> I agree that existing implementations will need to change to support
> this. However, is there a large enough base of deployed endpoints that
> support a) multiple RTP sessions and b) the other RTCWEB constrains
> (ICE, etc) to make this a significant concern? 

You are referring to SIP/XMPP clients using RTP and ICE, right? Well,
given that this has been IETF's default way of handling RTC for the past
few years I would assume that there are a bunch yes.

Besides, if we ever make the ICE part optional (like XEP-0177 does for
XMPP) then we'd have media-layer interoperability with any VoIP client
out there and this would definitely be a great win.

Emil

-- 
http://jitsi.org

---

From: Randell Jesup <[email-masked]>
Date: Tue, 19 Jul 2011 13:05:29 -0400

> peer MUST be able to send them as individual flows.

C).   I understand the appeal of a).  However, we already have minor 
issues with handling
congestion across multiple streams - i.e. congestion notification goes 
to one flow, but not
generally to the other flows that make up the total traffic between 
these two points.  This
leads to sub-optimal response to congestion, and as the number of flows 
increases this
would get worse.  Now, this really isn't mandated by the 
multiplex/non-multiplex nature; it's
just easier and more obvious when they're multiplexed.

The other big issue is as you add more than a single or two streams, the 
overhead and
side-issues around ICE/etc get more ... interesting.  In particular you 
can get (I assume) cases
where different streams that are meant to be synchronized and follow the 
same route
go over different paths/interfaces.

If we only had to deal with 1 audio stream, or 1 audio and 1 video 
stream, the balance would be
different.  But in the world this spec will work in I strongly believe 
more than 2 streams
will often exist.  (Witness Google's "Huddle").

-- 
Randell Jesup
[email-masked]

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 19 Jul 2011 17:05:52 -0700

I'd like to see it that devices MUST support multiplexing and I'd probably be in favor of also MUST support non multiplexed flows. The multiplexing is a big advantage for any device that needs to deal with NATs so I suspect many "legacy" devices would be moving to this even if there was no need to deal with the browser like devices. I also think that allowing this type of multiplexing will make firewall traversal easier where a device can configure a firewall to open just one specified port and use it for RTP. 

The number places where RSVP is usable does not really overlap with places one uses ICE so it does not bother me in the slightest that RSVP won't work with this. It's pretty easy to imagine an extension to RSVP to support reservations based on SSRC inside the flow. Diff Serv based qos will work which is what more commonly used in situations where we are using NATs. 

The primary reason I want multiplexing is the time to set up all the ICE connections. The fact of the matter is that NATs limit how quickly you can create new flows. If all you need is something simple with just 2 flows, one for audio and one for video, this is workable. But as soon as you move to trying to have a transition strategy from V4 to V6 and multiple person video or even just multi screen video, the user experience goes downhill rapidly. 

Much of the IETF thinking around ICE was done on the assumption that ICE could be done before ringing started or during ringing but the deployments I am seeing are moving more towards not doing ICE until after a call is answered. There are many reasons for this but one of them is that if Alice calls Bob, and Bob is on a mobile internet device, you don't want to reveal Bob's location to Alice before Bob even decides to answer the call. Given an IP address more or less reveals location these days that a bit of issues to do ICE before answering. This puts more pressure on being able to do ICE quickly for a good user experience. 

Cullen <with my individual contributor hat on>

> Now it is your time to make your opinion heard!

Cullen Jennings
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html

---

From: Dzonatas Sol <[email-masked]>
Date: Tue, 19 Jul 2011 17:33:36 -0700

> The primary reason I want multiplexing is the time to set up all the ICE connections. The fact of the matter is that NATs limit how quickly you can create new flows. If all you need is something si…

V9 is quick, yet the impression gives the vaporware experience because 
it is solid state less-than-best solution. For real-time, probably best 
to just expose if V4 is fast or slow, and we can assume the opposite 
about V6.

> Much of the IETF thinking around ICE was done on the assumption that ICE could be done before ringing started or during ringing but the deployments I am seeing are moving more towards not doing ICE…

V9 crystallization technique simply wants to know if anything changed 
while 'do ICE quickly' is done. Consider that quality of service for 
dynamic-compilers, especially those that need what weight of energy 
before any real-time process is done.

If we know the process and data does not contain any location, sex, or 
age, then pressure relief exists with that process and data. I see three 
flows for ICE, not 2, and one separate individual flow.

Safe to multiplex those flows? Not by these standards in this WG.

> Cullen<with my individual contributor hat on>
>
>> F�r�gatan 6                | Mobile +46 73 0949079

--

---

From: Stefan Håkansson LK <[email-masked]>
Date: Wed, 20 Jul 2011 08:36:00 +0200

> support turning on and off the multiplexing shim as well, something I'm
The APIs being discussed now deal with "MediaStreams" that contain both 
audio (zero to many) and video (zero to many). This has been found to be 
a reasonable level of abstraction for the web app developer. The web app 
developer would have difficulties to understand how many RTP streams a 
MediaStream would generate. So it is not so simple to deal with this by 
creating several peer conn objects.

Besides, it has been discussed that A/V synch should be maintained 
within a MediaStream. This is something you would lose if you split it 
up and transport it using separate peer conn objects.

Stefan

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 20 Jul 2011 09:44:01 +0200

> support turning on and off the multiplexing shim as well, something I'm 

I am far from certain that it is that simple. First of all, two
different RTP sessions, one for audio and one for video that goes over
different UDP flows, still has an association, for example the used
CNAME to indicate that these flows can be synchronized and are from the
same end-point. If the API definition is such that a stream object
implies a synchronization context when sending, then moving the streams
to two different objects fails to have that separation in semantics.

I think the MUST is important in the aspect that it is a clear agreement
to get this to work. How, that is for us to work out together with W3C.

Thus, I wouldn't rule out that actual need for some option that you can
set to get non-multiplexed. It might be conceptually much simpler for a
developer than to actually understand how you need to spit streams and
then add additional binding properties to these media streams which
could be an alternative design.

Cheers

Magnus Westerlund

---

From: Stefan Håkansson LK <[email-masked]>
Date: Wed, 20 Jul 2011 11:36:59 +0200

FWIW I'm in camp A (but of course C is fine as well). Flow based QoS is 
part of modern cellular networks, and I would like rtcweb to work well 
on them (our experience is also that when the throughput is reduced you 
prefer the video to be degraded rather than having the audio being 
chopped - so you would need them to be individual flows).

Stefan

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 20 Jul 2011 11:49:19 +0200

Magnus, I cannot parse your list of choices.

In the case where you have

Sender A
Recipient B
Three video streams going from A to B
Three audio streams going from A to B
One video stream going from B to A
One audio stream going from B to A

with the application not requiring the network to treat the packets any 
differently, and with the application using the CNAME mechanism to match 
audio streams with corresponding video streams

it is completely unclear to me whether your choice A) requires 8 
transport flows, 4 transport flows or 2 transport flows, and it is 
completely unclear to me what C) actually means. (I assume "transport 
flow" = "RTP sessions" here)

Please - can you clarify what your question is intended to mean?

                  Harald

---

From: Emil Ivov <[email-masked]>
Date: Wed, 20 Jul 2011 13:08:50 +0200

На 20.07.11 02:05, Cullen Jennings написа:
> I'd like to see it that devices MUST support multiplexing and I'd
> probably be in favor of also MUST support non multiplexed flows. The
> multiplexing is a big advantage for any device that needs to deal
> with NATs so I suspect many "legacy" devices would be moving to this
> even if there was no need to deal with the browser like devices. I
> also think that allowing this type of multiplexing will make firewall
> traversal easier where a device can configure a firewall to open just
> one specified port and use it for RTP.

I don't really agree with this part. Devices would still need to handle
candidates that come from  STUN, TURN, UPnP, host IPv4, host IPv6, and
potentially VPN, MIPv6 and other tunnelling mechanisms. Whether or not
this only happens for a single component/stream doesn't really simplify
code does it?

According to my personal experience with our work on ice4j.org, managing
multiple checklists was far less resource consuming than implementing
TURN tunnels and persmissions, STUN multiplexing, synchronisation and
even getting pacing right.

<snip>

> The primary reason I want multiplexing is the time to set up all the
> ICE connections. The fact of the matter is that NATs limit how
> quickly you can create new flows. If all you need is something simple
> with just 2 flows, one for audio and one for video, this is workable.
> But as soon as you move to trying to have a transition strategy from
> V4 to V6 and multiple person video or even just multi screen video,
> the user experience goes downhill rapidly.

I agree more with this argument, although freezing candidates
significantly brings down the overhead incurred by having multiple
streams and components and I am not sure that the actual win here would
be good enough to pay for implementing multiplexing and for breaking
interoperability with those that don't have it.

> Much of the IETF thinking around ICE was done on the assumption that
> ICE could be done before ringing started or during ringing but the
> deployments I am seeing are moving more towards not doing ICE until
> after a call is answered. There are many reasons for this but one of
> them is that if Alice calls Bob, and Bob is on a mobile internet
> device, you don't want to reveal Bob's location to Alice before Bob
> even decides to answer the call. Given an IP address more or less
> reveals location these days that a bit of issues to do ICE before
> answering. This puts more pressure on being able to do ICE quickly
> for a good user experience.

Agreed, but is multiplexing the only way of bringing the time down? The
XMPP crowd for example has been using a mechanism that allows candidates
to be sent as they become available rather than wait for all harvesting
to complete. Maybe this is worth exploring and integrating in the
official ICE (although I don't see how it would work with SIP).

Cheers,
Emil

>> various protocols over single lower layer transport flow, such as a
>> UDP flow. I will attempt to split up the questions into different
>> emails.
>> 
>> The first question I think is reasonably easy to get answered, but
>> I think it is time we determine if my belief in the answer is
>> correct or not.
>> 
>> The traffic between two RTCWEB peers from the various components,
>> such as RTP sessions, datagram service:
>> 
>> c) SHOULD be multiplexed into a single transport flow, but the
>> RTCWEB peer MUST be able to send them as individual flows.
>> 
>> I personally prefer A as it it is simplest in all aspect except the
>> NAT traversal. - It allows for flow based QoS. - It is the what the
>> implementation that exist mostly do - Signaling protocols that
>> exist support it, no extra functionality - People are used to the
>> concept - It minimizes the difference to legacy.
>> 
>> Thus it is the quickest road to define something with the least
>> formal push back and concern over maturity of any solution.
>> 
>> The downside with B and C is that we do have to solve the
>> multiplexing and get an agreement that gets through all the
>> hurdles.
>> 
>> Of these two opens I do prefer C.  Although it results in the
>> extra complexities of having both alternatives, it will give us
>> both a fallback, flow based QoS and better legacy support.
>> 
>> Färögatan 6                | Mobile +46 73 0949079 SE-164 80
>> Stockholm, Sweden| mailto: [email-masked] 
>> rtcweb mailing list [email-masked] 
> Cullen Jennings For corporate legal information go to: 
> _______________________________________________ rtcweb mailing list 
> [email-masked] https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 20 Jul 2011 13:10:02 +0200

> Please - can you clarify what your question is intended to mean?

With choice A I do mean that one for each RTP session use one UDP flow
in each direction. And that these two UDP flows will be symmetric, i.e.
a bi-directional UDP flow.

This in contrast to B and C when you use some multiplexing to map all
RTP sessions, independently of how many you have onto one bi-directional
UDP flow.

The above example you give is at least two RTP sessions, one for audio
and one for video. It might be more than these two if the reason you are
sending multiple video streams is because the streams should be
differently treated, for example in a RTP mixer. Where you perform
composition of video streams of the participants and simply switch a
demonstration camera. Or it may be that entity A above is transport
relay and it is forwarding the camera from three different conference
participants and all video streams are the same RTP session.

It is the application that has do make these choices about if it needs
more than one RTP session or not.

Cheers

Magnus Westerlund

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 20 Jul 2011 14:58:14 +0200

> more than one RTP session or not.
Magnus, if the number of UDP flows is constant (even if it's slightly 
larger than 1) when the number of video streams changes under your 
solution A, I think this is a viable option.

If the number of UDP flows was linear with the number of video streams, 
I believe that a number of scenarios we want RTCWEB to support, 
including stuff like the Google+ Hangouts, would be basically unsustainable.

So - with the understanding that an application is going to put as many 
video streams it wants down one RTP session if that's appropriate for 
the purposes of the application - I am in favour of the "no 
multiplexing" solution.

                      Harald

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Wed, 20 Jul 2011 06:58:26 -0700

> up and transport it using separate peer conn objects.

Actually, one of the major design goals of the MediaStream API is to 
allow the synchronization of multiple, separate MediaStreams, and I've 
been working with roc to ensure that this is, in fact, relatively 
straightforward to do.

Which is not an argument against a more explicit API for dealing with 
multiplexing (that argument belongs on a different list anyway).

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 20 Jul 2011 07:11:04 -0700

> even getting pacing right.

Ahh, I think we have come to the key issues here - perhaps I have not been explaining myself very well. So in your experience roughly how long does it take to set up a for two scenarios using ICE as is today. 

First scenario: I have two devices that want to set up a single audio stream and they each have the candidates from: local v4 IP address, v4 stun, v4 turn, local v6, turn v6. Obviously they could have a lot more but that seems like a reasonable starting point. 

Second scenario: same as first address wise but instead of a single audio stream they want to set up a single audio stream  to a conference bridge plus 7 video streams for the video from the 5 people on the bridge plus a presentation stream and stream of video from active speaker. I don't really care much about the scenario other than there are 8 streams being set up. But this type of scenario is becoming very common for multi party video chat as it allows you to see perhaps small versions of everyone plus a large version of active speaker. 

My experience is the answer to the first scenario is not as quick as you would like and the answer to how long the second takes is about 8 times longer than the first one. You might do a bit better than that depending on how clever your implementation is but it still a lot longer. 

I totally agree with you that multiplexing may not hugely simplify the code and you still have to deal with the hard parts to implement. WHat is helps with is it makes the second scenario take exactly the same time as the first scenario because instead of taking 8 times a long. It's purely a user experience thing. Does that make sense? 

> interoperability with those that don't have it.

Use multiplexing would be negotiated over the SDP, just like rtcp/rtp mux is negotiated today, so it would not break interoperability with stuff that did not support it. 

> official ICE (although I don't see how it would work with SIP).

I am a fan of sending candidates as they become available but I don't think this helps much as most devices can gather the addresses before the user even initiates a call and the gather it typically very quick relative to the rest of the ICE process. 

Cullen Jennings

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 20 Jul 2011 07:24:25 -0700

> My experience is the answer to the first scenario is not as quick as you would like and the answer to how long the second takes is about 8 times longer than the first one. You might do a bit better…

[BA] Yes, that is correct.  However at the moment I'd characterize that as a "high quality problem", since it isn't even clear that the current WHATWG API can handle that case, due to glare and ICE timing issues.

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 20 Jul 2011 07:40:06 -0700

> [BA] Yes, that is correct.  However at the moment I'd characterize that as a "high quality problem", since it isn't even clear that the current WHATWG API can handle that case, due to glare and ICE…

Agree - but I think we need to fix that regardless of what we decide with this.

---

From: Dzonatas Sol <[email-masked]>
Date: Wed, 20 Jul 2011 07:53:57 -0700

I found the answer through the continued discussion in other messages 
sent to this WG list.

There is no semantics between spectral modes and multiplex modes.

In fact, news yesterday reveals "they're getting it" now, so I may not 
seem so much like a pest...

"A team of researchers at the Oak Ridge National Laboratory have found 
that it's hydrogen rather than carbon which plays a fundamental part in 
creating a uniform crystalline form."

Read more: 
http://www.techeye.net/science/graphene-discovery-points-to-large-scale-production#ixzz1SelXjqyn

Yes, we have been thinking in light ("spectral" crystal ICs) and 
hydrogen economy for years (stateful vs stateless protocols), and 
realized many do not, were stuck, or haven't yet realized this much at 
this level because their trained to think in unlimited resources. 
Practicality and duplication is near-term now.

...well, a resource-pest around standards to determine sustainable 
technology. Real-time and dates are always an issue with assets.

If the above still doesn't make think, realize that generally hydrogen 
has no shape ("nobody"/nothing holds it), therefore considered 
unreliable for transport, until now...

Thanks.

On 07/19/2011 08:30 AM, Dzonatas Sol wrote:
> example, allows us to multiplex in a TCP-less way, simply by 
> fulfillment of the flows to more than one individual address. Those 
> addresses could constitute one spectrum in a stateful manner, or not.
>

--

---

From: "Elwell, John" <[email-masked]>
Date: Wed, 20 Jul 2011 17:38:30 +0200

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Magnus Westerlund
> Sent: 19 July 2011 15:28
> To: [email-masked]
> Subject: [rtcweb] To multiplex or not!
> 
> Hi,
> 
[JRE] As soon as we start talking about what an RTCWEB peer MUST support, the question arises whether this applies to the browser, the script or both. For example, a browser might implement each PeerConnection object over separate transports, but if the script chooses to use a single PeerConnection object for both audio and video it might not be interoperable with a peer that doesn't support multiplexing.

So whilst I agree that picking something along the lines of c) might be possible, we would need to define more precisely what we mean before coming to such an agreement.

John

John Elwell
Tel: +44 1908 817801 (office and mobile)
Email: [email-masked]
http://www.siemens-enterprise.com/uk/

Siemens Enterprise Communications Limited.
Registered office: Brickhill Street, Willen Lake, Milton Keynes, MK15 0DJ.
Registered No: 5903714, England.

Siemens Enterprise Communications Limited is a Trademark Licensee of Siemens AG.

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 20 Jul 2011 17:50:21 +0200

> [JRE] As soon as we start talking about what an RTCWEB peer MUST support, the question arises whether this applies to the browser, the script or both. For example, a browser might implement each Pe…
It's not at all obvious to me how we map PeerConnection objects to RTP 
sessions. It might be that there can only be a single PeerConnection 
object to a specific remote peer, and that the PeerConnection object 
decides internally how to map media streams to RTP sessions (for 
instance). Or indeed where that mapping ought to be specified.

In the IETF, we may want to concentrate on what the packets should be on 
the wire; in the W3C, we may want to concentrate on making the API 
powerful enough and easy enough to use - sooner or later, all the pieces 
have to match up, though.

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 20 Jul 2011 08:53:19 -0700

> So whilst I agree that picking something along the lines of c) might be possible, we would need to define more precisely what we mean before coming to such an agreement.

[BA] .MUST support sending as individual flows makes sense to me, but making normative statements about multiplexing onto a single transport doesn't.   If both sides support multiplexing into a single transport flow, then it can be negotiated and used.

---

From: Colin Perkins <[email-masked]>
Date: Wed, 20 Jul 2011 23:21:00 +0100

On 20 Jul 2011, at 15:11, Cullen Jennings wrote:
>> I don't really agree with this part. Devices would still need to handle candidates that come from  STUN, TURN, UPnP, host IPv4, host IPv6, and potentially VPN, MIPv6 and other tunnelling mechanisms. Whether or not this only happens for a single component/stream doesn't really simplify code does it?
>> 
>> According to my personal experience with our work on ice4j.org, managing multiple checklists was far less resource consuming than implementing TURN tunnels and persmissions, STUN multiplexing, synchronisation and even getting pacing right.
> 

I may be misunderstanding, but why would this take 8 times as long to set up? Wouldn't you implement it as one UDP flow for the RTP audio, plus one UDP flow for the RTP video (with the 7 video streams multiplexed by SSRC, since RTP has always allowed multiparty flows of the same media type in a single RTP session)? This might take twice as long to setup as a single audio flow in the worst case, but not eight times as long.

Cheers,
Colin

---

From: Colin Perkins <[email-masked]>
Date: Wed, 20 Jul 2011 23:24:44 +0100

If you're not doing network-based QoS to prioritise certain flows, then this is requires two bidirectional UDP flows: one for the audio and one for video, using the standard RTP mechanisms (which is what I understand Magnus' option A to be).

Colin

On 20 Jul 2011, at 10:49, Harald Alvestrand wrote:
> with the application not requiring the network to treat the packets any differently, and with the application using the CNAME mechanism to match audio streams with corresponding video streams
> 
> it is completely unclear to me whether your choice A) requires 8 transport flows, 4 transport flows or 2 transport flows, and it is completely unclear to me what C) actually means. (I assume "transport flow" = "RTP sessions" here)
> 

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Wed, 20 Jul 2011 23:39:01 +0100

My preference is for option A: send each RTP session on a separate lower layer flow (e.g., a separate UDP flow). Yes, this means that your typical multiparty audio/video conference uses two UDP ports rather than 1, but I do not believe this is a significant problem, since most browsers open far more than 2 TCP ports simultaneously when fetching web resources.

That said, I understand the desire to reduce ICE negotiation times. If this is a real concern (and I'm not convinced it should be, since I don't see that this adds more than 1 RTT to the negotiation in the usual case), then some means of explicitly multiplexing RTP sessions might be useful to develop. I do not believe SSRC-based multiplexing of RTP sessions works though: we would need to develop a multiplexing mechanism that conforms to RTP semantics, or get AVTCORE to agree to a change in RTP with all the backwards compatibility issues that raises.

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: Matthew Kaufman <[email-masked]>
Date: Wed, 20 Jul 2011 15:40:44 -0700

On Jul 20, 2011, at 3:39 PM, Colin Perkins wrote:

> we would need to develop a multiplexing mechanism that conforms to RTP semantics, or get AVTCORE to agree to a change in RTP with all the backwards compatibility issues that raises.

Or decide that RTP is old and broken and not use it at all.

Matthew Kaufman

---

From: Stefan Håkansson LK <[email-masked]>
Date: Thu, 21 Jul 2011 07:49:54 +0200

On 2011-07-20 16:24, Bernard Aboba wrote:

>  > My experience is the answer to the first scenario is not as quick as
> you would like and the answer to how long the second takes is about 8
> times longer than the first one. You might do a bit better than that
> depending on how clever your implementation is but it still a lot longer.
(I think Colin concluded it would be 2 not 8)
>
> [BA] Yes, that is correct. However at the moment I'd characterize that
> as a "high quality problem", since it isn't even clear that the current
> WHATWG API can handle that case, due to glare and ICE timing issues.
Not sure what you're meaning. We've made an experimental implementation, 
and do not see glare or ICE timing issues.

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 22 Jul 2011 15:03:30 +0200

Cullen,

I mostly agree with Colin here. But looking at the advanced use case
this might be considered, and a bit on the control hierachy around it I
would say that this is either 2, 3 or 4 RTP Sessions. It is primarily
about how you would write your application and what type of RTP
conference mixer you have and what behavior it has.

For an application that has over the top labeling of the streams and
where the RTP mixer also can handle these streams appropriate it can be
only two RTP sessions. One for audio and one for video using SSRC
multiplexing to mux all the video streams.

In case you have your 5 participants being composited together in the
mixer and delivered as one stream, or some other behavior in the central
node you likely need to have one RTP session for that behavior. The
Active speaker and the presentation could simply be relayed as equal
SSRC multiplexed streams in one RTP session. Thus giving us one audio
and two video.

In worst case from a session perspective, the application writer has
decided that it is simplest to put map the stream semantics to the RTP
sessions, thus creating one Video for conference participants, one for
the presentation, and one for the active speaker(presenter) video. Thus
resulting in four streams.

There is two main things affecting the RTP session usage in an
particular application. The need of the application itself, and the
capabilities of any central node to perform the behavior desired by the
application.

Yes, I know that RTP Sessions appear a difficult concept. But, I would
recommend that everyone remembers that:

- An application can contain any number of RTP sessions.
- An RTP session can carry multiple media streams of the same type
- Multiple RTP sessions for the same media type can be used to separate
streams of different usages/behavior/processing in the applications or
any central node.

Cheers

Magnus

>> Second scenario: same as first address wise but instead of a single
>> audio stream they want to set up a single audio stream  to a
>> conference bridge plus 7 video streams for the video from the 5
>> people on the bridge plus a presentation stream and stream of video
>> from active speaker. I don't really care much about the scenario
>> other than there are 8 streams being set up. But this type of
>> scenario is becoming very common for multi party video chat as it
>> allows you to see perhaps small versions of everyone plus a large
>> version of active speaker.
>> 
>> My experience is the answer to the first scenario is not as quick
>> as you would like and the answer to how long the second takes is
>> about 8 times longer than the first one. You might do a bit better
>> than that depending on how clever your implementation is but it
>> still a lot longer.
> 
> I may be misunderstanding, but why would this take 8 times as long to
> set up? Wouldn't you implement it as one UDP flow for the RTP audio,
> plus one UDP flow for the RTP video (with the 7 video streams
> multiplexed by SSRC, since RTP has always allowed multiparty flows of
> the same media type in a single RTP session)? This might take twice
> as long to setup as a single audio flow in the worst case, but not
> eight times as long.
> 
-- 

Magnus Westerlund

---

From: "Avasarala, Ranjit" <[email-masked]>
Date: Tue, 26 Jul 2011 13:28:54 +0800

Hi all

By when can we expect WebRTC code (Google code) to support Websockets?

Regards
Ranjit

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 26 Jul 2011 08:50:52 -0400

> By when can we expect WebRTC code (Google code) to support Websockets?
Implementation issues with the Google WebRTC code should be addressed to 
the Google WebRTC implementation support mailing list: 
[email-masked]

In this forum, it may be appropriate to describe exactly how you see 
Websockets fitting in with the RTCWEB specification suite; so far, it's 
seemed to be not very closely related.