Subject: [rtcweb] Fwd: I-D Action: draft-rosenberg-rtcweb-rtpmux-00.txt

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 04 Jul 2011 18:19:00 -0400

We put together the draft below to argue in favor of using a single port 
for multiplexing voice and video traffic over RTP.

Looking forward to a lively discussion on this, I am sure.. ;)

Thanks,
Jonathan R.

-------- Original Message --------
Subject: I-D Action: draft-rosenberg-rtcweb-rtpmux-00.txt
Date: Mon, 04 Jul 2011 15:14:25 -0700
From: [email-masked]
Reply-To: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.

	Title           : Multiplexing of Real-Time Transport Protocol (RTP) 
Traffic for Browser based Real-Time Communications (RTC)
	Author(s)       : Jonathan Rosenberg
                           Cullen Jennings
                           Jon Peterson
                           Matthew Kaufman
                           Eric Rescorla
                           Tim Terriberry
	Filename        : draft-rosenberg-rtcweb-rtpmux-00.txt
	Pages           : 12
	Date            : 2011-07-04

    This document argues that multiplexing of voice and video traffic
    over a single RTP session should be specified as the baseline mode of
    operation for multimedia traffic in RTC web.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-rosenberg-rtcweb-rtpmux-00.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-rosenberg-rtcweb-rtpmux-00.txt
_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: Henry Sinnreich <[email-masked]>
Date: Wed, 06 Jul 2011 15:21:45 -0500

+1
Though usually quite skeptical of I-D's not supported by open, available
code and test results, I find this I-D is quite refreshing, realistic and
based on the huge deployment experience of the products supported by the
authors.

The only nit at first reading is not considering audio/video/IM chat as the
normal minimal deployment scenario, either CS or P2P.

Thanks, Henry

On 7/4/11 5:19 PM, "Jonathan Rosenberg" <[email-masked]> wrote:

> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 6 Jul 2011 15:26:55 -0600

On Jul 6, 2011, at 2:21 PM, Henry Sinnreich wrote:

> code and test results

I hope to have open, available code, and results by later this year ...

---

From: Matthew Kaufman <[email-masked]>
Date: Sun, 24 Jul 2011 16:44:26 -0400

I've been looking for technical reasons and/or hard requirements that would argue against doing RTP multiplexing as proposed in this draft.

So far my conclusion is that the RTP specifications recommend against this, but do not prohibit it.

Supporting evidence:
RFC 3550 Section 2.2 notes that multiplexing A&V together may be a problem if you want to receive one and not together (presumably in the multicast case) and refers to Section 5.2 for more support.

RFC 3550 Section 5.2 says "Separate audio and video streams SHOULD NOT be carried in a single RTP session and demultiplexed based on the payload type or SSRC fields." and provides five reasons. Note the "SHOULD NOT" as opposed to "MUST NOT".

The same section says that the five reasons apply to multiplexing on payload type, only the last two apply to multiplexing on SSRC. These remaining reasons are:

   4. An RTP mixer would not be able to combine interleaved streams of
      incompatible media into one stream.

Not relevant to the RTCWEB peer-to-peer case as the browser is not an RTP mixer. Also not relevant to any RTCWEB use cases that I can identify. And, if an RTP mixer at one end it can simply not agree to use multiplexing.

   5. Carrying multiple media in one RTP session precludes: the use of
      different network paths or network resource allocations if
      appropriate;

Appears to not be relevant to the RTCWEB case where a unicast path will be negotiated using ICE.

      reception of a subset of the media if desired, for
      example just audio if video would exceed the available bandwidth;

Appears to not be relevant to anything but the multicast cases. Any unicast case, including RTCWEB, can simply not have that component of the stream sent to it.

      and receiver implementations that use separate processes for the
      different media, whereas using separate RTP sessions permits
      either single- or multiple-process implementations.

Appears to not be relevant to the RTCWEB use case. Additionally, if it is necessary for there to be separate processes or even devices at the far end this can be solved by again simply not agreeing to use multiplexing.

In all my reading today I have not been able to find anything more concrete than the "SHOULD NOT" in section 5.2 of RFC3550. PLEASE follow up if you are aware of any other relevant specifications that would argue against using SSRC to multiplex audio and video streams over a single RTP session between a pair of compatible endpoints that agree to do so.

Matthew Kaufman

---

From: Randell Jesup <[email-masked]>
Date: Sun, 24 Jul 2011 20:15:00 -0400

> Not relevant to the RTCWEB peer-to-peer case as the browser is not an RTP mixer. Also not relevant to any RTCWEB use cases that I can identify. And, if an RTP mixer at one end it can simply not agr…

Well, there's no reason a browser using rtcweb couldn't be acting as a 
mixer for sessions with multiple other rtcweb clients/browsers.  
However, a) it could disable multiplexing (if we provide a control for 
that), and b) I'm not sure that I agree with the reasoning in 4) above.  
Perhaps someone could explain why that would be a problem (not having 
played with mixers much, I can't think of a reason off the top of my head).

> Appears to not be relevant to the RTCWEB case where a unicast path will be negotiated using ICE.

Well, 5) is relevant, though you can decide you don't care.  For example 
the concern raised about network-applied QOS to different streams 
from/to a client.  The different network paths isn't important here.

> Appears to not be relevant to anything but the multicast cases. Any unicast case, including RTCWEB, can simply not have that component of the stream sent to it.

Right, and no bandwidth will be saved unless you stop them from sending 
the stream.  Given offer/answer/etc this issue is moot.

> Appears to not be relevant to the RTCWEB use case. Additionally, if it is necessary for there to be separate processes or even devices at the far end this can be solved by again simply not agreeing…

Agreed, not relevant to our case.

> In all my reading today I have not been able to find anything more concrete than the "SHOULD NOT" in section 5.2 of RFC3550. PLEASE follow up if you are aware of any other relevant specifications t…

People certainly have done it, and people have also (though perhaps 
inadvisedly) had clients that would have several sessions with different 
clients terminating at the same port -- though people having done it 
doesn't mean it's good.

-- 
Randell Jesup
[email-masked]

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 25 Jul 2011 08:06:53 -0400

> In all my reading today I have not been able to find anything more concrete than the "SHOULD NOT" in section 5.2 of RFC3550. PLEASE follow up if you are aware of any other relevant specifications t…
I have found *one* reason not mentioned in the draft:

An RTP session with both "audio" and "video" media types cannot be 
represented by an SDP description, since SDP ties RTP sessions 1-1 to 
the "m" line of the description, which contains the top-level type, and 
the codec descriptions omit the top-level type in their codec naming.

I've said elsewhere that I consider this to be a design mistake for 
entirely different reasons, but this debate has reinforced my opinion 
that it is.

                      Harald

---

From: Emil Ivov <[email-masked]>
Date: Mon, 25 Jul 2011 14:17:25 +0200

> the codec descriptions omit the top-level type in their codec naming.

I am not sure I understand this. Why would one need to use the same "m"
line? What would be the consequences of using two different "m" lines?

Personally, I am still not convinced that multiplexing brings
significant improvement over the status quo but if we accept it then it
definitely needs to keep separate "m" lines for tons of reasons.

Emil

-- 
http://jitsi.org

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 25 Jul 2011 08:30:20 -0400

> definitely needs to keep separate "m" lines for tons of reasons.
RFC 4566 section 5:

    An SDP session description consists of a session-level section
    followed by zero or more media-level sections.  The session-level
    part starts with a "v=" line and continues to the first media-level
    section.  Each media-level section starts with an "m=" line and
    continues to the next media-level section or end of the whole session
    description.

....

5.14.  Media Descriptions ("m=")

       m=<media> <port> <proto> <fmt> ...

    A session description may contain a number of media descriptions.
    Each media description starts with an "m=" field and is terminated by
    either the next "m=" field or by the end of the session description.
    A media field has several sub-fields:

<media> is the media type.  Currently defined media are "audio",
       "video", "text", "application", and "message", although this list
       may be extended in the future (see Section 8).

<port> is the transport port to which the media stream is sent.

In other words: an "m" line is tied to a port number, and (from RTP) the 
port number identifies the RTP session. The ICE parameters can override 
the port number, but they're carried within the "m" block, so they're 
tied to a single "m" line too.

I do not know what would happen if you used the same port number and ICE 
parameters with two "m" lines.

---

From: Randell Jesup <[email-masked]>
Date: Mon, 25 Jul 2011 08:31:11 -0400

> definitely needs to keep separate "m" lines for tons of reasons.

I've been assuming the method of indicating multiplexing would be two m= 
lines with
the same c= destination and the same port specified.  This is a 
mechanism that has
been discussed before on the AVT list and/or in drafts (and I'm sure has 
also been
done before).  It does require avoiding using the same payload values in 
the two m= lines,
but beyond that it's fairly straightforward.  It also allows for the 
answerer to clearly reject a
media type explicitly, instead of an implicit rejection by removing the 
payloads of a type.

-- 
Randell Jesup
[email-masked]

---

From: Colin Perkins <[email-masked]>
Date: Mon, 25 Jul 2011 08:35:01 -0400

Section 2.2 of draft-perkins-rtcweb-rtp-usage-02 lists several reasons why this sort of multiplexing is problematic. I would suggest, again, that you read that draft. 

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: Emil Ivov <[email-masked]>
Date: Mon, 25 Jul 2011 14:46:03 +0200

> tied to a single "m" line too.

OK, but this simply brings the discussion back to whether or not we can
have audio and video on the same port. I still don't understand how it
adds extra constraints and why two m-lines with the same port would be
prohibited by SDP itself.

> parameters with two "m" lines.

Aha! Right!

ICE itself would be able to handle this based on the user credentials.
However, having two sets of ICE candidates in both media descriptions
would simply trigger ICE processing for both of them.

Given that the whole idea behind multiplexing is to simplify ICE
signalling, I suppose this kind of kills it indeed.

Emil

-- 
http://jitsi.org

---

From: Colin Perkins <[email-masked]>
Date: Mon, 25 Jul 2011 08:50:35 -0400

> Not relevant to the RTCWEB peer-to-peer case as the browser is not an RTP mixer. Also not relevant to any RTCWEB use cases that I can identify. And, if an RTP mixer at one end it can simply not agr…

This would work if you are directly talking to a mixer. It precludes the cases where you are talking indirectly to a mixer via a gateway, unless you define additional signalling to determine that there is a mixer on the remote side of the gateway. 

> Appears to not be relevant to the RTCWEB case where a unicast path will be negotiated using ICE.

An RTCWeb client using ICE could negotiate different network paths for the audio and video flows, and could use network-based QoS mechanisms, if flows were sent separately. Those mechanisms work just fine in unicast.

> Appears to not be relevant to anything but the multicast cases. Any unicast case, including RTCWEB, can simply not have that component of the stream sent to it.

Relevant for proxies which do in-network adaptation.

> In all my reading today I have not been able to find anything more concrete than the "SHOULD NOT" in section 5.2 of RFC3550. PLEASE follow up if you are aware of any other relevant specifications t…

   Timing and Sequence Number Space:  An RTP SSRC is defined to identify
      a single timing and sequence number space.  Interleaving multiple
      payload types would require different timing spaces if the media
      clock rates differ and would require different sequence number
      spaces to tell which payload type suffered packet loss.  Using
      multiple clock rates in a single RTP session is problematic, as
      discussed in [I-D.ietf-avtext-multiple-clock-rates].  This can be
      avoided by partitioning the SSRC space between the two sessions,
      but that causes other problems as discussed below.

   RTCP Reception Reports:  RTCP sender reports and receiver reports can
      only describe one timing and sequence number space per SSRC, and
      do not carry a payload type field.  Multiplexing sessions based on
      the payload type breaks RTCP.  This can be avoided by partitioning
      the SSRC space between the two sessions, but that causes other
      problems as discussed below.

   Scalability:  RTP was built with media scalability in consideration.
      The simplest way of achieving separation between different
      scalability layers is placing them in different RTP sessions, and
      using the same SSRC and CNAME in each session to bind them
      together.  This is most commonly done in multicast, and not
      particularly applicable to RTC-Web, but gatewaying of such a
      session would then require more alterations and likely stateful
      translation.

   RTP Retransmission in Session Multiplexing mode:  RTP Retransmission
      [RFC4588] does have a mode for session multiplexing.  This would
      not be the main mode used in RTC-Web, but for interoperability and
      reduced cost in translation support for different RTP Sessions are
      beneficial.

   Forward Error Correction:  The "An RTP Payload Format for Generic
      Forward Error Correction" [RFC2733] and its update [RFC5109] can
      only be used on media formats that produce RTP packets that are
      smaller than half the MTU if the FEC flow and media flow being
      protected are to be sent in the same RTP session, this is due to
      "RTP Payload for Redundant Audio Data" [RFC2198].  This is because
      the SSRC value of the original flow is recovered from the FEC
      packets SSRC field.  So for anything that desires to use these
      format with RTP payloads that are close to MTU needs to put the
      FEC data in a separate RTP session compared to the original
      transmissions.  The usage of this type of FEC data has not been
      decided on in RTCWEB.

   SSRC Allocation and Collision:  The SSRC identifier is a random 32-
      bit number that is required to be globally unique within an RTP
      session, and that is reallocated to a new random value if an SSRC
      collision occurs between participants.  If two or more RTP
      sessions share a transport layer flow, there is no guarantee that
      their choice of SSRC values will be distinct, and there is no way
      in standard RTP to signal which SSRC values are used by which RTP
      session.  RTP is explicitly a group-based communication protocol,
      and new participants can join an RTP session at any time; these
      new participants may chose SSRC values that conflict with the SSRC
      values used in any of the multiplexed RTP sessions.  This problem
      can be avoided by partitioning the SSRC space, and signalling how
      the space is to be subdivided, but this is not backwards
      compatible with any existing RTP system.  In addition, subdividing
      the SSRC space makes it difficult to gateway between multiplexed
      RTP sessions and standard RTP sessions: the standard sessions may
      use parts of the SSRC space reserved in the multiplexed RTP
      sessions, requiring the gateway to rewrite RTCP packets, as well
      as the SSRC and CSRC list in RTP data packets.  Rewriting RTCP is
      a difficult task, especially when one considers extensions such as
      RTCP XR.

   Conflicting RTCP Report Types:  The extension mechanisms used in RTCP
      depend on separation of RTP sessions for different media types.
      For example, the RTCP Extended Report block for VoIP is suitable
      for conversational audio, but clearly not useful for Video.  This
      may cause unusable or unwanted reports to be generated for some
      streams, wasting capacity and confusing monitoring systems.  While
      this is problem may be unlikely for VoIP reports, it may be an
      issue for the more detailed media agnostic reports which are
      sometimes be used for different media types.  Also, this makes the
      implementation of RTCP more complex, since partitioning the SSRC
      space by media type needs not only to be one the media processing
      side, but also on the RTCP reporting

   RTCP Reporting and Scheduling:  The RTCP reporting interval and its
      packet scheduling will be affected if several RTP sessions are
      multiplexed onto the same transport layer flow.  The reporting
      interval is determined by the session bandwidth, and the reporting
      interval chosen for a high-rate video session will be different to
      the interval chosen by a low-rate VoIP session.  If such sessions
      are multiplexed, then participants in one session will see the
      SSRC values of the other session.  This will cause them to
      overestimate the number of participants in the session by a factor
      of two, thus doubling their RTCP reporting interval, and making
      their feedback less timely.  In the worst case, when an RTP
      session with very low RTCP bandwidth is multiplexed with an RTP
      session with high RTCP bandwidth, this may cause repeated RTCP
      timer reconsideration, leading to the members of the low bandwidth
      session timing out.  Participants in an RTP session configured
      with high bandwidth (and short RTCP reporting interval) will see
      RTCP reports from participants in the low bandwidth session much
      less often than expected, potentially causing them to repeatedly
      timeout and re-create state for those participants.  The split of
      RTCP bandwidth between senders and receivers (where at least 25%
      of the RTCP bandwidth is allocated to senders) will be disrupted
      if a session with few senders (e.g., a VoIP session) is
      multiplexed with a session with many senders (e.g., a video
      session).  These issues can be resolved if the partition of the
      SSRC is signalled, but this is not backwards compatible with any
      existing RTP system.  The partition would require re-implementing
      large part of the RTCP processing to take the individual sessions
      into account.

   Sampling Group Membership:  The mechanism defined in RFC2762 to
      sample the group membership, allowing participants to keep less
      state, assumes a single flat 32-bit SSRC space, and breaks if the
      SSRC space is shared between several RTP sessions.

-- 
Colin Perkins
http://csperkins.org/

---

From: "Muthu Arul Mozhi Perumal (mperumal)" <[email-masked]>
Date: Mon, 25 Jul 2011 18:27:34 +0530

A few options to indicate this multiplexing in SDP and being backward compatible with legacy implementations:

1) The simplest option is to use the same port value across multiple "m=" lines in the SDP. Unfortunately, that won't be backward compatible with legacy implementations because of RFC-4566 section 5.14:
<snip>
   The semantics of multiple "m=" lines using the same transport address are undefined.
</snip>

So, if a legacy implementation receives such an offer, the behavior would be unpredictable.

2) Another option is to simply list all the (audio/video) codes in a single "m=" line and define a new (optional) SDP attribute to indicate multiplexing. This also is no way going to be backward compatible with legacy implementations, since a legacy endpoint might typically send an answer with a single codec, resulting in an audio-only/video-only call, when audio + video was intended.

3) Another options is SDP grouping. Unfortunately, RFC-3388 section 7.5.3 forbids grouping multiple "m=" lines using the same transport address:
<snip>
   If several codecs have to be sent to the same IP address and port,
   the traditional SDP syntax of listing several codecs in the same "m"
   line MUST be used.  FID MUST NOT be used to group "m" lines with the
   same IP address/port.  Therefore, an SDP like the one below MUST NOT
   be generated.

       v=0
       o=Laura 289083124 289083124 IN IP4 six.example.com
       t=0 0
       c=IN IP4 131.160.1.112
       a=group:FID 1 2
       m=audio 30000 RTP/AVP 0
       a=mid:1
       m=audio 30000 RTP/AVP 8
       a=mid:2

   The correct SDP for the session above would be the following one:

       v=0
       o=Laura 289083124 289083124 IN IP4 six.example.com
       t=0 0
       c=IN IP4 131.160.1.112
       m=audio 30000 RTP/AVP 0 8

   If two "m" lines are grouped using FID they MUST differ in their
   transport addresses (i.e., IP address plus port).
</snip>

In addition, using the transport address across multiple "m=" lines by itself doesn't define a new group. So, using grouping doesn't look right.

4) The best option I can think of is to define a new media-level SDP attribute (say, a=muxport: <port>) that indicates a multiplexed port where the offerer/answerer is willing to receive media streams, in-addition to the port indicated in the "m=" lines.

So, the offer could look like this:
      m=audio 49170 RTP/AVP 0 8 97
      a=rtpmap:0 PCMU/8000
      a=rtpmap:8 PCMA/8000
      a=rtpmap:97 iLBC/8000
      a=muxport:5000 
      m=video 51372 RTP/AVP 31 32
      a=rtpmap:31 H261/90000
      a=rtpmap:32 MPV/90000
      a=muxport:5000

An endpoint that doesn't support multiplexing would ignore a=muxport and send a answer like this:
      m=audio 49174 RTP/AVP 0
      a=rtpmap:0 PCMU/8000
      m=video 49170 RTP/AVP 32
      a=rtpmap:32 MPV/90000

Whereas an endpoint that understand multiplexing would understand a=muxport and send an answer like this:
      m=audio 40000 RTP/AVP 0
      a=rtpmap:0 PCMU/8000
      a=muxport:4000
      m=video 40000 RTP/AVP 32
      a=rtpmap:32 MPV/90000
      a=muxport:4000

However, the overhead for new implementations is that it needs to listen on both the port listed in the "m=" lines and the port specified in a=muxport, in addition to gathering both candidates for ICE, till it receives the answer. If it wants to be backward compatible with legacy implantations, it needs to pay for it, anyway..

Muthu

|-----Original Message-----
|From: [email-masked] [mailto:[email-masked]] On Behalf Of Randell Jesup
|Sent: Monday, July 25, 2011 6:01 PM
|To: [email-masked]
|Subject: Re: [rtcweb] Reasons not to multiplex audio with video (Re: Fwd: I-D Action: draft-rosenberg-
|rtcweb-rtpmux-00.txt)
|
|On 7/25/2011 8:17 AM, Emil Ivov wrote:
|> На 25.07.11 14:06, Harald Alvestrand написа:
|>> On 07/24/11 16:44, Matthew Kaufman wrote:
|>>> In all my reading today I have not been able to find anything more concrete than the "SHOULD NOT"
|in section 5.2 of RFC3550. PLEASE follow up if you are aware of any other relevant specifications that
|would argue against using SSRC to multiplex audio and video streams over a single RTP session between
|a pair of compatible endpoints that agree to do so.
|>> I have found *one* reason not mentioned in the draft:
|>>
|>> An RTP session with both "audio" and "video" media types cannot be
|>> represented by an SDP description, since SDP ties RTP sessions 1-1 to
|>> the "m" line of the description, which contains the top-level type, and
|>> the codec descriptions omit the top-level type in their codec naming.
|> I am not sure I understand this. Why would one need to use the same "m"
|> line? What would be the consequences of using two different "m" lines?
|>
|> Personally, I am still not convinced that multiplexing brings
|> significant improvement over the status quo but if we accept it then it
|> definitely needs to keep separate "m" lines for tons of reasons.
|
|I've been assuming the method of indicating multiplexing would be two m=
|lines with
|the same c= destination and the same port specified.  This is a
|mechanism that has
|been discussed before on the AVT list and/or in drafts (and I'm sure has
|also been
|done before).  It does require avoiding using the same payload values in
|the two m= lines,
|but beyond that it's fairly straightforward.  It also allows for the
|answerer to clearly reject a
|media type explicitly, instead of an implicit rejection by removing the
|payloads of a type.
|
|--
|Randell Jesup
[email-masked]
|
|_______________________________________________
|rtcweb mailing list
[email-masked]
|https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Matthew Kaufman <[email-masked]>
Date: Mon, 25 Jul 2011 09:04:56 -0400

On Jul 25, 2011, at 8:06 AM, Harald Alvestrand wrote:

> An RTP session with both "audio" and "video" media types cannot be represented by an SDP description, since SDP ties RTP sessions 1-1 to the "m" line of the description, which contains the top-level type, and the codec descriptions omit the top-level type in their codec naming.
> 
> I've said elsewhere that I consider this to be a design mistake for entirely different reasons, but this debate has reinforced my opinion that it is.

Good point, but perhaps it just argues for either A) not using SDP or B) extending SDP if we use it for RTCWEB (we probably need to do that anyway if we use SDP, given all the discussion about "hints" that we want to send from one end to the other)

Matthew Kaufman

---

From: "Muthu Arul Mozhi Perumal (mperumal)" <[email-masked]>
Date: Mon, 25 Jul 2011 19:05:55 +0530

c) Using SDP for legacy interop, but extending STUN, by defining a
(comprehension optional) STUN attribute that indicates it is a
multiplexed media flow and that the SSRC space has been partitioned to
enable DPI. Since rtcweb apps are expected to use STUN, it should be
trivial for them to support this STUN attribute. In addition, the
software in on-path routers would have to be upgraded anyway to support
this king of DPI, so they could also be enhanced to detect STUN and look
for this attribute.

One another benefit to indicating this in STUN is that even if both
rtcweb endpoints said they supported this mechanism via SDP, the routers
where QoS, traffic engineering etc are applied to media packets many not
be there in the signaling path to understand that it is a multiplexed
media flow using the same port, and in particular that the SSRC space
has been partitioned as proposed in draft-rosenberg-rtcweb-rtpmux to
enable them to perform DPI and figure out the media type (audio/video
etc).

Muthu

|-----Original Message-----
|From: [email-masked] [mailto:[email-masked]] On
Behalf Of Matthew Kaufman
|Sent: Monday, July 25, 2011 6:35 PM
|To: Harald Alvestrand
|Cc: [email-masked]
|Subject: Re: [rtcweb] Reasons not to multiplex audio with video (Re:
Fwd:I-D Action: draft-rosenberg-
|rtcweb-rtpmux-00.txt)
|
|
|On Jul 25, 2011, at 8:06 AM, Harald Alvestrand wrote:
|
|> On 07/24/11 16:44, Matthew Kaufman wrote:
|>> In all my reading today I have not been able to find anything more
concrete than the "SHOULD NOT"
|in section 5.2 of RFC3550. PLEASE follow up if you are aware of any
other relevant specifications that
|would argue against using SSRC to multiplex audio and video streams
over a single RTP session between
|a pair of compatible endpoints that agree to do so.
|> I have found *one* reason not mentioned in the draft:
|>
|> An RTP session with both "audio" and "video" media types cannot be
represented by an SDP
|description, since SDP ties RTP sessions 1-1 to the "m" line of the
description, which contains the
|top-level type, and the codec descriptions omit the top-level type in
their codec naming.
|>
|> I've said elsewhere that I consider this to be a design mistake for
entirely different reasons, but
|this debate has reinforced my opinion that it is.
|
|Good point, but perhaps it just argues for either A) not using SDP or
B) extending SDP if we use it
|for RTCWEB (we probably need to do that anyway if we use SDP, given all
the discussion about "hints"
|that we want to send from one end to the other)
|
|Matthew Kaufman
|_______________________________________________
|rtcweb mailing list
[email-masked]
|https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 25 Jul 2011 09:50:21 -0400

On 07/25/11 09:35, Muthu Arul Mozhi Perumal (mperumal) wrote:
> enable DPI.
I think that's a layering violation. STUN has no references to RTP; 
there is no assumption in STUN that the content of the flow will be RTP. 
I think we should keep it that way.
>   Since rtcweb apps are expected to use STUN, it should be

---

From: Matthew Kaufman <[email-masked]>
Date: Mon, 25 Jul 2011 10:45:55 -0400

> this king of DPI, so they could also be enhanced to detect STUN and look

Agree with Harald that this is a layering violation, but I would not be averse to putting a hint in STUN for this, or indeed a hint in STUN (as a non-mandatory attribute) saying that we're doing other RTCWEB things, for devices that can only see the RTP/RTCP/STUN packets.

That'd get yet another WG into the mix, of course.

Matthew Kaufman

---

From: Jonathan Lennox <[email-masked]>
Date: Mon, 25 Jul 2011 13:34:46 -0400

> This would work if you are directly talking to a mixer. It precludes the cases where you are talking indirectly to a mixer via a gateway, unless you define additional signalling to determine that t…

Clearly, the gateway which is talking to the mixer needs to perform signaling negotiation with it, so it understands the RTP sessions at all, and so it will know if the mixer understands muxing or not.

> An RTCWeb client using ICE could negotiate different network paths for the audio and video flows, and could use network-based QoS mechanisms, if flows were sent separately. Those mechanisms work ju…

Yes, it could use these mechanisms, if they're appropriate.  If it's not appropriate, it wouldn't need to.

>> In all my reading today I have not been able to find anything more concrete than the "SHOULD NOT" in section 5.2 of RFC3550. PLEASE follow up if you are aware of any other relevant specifications…

[Quote from Colin's draft omitted]

A lot of these are responses to the ideas of doing PT muxing, or muxing several distinct RTP sessions onto a single transport flow.  They don't seem, mostly, to be addressing muxing RTP audio and video sources onto a single (proper) RTP session, i.e. with a single SSRC space, single (global) RTCP bandwidth value, single value for RTCP timing intervals, etc.

It's clearly the case that audio's share of 0.05 * (audio bw + video bw) is going to be much greater than 0.05 * (audio bw), so you're going to be sending audio RTCP a lot more frequently with mux'ed media than you would in an audio-only RTP session, whereas you'll send video RTCP slightly less frequently than a video-only session.  But this isn't, as far as I can tell, a problem, especially if you set a sensible value for trr_int (assuming AVPF).

In SSRC-muxing, RTX and FEC require a way of binding together original and repair streams, but RFC 5576 already defines a mechanism (ssrc-group) for doing this in SDP, and Magnus's SRCNAME proposal could alternately do it in RTCP.

I admit to not understanding the XRBlocks well enough to say whether it expects to send VoIP feedback reports for every source in a session that it thinks is audio, but presumably a monitoring device that's monitoring a VoIP RTP session already needs to be told whether the session it's monitoring is audio, video, or other, and would be able to (or have to) ignore a mux'd session.

--
Jonathan Lennox
[email-masked]

---

From: Jonathan Lennox <[email-masked]>
Date: Mon, 25 Jul 2011 13:54:14 -0400

> A few options to indicate this multiplexing in SDP and being backward compatible with legacy implementations:

Unfortunately, I think the "correct" way to do this would be SDP media capability negotiation -- you'd offer an alternative of either two m= lines (one audio and one video), or else a single, mux'd m= line.  Anything else means that your SDP is, effectively, lying about what it's actually doing, and I'm really worried about how that would interact with other SDP extensions.

I'm certainly not crazy about this suggestion, though, because SDP Capability Negotiation is pretty ugly, and requires an additional offer/answer exchange to converge.

For RTCWeb proper, the need not to have worry about compatibility with legacy implementations means that much of this could potentially be avoided.

--
Jonathan Lennox
[email-masked]

---

From: "Muthu Arul Mozhi Perumal (mperumal)" <[email-masked]>
Date: Tue, 26 Jul 2011 21:55:14 +0530

First refinement:
Define a new SDP attribute "a=muxed-candidate" that is similar to the
"a=candidate" attribute defined by ICE, except that the same
muxed-candidate can appear under multiple media stream descriptions in
SDP. The muxed-candidate identifies the transport address where the
RTCWEB endpoint would like to receive all these (multiplexed) media
streams. Non-RTCWEB endpoints would ignore this attribute.

Define a new comprehension-optional STUN attribute MUXED-CANDIDATE
(similar to the USE-CANDIDATE attribute). This attribute has no content
(the Length field of the attribute is zero) -- it serves as a flag.  It
has an attribute value of 0x802C (the first unassigned value from the
first half of the comprehension-optional range). It can be present in
the STUN connectivity check(s) sent when ICE concludes (i.e those STUN
Binding requests that contain the USE-CANDIDATE attribute) and serves as
an indication to routers and firewalls that any media stream that flows
between the selected pair of candidates is a multiplexed media stream --
for RTP it means that the SSRC space has been partitioned as described
in draft-rosenberg-rtcweb-rtpmux to enable DPI.

IMO, this isn't a layer violation per se, since we aren't carrying any
RTP stuff in STUN, rather just providing an indication for a particular
RTP usage.

Once issue, though -- comprehension-optional STUN attribute aren't
protected by STUN MESSAGE-INTEGRITY. So, MUXED-CANDIDATE would have to
protect itself or a new comprehension-optional STUN needs to be defined
that protects other comprehension-optional STUN attributes.

Muthu

|-----Original Message-----
|From: Matthew Kaufman [mailto:[email-masked]]
|Sent: Monday, July 25, 2011 8:16 PM
|To: Muthu Arul Mozhi Perumal (mperumal)
|Cc: Harald Alvestrand; [email-masked]
|Subject: Re: [rtcweb] Reasons not to multiplex audio with video (Re:
Fwd:I-D Action: draft-rosenberg-
|rtcweb-rtpmux-00.txt)
|
|
|On Jul 25, 2011, at 9:35 AM, Muthu Arul Mozhi Perumal (mperumal) wrote:
|
|> c) Using SDP for legacy interop, but extending STUN, by defining a
|> (comprehension optional) STUN attribute that indicates it is a
|> multiplexed media flow and that the SSRC space has been partitioned
to
|> enable DPI. Since rtcweb apps are expected to use STUN, it should be
|> trivial for them to support this STUN attribute. In addition, the
|> software in on-path routers would have to be upgraded anyway to
support
|> this king of DPI, so they could also be enhanced to detect STUN and
look
|> for this attribute.
|
|Agree with Harald that this is a layering violation, but I would not be
averse to putting a hint in
|STUN for this, or indeed a hint in STUN (as a non-mandatory attribute)
saying that we're doing other
|RTCWEB things, for devices that can only see the RTP/RTCP/STUN packets.
|
|That'd get yet another WG into the mix, of course.
|
|Matthew Kaufman

---

From: Ross Finlayson <[email-masked]>
Date: Tue, 26 Jul 2011 16:55:32 -0400

When I first started thinking about the 'multiplexing' question, I 
tended to support Colin's point-of-view.  Carrying audio and video on 
separate ports gives a system more flexibility, is compatible with 
existing implementations (including mine :-), and is in line with 
long-standing assumptions about the use of the SSRC, RTCP etc.

But over time I have become increasingly sympathetic to Jonathan's 
arguments - especially regarding NAT traversal - in particular, setup 
time, use of resources (# of ports), and failure modes.  We have to 
assume that many, if not most users of RTCWEB systems will be on 
unsophisticated, heavily NATed IPv4 home (or hotel, or coffee shop 
etc.) networks.  As IPv4 addresses become scarcer, this situation 
will likely only get worse.

However, as someone noted (in the context of "security") during 
today's RTCWEB session, we're not the first people to be designing a 
(hopefully) widely-deployed consumer-oriented peer-to-peer A/V chat 
system.  There are several existing systems out there, so it might be 
instructive to look at how they address the 'media multiplexing' 
issue - in particular:

1/ Apple's "FaceTime".  OK, this is not browser-based (although with 
a bit of work it probably could have been), but it does use RTP.  Is 
it using separate ports for audio and video (and two more for RTCP)? 
If so, has NAT traversal been a problem, and would it have been 
beneficial to have been able to reduce the number of ports used?  (Is 
Dave Singer on this mailing list?)

2/ Facebook's new video chat.  OK, this uses Skype, and thus 
presumably not RTP (at least, not now).  But does this multiplex 
audio+video on a single port?  (And if so, did this help motivate 
Jonathan's point-of-view?)

3/ Google Plus "Hangouts".  Similar question to 1/.

4/ Any others (whether RTP-based or not)?

-- 

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

---

From: Matthew Kaufman <[email-masked]>
Date: Tue, 26 Jul 2011 17:34:43 -0400

On Jul 26, 2011, at 4:55 PM, Ross Finlayson wrote:
> 
> 2/ Facebook's new video chat.  OK, this uses Skype, and thus presumably not RTP (at least, not now).  But does this multiplex audio+video on a single port?

Yes.

>  (And if so, did this help motivate Jonathan's point-of-view?)

I can't answer this.

> 4/ Any others (whether RTP-based or not)?

All systems that use Flash Player's RTMFP for peer-to-peer audio/video communication (an early example was Chatroulette) multiplex audio+video on a single port *and* in fact use that single port to talk to all the other participants in the multi-party cases. (So for instance, when you "next" someone in Chatroulette, your local address+port are the same as they were for the last guy.)

Matthew Kaufman

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 26 Jul 2011 17:46:04 -0400

> 3/ Google Plus "Hangouts".  Similar question to 1/.
Google+ Hangouts uses SRTP, with one port for video and one port for 
audio. Many audio and video channels (multiplexed by SSRC) in the 
server->client direction.

We multiplex RTCP onto the RTP port as specified in the relevant RFC.

                       Harald

---

From: Bala Pitchandi <[email-masked]>
Date: Tue, 26 Jul 2011 17:48:30 -0400

> 1/ Apple's "FaceTime".  OK, this is not browser-based (although with a bit of work it probably could have been), but it does use RTP.  Is it using separate ports for audio and video (and two more for RTCP)? 
> If so, has NAT traversal been a problem, and would it have been beneficial to have been able to reduce the number of ports used?  (Is Dave Singer on this mailing list?)

I believe FaceTime multiplexes audio & video on to the same port. In fact, it even multiplexes the signaling on to the same port. See: http://blog.roychowdhury.org/2010/06/25/facetime-on-iphone-4-vanilla-unencrypted-stun-and-sip/

-- Bala

---

From: Aron Rosenberg <[email-masked]>
Date: Tue, 26 Jul 2011 14:58:11 -0700

Facetime muxes everything onto the same port using RTP SSRC and signals that
in SDP using an a=rtpID attribute inside each m= line. Each m= line has the
same <port> value.

-Aron

On Tue, Jul 26, 2011 at 2:48 PM, Bala Pitchandi <[email-masked]> wrote:

> > 1/ Apple's "FaceTime".  OK, this is not browser-based (although with a
> bit of work it probably could have been), but it does use RTP.  Is it using
> separate ports for audio and video (and two more for RTCP)?
> beneficial to have been able to reduce the number of ports used?  (Is Dave
> Singer on this mailing list?)
>
> I believe FaceTime multiplexes audio & video on to the same port. In fact,
> it even multiplexes the signaling on to the same port. See:
> http://blog.roychowdhury.org/2010/06/25/facetime-on-iphone-4-vanilla-unencrypted-stun-and-sip/
>
> -- Bala
>