Subject: Re: [rtcweb] Regarding cloning of PeerConnections when multiple answers are received

From: Justin Uberti <[email-masked]>
Date: Sun, 1 Apr 2012 23:21:49 -0400

On Thu, Mar 29, 2012 at 3:57 AM, Magnus Westerlund <
[email-masked]> wrote:

> WG, as individual
>
> As just discussed in the meeting Justin and Hadriel didn't think that
> you could clone PeerConnections properly. I do believe it is possible.
>
> If the API provides a method where you can clone a current
> peerConnection you can actually share the transport layer information at
> one end-point for multiple peer connections assuming that the other
> end-points transport addresses are different between the different
> peerConnections being cloned.
>
> This works as the complete ICE username is the concatenation of both
> sides, as long as an secondary answer contains a different username and
> has candidates with different transport addresses (address+port). Then
> end-point A receiving the second answer can have the ICE stack accept
> both usernames and if connectivity checks works then you create
> different 5-tuple filter to divide any media streams to the right peer
> connections media handler.
>
> When you clone a PC then must allocate the media decoding resources for
> each cloning. And that is something the cloning function may block on.
>
>
Thanks Magnus. This sounds reasonable; it might even be possible to make
cloning work using the existing API by taking the local description from
one PC and installing it as the local description for a new PC; the browser
could recognize that the candidates refer to existing ports and behave as
you mention above.

However, given that this seems to be mostly an edge case, I think we can
probably save this for v2.

---

From: Hadriel Kaplan <[email-masked]>
Date: Mon, 2 Apr 2012 13:02:33 +0000

Hi,
I was thinking the tricky part is more about the local transport stuff getting cloned - i.e., imagine if you're using a TCP-based TURN connection.
But maybe it's a non-issue.

-hadriel

On Mar 29, 2012, at 3:57 AM, Magnus Westerlund wrote:

> Cheers
> Magnus Westerlund
> ----------------------------------------------------------------------
> Multimedia Technologies, Ericsson Research EAB/TVM
> ----------------------------------------------------------------------
> Ericsson AB                | Phone  +46 10 7148287
> Färögatan 6                | Mobile +46 73 0949079
> SE-164 80 Stockholm, Sweden| mailto: [email-masked]
> ----------------------------------------------------------------------
> 
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Roman Shpount <[email-masked]>
Date: Mon, 2 Apr 2012 12:09:22 -0400

On Mon, Apr 2, 2012 at 9:02 AM, Hadriel Kaplan <[email-masked]>wrote:

>
> Hi,
> I was thinking the tricky part is more about the local transport stuff
> getting cloned - i.e., imagine if you're using a TCP-based TURN connection.
This really depends on how the TURN support is implemented, but typically
using the same TCP connection or any other TURN allocation for multiple PC
should be no more of an issue then using same connection for multiple
streams within the PC.
_____________
Roman Shpount

---

From: Randell Jesup <[email-masked]>
Date: Mon, 02 Apr 2012 13:22:03 -0400

> one PC and installing it as the local description for a new PC; the
> browser could recognize that the candidates refer to existing ports and
> behave as you mention above.

Yes, this seems like it would work.

> probably save this for v2.

So the question for me is "what scenarios/use-cases does this enable?"

1) Mesh Conferencing

While there are many ways to handle conferencing, if you have a mesh 
(full or partial), you need to create new PeerConnections for existing 
members of the mesh when someone joins.  There are two ways to do this:
a) somehow application-specific tell the new member who to send invites 
to, and then start N PeerConnection calls with some "I'm in the 
conference" token;
b)have them send N separate generic invites to the service provider 
which either forwards it to an existing member, or returns it with "it's 
ok, I've already forwarded invites for it);
c) somehow tell all the existing members to invite the new member; or
d) have the service provider fork the OFFER from the new member to all 
the existing members, and have all of them reply with ANSWERs.  This 
requires PeerConnection cloning to work well.

I strongly prefer d), and it also reduces conference join time (less 
round-trips, at least in theory - perhaps noticeably less by sharing the 
ICE candidate generation).

2) "Hanging" invites at a service provider

You send an OFFER, which is "held" at the service provider and then 
"forked"/forwarded to other clients when the server decides to.

This is of use in cases where there is external state (a game, virtual 
environment ala Second Life, virtual conference hall, etc) where 
something at the server decides when you should actually be connected 
and to whom (or multiple people).  Example (applies to several of 
these): as you move around a game/virtual-environment, your app is 
connected to other nearby players when they're in audible range.  This 
could be done by having the server tell your client to generate an OFFER 
(with attendant ICE discovery delay) on demand for each other person who 
comes in range, but that would be sub-optimal and again involve setup 
delay.  It also allows permission to be dealt-with up front and not 
delay actual connection.

I imagine there are others, but those are two which I think are pretty 
likely uses (we've been talking mesh conferences since the beginning, 
and this sort of environmental audio/teamspeak type stuff is an obvious 
use).  Our game-table demo showed chess, but other uses would involve 
more players (4 for bridge, etc) where you might want a full-mesh 
conference).

So, I'd really like to push to support cloning if possible.

-- 
Randell Jesup
[email-masked]

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 09 Apr 2012 13:02:24 +0200

On 04/02/2012 07:22 PM, Randell Jesup wrote:
> a) somehow application-specific tell the new member who to send 
> invites to, and then start N PeerConnection calls with some "I'm in 
> the conference" token;
> which either forwards it to an existing member, or returns it with 
> "it's ok, I've already forwarded invites for it);
> round-trips, at least in theory - perhaps noticeably less by sharing 
> the ICE candidate generation).

Hmm.... for one new member, I don't think d) is actually different from 
the case where the client sends an "extra" OFFER when he joins, and the 
server uses the "hanging OFFER" to signal the new member.
This is a windowing protocol, though - the client would need to send a 
new "hanging OFFER" after the current "hanging OFFER" has been "used 
up". Cloning allows you to connect multiple new conference mebers as 
fast as the Javascript can accept them. (Which may have issues with 
congestion, but that might be a price worth paying).

It's not clear to me whether hanging OFFERs work with NAT traversal; 
would the ICE machine send keepalives for an association that hasn't 
been associated yet?

                    Harald

---

From: Randell Jesup <[email-masked]>
Date: Mon, 09 Apr 2012 10:19:33 -0400

On 4/9/2012 7:02 AM, Harald Alvestrand wrote:
> up".

Right, and there would be complication when a new "hanging offer" hadn't 
yet been sent when another member joins (and it would be very easy for 
bugs to creep in there, as it's evil race-condition testing).  "hanging 
offers" is not a protocol that I would like to implement.

> Cloning allows you to connect multiple new conference mebers as

And removes a whole passel of error paths and handling.

> would the ICE machine send keepalives for an association that hasn't

This certainly needs investigation; I could see it needing some type of 
occasional "still holding it" notification from the server anyways to 
keep the state active in the client.

-- 
Randell Jesup
[email-masked]

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 10 Apr 2012 10:18:29 +0200

On 2012-04-02 18:09, Roman Shpount wrote:
> 
> On Mon, Apr 2, 2012 at 9:02 AM, Hadriel Kaplan <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
> 
>     Hi,
>     I was thinking the tricky part is more about the local transport
>     stuff getting cloned - i.e., imagine if you're using a TCP-based
>     TURN connection.
> This really depends on how the TURN support is implemented, but
> typically using the same TCP connection or any other TURN allocation for
> multiple PC should be no more of an issue then using same connection for
> multiple streams within the PC.

Actually TURN forces one to setup at least new permissions for each peer
that you want to accept flows from. Thus, even cloning an existing local
state will require additional actions for any TURN candidates. I haven't
looked into the detail for TURN TCP.

Cheers

Magnus Westerlund

---

From: Roman Shpount <[email-masked]>
Date: Tue, 10 Apr 2012 14:56:16 -0400

> looked into the detail for TURN TCP.
You are correct, when a new remote peer is added to a given allocation TURN
allocation, a new permission needs to be added if remote peer address is
different (permission is per address), new channel needs to be
created(optional but highly recommended for RTP), and ICE exchange should
be completed to confirm the connectivity. This procedure is the same for
adding new remote peer due to normal session re-negotiation (you can send
an offer with the same SDP on existing connetion and get a new response
with different peers back), reusing TURN allocation in case of bundle, or
in case we discuss -- forking. I would expect that a functional WebRTC
implementation must be able to reuse the same TURN allocation for multiple
connections within the same call. I do not think that adding support for
this to enable reuse of the same allocation across multiple calls would
complicate things too much.
_____________
Roman Shpount