Subject: [rtcweb] Proposal for PeerConnection cloning

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Sun, 6 May 2012 13:54:16 -0500

Harald asked for a proposal for PeerConnection cloning, so here it goes:

I propose to create a new constructor "ClonePeerConnection" (CPC below) with the following semantics.

 1.  CPC will create a new PeerConnection object when successful.  The resulting PC behaves just like any other PC except as described below.
 2.  CPC will take as input a reference to an existing PeerConnection (PC) object (no configuration or IceCallback parameters).
 3.  CPC can clone any other PC (including a cloned PC).  The sequence of cloning is not important and parents have no particular status different from any generation of clone.
 4.  CPC will fail if either the local description of the parent PC is not set or the parent PC ICE is in ICE_CLOSED or ICE_GATHERING state.  The parent PC can be in the OPENING or ACTIVE state.
 5.  The CPC object will inherit the local streams, local ICE candidates, and local description of the PC.
 6.  The remote streams and remote description for the CPC object will be set to empty.
 7.  The CPC object will be set to the OPENING state to reflect that only the local description is set.
 8.  The ICE states other than CLOSED and GATHERING will be handled independently for each PC and its clones (as is true in standard forking scenarios).
 9.  The ICE state for this clone will be set to ICE_WAITING to reflect that all candidates are available but the remote configuration is not yet set.
 10. The PC and its clone(s) use a common pool of media resources.
 11. If the parent PC object has already released unused resources (final ANSWER), resources are reallocated as available to reflect the capabilities for each stream (as they would be reflected in a createOffer).
 12. The local streams might multicast toward the remote targets depending on the directionality attributes independently set for each PC and clone.
 13. The application should manage the directionality attributes for remote streams from different targets to avoid resource conflicts.

CPC will be used primarily if forking is anticipated or actually occurs.  It can also be used to clone a stable PC if desired for other reasons.  When used for SIP forking, creation of the clone can be delayed until an ANSWER actually arrives from a 2nd target as long as final ANSWER hasn't been applied to the parent (PRANSWER is OK).  The parent always handles the first target; the first clone handles the 2nd target; etc.  The application can even try to clone for forking after the first ANSWER is applied to the parent and resources are released, as long as the local description has not changed, at the risk that some resources needed for the 2nd target are no longer available and must be renegotiated.

Comments on this proposal are welcome.

Richard

---

From: Christer Holmberg <[email-masked]>
Date: Sun, 6 May 2012 21:01:12 +0200

Hi Richard,

Interesting proposal.

In bullet 11, you say that, if a final answer has been received (and unused resources released), resources would have to be reallocated once a clone is created.

A couple of questions:

1. Is there a need to allow cloning once a final answer has been received?

2. Once a final answer is received on a given PC, would it automatically remove all associated CPCs?

The basic question is really whether there is a need for clones once a final answer has been received. After all, there is a reason we call it "final" :)

Regards,

Christer

________________________________
From: [email-masked] [[email-masked]] On Behalf Of Ejzak, Richard P (Richard) [[email-masked]]
Sent: Sunday, May 06, 2012 9:54 PM
To: [email-masked]
Subject: [rtcweb] Proposal for PeerConnection cloning

I propose to create a new constructor “ClonePeerConnection” (CPC below) with the following semantics.

CPC will be used primarily if forking is anticipated or actually occurs.  It can also be used to clone a stable PC if desired for other reasons.  When used for SIP forking, creation of the clone can be delayed until an ANSWER actually arrives from a 2nd target as long as final ANSWER hasn’t been applied to the parent (PRANSWER is OK).  The parent always handles the first target; the first clone handles the 2nd target; etc.  The application can even try to clone for forking after the first ANSWER is applied to the parent and resources are released, as long as the local description has not changed, at the risk that some resources needed for the 2nd target are no longer available and must be renegotiated.

Richard

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Sun, 6 May 2012 15:40:35 -0500

Hi Christer,
Question 1: I agree that for parallel forking, as long as a PC is cloned before the unused parent resources are released, it is not necessary to allow cloning later.  I would be ok to remove this feature.  I threw it in because it seemed not too expensive to implement (others have to comment on this) and it might be useful to allow delaying of cloning to the last possible instant (and for potentially other applications).  

Question 2: I do not think we want to automatically remove all related PCs when one gets a final answer, else we could not support Partha's nested O/A scenario.  I propose that this be done manually (so it is not mentioned in the proposal).  This allows ICE and DTLS to progress independently with multiple targets until a winner is selected and the others are closed.

BR, Richard

-----Original Message-----
From: Christer Holmberg [mailto:[email-masked]] 
Sent: Sunday, May 06, 2012 2:01 PM
To: Ejzak, Richard P (Richard); [email-masked]
Subject: RE: [rtcweb] Proposal for PeerConnection cloning

Hi Richard,

Interesting proposal.

A couple of questions:

Regards,

Christer

Richard

---

From: Martin Thomson <[email-masked]>
Date: Sun, 6 May 2012 15:19:57 -0700

Such a complicated process would not be necessary if the PeerConnection
object were not home to two largely independent state machines.

Nor would it be necessary if this scenario were considered out of scope.

Certainly, if you consider the scenario interesting enough to do something
about (I don't), then you news some support from the browser in setting up
two ICE associations on the same local port, but that is easier without
having to worry about the state of the media negotiation.

Let's assume you could have transport and media separate. Then what you
need is a clone operation for the transport object. You can simply start
the media negotiation, as if from scratch on the cloned transport.

The state of the media has no real bearing on this then, and Christer's
concern becomes irrelevant.  ...Though it is certainly valid given the
current proposal.

Other considerations:
-does the clone get a copy of local streams if they are added to the
original? I'm guessing not.
-the transition from PR_ANSWER to OFFER will be difficult given that this
is the obvious way to implement this
-how does the application learn that cloning fails due to lack of
resources? Normally, an offer can't include capabilities that cannot be
provided, but this implies it.

An alternative, within the constraints offered by JSEP, is to create a new
PeerConfection and bind it to an existing one. What reasons can you offer
for avoiding this choice?

--Martin
 On May 6, 2012 11:54 AM, "Ejzak, Richard P (Richard)" <
[email-masked]> wrote:

>  Harald asked for a proposal for PeerConnection cloning, so here it goes:*
> ***
>
> ** **
>
> I propose to create a new constructor “ClonePeerConnection” (CPC below)
> with the following semantics.****
>
> ** **
>
>    1. CPC will create a new PeerConnection object when successful.  The
>    resulting PC behaves just like any other PC except as described below.*
>    ***
>    2. CPC will take as input a reference to an existing PeerConnection
>    (PC) object (no configuration or IceCallback parameters).****
>    3. CPC can clone any other PC (including a cloned PC).  The sequence
>    of cloning is not important and parents have no particular status different
>    from any generation of clone.****
>    4. CPC will fail if either the local description of the parent PC is
>    not set or the parent PC ICE is in ICE_CLOSED or ICE_GATHERING state.  The
>    parent PC can be in the OPENING or ACTIVE state.****
>    5. The CPC object will inherit the local streams, local ICE
>    candidates, and local description of the PC.****
>    6. The remote streams and remote description for the CPC object will
>    be set to empty.****
>    7. The CPC object will be set to the OPENING state to reflect that
>    only the local description is set.****
>    8. The ICE states other than CLOSED and GATHERING will be handled
>    independently for each PC and its clones (as is true in standard forking
>    scenarios).****
>    9. The ICE state for this clone will be set to ICE_WAITING to reflect
>    that all candidates are available but the remote configuration is not yet
>    set.****
>    10. The PC and its clone(s) use a common pool of media resources.****
>    11. If the parent PC object has already released unused resources
>    (final ANSWER), resources are reallocated as available to reflect the
>    capabilities for each stream (as they would be reflected in a createOffer).
>    ****
>    12. The local streams might multicast toward the remote targets
>    depending on the directionality attributes independently set for each PC
>    and clone.****
>    13. The application should manage the directionality attributes for
>    remote streams from different targets to avoid resource conflicts.****
>
> ** **
>
> CPC will be used primarily if forking is anticipated or actually occurs.
>  It can also be used to clone a stable PC if desired for other reasons.
>  When used for SIP forking, creation of the clone can be delayed until an
> ANSWER actually arrives from a 2nd target as long as final ANSWER hasn’t
> been applied to the parent (PRANSWER is OK).  The parent always handles the
> first target; the first clone handles the 2nd target; etc.  The
> application can even try to clone for forking after the first ANSWER is
> applied to the parent and resources are released, as long as the local
> description has not changed, at the risk that some resources needed for the
> 2nd target are no longer available and must be renegotiated.****
>
> ** **
>
> Comments on this proposal are welcome.****
>
> ** **
>
> Richard****
>
> ** **
>
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>
>

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Sun, 6 May 2012 19:26:06 -0500

Hi Martin,
I'm not sure that separating the ICE and media state machines would help very much.  ICE has to handle all the streams and components of a PeerConnection at the same time anyway.

Once cloning occurs, operations on them (in my proposal) are independent except in the sharing of resources allocated to the parent at the time of cloning.

If we accept Christer's simplification then we should never need to fail the cloning operation due to lack of resources.  I didn't think that reporting failure would be a problem (if necessary).

You suggest that creating a separate PeerConnection and "binding" it to the parent is somehow a better alternative for cloning.  I look forward to reviewing an alternative proposal with sufficient detail to make a comparison (as Harald challenged me to do).  I took my best shot and certainly didn't "avoid" any alternatives that I was aware of and thought might be superior.

Richard

________________________________
From: Martin Thomson [mailto:[email-masked]]
Sent: Sunday, May 06, 2012 5:20 PM
To: Ejzak, Richard P (Richard)
Cc: [email-masked]
Subject: Re: [rtcweb] Proposal for PeerConnection cloning

Such a complicated process would not be necessary if the PeerConnection object were not home to two largely independent state machines.

Certainly, if you consider the scenario interesting enough to do something about (I don't), then you news some support from the browser in setting up two ICE associations on the same local port, but that is easier without having to worry about the state of the media negotiation.

Let's assume you could have transport and media separate. Then what you need is a clone operation for the transport object. You can simply start the media negotiation, as if from scratch on the cloned transport.

The state of the media has no real bearing on this then, and Christer's concern becomes irrelevant.  ...Though it is certainly valid given the current proposal.

Other considerations:
-does the clone get a copy of local streams if they are added to the original? I'm guessing not.
-the transition from PR_ANSWER to OFFER will be difficult given that this is the obvious way to implement this
-how does the application learn that cloning fails due to lack of resources? Normally, an offer can't include capabilities that cannot be provided, but this implies it.

An alternative, within the constraints offered by JSEP, is to create a new PeerConfection and bind it to an existing one. What reasons can you offer for avoiding this choice?

--Martin
On May 6, 2012 11:54 AM, "Ejzak, Richard P (Richard)" <[email-masked]<mailto:[email-masked]>> wrote:
Harald asked for a proposal for PeerConnection cloning, so here it goes:

Richard

_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Randell Jesup <[email-masked]>
Date: Mon, 07 May 2012 01:33:06 -0400

> Question 2: I do not think we want to automatically remove all related PCs when one gets a final answer, else we could not support Partha's nested O/A scenario.  I propose that this be done manuall…

If you want to support the usecase I suggested, where a single offer can 
result in parallel forking, but where you don't want to lock down to a 
single answerer: then you obviously need to support forking, and likely 
you need/want to support it irrespective of the 'final' state of the 
original PeerConnection.  You *can* require cloning before the final 
answer is applied, though I would argue that you don't gain much by 
doing so.

So this is the usecase: you send a single offer (say to a conference 
server controller, or to a game server) and you get a separate answer 
from each other endpoint, such as in a mesh conference, or who are in 
the same 'area' of the game, or in hearing (at that moment) in a VR 
environment.  You might get these answers well after the initial answer 
or burst of answers.

Using CPC, the sequence would be to:  ClonePeerConnection(), apply the 
new answer to the new CPC-created PeerConnection.  Nothing requires you 
to do so; you can reject the new answerer because you have a final 
already (or if the PeerConnection was made speculatively, you could 
close them if any other one of the clones received an answer).  I 
*think* that CPC will reduce the amount of ICE traffic (in some cases) 
and the number of external ports used (depending on NATs), and therefore 
might reduce call setup-time overhead.

Without CPC, the server will need to be able to ask endpoints to 
generate offers in anticipation of using later, or even immediately 
("send me an offer *now* to forward to the new user, or "send me 8 more 
offers just like that one to send of existing users").

p.s. Thanks for the concrete proposal about how to handle this!  We've 
been talking about it (on occasion) for a year-ish, but no one has 
proposed an actual set of semantics for it.

-- 
Randell Jesup
[email-masked]

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 7 May 2012 08:22:03 +0200

Hi,

>> Hi Christer,
>> Question 1: I agree that for parallel forking, as long as a PC is cloned before the unused parent resources are
>> released, it is not necessary to allow cloning later.  I would be ok to remove this feature.  I threw it in because 
>> it seemed not too expensive to implement (others have to comment on this) and it might be useful to allow
>> delaying of cloning to the last possible instant (and for potentially other applications).
>>
>> Question 2: I do not think we want to automatically remove all related PCs when one gets a final answer, else we
>> could not support Partha's nested O/A scenario.  I propose that this be done manually (so it is not mentioned in the
>> proposal).  This allows  ICE and DTLS to progress independently with multiple targets until a winner is selected 
>> and the others are closed.
>
> If you want to support the usecase I suggested, where a single offer can result in parallel forking, but where you don't
> want to lock down to a single answerer: then you obviously need to support forking, and likely you need/want to support
> it irrespective of the 'final' state of the original PeerConnection.  You *can* require cloning before the final answer is 
> applied, though I would argue that you don't gain much by doing so.
>
> So this is the usecase: you send a single offer (say to a conference server controller, or to a game server) and you get 
> a separate answer from each other endpoint, such as in a mesh conference, or who are in the same 'area' of the game, 
> or in hearing (at that moment) in a VR environment.  You might get these answers well after the initial answer or burst 
> of answers.

Yes. And, my idea would be that each of those answers are provided to the browser as PRE_ANSWERS.

Then, at some point, you decide that you are not going to accept any more answers, and you provide an ANSWER to one of the (C)PCs, which will release all other associated (C)PCs.

Regards,

Christer

-- 
Randell Jesup
[email-masked]

---

From: Randell Jesup <[email-masked]>
Date: Mon, 07 May 2012 02:53:15 -0400

> Then, at some point, you decide that you are not going to accept any more answers, and you provide an ANSWER to one of the (C)PCs, which will release all other associated (C)PCs.

Sorry, I think you missed the point: the application wants to end up 
with N connected, active PeerConnections, not one.  This is a mesh 
conference (for example), not a central-mixer conference.  And for added 
fun, we don't necessarily know there won't be more coming later, perhaps 
much later (but that's a possibly-separable feature).

-- 
Randell Jesup
[email-masked]

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 7 May 2012 09:08:27 +0200

Hi,

>>> If you want to support the usecase I suggested, where a single offer 
>>> can result in parallel forking, but where you don't want to lock down 
>>> to a single answerer: then you obviously need to support forking, and 
>>> likely you need/want to support it irrespective of the 'final' state of the original PeerConnection.  You *can* require cloning before the final answer is applied, though I would argue that you don't gain much by doing so.
>>>
>>> the same 'area' of the game, or in hearing (at that moment) in a VR environment.  You might get these answers well after the initial answer or burst of answers.
> Sorry, I think you missed the point: the application wants to end up with N connected, active PeerConnections, not one.  This is a 
> mesh conference (for example), not a central-mixer conference.  And for added fun, we don't necessarily know there won't be more 
> coming later, perhaps much later (but that's a possibly-separable feature).

Speaking in SIP terms, I don't think it is possible to implement mesh conference that way, because clients will normally, once they've received one 200 OK, ACK+BYE any additional ones. Or, a forking proxy will CANCEL all other forked legs once a 200 OK has been forwarded on one of them.

Regards,

Christer

---

From: Randell Jesup <[email-masked]>
Date: Mon, 07 May 2012 03:36:23 -0400

> Speaking in SIP terms, I don't think it is possible to implement mesh conference that way, because clients will normally, once they've received one 200 OK, ACK+BYE any additional ones. Or, a forkin…

Why would SIP come into this?  Yes, classic PSTN telecom is focused on 
endpoint to one other connection, and any form of more complex 
arrangement is implemented with a mixing server or equivalent, and thus 
SIP flows are designed with that assumption in mind.  However, we are 
NOT constrained to mirror SIP, and we should be able to build 
applications that go beyond what SIP makes easy to do.

I gave some direct use-cases we've talked about since the beginning or 
near: Mesh conferencing, multiplayer games - where parallel forking 
(without collapse to a single session/connection) is desirable/useful.  
If there's an equally-usable way to implement those cases avoiding 
cloning, please detail that.  I think cloning makes initiation of these 
sorts of mesh connections much simpler.

-- 
Randell Jesup
[email-masked]

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 7 May 2012 12:58:19 +0200

Hi,

>>>>> If you want to support the usecase I suggested, where a single 
>>>>> offer can result in parallel forking, but where you don't want to 
>>>>> lock down to a single answerer: then you obviously need to support 
>>>>> forking, and likely you need/want to support it irrespective of the 'final' state of the original PeerConnection.  You *can* require cloning before the final answer is applied, though I would argue that you don't gain much by doing so.
>>>>>
>>>>> So this is the usecase: you send a single offer (say to a 
>>>>> conference server controller, or to a game server) and you get a 
>>>>> separate answer from each other endpoint, such as in a mesh 
>>>>> conference, or who are in the same 'area' of the game, or in hearing (at that moment) in a VR environment.  You might get these answers well after the initial answer or burst of answers.
>>> conference (for example), not a central-mixer conference.  And for added fun, we don't necessarily know there won't be more coming later, perhaps much later (but that's a possibly-separable feature).
>> Speaking in SIP terms, I don't think it is possible to implement mesh conference that way, because clients will normally, once they've received one 200 OK, ACK+BYE any
>> additional ones. Or, a forking proxy will CANCEL all other forked legs once a 200 OK has been forwarded on one of them.
>
> Why would SIP come into this?  Yes, classic PSTN telecom is focused on endpoint to one other connection, and any form of more complex arrangement is 
> implemented with a mixing server or equivalent, and thus SIP flows are designed with that assumption in mind.  However, we are NOT constrained to mirror 
> SIP, and we should be able to build applications that go beyond what SIP makes easy to do.
>
> I gave some direct use-cases we've talked about since the beginning or near: Mesh conferencing, multiplayer games - where parallel forking 
> (without collapse to a single session/connection) is desirable/useful.  If there's an equally-usable way to implement those cases avoiding cloning, please
> detail that.  I think cloning makes initiation of these sorts of mesh connections much simpler.

I didn't say you can't do it with CPC, I simply said that you would have issues if doing it with SIP. But, of course you can use whatever protocol on the wire.

Maybe there could be an API parameter to indicate, when a final answer is provided to the browser, whether it should do a CPC "clean-up". It would make life easier for the JS App developers in non-mesh-conf cases.

Regards,

Christer

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Mon, 7 May 2012 07:31:46 -0500

Hi Christer,
As I mentioned in a previous response to you, you cannot mandate that clones be cleared out once a final ANSWER is applied since that prevents you from applying independent offer/answer sequences to each clone (as required in some SIP scenarios).  The optional "cleanup" indicator you propose to associate with final ANSWER will not work in the case where a final ANSWER has already been applied to a clone to allow another offer/answer sequence on that clone before a final "winner" is selected.  I would prefer to just clean up the "losers" manually or create a new method for this.

There is at least one ERROR that I found in my proposal.  I forgot that the clone will continue to need to be able to report PEER-REFLEXIVE candidates after the cloning operation, since this occurs after ICE exits the ICE_GATHERING state (this should be made clearer in the JSEP text).  So the CPC still needs the IceCallback.

The SIP example in JSEP is also missing the 2nd offer/answer needed by ICE to update the default candidates.  How is the application triggered to generate this offer?

BR, Richard

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christer Holmberg
Sent: Monday, May 07, 2012 5:58 AM
To: Randell Jesup; [email-masked]
Subject: Re: [rtcweb] Proposal for PeerConnection cloning

Hi,

Regards,

Christer

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 7 May 2012 13:33:16 +0200

> Maybe there could be an API parameter to indicate, when a final answer is provided to the browser, whether it should do a CPC "clean-up". It would make life easier for the JS App developers in non-…

I agree. IMHO it's much better if the API offers a simple mechanism
for building/imitating existing VoIP technologies (i.e. SIP and
XMPP-Jingle). Of course the API could be so powerful that it allows
receiving late responses and use them (something that makes no sense
in SIP world) but giving a simple parameter "just_one_final_response:
true" in the constructor of the first PC, or offering an API function
for discarding all the previous (C)PC's would be nice.

Regards.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Randell Jesup <[email-masked]>
Date: Mon, 07 May 2012 09:44:44 -0400

> As I mentioned in a previous response to you, you cannot mandate that clones be cleared out once a final ANSWER is applied since that prevents you from applying independent offer/answer sequences t…

It doesn't have to be via this mechanism, but we could have an onfinal 
notification, or a manually polled jsepState checked (on all 
PeerConnections the app has) after installing an answer on one.  (Unless 
the app knows that all it has are clones, in which case it could just 
kill off the others).  jsepState is probably all that is needed (and 
probably needed even with onfinal, unless it gives a list of all clones 
of a finalized PC).

-- 
Randell Jesup
[email-masked]

---

From: Roman Shpount <[email-masked]>
Date: Mon, 7 May 2012 12:18:13 -0400

On Mon, May 7, 2012 at 6:58 AM, Christer Holmberg <
[email-masked]> wrote:

> endpoint to one other connection, and any form of more complex arrangement
> is
> > implemented with a mixing server or equivalent, and thus SIP flows are
> designed with that assumption in mind.  However, we are NOT constrained to
> mirror
> cloning, please
> I didn't say you can't do it with CPC, I simply said that you would have
> issues if doing it with SIP. But, of course you can use whatever protocol
> on the wire.
>
> Maybe there could be an API parameter to indicate, when a final answer is
> provided to the browser, whether it should do a CPC "clean-up". It would
> make life easier for the JS App developers in non-mesh-conf cases.
>
>
First of all, even in classic SIP use case, proxy will not hang up or
cancel other call legs when 200 OK is received. You can setup a mesh
conference this way if your end point supports this. Most of the end points
do not, so they hang up the calls, but I have dealt with a few that
actually start multiple calls this way and allow user to switch between
them or conference them together.

Second, you do need to be able to provide a final answer on the primary or
cloned peer connection to implement Partha's scenario, since for this
scenario you need to process another O/A while the original O/A is still in
progress. This means you need to provide a final response on the call leg
where you received the offer, but still be able to receive answers on other
call legs.

_____________
Roman Shpount

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 07 May 2012 23:24:22 +0200

On 05/07/2012 03:44 PM, Randell Jesup wrote:
>> As I mentioned in a previous response to you, you cannot mandate that 
>> clones be cleared out once a final ANSWER is applied since that 
>> prevents you from applying independent offer/answer sequences to each 
>> clone (as required in some SIP scenarios).  The optional "cleanup" 
>> indicator you propose to associate with final ANSWER will not work in 
>> the case where a final ANSWER has already been applied to a clone to 
>> allow another offer/answer sequence on that clone before a final 
>> "winner" is selected.  I would prefer to just clean up the "losers" 
>> manually or create a new method for this.
>
> PeerConnections the app has) after installing an answer on one.  
> (Unless the app knows that all it has are clones, in which case it 
> could just kill off the others).  jsepState is probably all that is 
> needed (and probably needed even with onfinal, unless it gives a list 
> of all clones of a finalized PC).
>
It seems to me that

for each Pc in ClonedConnections {
   if (Pc !== winner)
      Pc.close()
}

is simple enough to code that we should simply say that "the JS has to 
do cleanup when he decides he only wants one of the PeerConnection".

Creating mechanisms for things to happen behind the scenes seems to me 
to make life for the programmer more complex, not less; "why did the 
voice of person 2 go silent? Oh - you applied an ANSWER instead of a 
PR-ANSWER to person 1"....

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 07 May 2012 23:32:29 +0200

Richard, thank you for doing this! It's definitely the most 
thought-through proposal I've seen on this question!

Two questions that are different from what others have raised:

On 05/06/2012 08:54 PM, Ejzak, Richard P (Richard) wrote:
> I propose to create a new constructor ?ClonePeerConnection? (CPC 
> below) with the following semantics.
>
>    1. CPC will create a new PeerConnection object when successful. 
>       The resulting PC behaves just like any other PC except as
>       described below.
>       (PC) object (no configuration or IceCallback parameters).
>    3. CPC can clone any other PC (including a cloned PC).  The
>       sequence of cloning is not important and parents have no
>       particular status different from any generation of clone.
>    4. CPC will fail if either the local description of the parent PC
>       is not set or the parent PC ICE is in ICE_CLOSED or
>       ICE_GATHERING state.  The parent PC can be in the OPENING or
>       ACTIVE state.
>       candidates, and local description of the PC.
>
when it inherits the local streams, do you assume that unless the JS 
applies some setting, they will also inherit the "enabled/disabled" 
state, but that the "enabled/disabled" state can be changed 
independently for each PeerConnection's local streams after the cloning?
>
>   1.
>
>
>    2. The remote streams and remote description for the CPC object
>       will be set to empty.
>    3. The CPC object will be set to the OPENING state to reflect that
>       only the local description is set.
>    4. The ICE states other than CLOSED and GATHERING will be handled
>       independently for each PC and its clones (as is true in standard
>       forking scenarios).
>    5. The ICE state for this clone will be set to ICE_WAITING to
>       reflect that all candidates are available but the remote
>       configuration is not yet set.
>    6. The PC and its clone(s) use a common pool of media resources.
>    7. If the parent PC object has already released unused resources
>       (final ANSWER), resources are reallocated as available to
>       reflect the capabilities for each stream (as they would be
>       reflected in a createOffer).
>    8. The local streams might multicast toward the remote targets
>       depending on the directionality attributes independently set for
>       each PC and clone.
>
So far, we've stayed away from anything that involves IP multicast. Are 
you sure it's worth the potential complexity to try to use this here?
Remember that using multicast changes a *lot* of things on both ends of 
the connections - the sender has to have IP multicast address 
allocations, he has to decide between SSM and ASM, the recipients have 
to use a multicast joining protocol, the infrastructure has to be 
present to carry the multicast groups, we have to have a fallback in 
case it isn't, the appropriate congestion control protocols are probably 
different ..... I would prefer to not bring this in scope for this round 
of WebRTC.
Is there a compelling reason you see for embarking on this particular 
adventure?
>
>   1.
>
>
>    2. The application should manage the directionality attributes for
>       remote streams from different targets to avoid resource conflicts.
>
> CPC will be used primarily if forking is anticipated or actually 
> occurs.  It can also be used to clone a stable PC if desired for other 
> reasons.  When used for SIP forking, creation of the clone can be 
> delayed until an ANSWER actually arrives from a 2^nd target as long as 
> final ANSWER hasn?t been applied to the parent (PRANSWER is OK).  The 
> parent always handles the first target; the first clone handles the 
> 2^nd target; etc.  The application can even try to clone for forking 
> after the first ANSWER is applied to the parent and resources are 
> released, as long as the local description has not changed, at the 
> risk that some resources needed for the 2^nd target are no longer 
> available and must be renegotiated.
>

---

From: Randell Jesup <[email-masked]>
Date: Mon, 07 May 2012 17:53:10 -0400

> thought-through proposal I've seen on this question!

Ditto!

> independently for each PeerConnection's local streams after the cloning?

I would assume that personally, and that typically after cloning you'd 
do "appropriate" tweaking to new (and old).

> you sure it's worth the potential complexity to try to use this here?

Richard - do you mean "IP Multicast" (tm) or 'multicast' by sending the 
same encoded data to more than one destination in separate RTP streams? 
  (i.e. one encoding being used to simulcast to multiple receivers?)

Simulcast is possible and useful (almost required for Mesh 
Conferencing); IP Multicast has been pretty much ruled out-of-scope IIRC.

-- 
Randell Jesup
[email-masked]

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Mon, 7 May 2012 19:59:20 -0500

Thanks, Harald, Randell.

Question 1: I have no strong opinion on whether each local stream should inherit its parent's state or get reset to a default value, but inheriting the state seems cleaner (today).  It must be possible to control the state of each cloned local stream separately.

Question 2: I should have been more careful in my choice of words.  Randell interpreted my meaning correctly.  My meaning is better indicated with the word "simulcast", as Randell suggests.  I NEVER intended the use of "IP multicast addressing".

Richard

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Harald Alvestrand
Sent: Monday, May 07, 2012 4:32 PM
To: [email-masked]
Subject: Re: [rtcweb] Proposal for PeerConnection cloning

Richard, thank you for doing this! It's definitely the most thought-through proposal I've seen on this question!

when it inherits the local streams, do you assume that unless the JS applies some setting, they will also inherit the "enabled/disabled" state, but that the "enabled/disabled" state can be changed independently for each PeerConnection's local streams after the cloning?

 1.
 2.  The remote streams and remote description for the CPC object will be set to empty.
 3.  The CPC object will be set to the OPENING state to reflect that only the local description is set.
 4.  The ICE states other than CLOSED and GATHERING will be handled independently for each PC and its clones (as is true in standard forking scenarios).
 5.  The ICE state for this clone will be set to ICE_WAITING to reflect that all candidates are available but the remote configuration is not yet set.
 6.  The PC and its clone(s) use a common pool of media resources.
 7.  If the parent PC object has already released unused resources (final ANSWER), resources are reallocated as available to reflect the capabilities for each stream (as they would be reflected in a createOffer).
 8.  The local streams might multicast toward the remote targets depending on the directionality attributes independently set for each PC and clone.
So far, we've stayed away from anything that involves IP multicast. Are you sure it's worth the potential complexity to try to use this here?
Remember that using multicast changes a *lot* of things on both ends of the connections - the sender has to have IP multicast address allocations, he has to decide between SSM and ASM, the recipients have to use a multicast joining protocol, the infrastructure has to be present to carry the multicast groups, we have to have a fallback in case it isn't, the appropriate congestion control protocols are probably different ..... I would prefer to not bring this in scope for this round of WebRTC.
Is there a compelling reason you see for embarking on this particular adventure?

 1.
 2.  The application should manage the directionality attributes for remote streams from different targets to avoid resource conflicts.

Richard

rtcweb mailing list

---

From: Roman Shpount <[email-masked]>
Date: Mon, 7 May 2012 21:36:37 -0400

One question I have is when should TURN allocations be released? TURN would
not be used for 99% of PeerConnections but it cannot be released until it
is guaranteed that the offer that used this TURN candidate would not be
used to create any more PeerConnections. We would need a separate method to
release the TURN candidates, which can be complicated by the fact that you
can have several offers for a given peerconnection in progress, each with
its own TURN candidates that can have overlapping lifetimes.

I would actually think that the model where an offer object is created and
each PeerConnection is created  using the offer object and the answer would
be cleaner.

One way to implement this, is to define a factory object is used to create
offers. SDP is extracted from an offer object. Once answer is received, a
method in the offer object is used to create the PeerConnection. Once no
more answers are expected offer object should be released.

The other option, that would be less disruptive to the curretn API,  is
that instead of cloning peerconnection method, a method that returns
reference to the current offer is used. A cloned peerconnection can be
created using this offer. Once final answer is processed by the
peerconnection, offer object is released by it. If multiple answers are
expected, offer object can be retrieved using the get offer method and
stored separately until the moment when answers for this particular offer a
no longer expected.
_____________
Roman Shpount

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Mon, 7 May 2012 21:10:56 -0500

You raise a good point since some candidates may be released before cloning occurs.  So this is another bug in the procedure.  I would suggest that the configuration also be passed to the CPC and that if any of the required candidates are released before cloning occurs that they be reallocated.

Alternately, we could recommend that candidates be kept allocated longer than usual and that if cloning is attempted after any of the candidates are released then the cloning procedure fails.

The latter approach should be sufficient for SIP forking scenarios, but others may prefer the generality of the former.  Or maybe both?  Any votes?

Richard

________________________________
From: Roman Shpount [mailto:[email-masked]]
Sent: Monday, May 07, 2012 8:37 PM
To: Ejzak, Richard P (Richard)
Cc: [email-masked]
Subject: Re: [rtcweb] Proposal for PeerConnection cloning

One question I have is when should TURN allocations be released? TURN would not be used for 99% of PeerConnections but it cannot be released until it is guaranteed that the offer that used this TURN candidate would not be used to create any more PeerConnections. We would need a separate method to release the TURN candidates, which can be complicated by the fact that you can have several offers for a given peerconnection in progress, each with its own TURN candidates that can have overlapping lifetimes.

I would actually think that the model where an offer object is created and each PeerConnection is created  using the offer object and the answer would be cleaner.

One way to implement this, is to define a factory object is used to create offers. SDP is extracted from an offer object. Once answer is received, a method in the offer object is used to create the PeerConnection. Once no more answers are expected offer object should be released.

The other option, that would be less disruptive to the curretn API,  is that instead of cloning peerconnection method, a method that returns reference to the current offer is used. A cloned peerconnection can be created using this offer. Once final answer is processed by the peerconnection, offer object is released by it. If multiple answers are expected, offer object can be retrieved using the get offer method and stored separately until the moment when answers for this particular offer a no longer expected.
_____________
Roman Shpount

---

From: Stefan Hakansson LK <[email-masked]>
Date: Tue, 8 May 2012 08:51:05 +0200

> do "appropriate" tweaking to new (and old).

As the APIs are designed right now, this is not possible AFAIK. The 
"local streams" is more or less just a list of the streams that has been 
added to the PeerConnection (PC) using "addStream". So if you have one 
PC with one MediaStream added, then clones the PC, the result would be 
that that MediaStream now has two consumers. And if you disable a track, 
that track would be disabled for all consumers.

I think a more flexible solution would be that the new PC does _not_ 
inherit the streams. This way the application can choose to clone the 
MediaStream's attached to the first PC, and add the cloned MS's to the 
new PC, and thereby get independent control of e.g. enable/disable 
state. After all, it is the application that initiates the PC cloning, 
so it knows exactly when it happens and the context, so adding the req. 
to the app to add the stream(s) that should be added would not make it 
more complex IMO.

The possible problem I can see with this approach is that if the set of 
MS's added to the cloned PC is not exactly the same as for the original 
PC, the number of ICE connections needed might not match. But is that 
problem not already there, and must be handled by the ICE agent? The 
cloned PC may connect to an endpoint that does not support multiplexing 
of RTP-streams, or new streams might be added to the cloned PC at any 
time after cloning.

---

From: Roman Shpount <[email-masked]>
Date: Tue, 8 May 2012 09:40:30 -0400

On Mon, May 7, 2012 at 10:10 PM, Ejzak, Richard P (Richard) <
[email-masked]> wrote:

>  You raise a good point since some candidates may be released before
> cloning occurs.  So this is another bug in the procedure.  I would suggest
> that the configuration also be passed to the CPC and that if any of the
> required candidates are released before cloning occurs that they be
> reallocated.****
>
> ** **
>
> Alternately, we could recommend that candidates be kept allocated longer
> than usual and that if cloning is attempted after any of the candidates are
> released then the cloning procedure fails.****
>
> ** **
>
> The latter approach should be sufficient for SIP forking scenarios, but
> others may prefer the generality of the former.  Or maybe both?  Any votes?
> ****
>
> ** **
>

My suggestion was that instead of the clone method to have a method that
returns reference to the offer object. This separate object holds the
candidates while references to it exist. When this object is passed during
creation of peer connection, peer connection is initialized as a clone of
an existing peer connection This way application has the explicit control
of the ICE candidate lifetimes. I would think this would be the most
flexible.
_____________
Roman Shpount

---

From: Martin Thomson <[email-masked]>
Date: Tue, 8 May 2012 08:49:03 -0700

On 7 May 2012 19:10, Ejzak, Richard P (Richard)
<[email-masked]> wrote:
> You raise a good point since some candidates may be released before cloning
> occurs.  So this is another bug in the procedure.  I would suggest that the
> configuration also be passed to the CPC and that if any of the required
> candidates are released before cloning occurs that they be reallocated.

Releasing a TURN candidate is a real possibility.  The same could
apply to local candidates (and server reflexive), though that will
depend on implementation.  Getting the same one back is unlikely for
TURN, though it might be possible for local.

In all these cases, answers that arrive late will fail.

> released then the cloning procedure fails.

I'd be against making any such recommendation for browser
implementations.  The signaling application is more likely to know
best in this scenario.  In that case, it can make the call to hold
candidates open.  PR_ANSWER seems to have the right semantics.

--Martin