Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Tue, 1 May 2012 08:57:40 -0500

Hi Justin, Partha,
The call flow from Partha is a very reasonable one in the presence of SIP forking.  Let's assume that a first target responded with an SDP answer in a 183 that must be treated as a PRANSWER since other targets have not yet had a chance to respond.  Then the first target sends an UPDATE with offer2.

As far as SIP is concerned, there is no such thing as a PRANSWER.  The first target already responded with an SDP ANSWER and wants to UPDATE with a new offer.  This is perfectly legitimate and common SIP usage.  PRANSWER is a local construct that should be usable by the application to support more complex offer/answer cases like forking.

You must respond to this offer by cloning the peer connection, closing the clone with an ANSWER identical to the previous PRANSWER and then applying offer2 to the clone (or just applying the offer2 directly with the understanding that the previous transaction is closed).  The original peer connection must revert to the original OFFER state to allow for other targets to respond.

Alternately, the original peer connection can support the first target and a new clone created if there is the potential for other forked responses to the original offer.  Either way can be made to work.

This is a perfectly reasonable SIP use case that explains why we need to be able to clone the peer connection.

Richard

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Justin Uberti
Sent: Monday, April 30, 2012 9:51 PM
To: Nataraju A.B
Cc: [email-masked]
Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

Yes, I was under the impression that SIP enforced this requirement, although I am probably not aware of all the corner cases. Is there a real-world scenario where this flow is required?
On Mon, Apr 30, 2012 at 10:55 AM, Nataraju A.B <[email-masked]<mailto:[email-masked]>> wrote:
If the scenario considered is without reliable provisional responses. Then the first ANSWER must be in 200-INV and no more O/A allowed during INVITE(initial) transaction.

Basic requirement for reliable and unambiguous O/A is - At any point in time there could be only one O/A open.

Also only one O/A suggested during INVITE(initial) transaction.

For reference, rfc6337, list outs different O/A models...

<rfc6337>

            Offer                Answer             RFC    Ini Est Early

     -------------------------------------------------------------------

     1. INVITE Req.          2xx INVITE Resp.     RFC 3261  Y   Y    N

     2. 2xx INVITE Resp.     ACK Req.             RFC 3261  Y   Y    N

     3. INVITE Req.          1xx-rel INVITE Resp. RFC 3262  Y   Y    N

     4. 1xx-rel INVITE Resp. PRACK Req.           RFC 3262  Y   Y    N

     5. PRACK Req.           200 PRACK Resp.      RFC 3262  N   Y    Y

     6. UPDATE Req.          2xx UPDATE Resp.     RFC 3311  N   Y    Y

          Table 1: Summary of SIP Usage of the Offer/Answer Model
</rfc6337>

Thanks,
Nataraju A B

On Mon, Apr 30, 2012 at 12:31 PM, Ravindran, Parthasarathi <[email-masked]<mailto:[email-masked]>> wrote:
Justin/Cullen,

Could you please explain in case for an SDP_OFFER(1), the remote entity replies with SDP_PRANSWER followed by new SDP_OFFER (2) what is the expected behavior. The exact callflow is as follows:

Browser1-------------------------Browser2(SIP-JSEP gateway)
   |        SDP_OFFER(1)            |  INV with offer1
   |------------------------------->|--->
   |       SDP_PRANSWER(1)          |  183 with answer1
   |<-------------------------------|<---
   |       SDP_OFFER(2)             |   UPDATE with offer2
   |<-------------------------------|<---
   |       SDP_ANSWER(2?)           |
   |--------------------->?

The reason for this O/A callflow is due to UPDATE method (RFC 3311) mapping in Browser 2 (SIP-JSEP gateway).

Thanks
Partha

PS: For simplicity, PRACK message exchange is not chosen in SIP side.

_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

--
Thanks,
Nataraju A.B.

---

From: "Nataraju A.B" <[email-masked]>
Date: Tue, 1 May 2012 21:42:28 +0530

The original scenario mentioned by Partha is quite logical and acceptable,
since there is only one O/A open all the times.

If not handled properly, this additional UPDATE (with OFFER2) during the
lifetime of INVITE transaction lead to ambiguities. Hence rfc6337, don't
support this as a valid O/A.

On Tue, May 1, 2012 at 7:27 PM, Ejzak, Richard P (Richard) <
[email-masked]> wrote:

>  Hi Justin, Partha,****
>
> The call flow from Partha is a very reasonable one in the presence of SIP
> forking.  Let’s assume that a first target responded with an SDP answer in
> a 183 that must be treated as a PRANSWER since other targets have not yet
> had a chance to respond.  Then the first target sends an UPDATE with offer2.
> ****
>
> ** **
>
> As far as SIP is concerned, there is no such thing as a PRANSWER.  The
> first target already responded with an SDP ANSWER and wants to UPDATE with
> a new offer.  This is perfectly legitimate and common SIP usage.  PRANSWER
> is a local construct that should be usable by the application to support
> more complex offer/answer cases like forking.****
>
> ** **
>
> You must respond to this offer by cloning the peer connection, closing the
> clone with an ANSWER identical to the previous PRANSWER and then applying
> offer2 to the clone (or just applying the offer2 directly with the
> understanding that the previous transaction is closed).  The original peer
> connection must revert to the original OFFER state to allow for other
> targets to respond.****
>
> ** **
>
> Alternately, the original peer connection can support the first target and
> a new clone created if there is the potential for other forked responses to
> the original offer.  Either way can be made to work.****
>
> ** **
>
> This is a perfectly reasonable SIP use case that explains why we need to
> be able to clone the peer connection.****
>
> ** **
>
> Richard****
>
> ** **
>  ------------------------------
>
> *From:* [email-masked] [mailto:[email-masked]] *On
> Behalf Of *Justin Uberti
> *Sent:* Monday, April 30, 2012 9:51 PM
> *To:* Nataraju A.B
> *Cc:* [email-masked]
> *Subject:* Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in
> JSEP****
>
> ** **
>
> Yes, I was under the impression that SIP enforced this requirement,
> although I am probably not aware of all the corner cases. Is there a
> real-world scenario where this flow is required?****
>
> On Mon, Apr 30, 2012 at 10:55 AM, Nataraju A.B <[email-masked]>
> wrote:****
>
> If the scenario considered is without reliable provisional responses. Then
> the first ANSWER must be in 200-INV and no more O/A allowed during
> INVITE(initial) transaction.****
>
> ** **
>
> Basic requirement for reliable and unambiguous O/A is -* At any point in
> time there could be only one O/A open. *****
>
> ** **
>
> Also only one O/A suggested during INVITE(initial) transaction.****
>
> ** **
>
> For reference, rfc6337, list outs different O/A models...****
>
> ** **
>
> <rfc6337>****
>
> ** **
>
> ** **
>
>             Offer                Answer             RFC    Ini Est Early****
>
>      -------------------------------------------------------------------****
>
>      1. INVITE Req.          2xx INVITE Resp.     RFC 3261  Y   Y    N****
>
>      2. 2xx INVITE Resp.     ACK Req.             RFC 3261  Y   Y    N****
>
>      3. INVITE Req.          1xx-rel INVITE Resp. RFC 3262  Y   Y    N****
>
>      4. 1xx-rel INVITE Resp. PRACK Req.           RFC 3262  Y   Y    N****
>
>      5. PRACK Req.           200 PRACK Resp.      RFC 3262  N   Y    Y****
>
>      6. UPDATE Req.          2xx UPDATE Resp.     RFC 3311  N   Y    Y****
>
> ** **
>
>           Table 1: Summary of SIP Usage of the Offer/Answer Model ****
>
>  </rfc6337>****
>
> ** **
>
> Thanks,****
>
> Nataraju A B****
>
> ** **
>
> On Mon, Apr 30, 2012 at 12:31 PM, Ravindran, Parthasarathi <
> [email-masked]> wrote:****
>
>  Justin/Cullen,
>
> Could you please explain in case for an SDP_OFFER(1), the remote entity
> replies with SDP_PRANSWER followed by new SDP_OFFER (2) what is the
> expected behavior. The exact callflow is as follows:
>
>
> ****
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb****
>
>
>
> ****
>
> ** **
>
> --
> Thanks,****
>
> Nataraju A.B.****
>
> ** **
>
> ** **
>

-- 
Thanks,
Nataraju A.B.

---

From: "Ravindran, Parthasarathi" <[email-masked]>
Date: Wed, 2 May 2012 05:32:01 +0000

Nataraju,

RFC 6337 supports early dialog UPDATE scenario which I mentioned in this mail. The exact RFC
snippet with "Early column" explanation.

   The 'Early' column indicates which patterns may be used to modify the
   established session in an early dialog.  There are two ways to
   exchange a subsequent offer/answer in an early dialog.

Here, UPDATE MUST not be used for initiating the session but it shall be used in the early dialog.
AFAIK, UPDATE With SDP is mostly useful in early-dialog as RE-INVITE with SDP is possible in the
mid-dialog ("Est")

Justin,

Richard provided some of the usecase using forked responses.  This usecase is not the corner
case and it is very normal scenario in SIP O/A. I'll add further usecase for this scenario:

One of the main usecase of UPDATE with SDP in the early dialog is remote ringback tone (RRBT)
from media server Usecase. Multiple scenario involved in this usecase:

1)      First usecase, the first offer/answer is completed between  Originating endpoint and
        media server, UPDATE with second offer is sent towards Originating endpoint to update
        SDP with terminating Endpoint.

2)      Second usecase, the first offer/answer is completed with direction attribute as

Sendonly in the answer by which early dialog is only for remote ringback tone and  not

actual session establishment.  second offer in UPDATE is updated the direction attribute

as "sendrecv".

3)      Combination of First and second usecase wherein media server answer with sendonly,

UPDATE with SDP for "sendrecv".

Common Topology:

Originating
Browser--------------Gateway---------SoftSwitch/PBX(B2BUA)-------SIP UA (Media Server/IVR)
                                                                                      |
                                                                                      -------------------SIP UA (IP phone)

RFC 6337 has lot of other corner scenario. Could you please let me know how to map the second
OFFER and in PRANSWER state of the Originating endpoint (webbrowser in JSEP).

In case JSEP O/A is based on OFFER & ANSWER states only, this scenario will not hit.

Thanks
Partha

From: [email-masked] [mailto:[email-masked]] On Behalf Of Nataraju A.B
Sent: Tuesday, May 01, 2012 9:42 PM
To: Ejzak, Richard P (Richard)
Cc: [email-masked]
Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

The original scenario mentioned by Partha is quite logical and acceptable, since there is only one O/A open all the times.

If not handled properly, this additional UPDATE (with OFFER2) during the lifetime of INVITE transaction lead to ambiguities. Hence rfc6337, don't support this as a valid O/A.

On Tue, May 1, 2012 at 7:27 PM, Ejzak, Richard P (Richard) <[email-masked]<mailto:[email-masked]>> wrote:
Hi Justin, Partha,
The call flow from Partha is a very reasonable one in the presence of SIP forking.  Let's assume that a first target responded with an SDP answer in a 183 that must be treated as a PRANSWER since other targets have not yet had a chance to respond.  Then the first target sends an UPDATE with offer2.

Richard

________________________________
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Justin Uberti
Sent: Monday, April 30, 2012 9:51 PM
To: Nataraju A.B
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

<rfc6337>

Thanks,
Nataraju A B

Thanks
Partha

--
Thanks,
Nataraju A.B.

--
Thanks,
Nataraju A.B.

---

From: "Ravindran, Parthasarathi" <[email-masked]>
Date: Wed, 2 May 2012 13:01:41 +0000

Hi Charles,

UPDATE is allowed in the early dialog and I haven't shown in the callflow for simplicity as I mentioned in PS of below mail as " PS: For simplicity, PRACK message exchange is not chosen in SIP side."

Thanks
Partha

>-----Original Message-----
>From: Charles Eckel (eckelcu) [mailto:[email-masked]]
>Sent: Monday, April 30, 2012 9:53 PM
>To: Ravindran, Parthasarathi; Justin Uberti; Cullen Jennings
>Cc: [email-masked]
>Subject: RE: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in
>JSEP
>
>Hi Partha,
>
>I believe such a flow illegal, per RFC 3311:
>
>      o  If the UPDATE is being sent before the completion of the INVITE
>         transaction, and the initial INVITE contained an offer, the
>         UPDATE cannot be sent with an offer unless the callee has
>         generated an answer in a reliable provisional response, has
>         received a PRACK for that reliable provisional response, has
>         not received any requests (PRACK or UPDATE) with offers that it
>         has not answered, and has not sent any UPDATE requests
>         containing offers that have not been answered.
>
>Cheers,
>Charles
>
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On
>> Behalf Of Ravindran, Parthasarathi
>> Sent: Monday, April 30, 2012 12:02 AM
>> To: Justin Uberti; Cullen Jennings
>> Cc: [email-masked]
>> Subject: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP
>>
>> Justin/Cullen,
>>

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 02 May 2012 15:35:18 +0200

On 05/01/2012 03:57 PM, Ejzak, Richard P (Richard) wrote:
>
> Hi Justin, Partha,
>
> The call flow from Partha is a very reasonable one in the presence of 
> SIP forking.  Let's assume that a first target responded with an SDP 
> answer in a 183 that must be treated as a PRANSWER since other targets 
> have not yet had a chance to respond.  Then the first target sends an 
> UPDATE with offer2.
>
> first target already responded with an SDP ANSWER and wants to UPDATE 
> with a new offer.
>
In that case, PRANSWER seems like the wrong semantic.... if you want to 
support multiple answers, shouldn't you do the cloning before you start 
accepting answers?
>
>  This is perfectly legitimate and common SIP usage.  PRANSWER is a 
> local construct that should be usable by the application to support 
> more complex offer/answer cases like forking.
>
> You must respond to this offer by cloning the peer connection, closing 
> the clone with an ANSWER identical to the previous PRANSWER and then 
> applying offer2 to the clone (or just applying the offer2 directly 
> with the understanding that the previous transaction is closed).  The 
> original peer connection must revert to the original OFFER state to 
> allow for other targets to respond.
>
> Alternately, the original peer connection can support the first target 
> and a new clone created if there is the potential for other forked 
> responses to the original offer.  Either way can be made to work.
>

Hmm....
could you please propose a semantic for "clone" that allows us to 
evaluate whether/how it can be implemented?

Especially, I am worried about what the state of the peerconnection has 
to be before it is cloned, what the state of the peerconnection is after 
it is cloned, which peerconnection owns the various resources allocated 
(ports are the obvious part of it), and which peerconnection any streams 
(local or remote) attached to the peerconnection before the clone are 
attached to after the clone.

In an earlier thread, I suggested a peerconnection factory that could 
generate an offer, but required you to manufacture a real peerconnection 
before applying an answer (any answer).... since I'm not that interested 
in supporting forking, I haven't pursued that further. Others might want to.

I don't think any of this is impossible. I do think it requires rather 
precise definitions of what we want before we ask browser developers to 
do it.

            Harald

---

From: Roman Shpount <[email-masked]>
Date: Wed, 2 May 2012 11:40:09 -0400

On Wed, May 2, 2012 at 9:35 AM, Harald Alvestrand <[email-masked]>wrote:

> Especially, I am worried about what the state of the peerconnection has to
> be before it is cloned, what the state of the peerconnection is after it is
> cloned, which peerconnection owns the various resources allocated (ports
> are the obvious part of it), and which peerconnection any streams (local or
> remote) attached to the peerconnection before the clone are attached to
> after the clone.
>
>
I thought there was a discussion on this list about using the same set of
ports, ICE candidates and turn connection across multiple peerconnections.
I am actually curious if there is ever a down side of using the same set of
ICE candidates for all the streams within the all peerconnections for a
given web session. It should be possible to disambiguate those streams
based on remote IP/port and SSRC.

The state that must to be replicated with the peerconnection is the latest
offer information. It would probably be less disruptive if peerconnection
has some sort of clone method instead of using a factory. It should be
possible to clone the connection between the time offer is generated and
the final answer is received.
_____________
Roman Shpount

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 02 May 2012 17:54:00 +0200

On 05/02/2012 05:40 PM, Roman Shpount wrote:
>
> On Wed, May 2, 2012 at 9:35 AM, Harald Alvestrand 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Especially, I am worried about what the state of the
>     peerconnection has to be before it is cloned, what the state of
>     the peerconnection is after it is cloned, which peerconnection
>     owns the various resources allocated (ports are the obvious part
>     of it), and which peerconnection any streams (local or remote)
>     attached to the peerconnection before the clone are attached to
>     after the clone.
>
>
> I thought there was a discussion on this list about using the same set 
> of ports, ICE candidates and turn connection across multiple 
> peerconnections. I am actually curious if there is ever a down side of 
> using the same set of ICE candidates for all the streams within the 
> all peerconnections for a given web session. It should be possible to 
> disambiguate those streams based on remote IP/port and SSRC.
It does mean that the implementation will have to do reference counting 
to know when it can close the port - if one clones the socket and binds 
the clones to different remote ports, I think the OS will take care of 
it on Unix, I'm not sure how it goes for other OSes.

Does someone know what the semantic of bind() is here - whether one 
needs to have an unbound port to fork from in order to bind to different 
remote addresses? I've not tried this myself.

In the case of hardware codecs that must be allocated to a specific 
stream .... if one supports forking, which connection (if any) gets the 
codec? The first one to start using it? What happens to the second one?

>
> The state that must to be replicated with the peerconnection is the 
> latest offer information. It would probably be less disruptive if 
> peerconnection has some sort of clone method instead of using a 
> factory. It should be possible to clone the connection between the 
> time offer is generated and the final answer is received.
I kind of think it's less disruptive to the people who don't want to 
fork stuff if you must instantiate a different object in order to 
support forking. Then implementations that don't support forking can 
simply not offer a constructor for that object.

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 2 May 2012 18:13:17 +0200

Hi,

I thought that we had more or less agreed on the fact that, eventhough we would support forking, we would only support one "active" remote location - and typically that would be the location from where you got the previous answer.

So, if you receive an answer from R1, that becomes "active". Then, if you receive an answer from R2, that becomes "active".

Then, if you receive an UPDATE from R1, you would reject it, since R2 is "active".

...or, alternatively you could switch back to R1, and accept the UPDATE. That is implementation dependent, and nothing we need to specify.

Do people think that is not enough, and that we would need to support multiple remote locations simultanously (no matter whether it's implemented using cloning or something else)?

Regards,

Christer

________________________________
From: [email-masked] [[email-masked]] On Behalf Of Harald Alvestrand [[email-masked]]
Sent: Wednesday, May 02, 2012 6:54 PM
To: Roman Shpount
Cc: [email-masked]
Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

On Wed, May 2, 2012 at 9:35 AM, Harald Alvestrand <[email-masked]<mailto:[email-masked]>> wrote:
Especially, I am worried about what the state of the peerconnection has to be before it is cloned, what the state of the peerconnection is after it is cloned, which peerconnection owns the various resources allocated (ports are the obvious part of it), and which peerconnection any streams (local or remote) attached to the peerconnection before the clone are attached to after the clone.

I thought there was a discussion on this list about using the same set of ports, ICE candidates and turn connection across multiple peerconnections. I am actually curious if there is ever a down side of using the same set of ICE candidates for all the streams within the all peerconnections for a given web session. It should be possible to disambiguate those streams based on remote IP/port and SSRC.
It does mean that the implementation will have to do reference counting to know when it can close the port - if one clones the socket and binds the clones to different remote ports, I think the OS will take care of it on Unix, I'm not sure how it goes for other OSes.

Does someone know what the semantic of bind() is here - whether one needs to have an unbound port to fork from in order to bind to different remote addresses? I've not tried this myself.

In the case of hardware codecs that must be allocated to a specific stream .... if one supports forking, which connection (if any) gets the codec? The first one to start using it? What happens to the second one?

The state that must to be replicated with the peerconnection is the latest offer information. It would probably be less disruptive if peerconnection has some sort of clone method instead of using a factory. It should be possible to clone the connection between the time offer is generated and the final answer is received.
I kind of think it's less disruptive to the people who don't want to fork stuff if you must instantiate a different object in order to support forking. Then implementations that don't support forking can simply not offer a constructor for that object.

---

From: Roman Shpount <[email-masked]>
Date: Wed, 2 May 2012 12:23:27 -0400

On Wed, May 2, 2012 at 11:54 AM, Harald Alvestrand <[email-masked]>wrote:

> to know when it can close the port - if one clones the socket and binds the
> clones to different remote ports, I think the OS will take care of it on
> Unix, I'm not sure how it goes for other OSes.
>
> Does someone know what the semantic of bind() is here - whether one needs
> to have an unbound port to fork from in order to bind to different remote
> addresses? I've not tried this myself.
>

I actually do not think this cloning the socket will work properly even on
Unix. You can connect a cloned UDP socket to a different remote IP/port but
both sockets will continue to receive packets from ALL the remote IP/ports.

Even if it does work, this does not prevent the implementation from
reference counting, since the same TURN allocation would need to be reused
across multiple peerconnections, with a new set of permissions/channels
created for each connection.

In the case of hardware codecs that must be allocated to a specific stream
> .... if one supports forking, which connection (if any) gets the codec? The
> first one to start using it? What happens to the second one?
>

My understanding was that you do not really need to allocated hardware
resources until the answer arrives and ICE handshake completes. So, my
assumption was that in case of forking, original connection does not
allocate any codec related resources. If the connection is cloned, then it
does not allocation any codec related resources either and stores the
reference for the sockets and TURN allocations from the original
connection. When either connection receives an answer, it conducts the ICE
handshake and allocates resources as needed.

>  I kind of think it's less disruptive to the people who don't want to fork
> stuff if you must instantiate a different object in order to support
> forking. Then implementations that don't support forking can simply not
> offer a constructor for that object.
>

Alternatively clone can return and error/null if cloning is not supported.
_____________
Roman Shpount

---

From: Roman Shpount <[email-masked]>
Date: Wed, 2 May 2012 12:27:20 -0400

On Wed, May 2, 2012 at 12:13 PM, Christer Holmberg <
[email-masked]> wrote:

> Do people think that is not enough, and that we would need to support
> multiple remote locations simultanously (no matter whether it's implemented
> using cloning or something else)?
>
>
I never thought that was enough if we are to interop with anything that
support forking. Latest example from Partha shows that this is indeed the
case.

I actually think that adding support for forking is trivial (order of
magnitude less complcated then DTLS-SRTP or identity) and would simplify
interop as well as enable numerous additional calling scenarios.
_____________
Roman Shpount

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 2 May 2012 18:28:32 +0200

Hi,

>> Do people think that is not enough, and that we would need to support multiple remote locations 
>> simultanously (no matter whether it's implemented using cloning or something else)?
>
> I never thought that was enough if we are to interop with anything that support forking. Latest example from > Partha shows that this is indeed the case.
>
> I actually think that adding support for forking is trivial (order of magnitude less complcated then DTLS-
> SRTP or identity) and would simplify interop as well as enable numerous additional calling scenarios.

It's not about forking as such - I also agree we need to support that.

The question is whether we need to *simultanously* need to support multiple remote locations, or whether we only have one "active" remote location, but we allow switching from one to another (based on where the previous answer can from, or whatever policy).

Regards,

Christer

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 02 May 2012 18:38:36 +0200

On 05/02/2012 06:27 PM, Roman Shpount wrote:
>
> On Wed, May 2, 2012 at 12:13 PM, Christer Holmberg 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Do people think that is not enough, and that we would need to
>     support multiple remote locations simultanously (no matter whether
>     it's implemented using cloning or something else)?
>
>
> I never thought that was enough if we are to interop with anything 
> that support forking. Latest example from Partha shows that this is 
> indeed the case.
>
> magnitude less complcated then DTLS-SRTP or identity) and would 
> simplify interop as well as enable numerous additional calling scenarios.

I'm looking forward to reviewing your detailed proposal and your 
implementation.

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Wed, 2 May 2012 15:27:53 -0500

Christer,
Regarding whether we need to be able to simultaneously support multiple remote locations (forking), I have reluctantly concluded "yes" after thinking about it for a bit.

The UPDATE scenario from Partha is not the only reason we need the concept of peer connection cloning (or something similar), although it is sufficient in itself.  As soon as you get multiple answers from different forked targets, it is necessary to separately keep track of ICE and DTLS progress with each target, not to mention RTCP.  Otherwise ICE and DTLS results from one target will get overwritten by those from another and then need to be redone again if an earlier target is selected as the winner.  To me this is not acceptable, hence my reluctant conclusion.

Maybe it could work if we simply delay ICE, DTLS, RTCP and media until we decide on a final ANSWER, and any nested offer/answer transactions (i.e., Partha's example) are handled at the JS layer only without interaction with the peer connection.  Does anyone think this is an acceptable approach?  Not I.

Richard

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christer Holmberg
Sent: Wednesday, May 02, 2012 11:29 AM
To: Roman Shpount
Cc: [email-masked]
Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

Hi,

Regards,

Christer

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Wed, 2 May 2012 15:34:22 -0500

Harald,
Thanks for the challenge.

I do agree with Roman that there is considerable overlap between forking and bundle that we should be able to leverage in a solution.

I need a few days to think this through before I am ready to propose anything specific.

BR, Richard

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Harald Alvestrand
Sent: Wednesday, May 02, 2012 8:35 AM
To: [email-masked]
Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

As far as SIP is concerned, there is no such thing as a PRANSWER.  The first target already responded with an SDP ANSWER and wants to UPDATE with a new offer.
In that case, PRANSWER seems like the wrong semantic.... if you want to support multiple answers, shouldn't you do the cloning before you start accepting answers?

 This is perfectly legitimate and common SIP usage.  PRANSWER is a local construct that should be usable by the application to support more complex offer/answer cases like forking.

Hmm....
could you please propose a semantic for "clone" that allows us to evaluate whether/how it can be implemented?

In an earlier thread, I suggested a peerconnection factory that could generate an offer, but required you to manufacture a real peerconnection before applying an answer (any answer).... since I'm not that interested in supporting forking, I haven't pursued that further. Others might want to.

I don't think any of this is impossible. I do think it requires rather precise definitions of what we want before we ask browser developers to do it.

           Harald

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 2 May 2012 22:41:22 +0200

Hi,

>Regarding whether we need to be able to simultaneously support multiple remote locations (forking), I have >reluctantly concluded "yes" after thinking about it for a bit.
>
>The UPDATE scenario from Partha is not the only reason we need the concept of peer connection cloning (or >something similar), although it is sufficient in itself.  As soon as you get multiple answers from different forked >targets, it is necessary to separately keep track of ICE and DTLS progress with each target, not to mention >RTCP.

...unless you, when you receive an answer from a new target, terminate the leg with the previous forked target.

>Otherwise ICE and DTLS results from one target will get overwritten by those from another and then need to be >redone again if an earlier target is selected as the winner. 

I think that is unlikely to happen, as forking is normally done in a serial manner (to avoid other issues associated with parallel forking).

I am not saying that there aren't use-cases that would break, but my question is still whether we can live with that restriction.

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christer Holmberg
Sent: Wednesday, May 02, 2012 11:29 AM
To: Roman Shpount
Cc: [email-masked]
Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

Hi,

Regards,

Christer

---

From: Roman Shpount <[email-masked]>
Date: Wed, 2 May 2012 17:13:21 -0400

On Wed, May 2, 2012 at 4:41 PM, Christer Holmberg <
[email-masked]> wrote:

> I think that is unlikely to happen, as forking is normally done in a
> serial manner (to avoid other issues associated with parallel forking).
>

I am not sure why you would say that most forking is serial. It is quite
often parallel. What is commonly done to avoid support for multiple
parallel media streams is playing media from the newest received RTP
stream. In either case there is no guarantee that you picked the winning
stream, and you always need to make sure that you use the SDP received in
the dialog that got 200 OK and that media playback switches to RTP stream
that remains. Making this work is often problematic in SIP since there is
no way to associate received RTP with received SDP information due to RTP
being sent from a different IP/port from the one used in SDP. It should be
easier with WebRTC since ICE support is required and remote party IP/port
can be determined for each flow.

Finally, when you are dealing with forking you need to keep track of the
remote IP/ports and ICE candidates for each dialog. You can play media from
only one of them, but this remote connection state is required to complete
the call setup when 200 OK is received.
_____________
Roman Shpount

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 3 May 2012 08:02:44 +0200

Hi,

> I am not sure why you would say that most forking is serial. It is quite often parallel. What is commonly done to avoid support for multiple 
> parallel media streams is playing media from the newest received RTP stream. In either case there is no guarantee that you picked the 
> winning stream, and you always need to make sure that you use the SDP received in the dialog that got 200 OK and that media playback 
> switches to RTP stream that remains. Making this work is often problematic in SIP since there is no way to associate received RTP with 
> received SDP information due to RTP being sent from a different IP/port from the one used in SDP. It should be easier with WebRTC 
> since ICE support is required and remote party IP/port can be determined for each flow.
>
> Finally, when you are dealing with forking you need to keep track of the remote IP/ports and ICE candidates for each dialog. You can 
> play media from only one of them, but this remote connection state is required to complete the call setup when 200 OK is received.

You are absolutely right. I still question whether we need to support parallel forking. If you say it occurs often, I can't argue against you, eventhough my own experience is different :)

(Another option, if we want to support parallel forking, would of course be to create a completely new PeerConnection, with a *new* local IP address:port. But, you would of course then have to provide that information to the remote peer.)

I have no strong feeling on whether we want to do cloning, but do people agree that, for a given PeerConnection, we only need to support a single remote peer (which can be modified, though)?

Regards,

Christer

---

From: Roman Shpount <[email-masked]>
Date: Thu, 3 May 2012 11:01:04 -0400

On Thu, May 3, 2012 at 2:02 AM, Christer Holmberg <
[email-masked]> wrote:

> You are absolutely right. I still question whether we need to support
> parallel forking. If you say it occurs often, I can't argue against you,
> even though my own experience is different :)
>
>
You do end up with parallel forking when you allow registrations from
multiple devices. This is a policy option, but once you enable this,
parallel forking or B2BUA are the only options. Furthermore, even if you do
serial forking, you end up with multiple parallel media streams being
received by the client due to loose synchronization between RTP streams and
SIP signaling. The main discrepancy between current WebRTC signaling model
and SIP is that in SIP, when you do forking, you still have separate
dialogs that each maintain state. Even if only render media from one RTP
stream, black hole all the other media, and send media to one RTP
destination, you still have dialog specific SIP and ICE state that you can
update (for instance due to UPDATE transaction) and recover if the
currently selected dialog ended up not being the one that was selected in
200 OK. This is not an option with WebRTC.

> (Another option, if we want to support parallel forking, would of course
> be to create a completely new PeerConnection, with a *new* local IP
> address:port. But, you would of course then have to provide that
> information to the remote peer.)
>

This would probably do absolutely nothing to SIP interop. And if we do not
need SIP interop we can live without forking (at the cost of being slightly
less efficient in some call setup scenarios).

> I have no strong feeling on whether we want to do cloning, but do people
> agree that, for a given PeerConnection, we only need to support a single
> remote peer (which can be modified, though)?
>

I think if we do cloning we will only need to support a single remote peer
per PeerConnection. Strictly speaking even updates due to provisional
responses would not be necessary, since you can always clone the connection
and provide a new answer to it instead.
_____________
Roman Shpount

---

From: Randell Jesup <[email-masked]>
Date: Thu, 03 May 2012 12:37:27 -0400

On 5/3/2012 11:01 AM, Roman Shpount wrote:
>
> On Thu, May 3, 2012 at 2:02 AM, Christer Holmberg
> <[email-masked] <mailto:[email-masked]>>
> wrote:
>
>     You are absolutely right. I still question whether we need to
>     support parallel forking. If you say it occurs often, I can't argue
>     against you, even though my own experience is different :)
>
>
> parallel forking or B2BUA are the only options.

Correct; I brought this up multiple times last year, when we had one of 
the original discussions of cloning.

If I call [email-masked], and he has a phone, and a tablet, and a 
computer, and a star trek communicator all "attached" to google's JSEP 
signaling proxy, it's going to want to fork the offer to all of them. 
So I really believe parallel forking is needed for very basic use-cases. 
  And both Scotty (in the Away Team) and his assistant (sitting at the 
computer) might accept.  If his assistant accepts first, Scotty is out 
of luck unless some parallel forking is supported and PeerConnection 
cloning is done.  That doesn't mean you must render or encode data for 
all clones simultaneously, though that's a nice option and mirrors 
normal requirements for mesh conferencing.  You can send the same data 
to both (mesh), or possibly act as a conference bridge instead of a full 
mesh conference (ah-hoc bridged conference).

Another usecase I bandied about was "hanging" cloned offers being 
provided to new participants in a partial-mesh conference.  Think Second 
Life, or a virtual conference with positional locality (it doesn't have 
to be visual Virtual Reality ala Second Life; it can be abstract - but 
it's easiest to think of in a Second Life or MMORPG type of setting). 
I.e. the signaling server holds a cloned copy of your invite, and when 
someone new comes into hearing range of you it forwards your (cloned) 
OFFER to them (delayed parallel forking, if you will).  Otherwise, it 
would need a protocol to let the server tell you to generate a new offer 
that it could then forward, adding roughly 1 client-server-RTT to setup 
time, plus requiring extra application protocols and logic.

>     (Another option, if we want to support parallel forking, would of
>     course be to create a completely new PeerConnection, with a *new*
>     local IP address:port. But, you would of course then have to provide
>     that information to the remote peer.)
>
>
> This would probably do absolutely nothing to SIP interop. And if we do
> not need SIP interop we can live without forking (at the cost of being
> slightly less efficient in some call setup scenarios).

Ignoring SIP interop, this might work as an equivalent to cloning, but 
requiring an extra O/A exchange (and adding considerable chance of 
clipping on a second fork being activated).

>
>     I have no strong feeling on whether we want to do cloning, but do
>     people agree that, for a given PeerConnection, we only need to
>     support a single remote peer (which can be modified, though)?
>
>
> I think if we do cloning we will only need to support a single remote
> peer per PeerConnection. Strictly speaking even updates due to
> provisional responses would not be necessary, since you can always clone
> the connection and provide a new answer to it instead.

I believe that's correct.

-- 
Randell Jesup
[email-masked]

---

From: "Roy, Radhika R CIV (US)" <[email-masked]>
Date: Thu, 3 May 2012 17:34:36 +0000

Correct; I brought this up multiple times last year, when we had one of the original discussions of cloning.

[RRR] Did you also think about sub-conferencing as a part of the main conferencing as well?

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 3 May 2012 20:51:23 +0200

Hi,

> clipping on a second fork being activated).

Assuming you have to perform the ICE procedures anyway, I don't think it would cause clipping. It probably would cause some additinoal call setup delay, though, as you need to perform that extra O/A exchange before the ICE procedures can start.

> I believe that's correct.

Sure. But, that would also mean that you would have to clone the PeerConnection mid-session, if the remote entity changes.

Regards,

Christer

---

From: Roman Shpount <[email-masked]>
Date: Thu, 3 May 2012 15:09:34 -0400

On Thu, May 3, 2012 at 2:51 PM, Christer Holmberg <
[email-masked]> wrote:

> Sure. But, that would also mean that you would have to clone the
> PeerConnection mid-session, if the remote entity changes.
>

What I meant was that you do not need ability to update remote destination
multiple times per single offer. You should still be able to initiate
another O/A exchange on the same peer connection and change the remote
party this way.
_____________
Roman Shpount

---

From: Randell Jesup <[email-masked]>
Date: Thu, 03 May 2012 15:54:12 -0400

> Assuming you have to perform the ICE procedures anyway, I don't think it would cause clipping. It probably would cause some additinoal call setup delay, though, as you need to perform that extra O/…

Call setup delay causes clipping, unless you in some manner use feedback 
to the answerer to hold them off from talking.  The classic problem you 
probably all know is "pickup receiver, say "hello" - if call setup takes 
(or media is clipped by) more than a couple hundred ms, the clipping is 
audible to the caller.  And it gets worse (100ms or perhaps even less) 
for "click-to-answer".  The only thing to help you there is possible 
visual indication that it's not connected yet to encourage the answerer 
to wait before talking, and in some uses that's not possible or isn't 
sufficiently obvious.  (audio-only connections, especially in a context 
where you're paying attention to other things as well (game, VR 
conference, etc)).

-- 
Randell Jesup
[email-masked]

---

From: Roman Shpount <[email-masked]>
Date: Thu, 3 May 2012 16:26:25 -0400

On Thu, May 3, 2012 at 3:54 PM, Randell Jesup <[email-masked]>wrote:

> audible to the caller.  And it gets worse (100ms or perhaps even less) for
> "click-to-answer".  The only thing to help you there is possible visual
> indication that it's not connected yet to encourage the answerer to wait
> before talking, and in some uses that's not possible or isn't sufficiently
> obvious.  (audio-only connections, especially in a context where you're
> paying attention to other things as well (game, VR conference, etc)).
>

Unfortunately this is unavoidable with ICE. Once you have ICE setup enabled
you normally end up with at least a RTT delay. If this is a call from east
to west coast in US, this typically means that about 100 ms worth of media
is clipped (pretty much means no initial hello). If you add DTLS setup on
top of this, you end up with more then half a second worth of audio
clipped, so this typically means people saying hello several times to see
if there is a person on the other side. All of this for the calls in
continental US. If you are talking to somebody over a satellite link,
situation is much worse. The way I've seen dealt with this was playing some
sort of audio and media to the answering party until call is actually
connected to prevent them from speaking.
_____________
Roman Shpount

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 3 May 2012 22:29:04 +0200

Hi,

You could also choose not to alert the remote user until the ICE procedures have finished - more or less using ICE with preconditions.

Of course, that is going to trigger actions in STUN/TURN servers, even if the called party won't accept the call, but at least from a user experience perspective that is still better than lifting the hook, and having to wait for whatever-time-it-takes-for-ICE-to-finish seconds before one can start to talk.

Regards,

Christer

________________________________
From: [email-masked] [[email-masked]] On Behalf Of Roman Shpount [[email-masked]]
Sent: Thursday, May 03, 2012 11:26 PM
To: Randell Jesup
Cc: [email-masked]
Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

On Thu, May 3, 2012 at 3:54 PM, Randell Jesup <[email-masked]<mailto:[email-masked]>> wrote:

Call setup delay causes clipping, unless you in some manner use feedback to the answerer to hold them off from talking.  The classic problem you probably all know is "pickup receiver, say "hello" - if call setup takes (or media is clipped by) more than a couple hundred ms, the clipping is audible to the caller.  And it gets worse (100ms or perhaps even less) for "click-to-answer".  The only thing to help you there is possible visual indication that it's not connected yet to encourage the answerer to wait before talking, and in some uses that's not possible or isn't sufficiently obvious.  (audio-only connections, especially in a context where you're paying attention to other things as well (game, VR conference, etc)).

Unfortunately this is unavoidable with ICE. Once you have ICE setup enabled you normally end up with at least a RTT delay. If this is a call from east to west coast in US, this typically means that about 100 ms worth of media is clipped (pretty much means no initial hello). If you add DTLS setup on top of this, you end up with more then half a second worth of audio clipped, so this typically means people saying hello several times to see if there is a person on the other side. All of this for the calls in continental US. If you are talking to somebody over a satellite link, situation is much worse. The way I've seen dealt with this was playing some sort of audio and media to the answering party until call is actually connected to prevent them from speaking.
_____________
Roman Shpount

---

From: Roman Shpount <[email-masked]>
Date: Thu, 3 May 2012 16:34:21 -0400

On Thu, May 3, 2012 at 4:29 PM, Christer Holmberg <
[email-masked]> wrote:

> You could also choose not to alert the remote user until the ICE
> procedures have finished - more or less using ICE with preconditions.
>
> Of course, that is going to trigger actions in STUN/TURN servers, even if
> the called party won't accept the call, but at least from a user experience
> perspective that is still better than lifting the hook, and having to wait
> for whatever-time-it-takes-for-ICE-to-finish seconds before one can start
> to talk.
>
>
This also has a downside of disclosing user's IP to the caller without the
user confirmation. For a lot of applications this can be serious security
flaw.
_____________
Roman Shpount

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 3 May 2012 22:50:05 +0200

Hi,

>> Of course, that is going to trigger actions in STUN/TURN servers, even if the called party won't accept the call, but at least from a user 
>> experience perspective that is still better than lifting the hook, and having to wait for whatever-time-it-takes-for-ICE-to-finish seconds before one can start to talk.
>
> This also has a downside of disclosing user's IP to the caller without the user confirmation. For a lot of applications this can be serious security flaw. 

The client can still log when ICE procedures occur.

Because, even if I wait until your phone starts to ring, most likely I would still get your IP address without user confirmation (speaking in SIP terms, phones normally don't ask for user permission before they send 18x with SDP), eventhough you would easier be made aware of that it happens.

Another alternative is of course to do ICE with an SBC, and/or having an SBC doing ICE on behalf of you :)

Regards,

Christer

---

From: Roman Shpount <[email-masked]>
Date: Thu, 3 May 2012 17:01:52 -0400

On Thu, May 3, 2012 at 4:50 PM, Christer Holmberg <
[email-masked]> wrote:

> >> Of course, that is going to trigger actions in STUN/TURN servers, even
> if the called party won't accept the call, but at least from a user
> >> experience perspective that is still better than lifting the hook, and
> having to wait for whatever-time-it-takes-for-ICE-to-finish seconds before
> one can start to talk.
> >
> > This also has a downside of disclosing user's IP to the caller without
> the user confirmation. For a lot of applications this can be serious
> security flaw.
>
> Because, even if I wait until your phone starts to ring, most likely I
> would still get your IP address without user confirmation (speaking in SIP
> terms, phones normally don't ask for user permission before they send 18x
> with SDP), eventhough you would easier be made aware of that it happens.
>

This is true for SIP but was as far as I know was specifically designed
around in WebRTC. WebRTC signaling server acts as B2BUA so any type of
ringing notification goes through the web server and does not need to carry
any type of client address information. The client address information is
only provided when SDP answer is sent or ICE negotiation is started.
_____________
Roman Shpount

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 3 May 2012 23:16:38 +0200

Hi,

> This is true for SIP but was as far as I know was specifically designed around in WebRTC. WebRTC signaling server acts as B2BUA so any type of ringing notification goes through the web server and does not need to carry any type of client address information. The client address information is only provided 
> when SDP answer is sent or ICE negotiation is started. 

Assuming you are only going to make user confirmation (read: accept calls) in cases where you absolutely sure that the caller isn't someone trying to get your IP address.

But, never the less, having a solution where I first have to give user confirmation, and then wait until I can start to talk, is probably something most people want to avoid. Depending upon, of course, how long the waiting time is.

Regards,

Christer

---

From: Justin Uberti <[email-masked]>
Date: Tue, 8 May 2012 11:09:58 -0400

The conclusion on this thread seems to be that the right way to address
this problem is via PeerConnection cloning, and that no changes are needed
to the JSEP state machine. I'll work with Richard to figure out how we
should extend JSEP to support cloning.

On Thu, May 3, 2012 at 5:16 PM, Christer Holmberg <
[email-masked]> wrote:

> >>>> Of course, that is going to trigger actions in STUN/TURN servers,
> even if the called party won't accept the call, but at least from a user
> >>>> experience perspective that is still better than lifting the hook,
> and having to wait for whatever-time-it-takes-for-ICE-to-finish seconds
> before one can start to talk.
> >>>
> only provided
> Assuming you are only going to make user confirmation (read: accept calls)
> in cases where you absolutely sure that the caller isn't someone trying to
> get your IP address.
>
> But, never the less, having a solution where I first have to give user
> confirmation, and then wait until I can start to talk, is probably
> something most people want to avoid. Depending upon, of course, how long
> the waiting time is.
>
> Regards,
>
> Christer

---

From: "Ravindran, Parthasarathi" <[email-masked]>
Date: Wed, 9 May 2012 06:34:33 +0000

Hi Justin,

Sorry for the delay in reply. There are two issues associated with this scenario:

Issue 1) SIP Forking : This is solved by peerconnection cloning

Issue 2)  UPDATE support within SIP early dialog in the non-forking scenario. Here, the cloning will not solve issue as the answerer originated the new offer during SDP_PRANSWER state . Issue 2 is the title of this mail thread.

IMO, SDP_PRANSWER will complicate the offer/answer state machine.

Thanks
Partha

From: [email-masked] [mailto:[email-masked]] On Behalf Of Justin Uberti
Sent: Tuesday, May 08, 2012 8:40 PM
To: Christer Holmberg
Cc: Randell Jesup; [email-masked]
Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

The conclusion on this thread seems to be that the right way to address this problem is via PeerConnection cloning, and that no changes are needed to the JSEP state machine. I'll work with Richard to figure out how we should extend JSEP to support cloning.
On Thu, May 3, 2012 at 5:16 PM, Christer Holmberg <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

Regards,

Christer

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 10 May 2012 08:13:37 -0600

On May 3, 2012, at 2:34 PM, Roman Shpount wrote:

> On Thu, May 3, 2012 at 4:29 PM, Christer Holmberg <[email-masked]> wrote:

The initial media flow could be forced to provide only relay ICE candidates to help mitigate this. (I think we decided awhile back that was one of the things we would provide support for in the API). Later with the user answers or consents in same way, the ICE can move the media to a non relay flow.

---

From: Roman Shpount <[email-masked]>
Date: Thu, 10 May 2012 12:40:21 -0400

On Thu, May 10, 2012 at 10:13 AM, Cullen Jennings <[email-masked]> wrote:

> The initial media flow could be forced to provide only relay ICE
> candidates to help mitigate this. (I think we decided awhile back that was
> one of the things we would provide support for in the API). Later with the
> user answers or consents in same way, the ICE can move the media to a non
> relay flow.
>
> Just to make this clear, in this scenario, once application receives the
offer, it will generate a provisional answer with only relay candidates,
send this answer to the calling party, completes the ICE process on the
called party side, and then alert the called party. Once called user
accepts the call, application will send the final answer with relay,
reflexive and local candidates. This can also be implemented with final
answer with only relay candidates by the called party before user accepts
the the call, and O/A in the opposite direction once the call is accepted.
There is still a slight chance of clipping the media from the calling party
towards the called party since the answer might not arrive to the calling
party or ICE process from the calling party to the called party might not
complete before the called party answers. To avoid this, application will
need to send a separate message via signaling or data channel once the ICE
process on the calling side completes and only alert the called party once
this signaling message is received. What it boils down to is that
application should setup the media channel (provide relay candidates only
from the called side, if IP privacy is a concern), send a separate
signaling message when it is done, and then alert the user. This will
significantly slow down the actual call setup and will use a bunch of extra
resources, but will solve the clipping problem.
_____________
Roman Shpount

---

From: Martin Thomson <[email-masked]>
Date: Thu, 10 May 2012 10:03:39 -0700

> The initial media flow could be forced to provide only relay ICE candidates to help mitigate this. (I think we decided awhile back that was one of the things we would provide support for in the API…

That's right.  But you pretty much have to leave this in the hands of
the application.

Unless you want to provide an authenticated indication of user answer
on the media path so that when the user picks up the phone you can
provide other local candidates...

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 10 May 2012 13:05:51 -0600

I'm a very confused on what type of forking (with ICE) use case people want to satisfy. 

I agree with you that PRANSWER does make things more complicated but I presented at several of the meetings about how things just don't work without it or something like it so I don't see how to avoid it. 

On May 9, 2012, at 12:34 AM, Ravindran, Parthasarathi wrote:

> On Thu, May 3, 2012 at 5:16 PM, Christer Holmberg <[email-masked]> wrote:
> Hi,
>

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 10 May 2012 13:06:32 -0600

Uh, I don't think we are ready to design anything yet - I am still 100% confused on what the use case we are trying to solve is. Is it forking on at the browser, in intermediate node, in SIP, parallel, sequential or what? I want to see the end user use cases we need to deal with. The example Randell gave seems reasonable but seem like it would best be handled with just multiple independent PeerConnection. The combination of requiring ICE and SIP parallel forking is very complicated - particularly without Reliable provisions responses and all the complexity that ensues.

So before we go down this path, lets get clearer on what we are trying to accomplish.

Cullen

---

From: Roman Shpount <[email-masked]>
Date: Thu, 10 May 2012 15:51:29 -0400

On Thu, May 10, 2012 at 3:06 PM, Cullen Jennings <[email-masked]> wrote:

>
> Uh, I don't think we are ready to design anything yet - I am still 100%
> confused on what the use case we are trying to solve is. Is it forking on
> at the browser, in intermediate node, in SIP, parallel, sequential or what?
> I want to see the end user use cases we need to deal with. The example
> Randell gave seems reasonable but seem like it would best be handled with
> just multiple independent PeerConnection. The combination of requiring ICE
> and SIP parallel forking is very complicated - particularly without
> Reliable provisions responses and all the complexity that ensues.
>
One use case is interop with SIP with parallel forking, reliable
provisioning responses, UPDATE, and ICE. The big question regarding this
use case is availability of anything that supports parallel forking,
reliable provisioning and UPDATE. I actually think ICE makes implementation
of this use case easier, not harder.

The real end user use case is any type of service where calling destination
is a small group of people. This allows to initiate a single offer and send
it to all the members of the group. In most cases only one person will
reply, but, in a smaller set of cases, several people from the group will
answer. In the later case, you need a mechanism to clone the original peer
connection (or to create additional peer connections based on the original
offer) and deal with additional answers independently.

This is a very common use case for any type of service where people use
multiple devices. For instance if we add voice/video calling to a web mail
program, I can be logged in from multiple devices. It can also be a shared
account (like [email-masked]) where several different people are logged
in to the same account. When someone places a call to a shared account,
several people can answer at the same time.

I know this can be addressed by sending a separate message  and reversing
the call setup direction (each accepting party creates an offer and sends
it back), but this causes another set of issues related to slow call setup
and media clipping.
_____________
Roman Shpount

---

From: Martin Thomson <[email-masked]>
Date: Thu, 10 May 2012 12:53:13 -0700

> I'm a very confused on what type of forking (with ICE) use case people want to satisfy.

Here's the use case as I understand it:

create offer and send to some thing
the offer actually hits multiple things
multiple answers arrive
want to establish sessions with all of them

That's the use case.  Adding it to the use cases draft (or not) is
probably something that needs to be debated more.

--Martin

p.s. How the offer reaches multiple things doesn't matter much.  I see
too many emails on these lists that include the letters S, I and P
together.

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 10 May 2012 14:55:16 -0600

On May 10, 2012, at 1:51 PM, Roman Shpount wrote:

> One use case is interop with SIP with parallel forking, reliable provisioning responses, UPDATE, and ICE. The big question regarding this use case is availability of anything that supports parallel forking, reliable provisioning and UPDATE. I actually think ICE makes implementation of this use case easier, not harder.
> 
> The real end user use case is any type of service where calling destination is a small group of people. This allows to initiate a single offer and send it to all the members of the group. In most cases only one person will reply, but, in a smaller set of cases, several people from the group will answer. In the later case, you need a mechanism to clone the original peer connection (or to create additional peer connections based on the original offer) and deal with additional answers independently. 
> 
> This is a very common use case for any type of service where people use multiple devices. For instance if we add voice/video calling to a web mail program, I can be logged in from multiple devices. It can also be a shared account (like [email-masked]) where several different people are logged in to the same account. When someone places a call to a shared account, several people can answer at the same time.
> 
> I know this can be addressed by sending a separate message  and reversing the call setup direction (each accepting party creates an offer and sends it back), but this causes another set of issues related to slow call setup and media clipping.
> _____________
> Roman Shpount
> 

So let me try and make sure I understand this uses case. The web browser sends some signaling to a server that convert it to a sip call to the [email-masked] address. This sends it to the example.com sip server that parallel forks it to two SIP UASs  that are so two phones ring at the same time. Now both theses phones are going to negotiate ICE by using reliably provisional responses. 

What products actually do parallel forking like this and return provisional reliable responses with ICE. I'm sort of questioning this is a common use case. The Nortel, Cisco, Avaya, Siemens etc enterprises PBX systems I have seen that support ringing multiple phones at the same time don't seem to do this way. They manage to hide the parallel forking with a B2BUA model and make it look like sequential forking.

---

From: Justin Uberti <[email-masked]>
Date: Thu, 10 May 2012 16:56:07 -0400

The other use case is the one Partha originally mentioned, and recently
resurfaced:

Issue 2)  UPDATE support within SIP early dialog in the non-forking
scenario. Here, the cloning will not solve issue as the answerer originated
the new offer during SDP_PRANSWER state . Issue 2 is the title of this mail
thread.

which I think expands to:
1. local PC1 sends offer, forks to remote PC1 and PC2
2. remote PC1 receives offer, sends pranswer
3. local PC1 applies this pranswer
4. remote PC2 receives offer, sends pranswer
5. local PC1 applies this pranswer
6. remote PC1 sends update (i.e. new offer)
7. local PC1 does ???

Richard mentioned that this could be solved by cloning, and I think I agree.
local PC1 can clone at step 5, and create local PC2. local PC2 stays in the
pranswer state; local PC1 applies the previous pranswer as final, and then
handles the new offer (update).

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 10 May 2012 14:56:36 -0600

I was looking more for the actually sort of end user use case where this might happen.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 10 May 2012 14:07:35 -0700

> I was looking more for the actually sort of end user use case where this might happen.

I ring you.
Your mobile and home phone both ring.
You answer your mobile, your daughter answers the home phone.
I get to talk to both of you.

Of course, that's not such a great outcome.  I can hear both of you,
but you can't hear each other unless I have really bad echo
cancellation.

I tend to think of this as solving a really interesting signaling
problem that doesn't really have any practical application.  If you
want to talk to both people, then you should signal the creation of a
conference call.

--Martin

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Thu, 10 May 2012 23:12:41 +0200

2012/5/10 Cullen Jennings <[email-masked]>:
> The Nortel, Cisco, Avaya, Siemens etc enterprises PBX systems I have seen that support ringing multiple phones at the same time don't seem to do this way. They manage to hide the parallel forking with a B2BUA model and make it look like sequential forking.

Hi Cullen, I strongly prefer that WebRTC use cases are based on real
standards (i.e. pure SIP proxies) instead of non-standard devices
(like B2BUA's that are defined nowhere).

At least for me, the use case in which a "call" arrives to two
different devices and both generate direct (early-media) to the origin
is a mandatory use case for WebRTC, and I don't think that we should
discard this use case by design in any way.

Regards.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Roman Shpount <[email-masked]>
Date: Thu, 10 May 2012 17:23:30 -0400

On Thu, May 10, 2012 at 4:55 PM, Cullen Jennings <[email-masked]> wrote:

> So let me try and make sure I understand this uses case. The web browser
> sends some signaling to a server that convert it to a sip call to the
> [email-masked] address. This sends it to the example.com sip server
> that parallel forks it to two SIP UASs  that are so two phones ring at the
> same time. Now both theses phones are going to negotiate ICE by using
> reliably provisional responses.
>
> What products actually do parallel forking like this and return
> provisional reliable responses with ICE. I'm sort of questioning this is a
> common use case. The Nortel, Cisco, Avaya, Siemens etc enterprises PBX
> systems I have seen that support ringing multiple phones at the same time
> don't seem to do this way. They manage to hide the parallel forking with a
> B2BUA model and make it look like sequential forking.
>
>
Let's say a standard SIP proxy (call it opensips for specificity) placing a
forked call to two ICE enabled IP phones. Both phones answer with 200 OK.

_____________
Roman Shpount

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 10 May 2012 16:29:21 -0600

On May 10, 2012, at 3:23 PM, Roman Shpount wrote:

> So let me try and make sure I understand this uses case. The web browser sends some signaling to a server that convert it to a sip call to the [email-masked] address. This sends it to theexample.com sip server that parallel forks it to two SIP UASs  that are so two phones ring at the same time. Now both theses phones are going to negotiate ICE by using reliably provisional responses.
> 
> Let's say a standard SIP proxy (call it opensips for specificity) placing a forked call to two ICE enabled IP phones. Both phones answer with 200 OK.

With a standard SIP proxy, only one of the 200 would be sent back to the web browser.

---

From: Roman Shpount <[email-masked]>
Date: Thu, 10 May 2012 18:40:39 -0400

On Thu, May 10, 2012 at 6:29 PM, Cullen Jennings <[email-masked]> wrote:

> [email-masked] address. This sends it to theexample.com sip server that
> parallel forks it to two SIP UASs  that are so two phones ring at the same
> time. Now both theses phones are going to negotiate ICE by using reliably
> provisional responses.
> >
> > Let's say a standard SIP proxy (call it opensips for specificity)
> placing a forked call to two ICE enabled IP phones. Both phones answer with
> 200 OK.
>

AFAIK a standard proxy will forward multiple 200 OK responses (see section
16.7 of RFC 3261). We have the same problem with 183 responses from two
phones anyway.
_____________
Roman Shpount

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 10 May 2012 16:47:50 -0600

> want to talk to both people, then you should signal the creation of a

That an good use cases, except the people need to be able to hear each other. It gets called lots of things but some PBXs call it bridged line appearance. Variation of it have been discussed in bliss and draft-ietf-bliss-shared-appearances. It really comes down to where you think the conferencing should happen. If it does not happened on the the callers phone, seems like existing jsep idea should be able to do this.  (given the callee is the phone that is most likely out of administrative control of the other two phones that set up bridged line a appearance, assuming that phone can and is willing to do the conferencing seems like a bad architecture for this problem). Again lot of systems do this today in SIP and seem to do it without needing forcing the callers phones to deal with multiple simultaneous calls signaled in some way I don't even know how to signal in SIP.

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Fri, 11 May 2012 09:26:24 -0500

Support for multiple registered contacts and (parallel) forking are pretty fundamental to SIP.  This has little to do with conferencing.  Some systems (not all) may support only serial forking or may serialize media interactions associated with parallel forking, but forking remains widely deployed and useful.

The point is not to support simultaneous media flow with all forked targets (your talking to me and my daughter at the same time in Martin's example), but to support the ability to negotiate ICE and DTLS independently with each target before one of them "answers", to minimize the potential for clipping.

True parallel forking has been around for a long time and is supported in some (not all) existing systems.  We do not mandate ICE support (for example) because it is commonly deployed, but because it is useful; the same principle should apply here.  I have not heard that simultaneous alerting of multiple registered contacts is not useful, but just that some existing systems find ways of serializing the media interactions.  That is relatively easier to do when you don't support ICE and DTLS since media can flow almost as soon as you signal the SDP changes.

But things are different with ICE and DTLS, particularly if you want to avoid media intermediaries.  Significant clipping is a real possibility if we have to restart ICE/DTLS after the "winner" responds with 200 OK.

This is not a separate use case, but a given aspect of interworking with SIP.

Richard

---

From: Neil Stratford <[email-masked]>
Date: Fri, 11 May 2012 16:04:31 +0100

> This is not a separate use case, but a given aspect of interworking with SIP.

The whole problem of clipping audio due to the delay between the 200OK and the media actually flowing raises a flag that we are triggering upper layers of the stack on the wrong event. 

Why is anyone sending media before the media channel is fully established? Why tell the user (or IVR etc) that the call is answered before we have a working media channel?

Neil

---

From: Randell Jesup <[email-masked]>
Date: Fri, 11 May 2012 12:41:30 -0400

On 5/10/2012 6:47 PM, Cullen Jennings wrote:
>> On 10 May 2012 13:56, Cullen Jennings<[email-masked]>  wrote:

I'll note I've built systems that simply handle in-phone conferencing; 
while there are some annoyances it's most certainly doable.  If the 
caller acts as the MCU you get OWD1+jitter+etc+encode+OWD2 
(OWD=one-way-delay) between the two leaf nodes (though this is no 
different than a normal conference).  Or your app, after the initial 
second answer tells the second answerer to invite the first into a mesh 
conference, if that works - and it may not, so you need to mix or 
forward if you want a conference.   (Forwarding the RTP traffic works at 
the cost of bandwidth and complexity, but simpler media processing and 
less leaf-to-leaf delay.)

If this *is* already a mesh conference you've joined (different usecase 
than above), no mixing or forwarding is needed - you render all the 
streams, and you send media to all the answerers.  This is exactly what 
you want.

However, if two people answer, you do not have to have a conference.  
First of all, the devices rtcweb is intended for generally have enough 
UI available that the user can manage multiple sessions reasonably, 
unlike POTS-type phones.  You can handle it much like a call-waiting 
scenario, though the second answerer might be confused unless you send 
some sort of hold/please-wait message (which, this not being POTS 
devices, you can).

In other cases/applications, the answers may be automated, and the 
channels may not be used immediately - think the VR example, where your 
OFFER is forwarded to any person who comes in "hearing" range, enters 
the room, whatever.  This may be a form of partial-mesh conference 
managed by the server.  These channels may be opened before they're 
actually needed in order to hide setup delay.

My point is to show that the possible uses of parallel forking exist, 
and don't always map to SIP or to classic telephony.  If cloning is 
supported, then these scenarios work.

I'm open to alternative proposals that cover this class of use-cases, if 
they make either "standard" apps simpler to a worthwhile degree, or if 
they make the implementation of webrtc itself noticably simpler, while 
still making these uses possible.  Alternatives may have downsides in 
some cases, in extra call setup delay/clipping or requiring re-invites 
or requiring special application protocols to request a offerer send a 
new invite, and added complexity to apps that wish to handle this.

-- 
Randell Jesup
[email-masked]

---

From: "Ejzak, Richard P (Richard)" <[email-masked]>
Date: Fri, 11 May 2012 15:19:25 -0500

Neil,
That's the point.  This discussion is with reference to parallel forking.  To avoid clipping, you need to establish the media channels with the registered targets before they are alerted because you don't know which one will answer.  This requires MediaConnection cloning so you can properly support parallel forking and do ICE and DTLS with each target in parallel before 200 OK from one of the targets.  Delaying delivery of answer indication to the caller (as you seem to imply) does not help.

Richard

-----Original Message-----
From: Neil Stratford [mailto:[email-masked]] On Behalf Of Neil Stratford
Sent: Friday, May 11, 2012 10:05 AM
To: Ejzak, Richard P (Richard)
Cc: [email-masked]
Subject: Re: [rtcweb] SDP_PRANSWER followed by SDP_OFFER scenario in JSEP

Neil

---

From: "Ravindran, Parthasarathi" <[email-masked]>
Date: Mon, 14 May 2012 08:39:20 +0000

Cullen,

PRANSWER ISSUE:
In case PRANSWER is designed for interop with SIP forking with ICE handling, 
then PRANSWER is not required because SIP (RFC 3261) never allows multiple
answer within the same dialog. In case of each forked response (18x/200 with 
new To-Tag), SIP UA creates new dialog and those dialog has to be handled 
separately. How grouping of those SIP dialogs works together is a implementation
issue in SIP. So, there is no need of new O/A state (PRANSWER) just to make 
SIP interop happen. Please let me know any other usecase which compels 
PRANSWER state in JSEP.

FORKING ISSUE:
I agree with you that most of the deployed SIP end-point products does not 
Support parallel forking and its next hop B2BUA (PBX, SBC, GW) takes care
of the handling. Having said that, some of the well deployed SIP entity in 
service provider (SP) network is based on SIP forking which compels to 
support forking in actual End-point or B2BUA. I know that it is possible
to design JSEP without forking support and interop with SIP is possible which 
will be equivalent to H.323 to SIP gateway development wherein H.323 never
supports forking. Now the question is whether JSEP API supports forking 
or not. I'm lean towards forking because the same peerconnection cloning API 
may extend for browser media mixing later stage. If you explain about my 
mixing API idea is wrong, then I'll agree with you that cloning of peer 
connection is not required.

Thanks
Partha      

>From: Cullen Jennings [mailto:[email-masked]]
>Sent: Friday, May 11, 2012 12:36 AM
>To: Ravindran, Parthasarathi
>Cc: Justin Uberti; Christer Holmberg; Randell Jesup; [email-masked]
>I agree with you that PRANSWER does make things more complicated but I
>presented at several of the meetings about how things just don't work
>without it or something like it so I don't see how to avoid it.
>
>
>scenario. Here, the cloning will not solve issue as the answerer
>originated the new offer during SDP_PRANSWER state . Issue 2 is the
>title of this mail thread.
>>
>> Behalf Of Justin Uberti
>> The conclusion on this thread seems to be that the right way to
>address this problem is via PeerConnection cloning, and that no changes
>are needed to the JSEP state machine. I'll work with Richard to figure
>out how we should extend JSEP to support cloning.
>>
>> >>>> even if the called party won't accept the call, but at least from
>a user experience perspective that is still better than lifting the
>hook, and having to wait for whatever-time-it-takes-for-ICE-to-finish
>seconds before one can start to talk.
>> >>>
>> >>> This also has a downside of disclosing user's IP to the caller
>without the user confirmation. For a lot of applications this can be
>serious security flaw.
>> >>
>> >> Because, even if I wait until your phone starts to ring, most
>likely I would still get your IP address without user confirmation
>(speaking in SIP terms, phones normally don't ask for user permission
>before they send 18x with SDP), eventhough you would easier be made
>aware of that it happens.
>> >
>> > Another alternative is of course to do ICE with an SBC, and/or
>> > having an SBC doing ICE on behalf of you :)
>> >
>> >
>> > This is true for SIP but was as far as I know was specifically
>> > designed around in WebRTC. WebRTC signaling server acts as B2BUA so
>any type of ringing notification goes through the web server and does
>not need to carry any type of client address information. The client
>address information is only provided when SDP answer is sent or ICE
>negotiation is started.
>>
>> Assuming you are only going to make user confirmation (read: accept
>calls) in cases where you absolutely sure that the caller isn't someone
>trying to get your IP address.
>>

---

From: Tim Panton <[email-masked]>
Date: Mon, 14 May 2012 10:25:45 +0100

> Support for multiple registered contacts and (parallel) forking are pretty fundamental to SIP.  This has little to do with conferencing.  Some systems (not all) may support only serial forking or m…

I have a feeling that it is mostly useful because SIP is managing both the signalling and the identity. 
In most WebRTC cases, the real identity of the user will be established with a web protocol (perhaps OAuth).
The identity used in any browser based legacy signalling (e.g. SIP) will be a second-order derived or temporary identity.

In that case there is no need for SIP to use the same  temporary identity for both my phones, which (I think)
renders the forking case significantly less useful.

Tim.

---

From: Roman Shpount <[email-masked]>
Date: Mon, 14 May 2012 14:45:25 -0400

On Mon, May 14, 2012 at 5:25 AM, Tim Panton <[email-masked]> wrote:

> > Support for multiple registered contacts and (parallel) forking are
> pretty fundamental to SIP.  This has little to do with conferencing.  Some
> systems (not all) may support only serial forking or may serialize media
> interactions associated with parallel forking, but forking remains widely
> deployed and useful.
>
>
> The identity used in any browser based legacy signalling (e.g. SIP) will
> be a second-order derived or temporary identity.
>
I strongly disagree with this. I can still be calling somebody via, let's
say, their email address and end up talking to two people using two
different devices. I think there is always going to be the situation where
one identity (email address, support link on the web site) will map to
multiple physical end points. It is possible to resolve this identity to
end point mapping using some sort of HTTP signaling before media connection
is setup and then to generate offers for each end point, but this would
make signaling more complex, and would slow down the call connect process
by at least half the round trip time.
_____________
Roman Shpount

---

From: Martin Thomson <[email-masked]>
Date: Mon, 14 May 2012 13:31:26 -0700

On 14 May 2012 11:45, Roman Shpount <[email-masked]> wrote:
> It is possible to resolve this identity to end
> point mapping using some sort of HTTP signaling

I think that this was the point.  Your application has some concept of
identity that maps to one or more registered endpoints.  It can work
out the signaling such that all devices get notified of a session
request without the need for this complex cloning mess.

---

From: Roman Shpount <[email-masked]>
Date: Mon, 14 May 2012 16:40:45 -0400

> request without the need for this complex cloning mess.

I am not sure the cloning is a "complex mess". If ICE is the requirement
(so you know remote IP/port of each party) you can map each remote party to
the source IP/port. Then all you need to do is reference counting for local
sockets/TURN allocations, and you are done implementing it. PRANSWER would
probably require as much coding but would not solve all the problems that
cloning will.
_____________
Roman Shpount

---

From: Randell Jesup <[email-masked]>
Date: Tue, 15 May 2012 00:40:47 -0400

On 5/14/2012 4:31 PM, Martin Thomson wrote:
> On 14 May 2012 11:45, Roman Shpount<[email-masked]>  wrote:

That isn't the point of cloning (getting notified).  It's the server in 
this case that knows about the multiple endpoints.  (For that matter, 
one could be a WebRTC client, and another could be a WebRTC gateway to 
PSTN, and a third could be a network-based answering service where a 
pickup from an actual endpoint would abort the playback/recording - 
another case of a forked ANSWER after a final ANSWER where cloning is 
useful.)

-- 
Randell Jesup
[email-masked]