Subject: Re: [rtcweb] Another consideration about signaling

From: "Kevin P. Fleming" <[email-masked]>
Date: Fri, 07 Oct 2011 14:50:03 -0500

On 09/16/2011 10:42 AM, Iñaki Baz Castillo wrote:
> Hi all,
>
> Let's imagine that rtcweb defines a specific signaling protocol (i.e.
> SIP) so browsers MUST use it natively for signaling the media streams.
> Of course this would require a SIP proxy/server in server side (think
> about NAT) which IMHO seems a showstopper (how to deploy such a SIP
> proxy in shared web hostings? a "mod_sip" for Apache? should I make a
> XMPP<->SIP protocol gateway in order to intercommunicate web-browsers
> with pure XMPP clients?)
>
> But there is another important drawback with this assumption:
>
> A web site could be interested in drawing in the web the status of the
> different audio/video streams between users connected to the web. This
> could mean displaying in the web the active streams (audio/video),
> when a session is on hold, when it's resumed again, when a new stream
> is added to a multimedia session (i.e. offering video within an
> already established audio session).
>
> If the signaling uses a separate channel (i.e. SIP) then there is no
> way for the web server to know what happens during multimedia sessions
> (or it would be really difficult to achieve). So multimedia sessions
> would be completely separated from the web page itself. Is that what
> we want?

(resurrecting this old thread)

Regardless of the signaling protocol in use, it seems unlikely to me 
that a web application developer is going to want to be *involved* in 
the call signaling itself; they are going to pass that responsibility 
over to some existing code/library/server/system (which could mean just 
shuttling SIP messages back and forth between WebSocket connections and 
UDP connections).

If this assumption is true, then the web application developer who wants 
to achieve the result you describe above is going to choose a SIP 
library/server that allows them access to the call states and other 
pieces of information that they need, in preference to those who don't 
make that information available. I can't imagine that a practical 
solution would be to choose a SIP library/server that handles the calls 
in an 'opaque' fashion but then build a web application that is required 
to snoop on all the signaling to be able to monitor activity.

-- 
Kevin P. Fleming
Digium, Inc. | Director of Software Technologies
Jabber: [email-masked] | SIP: [email-masked] | Skype: kpfleming
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at www.digium.com & www.asterisk.org

---

From: José Luis Millán <[email-masked]>
Date: Sat, 8 Oct 2011 11:02:42 +0200

Kevin,

2011/10/7 Kevin P. Fleming <[email-masked]>:
> Regardless of the signaling protocol in use, it seems unlikely to me that a
> web application developer is going to want to be *involved* in the call
> signaling itself; they are going to pass that responsibility over to some
> existing code/library/server/system (which could mean just shuttling SIP
> messages back and forth between WebSocket connections and UDP connections).
>
> If this assumption is true, then the web application developer who wants to
> achieve the result you describe above is going to choose a SIP
> library/server that allows them access to the call states and other pieces
> of information that they need, in preference to those who don't make that
> information available. I can't imagine that a practical solution would be to
> choose a SIP library/server that handles the calls in an 'opaque' fashion
> but then build a web application that is required to snoop on all the
> signaling to be able to monitor activity.
>

Web developers have to use well defined APIs so they don't need to
know how the underlaying protocol works. Instead, they rely on the API
methods and information to achieve what Iñaki is stating.

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: "Kevin P. Fleming" <[email-masked]>
Date: Sat, 08 Oct 2011 09:17:35 -0500

On 10/08/2011 04:02 AM, José Luis Millán wrote:
> Kevin,
>
> 2011/10/7 Kevin P. Fleming<[email-masked]>:

I don't disagree with your statements here, but I'm not sure that those 
statements change anything :-)

This group is not defining APIs to be used on the *server* side, only on 
the browser side. On the server side, if a web application wants to be 
able to keep track of multimedia sessions and use that information for 
various purposes, then yes... it will require APIs to do so. Those APIs 
will be into whichever signaling library/server/component the web 
application developer has chosen to handle the signaling of those 
multimedia sessions. This was exactly my point, that the web application 
would not be actively monitoring the signaling *itself* in order to keep 
track of sessions... so if the signaling is just being routed by the web 
application over to another component/server, that's perfectly 
acceptable and doesn't interfere with the application's ability to keep 
track of the sessions.

---

From: José Luis Millán <[email-masked]>
Date: Sat, 8 Oct 2011 23:51:43 +0200

2011/10/8 Kevin P. Fleming <[email-masked]>:
>>> to
>>> pieces
> This group is not defining APIs to be used on the *server* side, only on the
> browser side.

On the server side, if a web application wants to be able to
> keep track of multimedia sessions and use that information for various
> purposes, then yes... it will require APIs to do so.

Sorry if I'm not underdstanding you well. I understand from the
paragraph above that the client may use these APIs to consult the
server about it's own state (client's state).

The web application does not need to monitor the signalling, but will
be aware of its own state (sessions..) by simply relying on the
signalling API. ie: Client executes the 'call' method of the API 'X',
this method executes the corresponding callbacks according to the
state of the running call: 'trying', 'ringing', 'ok'. This way the
client is aware of its own state.

If this is not what you meant, could you please clarify?

Regards

Those APIs will be into
> whichever signaling library/server/component the web application developer
> has chosen to handle the signaling of those multimedia sessions. This was
> exactly my point, that the web application would not be actively monitoring
> the signaling *itself* in order to keep track of sessions...so if the
> signaling is just being routed by the web application over to another
> component/server, that's perfectly acceptable and doesn't interfere with the
> application's ability to keep track of the sessions.

---

From: "Kevin P. Fleming" <[email-masked]>
Date: Mon, 10 Oct 2011 09:59:17 -0500

On 10/08/2011 04:51 PM, José Luis Millán wrote:
> 2011/10/8 Kevin P. Fleming<[email-masked]>:

> If this is not what you meant, could you please clarify?

I interpreted the paragraph above from Inaki's post very differently, so 
I'll try to describe it more concretely.

Imagine we have users Alice, Bob and Charlie, all of who are using a web 
application called XYZ that utilizes RTCWEB functionality to allow 
communication between its users. The XYZ application has two major 
components: a set of Javascript/HTML that it pushes out to the browsers, 
and a set of code (language/platform is irrelevant) that runs on the 
server that pushed the code out, and communicates with that JavaScript code.

When the XYZ application's Javascript wants to setup sessions to other 
users of the application it uses the local WebRTC APIs to access the 
browser's interfaces for media, and uses *some* sort of signaling 
protocol over HTTP/WebSocket to establish the sessions through the XYZ 
application server. This could be SIP, Jingle, or something else entirely.

In order for XYZ's Javascript running in Alice's browser to display the 
state of *her* sessions with the XYZ application, the Javascript doesn't 
need any help; it already knows what sessions it has established and 
their states. The signaling protocol is irrelevant here. If the XYZ 
application server implements the chosen signaling protocol itself, then 
it *also* knows the states of these sessions, and by extension it knows 
the states of all the other sessions that XYZ application users are 
involved in.

If the XYZ application wants to display in Alice's browser the states of 
other sessions that Alice is not involved in (this is what I believe 
Inaki was referring to above), then it can use this information that it 
has (by virtue of implementing the signaling protocol itself) in order 
to push information out to her browser for it to display.

The situation changes quite a lot if the XYZ application server does 
*not* implement the signaling protocol itself, though. If it just acts 
as a conduit for that signaling to be sent between the browser and a 
'communications server' of some type (I suspect this will be a very 
common deployment model, as web application developers aren't going to 
want to shoehorn an entire communications server into their web 
application containers if they can avoid it), then it doesn't know the 
state of any of these sessions at all; it just knows that it has seen 
session signaling messages going back and forth.

In this situation, if the XYZ application wants to display the states of 
other sessions, it will have to have some API between itself and the 
communications server (the browser is not involved here) in order to 
monitor (and potentially control) those sessions. This allows the web 
application developer to be completely insulated from the signaling 
protocol that is used, and instead just use an API provided by the 
communications server in order to track and control communications 
sessions. Thus, the choice of signaling protocol is irrelevant, as long 
as a hypothetical web application developer can find and use a suitable 
communications server that implements the chosen protocol *and* provides 
adequate APIs for the web application to fulfill its requirements.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 10 Oct 2011 18:41:50 +0200

2011/10/10 Kevin P. Fleming <[email-masked]>:
> communications server (the browser is not involved here) in order to monitor
> (and potentially control) those sessions. This allows the web application
> developer to be completely insulated from the signaling protocol that is
> used, and instead just use an API provided by the communications server in
> order to track and control communications sessions. Thus, the choice of
> signaling protocol is irrelevant, as long as a hypothetical web application
> developer can find and use a suitable communications server that implements
> the chosen protocol *and* provides adequate APIs for the web application to
> fulfill its requirements.

Hi Kevin. Yes. In fact, in case the signaling is carried via
WebSocket, the WebSocket server could be a separate server (not
colocated within the web server) so you get the web server and the
signaling server in different servers (and you would need some kind of
proprietary communication between them in order for the web server to
get the sessions status).

But that's changes nothing in the design on RTCweb IMHO.

Regards.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Kevin P. Fleming" <[email-masked]>
Date: Mon, 10 Oct 2011 11:46:49 -0500

On 10/10/2011 11:41 AM, Iñaki Baz Castillo wrote:
> 2011/10/10 Kevin P. Fleming<[email-masked]>:

That's correct; the only reason I brought it up is that the choice of 
SIP as the RTCWEB signaling protocol, or something else, does not impact 
this at all. Choosing Jingle does not make this easier for the web 
application developer, and choosing SIP does not make it harder... in 
either case, they either *are* the signaling endpoint at the server end, 
or they aren't. Originally you had stated that if SIP was chosen, it 
would make this sort of application harder to build, because the 
application would not be likely to implement a SIP UA itself, and so it 
wouldn't have access to the session state information.

I'm not advocating that SIP be chosen... but for entirely different 
reasons :-)

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 10 Oct 2011 18:57:50 +0200

2011/10/10 Kevin P. Fleming <[email-masked]>:
> That's correct; the only reason I brought it up is that the choice of SIP as
> the RTCWEB signaling protocol, or something else, does not impact this at
> all. Choosing Jingle does not make this easier for the web application
> developer, and choosing SIP does not make it harder... in either case, they
> either *are* the signaling endpoint at the server end, or they aren't.
> Originally you had stated that if SIP was chosen, it would make this sort of
> application harder to build, because the application would not be likely to
> implement a SIP UA itself, and so it wouldn't have access to the session
> state information.

Not exactly, I meant that, in case the browser speaks *native* SIP
with a centralized SIP proxy, then it would be hard to get sessions
information in the web server (as it would require some custom and
complex communication with the SIP proxy). The same if it's a XMPP
server or whatever.

But if the signaling is carried via HTTP or WebSocket, the siteadmin
can choose to pass all the signaling through the web server so it
would be fully aware of the sessions status.

So I meant the server rather than the client. The client, of course,
must know the sessions he is involved in :)

> I'm not advocating that SIP be chosen... but for entirely different reasons

The fact is that nobody advocates for a specific signaling protocol
(well, just a person which insists and insists...). We need no default
signaling protocol at all, but just freedom for each siteadmin to
provide the signaling protocol it desired (of course, it should be
carried via HTTP or WebSocket, as those protocols are the ones that a
webbrowser can speak being controlled via JavaScript).

Regards.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Ted Hardie <[email-masked]>
Date: Mon, 10 Oct 2011 11:51:02 -0700

On Mon, Oct 10, 2011 at 9:57 AM, Iñaki Baz Castillo <[email-masked]> wrote:

> We need no default signaling protocol at all, but just freedom for each
> siteadmin to
Taking off my various hats, I have to say I don't agree.   We're putting
together a system here in which a downloaded application must talk to the
server from which it is downloaded, the local system into which it is
downloaded, and the other endpoints with whom it wants to trade flows.  If
we decline to put any constraint at all on the signaling that goes between
the downdloaded system and the server, we are, in essence, forcing the API
between the downloaded code and the local environment to support any
possible signaling.  Since it is the local environment that will manage
codecs and emit RTP flows, that's a rough ask--especially if it has to
support signaling that might have radically different semantics.

If we agreed on a semantic approach (e.g. solicit/propose or offer/answer),
then the syntactic difference would be, I admit, largely a matter of taste.
I would see no real reason not to have a baseline syntax, but I would
probably agree to it being pseudo code of some sort.  But if we have no
agreed on *semantics* for the signaling, then the API between the downloaded
code and the  local system is unbounded.  That doesn't tend to make for
stable systems or interoperability.

If you must gateway between systems (something I agree is not job one),
unbounded semantics would also imply arbitrary complexity in the gateways.
Again, not good for stability or deployability.

Lastly, there are also some security issues here.  I had a chat with Jon
Peterson last week about why SIP identity went down the road it did
(something that, honestly, had always been a bit puzzling to me).  As he
walked me through it, one of the comments he made struck me as particularly
important:  if you are going to expect that you know who you are talking to
by anything other than the assertion of the intermediary, you must provide
both an assurance of identity and an assurance of the media path
destinations.  I'm not sure we do want this, honestly; the intermediaries in
our system are powerful enough that trying to provide anything better than
SSH-style identify may be foolish.  But if we do, we need a common method of
describing those destinations in order to get a common method of assuring
them.  That's a long, long way down the road to common signaling syntax;
having that without common semantics seems liable to some pretty funky
risks.

I am, personally, far from convinced that having no constraints on the
signaling is worth imposing unbounded complexity on the Javascrip/Browser
API and any gateway systems, as well as circumscribing our security
choices.  It may seem to be a dazzling amount of freedom, but it is only one
part of the system, and it ought to play well with others.

Just my personal opinion,

Ted Hardie

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Mon, 10 Oct 2011 22:34:58 +0200

> then the syntactic difference would be, I admit, largely a matter of taste.

Hi Ted. We should come to the reality:

This is about establishing media sessions, and this is based on SDP,
and SDP works as follows:

- Alice sends, via some signaling protocol, a SDP offer to Bob.
- Bob prompts the human user and replies a SDP anwser (if it accepts the call).
- After that media session(s) starts.
- At some point, Alice or Both could send a new SDP offer to modify
the session(s) (for example, for adding video, putting on hold or
whatever).

And that's all. SIP and XMPP/Jingle do that at the end. Both have
different semantics but similar target (exchange SDP information).

That's the only we need to assume for RTCweb.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Ted Hardie <[email-masked]>
Date: Mon, 10 Oct 2011 14:45:53 -0700

> different semantics but similar target (exchange SDP information).
If you believe that each RTCWeb Javascript/server pair needs to implement an
SDP-based offer/answer protocol, but that it may choose among different
syntaxes for carrying the SDP, then I think you're a lot closer to what I
would call "having standard signaling" than not.   That could be described
in a document in pseudo-code which could be translated to JSON structures or
whatever the current flavor of the month is easily enough.  Because each
offer/answer protocol had to support the same basic things the requirements
on the Javascript/Browser API go down to a common core, which is far easier
to get.

I personally would describe that in some existing offer/answer protocol,
just for clarity, but, as I said, that is largely a matter of taste.

Again, speaking without hats on,

regards,

Ted

---

From: Randell Jesup <[email-masked]>
Date: Mon, 10 Oct 2011 20:21:06 -0400

On 10/10/2011 5:45 PM, Ted Hardie wrote:
> On Mon, Oct 10, 2011 at 1:34 PM, Iñaki Baz Castillo <[email-masked]
> <mailto:[email-masked]>> wrote:
>
> If you believe that each RTCWeb Javascript/server pair needs to
> implement an SDP-based offer/answer protocol, but that it may choose
> among different syntaxes for carrying the SDP, then I think you're a lot
> closer to what I would call "having standard signaling" than not.

Yes - this was not what I had assumed Iñaki was arguing.

I made a similar proposal some time ago (and so did others - Harald, 
etc) for standardizing O-A in some manner, and I suggested providing an 
*optional* simple signalling package that developers could use to 
quickly get up and running.  It would not be a replacement for a 
fully-featured stack, but it might be a small subset of an existing full 
stack.  I also think I proposed a way to handle the O-A portion 
(modification of Harald's semantics) that would help resolve simple 
forking.  I'll dig the reference out, but it also helped solve the 
start-of-call-clipping problem.

-- 
Randell Jesup
[email-masked]

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 11 Oct 2011 09:41:09 +0200

2011/10/10 Ted Hardie <[email-masked]>:
> would call "having standard signaling" than not.

Hi Ted, alice from her browser can only establish a media session with
bob if both visit the same website so they already share the same JS
code dictaminating the signaling protocol to use (which can be SIP, or
XMPP or some custom JSON based protocol).

I just said that, regardless the chosen signaling protocol, such
protocol would require some way to send and receive a SDP (or
something that looks an SDP), so the JS API for dealing with media in
a RTCweb capable browser will deal, at the end, with SDP's. IMHO
that's obvious.

But of course, depending on the chosen protocol, it could be simpler
(don't allow parallel forking, don't allow renegotiation of the media
after a call is established...) or complex (building a full SIP
implementation). That's up to the site admin.

I'n not speaking about a "standard signaling", not at all.

>   That could be described

Sure. Taking a working signaling protocol to build the RTCweb JS API
on top of that is a good idea since we can find the requirements for
such API.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Saúl Ibarra Corretgé <[email-masked]>
Date: Tue, 11 Oct 2011 11:23:08 +0200

> a RTCweb capable browser will deal, at the end, with SDP's. IMHO

Sure, reinventing SDP would not be nice. What would be nice is to have some sort of RTCWeb API to manage something that corresponds to an SDP. Then it would be up to the signaling protocol implementors to convert to and from this entities to their real SDP representation, that is, plaintext for SIP and XML for XMPP, for example.

--
Saúl Ibarra Corretgé
AG Projects

---

From: Neil Stratford <[email-masked]>
Date: Tue, 11 Oct 2011 11:04:51 +0100

> support signaling that might have radically different semantics.

At the core of any RTC endpoint is an API to manage codecs and emit RTP,
they are the fundamental building blocks and in my view should be exposed to
javascript. Attempts to impose the semantics of a given signalling protocol
on the API will cause unnecessary complication and limit the ability to
innovate.

> stable systems or interoperability.

I believe that interoperability should be reduced to the media stream -
there should be no requirement that signalling from two different vendors
would need to interoperate unless they choose to do so.

> Again, not good for stability or deployability.

If vendors wish to interoperate with a common signalling protocol such as
SIP then they will design their protocols with semantics that make that an
easy task. The key difference between rtcweb and traditional RTC is that we
have the flexibility of the javascript execution engine and we can change
the signalling protocol from one page to another, or even one call to the
next. If you want to make a SIP call, then use SIP.js, if you want to call
another browser within the same web application, then use whatever
signalling is most appropriate for the application.

> part of the system, and it ought to play well with others.

I agree that we need to ensure that the API can support the signalling
protocols that we want it to, but I don't think that we should limit the API
in any way so that it can only support those protocols unless there are very
good reasons to do so.

Neil