Subject: [rtcweb] Review of draft-kaufman-rtcweb-traversal

From: Bernard Aboba <[email-masked]>
Date: Sat, 11 Jun 2011 13:43:08 -0700

Review of draft-kaufman-rtcweb-traversal

Overall, a very good document. 

Comments below:

Section 2.2.  Hampers Innovation

   One of the benefits of ICE is that it allows local implementation
   flexibility in the way candidates are gathered, offered and
   prioritized.  However, once ICE is baked into the browser, it is no
   longer possible for that innovation to take place - or at least, it
   leaves the hands of the voice application providers.  To date, there
   has been variability in this aspect of implementation, with different
   providers tuning it to tweak their needs and deployments.

[BA] It might be pointed out this flexibility has already proven useful
to enable support for HTTP failover as well as in implementations of
ICE supporting IPV6.  Given the wide variety of IPv6 deployments in 
progress, consideration experimentation and tweaking has been required 
to handle the wide variety of conditions that can be experienced
(e.g. IPv6 tunnels of various flavors, network devices or services 
with partial or no IPv6 support,  missing IPv6 routes, NAT 64, etc.). 

2.3.  Unneccesary Cost in some Cases

   There is a broad array of use cases for VoIP.  It is used for
   everything from consumer Internet services (like Skype) to small
   business phone systems.  Though clearly global consumer Internet
   services require the kind of traversal technology provided by full
   ICE, it is not needed in other cases.  One such use case is, in fact,
   enterprise telephony, where users make calls within the confines of
   their corporate network, and remote access is supported through VPN.
   Today, VoIP endpoints in these environments do not generally use ICE.

[BA] ICE has in fact proven quite popular in enterprise deployments 
involving remote users (e.g. mobile devices), so I'd reword the
last sentence to say "For example, in an enterprise deployment where
users make calls primarily within a corporate network without deployed 
NATs, VoIP endpoints will not require ICE." 

3.  Proposed Model

   For security purposes, the browser will refuse to send, or accept,
   media to or from a peer to which a STUN transaction has not completed
   successfully.  This ensures that the browser cannot be used as a DoS
   tool to launch a voice hammer attack.

[BA] This condition is necessary but not sufficient.  For example, it
would not make sense for the browser to authorize media to be sent to
an endpoint which completed a STUN transaction without ICE extensions.
If this were allowed, then it would be possible to DoS a STUN server. 

   With this model, there is now a great deal of flexibility in how NAT
   traversal can be done.  Some of the models which can now be supported
   are:

   ICE in Javascript:  A full ICE implementation is possible in
      Javascript itself.  Because the implementation resides in
      Javasript, it is trivially changed at any time.

   Server-Based ICE:  A full ICE implementation can execute in the
      server, using remote-control commands to inform the browser to
      send STUN transactions, and passing the results from the browser
      back to the server.  In essence - MGCP for ICE.

   STUN-Only:  For deployments where the peer is always publically
      reachable from clients - such as enterprises or PSTN termination
      services - the Javascript can do a single STUN transaction to
      create a permission in the browser, and then proceed to send
      media.

   Non-ICE:  Protocols similar to ICE, but not otherwise compliant, can
      also be implemented.  Negotiation of which NAT traversal mechanism
      is needed, is done by the application outside of the browser.

[BA] There are some interoperability implications to these
approaches that need to be explored.  For example, the ICE
Javascript approach should enable interoperability
between different browsers attached to the same service.  However,
it is not clear to me that this will necessarily work for 
inter-service communication (e.g. the clients could have 
incompatible ICE Javascript libraries). 

4.4.  Receipt of Response

   Upon receipt of a valid STUN transaction response from the responding
   client, the initiating client MUST call a callback function that
   delivers the attribute/value pairs received in the response, one of
   which is the reflexive address.  The response MUST be ignored if the
   receivedSocketAddress does not match the socket address to which the
   matching transaction ID was sent, as per ICE 7.1.3.2.  Upon receipt
   of a valid response the client also adds the now-verified address to
   the Transmit Whitelist, a list of socket addresses to which sending
   of media is now permissible.  

[BA] Does it really make sense to add the verified address of a responding
client to the transmit whitelist, even if the nominated flag is not
set in the response?

---

From: Harald Alvestrand <[email-masked]>
Date: Sat, 11 Jun 2011 23:23:04 +0200

Bernard,

at last week's call, the question was raised on whether it's actually 
possible to implement ICE in javascript according to the described model 
- the Javascript event loop has certain problems with precise timing.

The upshoot seemed to be that we'd like to see a demonstration that it's 
possible to create an ICE-interoperable implementation using Javascript 
as described before attempting to go further down this path.

Some other comments below.

> with partial or no IPv6 support,  missing IPv6 routes, NAT 64, etc.).

This is good information, and it seems reasonable that it should lead to 
a rework of ICE based on this experience. Is it written up anywhere you 
can share?

> incompatible ICE Javascript libraries).

If both javascript libraries implement ICE, they should by definition be 
interoperable - I am more worried about the case where one application 
supports ICE and another application supports some ICE-like mechanism.

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 11 Jun 2011 15:09:27 -0700

These kind of "performance" arguments tend to fall victim to improvements in technology, so even if they can be demonstrated to be true at a given point in time, the argument may prove invalid years or even months down the road.   

In this particular case, there have been some major improvements in Javascript performance in recent browser releases, and it is quite possible that those improvements may continue.  

So even if such a demonstration were to "fail" on current releases,  the target is not today's browser (which after all, doesn't support RTCWEB at all), but rather future releases that do (and may also incorporate Javascript performance improvements). 

To my mind, the real question isn't performance, it is interoperability.   The "monolithic" approach to ICE brings with it the potential for interoperability problems between different browsers operating on the same service.  Mathew's approach of Javascript implementation in ICE addresses that issue, but I'm concerned about the implications for cross-service interoperability.  

Date: Sat, 11 Jun 2011 23:23:04 +0200
From: [email-masked]
To: [email-masked]
CC: [email-masked]
Subject: Re: [rtcweb] Review of draft-kaufman-rtcweb-traversal

    

    Bernard,

    at last week's call, the question was raised on whether it's
    actually possible to implement ICE in javascript according to the
    described model - the Javascript event loop has certain problems
    with precise timing.

    The upshoot seemed to be that we'd like to see a demonstration that
    it's possible to create an ICE-interoperable implementation using
    Javascript as described before attempting to go further down this
    path.

      Comments below:

         One of the benefits of ICE is that it allows local
      implementation

         prioritized.  However, once ICE is baked into the browser, it
      is no

         longer possible for that innovation to take place - or at
      least, it

         leaves the hands of the voice application providers.  To date,
      there

         has been variability in this aspect of implementation, with
      different

      [BA] It might be pointed out this flexibility has already proven
      useful

      to enable support for HTTP failover as well as in implementations
      of

      ICE supporting IPV6.  Given the wide variety of IPv6 deployments
      in 

      progress, consideration experimentation and tweaking has been
      required 

      with partial or no IPv6 support,  missing IPv6 routes, NAT 64,
      etc.). 

    This is good information, and it seems reasonable that it should
    lead to a rework of ICE based on this experience. Is it written up
    anywhere you can share?

       ICE in Javascript:  A full ICE implementation is
      possible in

            server, using remote-control commands to inform the browser
      to

            send STUN transactions, and passing the results from the
      browser

            reachable from clients - such as enterprises or PSTN
      termination

            services - the Javascript can do a single STUN transaction
      to

            media.

         Non-ICE:  Protocols similar to ICE, but not otherwise
      compliant, can

            also be implemented.  Negotiation of which NAT traversal
      mechanism

            is needed, is done by the application outside of the
      browser.

    

    If both javascript libraries implement ICE, they should by
    definition be interoperable - I am more worried about the case where
    one application supports ICE and another application supports some
    ICE-like mechanism.

---

From: Dzonatas Sol <[email-masked]>
Date: Sat, 11 Jun 2011 16:12:50 -0700

+1 "a demonstration that it's possible to create an ICE-interoperable 
implementation using Javascript"

Especially if the media is intentionally legal size document (or legal 
size video...).

On 06/11/2011 02:23 PM, Harald Alvestrand wrote:
> possible to implement ICE in javascript according to the described 
> model - the Javascript event loop has certain problems with precise 
> timing.
>
> Javascript as described before attempting to go further down this path.
>
> This is good information, and it seems reasonable that it should lead 
> to a rework of ICE based on this experience. Is it written up anywhere 
> you can share?
>
> If both javascript libraries implement ICE, they should by definition 
> be interoperable - I am more worried about the case where one 
> application supports ICE and another application supports some 
> ICE-like mechanism.
>
>
>
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>    

-- 
--- https://twitter.com/Dzonatas_Sol ---
Web Development, Software Engineering, Virtual Reality, Consultant

---

From: Harald Alvestrand <[email-masked]>
Date: Sun, 12 Jun 2011 07:55:38 +0200

On 06/12/2011 12:09 AM, Bernard Aboba wrote:
> These kind of "performance" arguments tend to fall victim to 
> improvements in technology, so even if they can be demonstrated to be 
> true at a given point in time, the argument may prove invalid years or 
> even months down the road.
>
> In this particular case, there have been some major improvements in 
> Javascript performance in recent browser releases, and it is quite 
> possible that those improvements may continue.
Actually my worry is more the javascript model than absolute 
performance. My tests with current browsers indicate that the timer 
resolutions for timed events are on the order of magnitude of 10-50 
msec, depending on browser, and the single-threaded model of the 
Javascript event queue means that response to timers depends on nothing 
else happening in that Javascript context. While I'm not a Javascript 
expert, I haven't had any reassurance that we can get around this from 
Javascript experts I've asked.
>
> So even if such a demonstration were to "fail" on current releases,  
> the target is not today's browser (which after all, doesn't support 
> RTCWEB at all), but rather future releases that do (and may also 
> incorporate Javascript performance improvements).
We might be taking on a bit much if we, in addition to specifying RTCWEB 
protocols, also take on rewriting the Javascript runtime model.
If all it takes is performance tweaking, I might be more optimistic; a 
demo running ICE at half the speed/timing precision needed would 
probably convince me it could be done.
>
> To my mind, the real question isn't performance, it is 
> interoperability.   The "monolithic" approach to ICE brings with it 
> the potential for interoperability problems between different browsers 
> operating on the same service.  Mathew's approach of Javascript 
> implementation in ICE addresses that issue, but I'm concerned about 
> the implications for cross-service interoperability.
One of my implementor colleagues claims (in another context) that a main 
difference between an embedded-in-browser implementation and a 
Javascript one is that the browser one has less than 10 significant 
implementations that have to interoperate, while the Javascript one will 
have an essentially unbounded number of implementations ("millions of 
web pages"). His conclusion was that for the things we need to get right 
in order to have interoperability (for the case we were discussing, 
which was not ICE), it's better to have them embedded in the browser 
than to depend on Javascript.
>
> ------------------------------------------------------------------------

---

From: Matthew Kaufman <[email-masked]>
Date: Sun, 12 Jun 2011 11:05:29 +0200

On Jun 12, 2011, at 7:55 AM, Harald Alvestrand wrote:

> Actually my worry is more the javascript model than absolute performance. My tests with current browsers indicate that the timer resolutions for timed events are on the order of magnitude of 10-50 msec, depending on browser, and the single-threaded model of the Javascript event queue means that response to timers depends on nothing else happening in that Javascript context. While I'm not a Javascript expert, I haven't had any reassurance that we can get around this from Javascript experts I've asked.

The resolution of waking up for a timer on WinNT is 30 msec and only 15 msec on WinXP and ICE works just fine there.

I am not at all convinced that for our use cases we need anything better than ~100 msec, so your 50 msec is already twice as good.

> We might be taking on a bit much if we, in addition to specifying RTCWEB protocols, also take on rewriting the Javascript runtime model.
> If all it takes is performance tweaking, I might be more optimistic; a demo running ICE at half the speed/timing precision needed would probably convince me it could be done.

See above.

> One of my implementor colleagues claims (in another context) that a main difference between an embedded-in-browser implementation and a Javascript one is that the browser one has less than 10 significant implementations that have to interoperate, while the Javascript one will have an essentially unbounded number of implementations ("millions of web pages"). His conclusion was that for the things we need to get right in order to have interoperability (for the case we were discussing, which was not ICE), it's better to have them embedded in the browser than to depend on Javascript.

This is actually the only good argument I've heard... that if the "right" implementation is baked into browsers then it will "just work". On the other hand, we have numerous examples of things that are baked into browsers that would be much more useful/flexible if they weren't... or at the very least offered both options. (And the future flexibility requirements around IPv6 are certainly a hint of why we might need this, in addition to service providers with non-standard needs.)

What about having a full ICE implementation *and* the ability to run it "manually" under Javascript control?

(The example that keeps coming to mind is the difference between progressive playback of HTTP video (as is in Flash Player and most HTML5 video tag implementation) vs. adaptive HTTP streaming (which, in Flash Player was architected as low-level ActionScript primitives plus reference code, and which has already done several things that weren't originally imagined when the APIs were first added))

Matthew Kaufman

---

From: Tim Panton <[email-masked]>
Date: Sun, 12 Jun 2011 10:36:38 +0100

> This is actually the only good argument I've heard... that if the "right" implementation is baked into browsers then it will "just work". On the other hand, we have numerous examples of things that…

Maybe I've missed something, but to my mind there is no need for the javascript ICE's to interoperate with each other. All the users of XYZ-webconference service will load _exactly_the_same_ javascript - so for them to talk peer-to-peer the XYZ-ICE-lite implementation only has to interoperate with itself.

Are there important use-cases where we expect to have a visitor on XYZ.com web-page calling to ABC.com ? If we do then a baked in ICE 
may be needed.

Tim.

---

From: Harald Alvestrand <[email-masked]>
Date: Sun, 12 Jun 2011 11:57:59 +0200

On 06/12/2011 11:05 AM, Matthew Kaufman wrote:
>>> improvements in technology, so even if they can be demonstrated to 
>>> be true at a given point in time, the argument may prove invalid 
>>> years or even months down the road.
>>>
>> Javascript event queue means that response to timers depends on 
>> nothing else happening in that Javascript context. While I'm not a 
>> Javascript expert, I haven't had any reassurance that we can get 
>> around this from Javascript experts I've asked.
>
> The resolution of waking up for a timer on WinNT is 30 msec and only 
> 15 msec on WinXP and ICE works just fine there.
>
> I am not at all convinced that for our use cases we need anything 
> better than ~100 msec, so your 50 msec is already twice as good.
The shortest timer in RFC 5245 (as far as I can see) is Ta (section 
16.1), which has a lower bound of 20 msec; however, this is a lower 
bound on transmission intervals, so it's possible that an implementation 
that fires only every 100 msec would be perfectly interoperable with an 
application that fires every 20 msec.

Testing will show....

>> We might be taking on a bit much if we, in addition to specifying 
>> RTCWEB protocols, also take on rewriting the Javascript runtime model.
>> If all it takes is performance tweaking, I might be more optimistic; 
>> a demo running ICE at half the speed/timing precision needed would 
>>> the potential for interoperability problems between different 
>>> browsers operating on the same service.  Mathew's approach of 
>>> Javascript implementation in ICE addresses that issue, but I'm 
>>> concerned about the implications for cross-service interoperability.
>> One of my implementor colleagues claims (in another context) that a 
>> main difference between an embedded-in-browser implementation and a 
>> implementations that have to interoperate, while the Javascript one 
>> will have an essentially unbounded number of implementations 
>> ("millions of web pages"). His conclusion was that for the things we 
>> need to get right in order to have interoperability (for the case we 
>> were discussing, which was not ICE), it's better to have them 
>> embedded in the browser than to depend on Javascript.
>
> This is actually the only good argument I've heard... that if the 
> "right" implementation is baked into browsers then it will "just 
> work". On the other hand, we have numerous examples of things that are 
> baked into browsers that would be much more useful/flexible if they 
> weren't... or at the very least offered both options. (And the future 
> flexibility requirements around IPv6 are certainly a hint of why we 
> might need this, in addition to service providers with non-standard 
> needs.)
>
I'd be happy to have a spec that allows us to run an "under the covers" 
fully compatible ICE implementation in the "normal case" (like what's 
currently implemented in Google's libjingle/WebRTC code), and have an 
extension (or another API spec) that allows "manual" driving of the STUN 
machinery, which we can agree on once it's proved viable. I'd be very 
unhappy at this point to have only a spec that required a 
partly-Javascript ICE implementation.

But that's my preferences.
>
> (The example that keeps coming to mind is the difference between 
> progressive playback of HTTP video (as is in Flash Player and most 
> HTML5 video tag implementation) vs. adaptive HTTP streaming (which, in 
> Flash Player was architected as low-level ActionScript primitives plus 
> reference code, and which has already done several things that weren't 
> originally imagined when the APIs were first added))
>
> Matthew Kaufman

---

From: Emil Ivov <[email-masked]>
Date: Sun, 12 Jun 2011 12:02:44 +0200

На 12.06.11 11:36, Tim Panton написа:
> Maybe I've missed something, but to my mind there is no need for the
> javascript ICE's to interoperate with each other. All the users of
> XYZ-webconference service will load _exactly_the_same_ javascript -
> so for them to talk peer-to-peer the XYZ-ICE-lite implementation only
> has to interoperate with itself.
> 
> Are there important use-cases where we expect to have a visitor on
> XYZ.com web-page calling to ABC.com ? If we do then a baked in ICE 
> may be needed.

Unless we are planning on promoting a skype-like model where the entire
world is stuck with the same provider and where PSTN is the only way to
have some sort of interop ... then I'd say that inter-domain calls are
the only use case that we have to consider.

The Gmail web client is a great example for how things could work. A
bunch of clients out there, connected to random XMPP services, can talk
to users connected on Gmail and this is great for both Gmail users and
those who'd prefer to stick with those other clients.

Emil

---

From: Matthew Kaufman <[email-masked]>
Date: Sun, 12 Jun 2011 12:24:58 +0200

> Maybe I've missed something, but to my mind there is no need for the javascript ICE's to interoperate with each other. All the users of XYZ-webconference service will load _exactly_the_same_ javasc…

Definitely true.

> Are there important use-cases where we expect to have a visitor on XYZ.com web-page calling to ABC.com ? If we do then a baked in ICE 

Not the case. One can build a perfectly interoperable ICE using Javascript, or a combination of Javascript calls and server-side ICE logic.

But when XYZ.com is calling itself it might be able to either save some of the steps, or do an even better job of NAT/firewall traversal by using more techniques than were baked in.

Matthew Kaufman

---

From: Emil Ivov <[email-masked]>
Date: Sun, 12 Jun 2011 12:31:02 +0200

> partly-Javascript ICE implementation.

+1

Besides, unless I've missed something, most of the arguments against an
in-browser ICE implementation, like being able to use port-prediction,
IPv6 tweaks, etc. are about the way an application would harvest candidates.

If we agree on that then wouldn't it make sense to make that part
extensible by applications (i.e. javascript), and have the rest handled
by the browser? In other words, the browser could come with a set of
default harvesters (i.e. host, stun, turn) and allow applications to
provide additional ones.

Connectivity checks, which are arguably where 90% of the ICE complexity
is can be handled by the browser which would also worry timing and such.
We've followed a similar model in ice4j.org and it has proven to be
rather practical.

Emil

---

From: Tim Panton <[email-masked]>
Date: Sun, 12 Jun 2011 12:21:50 +0100

> the only use case that we have to consider.
Ok, we have explicit non-requirements that cross web site signalling 
and auth are compatible (We push these issues up to the web service), but we propose 
to mandate that the ICE is identical ? I'm not clear why.

Lets say in my hypothetical case of  an XYZ.com page viewer talking to an ABC.com viewer 
both XYZ and ABC decide to support gtalk. Their web pages will have to include an gtalk 
compatible auth component to get the user's gtalk identity. It will also have to include a gtalk
compatible signalling layer. But we are balking at loading a suitable ICE implementation?

I still don't see a case where the site supplying an ICE implementation is a problem, given that
it already has to supply the signalling and auth code.

Tim.

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 12 Jun 2011 07:44:21 -0700

On Sun, Jun 12, 2011 at 2:05 AM, Matthew Kaufman
<[email-masked]> wrote:
> These kind of "performance" arguments tend to fall victim to improvements in
> technology, so even if they can be demonstrated to be true at a given point
> in time, the argument may prove invalid years or even months down the
> road.
>
> Javascript performance in recent browser releases, and it is quite possible
> that those improvements may continue.
>
> Actually my worry is more the javascript model than absolute performance. My
> tests with current browsers indicate that the timer resolutions for timed
> events are on the order of magnitude of 10-50 msec, depending on browser,
> and the single-threaded model of the Javascript event queue means that
> response to timers depends on nothing else happening in that Javascript
> context. While I'm not a Javascript expert, I haven't had any reassurance
> that we can get around this from Javascript experts I've asked.
>
> I am not at all convinced that for our use cases we need anything better
> than ~100 msec, so your 50 msec is already twice as good.

Remember that JS is single-threaded, so other parts of the app can easily
starve you for much longer than this if they are carelessly written. That's
why I'd like to see a demonstration in a site that is actually doing substantial
other JS action.

-Ekr

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sun, 12 Jun 2011 16:52:26 +0200

>performance. My
>> The resolution of waking up for a timer on WinNT is 30 msec 
>and only 15 msec
>> on WinXP and ICE works just fine there.
And on a device at the lower end of the performance spectrum.
--Stefan

---

From: Harald Alvestrand <[email-masked]>
Date: Sun, 12 Jun 2011 17:45:35 +0200

> Are there important use-cases where we expect to have a visitor on XYZ.com web-page calling to ABC.com ? If we do then a baked in ICE
We've spent a number of messages touting scenarios where one application 
running in one browser would use a webserver to set up a media path to a 
different application running in another browser, possibly via gateways 
in the signalling path.

Basically all but the first scenario from Jonathan Rosenberg's 
presentation in Prague fall into this category (the one I copied my 
illustrations from in the presentation from last Tuesday).

Hm... in draft-holmberg-rtcweb-ucreqs, only the last scenario (telephony 
interoperation) describes this need, and that scenario could also be 
handled with a media transcoder. We may need to add some scenarios to 
that document if we want to make sure we capture the other scenarios in 
the requirements.

                         Harald

---

From: Randell Jesup <[email-masked]>
Date: Sun, 12 Jun 2011 12:31:06 -0400

> And on a device at the lower end of the performance spectrum.

Yes!  Please remember that while desktops have unheard-of performance 
compared to 10 or even 5 years ago, an ever-increasing use-case for 
browsers and applications are smartphones and tablets that have an order 
of magnitude (or more) less performance (even with JITs) compared to 
desktops.  They will get better in terms of performance, but even if 
they get fast (dual/quad core, higher clock rates, etc), that speed 
comes at a price - for mobile devices it's more an issue of energy 
expended once a certain amount of performance has been obtained.  So 
even if the mobile device *can* run a pure-JS implementation fast 
enough, it may expend several times more energy in order to do so.

Now, in some cases that doesn't matter - it's a one-time-ish operation.  
But in others it matters a lot (media), and there's a gray area 
inbetween (moderately-frequent or rare-but-expensive operations).  It's 
something that needs to be considered before a decision is made.  It 
doesn't mean JS is the wrong choice, just that raw performance 
(especially on a desktop) isn't the (only) bar to judge against.

The other recently-mentioned point is very important - lack of threading 
in JS may cause RT or semi-RT operations needed for "call" setup or 
in-"call" things to be seriously delayed.  We should verify this and 
options for dealing with it with some JS experts.  (Brendan, are you 
here? ;-)

(Note: still catching up and haven't read the proposal itself yet; just 
the last few messages discussing it.)

-- 
Randell Jesup
[email-masked]

---

From: Dzonatas Sol <[email-masked]>
Date: Sun, 12 Jun 2011 09:48:52 -0700

On 06/12/2011 02:57 AM, Harald Alvestrand wrote:
> bound on transmission intervals, so it's possible that an 
> implementation that fires only every 100 msec would be perfectly 
> interoperable with an application that fires every 20 msec.
>
> Testing will show....

If there is some simple way to add something like the "Expires:" header 
then this may help settle the case of what is "real" per second across 
the page (legal size or 16:9). I can think of arguments against that and 
possible expansion yet synchronization simplicity and reimplementation 
avoidance is the possibility I had in mind to avoid further cost. 
What-if there are multiple script engines (or multiple browsers) on one 
page, so speed of one single javascript instance alone is not the best 
measurement (even in smaller msec).

One oversimplified example, let's say there are multiple tapes with date 
stamp per frame, and they are all played randomly. In this example, the 
system aligns the frames (per second) despite no guarantee each source 
starts with the same datestamp. Now scale that to colosseum size with 
added info for GPS, focal length, and direction where everybody records 
or recorded some part of any sports event.

Another example is in skeletal animation of multiple dancers. There are 
options that help automate alignment of each playback of dance moves 
among couples or several groups of people across each viewer. The 
significance is is the possible synchronization of less than per "real" 
second. Complexity varies in ability to decentralize from simulcast, yet 
we can assume some kind of tether exists, physical or tangible (in 
traversal). Some may get the gist of this for other use-cases.

--

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 12 Jun 2011 10:30:02 -0700

Emil Ivov said:

[BA] Many of the choices arise in formulating (and responding) to an offer.
>From RFC 5245 Section 4:

   In order to send the initial offer in an offer/answer exchange, an
   agent must (1) gather candidates, (2) prioritize them, (3) eliminate
   redundant candidates, (4) choose default candidates, and then (5)
   formulate and send the SDP offer.

Emil Ivov also said:

[BA] I'd agree that the time-critical portions can be handled by the
browser while still preserving much of the flexibility that Mathew's
draft argues for.   So it isn't just a choice between "all in the browser"
or "all but STUN/TURN in Javascript".

Harald said:

   One of my implementor colleagues claims (in another context) that a main
   difference between an embedded-in-browser implementation and a javascript
   one is that the browser one has less than 10 significant implementations
   that have to interoperate

[BA] Based on the recent SIPIt summaries, it seems that we still have a way to go before demonstrating interop between 10 ICE implementations:

https://www.sipit.net/SIPit23_Summary
https://www.sipit.net/SIPit24_Summary
https://www.sipit.net/SIPit26_Summary
https://www.sipit.net/SIPit27_Summary

To understand the challenges, it might be useful to go over the ICE interop learnings at IETF 81. 

Harald also said:

    Hm... in draft-holmberg-rtcweb-ucreqs, only the last scenario (telephony 
interoperation) 
    describes this need, and that scenario could also be 
handled with a media transcoder. 
    We may need to add some scenarios to 
that document if we want to make sure we capture 
    the other scenarios in 
the requirements.

[BA] That sounds like a good idea.

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 12 Jun 2011 11:55:55 -0700

Emil said: 

[BA] The issue here is not just performance/flexibility, there are architectural implications as well.  If we assume that it necessary to allow variation in signaling, then the signaling and connectivity checking functions of ICE cannot be combined into a single "monolithic" implementation in the browser.   For example,  the actual SDP negotiation may be handled within a Javascript library supporting a given signaling approach (e.g. Strophe for XMPP).   However, to enable the building of an offer (and response), the ICE candidates need to be gathered, prioritized, etc.  This implies the creation of APIs to support this.  After the (implementation dependent) offer/answer is completed, then the connectivity checking portion can be handled natively in the browser, but you have to have some API support to enable getting to that point.

---

From: Emil Ivov <[email-masked]>
Date: Mon, 13 Jun 2011 00:16:11 +0200

На 12.06.11 20:55, Bernard Aboba написа:
> [BA] The issue here is not just performance/flexibility, there are
> architectural implications as well.  If we assume that it necessary to
> allow variation in signaling, then the signaling and connectivity
> checking functions of ICE cannot be combined into a single "monolithic"
> implementation in the browser. 

When you say signalling, do you mean the exchange of candidates between
peers? If so, and provided we are not to pick a signalling protocol
here, then of course, this should most definitely be outside the ICE
impl and left to the javascript, just as it would be the case with a
regular application and an ICE stack.

>  For example,  the actual SDP
> negotiation may be handled within a Javascript library supporting a
> given signaling approach (e.g. Strophe for XMPP).

Right.

>  However, to enable
> the building of an offer (and response), the ICE candidates need to be
> gathered, prioritized, etc.  This implies the creation of APIs to
> support this. 

Right.

> After the (implementation dependent) offer/answer is
> completed, then the connectivity checking portion can be handled
> natively in the browser, but you have to have some API support to enable
> getting to that point.

Yes. Basically the browser would act as the ICE stack and the javascript
as the application.

Emil

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 13 Jun 2011 09:59:57 +0200

On 06/12/11 20:55, Bernard Aboba wrote:
> checking functions of ICE cannot be combined into a single 
> "monolithic" implementation in the browser.   For example,  the actual 
> SDP negotiation may be handled within a Javascript library supporting 
> a given signaling approach (e.g. Strophe for XMPP).
SDP or mechanisms of similar expressive power, yes.
>   However, to enable the building of an offer (and response), the ICE 
> candidates need to be gathered, prioritized, etc.  This implies the 
> creation of APIs to support this.  After the (implementation 
> dependent) offer/answer is completed, then the connectivity checking 
> portion can be handled natively in the browser, but you have to have 
> some API support to enable getting to that point.
I think we agree on that point (that you have to have some API support 
to enable getting to that point). In the WHATWG proposal, that interface 
consists of SDP blobs thrown across the JS/browser interface; in the 
Google RTCWeb implementation, that interface consists of JSON blobs 
thrown in the same fashion. It's possible to write a JS app that just 
passes these blobs to a backend - or one can have a JS app that parses 
the blobs and does more with them before sending them off to wherever 
they need to go - but it's kind of obvious that these blobs need to 
contain the ICE candidates that the other end should learn about.

---

From: Christopher Blizzard <[email-masked]>
Date: Mon, 13 Jun 2011 10:42:37 -0700

On 6/11/2011 10:55 PM, Harald Alvestrand wrote:
> were discussing, which was not ICE), it's better to have them embedded 
> in the browser than to depend on Javascript.

Agreed on this point.  The level for this API is probably pretty high to 
reduce the matrix for testing an interop.

--Chris

---

From: Christopher Blizzard <[email-masked]>
Date: Mon, 13 Jun 2011 10:45:04 -0700

> originally imagined when the APIs were first added))

For this case you should know that we're interested in the web browser 
to do both - have a "just works" mode for people that want adaptive 
bandwidth and a low-level API for people who want to control it 
directly.  But keep in mind that the interop matrix there is small - 
we're still talking about a few products and the site has control of its 
entire stack.  Getting services talking to each other with a lower level 
API is very different from a single site.

--Chris

---

From: Dzonatas Sol <[email-masked]>
Date: Mon, 13 Jun 2011 12:41:25 -0700

On 06/13/2011 10:45 AM, Christopher Blizzard wrote:
>> HTML5 video tag implementation) vs. adaptive HTTP streaming (which, 
>> in Flash Player was architected as low-level ActionScript primitives 
>> plus reference code, and which has already done several things that 
>> weren't originally imagined when the APIs were first added))
>
> we're still talking about a few products and the site has control of 
> its entire stack.  Getting services talking to each other with a lower 
> level API is very different from a single site.
>
>

Yes and people try to build browser in a browser in a browser etc 
turtles on turtles as they sort out the layers. With hypervisors 
available, the concept of service stacks as an entire separate system on 
the same hardware is known, and browsers can use those instead of 
endless built-in extensions (or major re-implementations). I wouldn't be 
surprised to see hypervisors that designate one guest system per port, 
per protocol (API), per URI scheme, and/or per URN.

--

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 17 Jun 2011 12:01:46 -0600

On Jun 12, 2011, at 3:57 AM, Harald Alvestrand wrote:

> The shortest timer in RFC 5245 (as far as I can see) is Ta (section 16.1), which has a lower bound of 20 msec; however, this is a lower bound on transmission intervals, so it's possible that an implementation that fires only every 100 msec would be perfectly interoperable with an application that fires every 20 msec.
> 
> Testing will show....

One complication is that there is ICE in theory and ICE in practice. Some of the times in the theory are bit longer than some implementations use in practice. This theory numbers are larger due to congestion control concerns at IETF. Some of the practice numbers are a bit lower to reduce the latency to have the call set up. 

Testing will tell ... :-)