Subject: [rtcweb] Usecases for innovation.

From: Tim Panton <[email-masked]>
Date: Tue, 1 Nov 2011 08:30:21 +0000

I've repeatedly been asked for use-cases for innovative applications of webRTC
to justify my contention that we should be providing a low-level framework,
not an embedded legacy compatibility application.

It's a 'when did you stop beating your wife?' question, there is no good answer.
By definition we don't know what innovative uses are yet, so we are reduced to
guessing, which sounds unconvincing.

By chance, this weekend I was exposed to 2 innovative uses of real-time-communications
in a browser that _won't_ fit in the current looking-over-its-shoulder scheme.

1) H264 implementation in Javascript http://yfrog.com/nmng0z 
2) Kinect as an input device for a virtual receptionist in a real reception area
	(Voxeo's as it happens).

Neither of these are production ready - or indeed necessarily a good idea,
but the fact that neither (minor) innovation fits at all into our brand new framework
should give us pause for thought. (but given the pell-mell dash to be compatible
with last century's deskphones I don't imagine it will).

Tim.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 1 Nov 2011 15:18:51 +0100

> 1) H264 implementation in Javascript http://yfrog.com/nmng0z

Interesting but, how is that related to WebRTC? IMHO the RTP stack and
media manipulation (including codecs) must be built-in the browser
rather than doing it at JavaScript level.

> 2) Kinect as an input device for a virtual receptionist in a real reception area

This is cool, but IMHO it requires:

1) A system driver for the Kinect device (in the same way a webcam
requires a system driver).

2) A browser specification for handling those kind of devices (indeed
not all the RTC is audio and video).

3) A mechanism for passing in-the-wire the data retrieved from the
device (corporal position and so).

- Bullet 1 is out of the scope of WebRTC, however there are open
source drivers for Kinect in most of the operating systems.

- IMHO bullet 2 should be in the scope of some other (new?) Working
Group, and we need it to be somehow standarized (I can't imagine a
specification for handling *just* Kinect in browsers, those kind of
devices should be "standarized" before carrying them to the browsers).

- Bullet 3 can already be achieved in multiple ways (using HTTP
polling, HTTP Commet, AJAX, WebSocket... and in theory also the WebRTC
Data specification).

> with last century's deskphones I don't imagine it will).

Indeed most of the discussions in this WG are about how to make
***current*** SIP devices to work with a WebRTC scenario. It's a
limited vision of WebRTC in which the main interest is to make profit
of this new spec within telcos business. Bad IMHO. But, where are
those "innovative" and "crazy" Web folks right now? why 95% of people
discussing in this ***open*** WG are telcos? So this is also their
fault, the fault of people not participating here and now.

In the other side, let's take into account that 99% of Web folks know
absolutely nothing about RTC and/or VoIP. They live on top of HTTP and
port 80/443. WebRTC is about realtime communications, we need people
with RTC experience here. But indeed we need also the vision of Web
folks (in contrast to so much legacy telephony vision).

Anyhow I'm not so pesimist. I think that current WebRTC design is
flexible enough to make possible lot of features without being
constrained to follow a telco/telephony model at all.

Regards.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Wolfgang Beck <[email-masked]>
Date: Tue, 1 Nov 2011 17:02:20 +0100

>> not an embedded legacy compatibility application.
[..]
> fault, the fault of people not participating here and now.

Well, I'm working for a telco. Hadriel' employer sells equipment to
telcos and telephony providers.
Still our proposals to avoid standardized signaling and to depart from
classical telco signaling flows were not well
received by the crazy web folks. The world has turned upside down..

Wolfgang Beck

---

From: Igor Faynberg <[email-masked]>
Date: Tue, 01 Nov 2011 12:37:18 -0400

;)

On 11/1/2011 12:02 PM, Wolfgang Beck wrote:
> On Tue, Nov 1, 2011 at 3:18 PM, Iñaki Baz Castillo<[email-masked]>  wrote:
>> 2011/11/1 Tim Panton<[email-masked]>:
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Tue, 1 Nov 2011 18:27:01 +0100

2011/11/1 Wolfgang Beck <[email-masked]>:
> received by the crazy web folks.

Is this really true? If so, they don't know what they want.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Randell Jesup <[email-masked]>
Date: Wed, 02 Nov 2011 13:55:25 -0400

>> fault, the fault of people not participating here and now.

Well, the IETF's AVT group has a strong component of telephony (really 
VoIP/Video) people, so it's not that surprising.  Most of the "web" 
people would be focused on the W3C side of things, and have minimal 
interest in the plumbing. ;-)  Mostly only in how the exposed interfaces 
affect them, but they (generic website/app developers) haven't really 
seen what's proposed yet - and most of them don't even follow the W3C, 
which is more major site vendors and browser people I believe.

I straddle both - long-term core Mozilla contributor, now employee, but 
in the interim spent 7 years working on videophones. But I'm no expert 
on website/javascript-library design.  (Mozilla does have people more 
focused on that involved.)

> received by the crazy web folks. The world has turned upside down..

We (browser vendors) have a strong focus on getting something 
deployable, without unduly limiting potential innovation.  We also (as 
has been mentioned) have concerns over complexity exposed and the 
resultant reliance on (rarely-if-ever-updated) JS frameworks.

Quite honestly, the largest problem we have is NOT signalling; one way 
or another media and data channels can get open and negotiated.  The 
biggest problem and hardest one to "solve" (it's not truly "solvable") 
is security and privacy and the user-interaction aspects of that.  The 
technical issues of signalling, congestion control, and data channels 
are all eminently resolvable (even if there is no universally-agreed 
'best' solution).  Security and privacy are far harder, and the 
solutions less satisfying.

So, if you have brain cycles to born, spend them on security, IMHO.

(Note: personal opinions, not official Mozilla position, though 
"Mozilla" may agree with me)

-- 
Randell Jesup
[email-masked]

---

From: Ravindran Parthasarathi <[email-masked]>
Date: Thu, 3 Nov 2011 01:12:37 +0000

IMO, low level API are nothing but exposing (secured) RTP library API from browser to web developer. Of course, it helps in case the web development focused on special web-application like

1) Proprietary SDP next generation mechanism (if ROAP proposal is accepted as default API mechanism) implementation. Of course, lot of flames on SDP already and alternative proposal for the specific deployment is possible.
2) Signaling mechanism like H.245 (Media description of H.323) which does not direct well fit with SDP
3) RTP specific enhancement services like adding proprietary RTCP extensions.

But I don't whether the above application will be interesting to typical website developer (not telephony or UC provider) and also, Browser vendors spelt out in the last meeting that it is tough to maintain these API in browsers.

Thanks to Randell for sharing his experience and burning issue in WebRTC. I understand that security & privacy should be the main focus of this WG. 

I raised standard signaling issue as it is normal response in any real-time (telephony or unified communication) development that signaling is less important and let us focus on service but after couple of releases with this mentality, signaling manageability go for toss which calls for hack & re-structure in signaling layer. The proprietary signaling helps for time to market but provides the manageability & scalability issues in the long run. Lot of so-called SIP based mechanism works based on unsubscribed NOTIFY & INFO mechanism. I have seen these kind of proprietary implementation wherein the companies struggles to move out of it because of developed service on top of this protocol infrastructure. I'm more skeptical about "no standardizing signaling protocol" argument as the argument mostly comes from VoIP & telephony service providers or OEM.

Thanks
Partha

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On Behalf
>Of Randell Jesup
>Sent: Wednesday, November 02, 2011 11:25 PM
>To: [email-masked]
>Subject: Re: [rtcweb] Usecases for innovation.
>

---

From: Wolfgang Beck <[email-masked]>
Date: Thu, 3 Nov 2011 09:18:41 +0100

> 3) RTP specific enhancement services like adding proprietary RTCP extensions.
The API must hit the right level. If it is too low-level, security
will be hard to achieve. If it is too high-level, it can't be used for
anything new. What worries me about ROAP is this protocol stack:

SDP
ROAP
proprietary JS client protocol
HTTP
...
SDP+ROAP may require a complexity of the JS client protocol that is
not really needed by the actual application.
However, running code wins. Maybe this isn't a problem.

> be the main focus of this WG.
We have less security issues if both parties are using the same
server. With trapezoid style interconnection/federation security
relevant information can get lost. You can get into transit scenarios
where all your signaling is routed through providers that you don't
know. If Facebook or Google became big hubs to interconnect small
RTCWEB providers, how would they use this signaling information?

>
> The proprietary signaling helps for time to market but provides the manageability & scalability issues in the long run.
FYI, here's one of the presentations that motivated RTCWEB in the first place:
'The End Of Application Protocol Standardization (?)'
http://www.ietf.org/proceedings/80/slides/plenaryt-5.pdf

Wolfgang Beck

---

From: Randell Jesup <[email-masked]>
Date: Thu, 03 Nov 2011 08:52:11 -0400

On 11/3/2011 4:18 AM, Wolfgang Beck wrote:
>> Thanks to Randell for sharing his experience and burning issue in WebRTC. I understand that security&  privacy should
>> be the main focus of this WG.

The security and privacy issues I was referring to were mostly regarding 
the user interface and evil JS apps, not trapezoid security "plumbing" 
issues.

-- 
Randell Jesup
[email-masked]

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 8 Nov 2011 15:57:22 -0700

Hmm ... both of these seem to work fine with the direction we are going....both require the W3C getMedia API to allow JS access to the data so it's more of a W3C issue than IETF but my read of the tea leafs is that is the direction W3C is going. At the TPAC meeting Opera bought up good use cases for this and they seemed to get a good reception that people wanted to solve them.  My recollection was there were only one or two people at IETF objecting to JS access to media ( and come to think of they were all fans of the "low level" API :-) For purpose of argument, one could imagine implementing the Kinect stereo correlation in JS and just using any two camera along with auto calibration of geometry. That would be cool....

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 8 Nov 2011 17:05:17 -0800

[BA]  I'm trying to understand the "generic" issues presented by these cases.   For #1, is the generic issue the ability to add support for codecs not supported natively (e.g. ability to add and use a codec added by a plugin, for example)?   For #2, is the generic issue the ability to add support for new types of devices without standardized APIs, or is the issue being able to represent the input/output of a non-standard device in terms of a media stream?     Tim said: " 1) H264 implementation in Javascript http://yfrog.com/nmng0z 
2) Kinect as an input device for a virtual receptionist in a real reception area
 	(Voxeo's as it happens).
 
 Neither of these are production ready - or indeed necessarily a good idea,but the fact that neither (minor) innovation fits at all into our brand new framework
should give us pause for thought. (but given the pell-mell dash to be compatiblewith last century's deskphones I don't imagine it will)."

---

From: Tim Panton <[email-masked]>
Date: Wed, 9 Nov 2011 13:24:08 +0000

On 9 Nov 2011, at 01:05, Bernard Aboba wrote:

> [BA]  I'm trying to understand the "generic" issues presented by these cases.  
>  
> For #1, is the generic issue the ability to add support for codecs not supported natively (e.g. ability to add and use a codec added by a plugin, for example)?  
>  
> For #2, is the generic issue the ability to add support for new types of devices without standardized APIs, or is the issue being able to represent the input/output of a non-standard device in terms of a media stream? 

Well, in some sense kinect is just a lossy video encoder. So you could argue that they are the same problem.
Both cases point out the fact that tying codecs/networking/negotiation into one object and
only presenting a (somewhat opaque) blob of SDP to represent them limits what we can do 
at the javascript end. 
That may be a limitation we can live with, but at least we need to acknowledge that it is a limitation.

>  
>  
>  
>  
> Tim said:
>  
> " 1) H264 implementation in Javascript http://yfrog.com/nmng0z 
> with last century's deskphones I don't imagine it will)."
>

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 9 Nov 2011 15:11:23 -0800

Tim said:

"Both cases point out the fact that tying codecs/networking/negotiation into
one object and only presenting a (somewhat opaque) blob of SDP to represent
them limits what we can do at the javascript end. That may be a limitation
we can live with, but at least we need to acknowledge that it is a
limitation."

[BA] For webcams with hardware encode/decode, this is an issue.  Given the
performance/power issues, it is important to enable the device
capabilities/codec support to be integrated.  It's one thing to talk about
what codecs a browser needs to support, but it should be recognized that
hardware could supplement those capabilities, and the browser shouldn't be a
bottleneck preventing the hardware from being utilized.