Subject: [rtcweb] JS friendly codec compromise?

From: Zach Lym <[email-masked]>
Date: Tue, 1 Apr 2014 15:13:47 -0700

After reviewing the discussion on the codec mandate (particularly h.261) I
was reminded of ORBX.js.

While I have my reservations regarding ORBX, the approach (a JS friendly
codec) would be more backwards compatible than h.261 and give better
performance.  If a "submarine" patent surfaces the codec can be altered and
clients can be "updated" on the next page refresh.

I should stress the "JS friendly" aspect of the codec, "VP8 like H.264 does
not parallelize as well via WebGL as ORBX does, so a "JS shim" is not going
to compete."  - Brendan Eich

Thank you,
-Zach Lym
P.S. For those who are not familiar with ORBX, it is a codec which is
decoded in JS and transmitted over websockets.  Their timeline (
http://render.otoy.com/newsblog/?p=317) mentions a FOSS encoder sometime
this summer, funding for a pure WebGL/CL encoder, and fee-free AMI's for
video transcoding.

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 02 Apr 2014 00:25:05 +0200

>clients can be "updated" on the next page refresh.

Which aspect of "performance" do you mean here, and could you share some
references to support the claim? References to a legal analysis would be
helpful aswell.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Zach Lym <[email-masked]>
Date: Tue, 01 Apr 2014 16:16:31 -0700

> references to support the claim? References to a legal analysis would be

ORBX claims performance on par with H.264.  We all know that's probably
marketing hype, but it's hard to imagine not being able to outperform
H.261.  Even if the working group chose to hack together a custom codec
which is worse than H.261, an upgradeable client means we can update the
spec as performance improvements are made by the browser vendors.

I know of no formal legal analysis of ORBX, but being able to update the
decoder means that any submarine patent would have to knock out every
known video encoding technique.  Software patents are generally simple
to work around, codecs may be more so but Google's due-diligence team
thought that VP8 managed to pull it off.  They paid shut-up money to
make the FUD go away but if you are really that concerned about
potential patents we could partner with Daala.  Mozilla was smart with
Opus and patented their "new methods" so they could force
cross-licensing deals with the trolls.  They are doing the same with
Daala, so the FUD around a submarine threat  against Daala is no better
or worse than with H.264.  I don't know how well Daala's encoding could
be adapted to WebCL, but we could include some patented methods of the
codec.

Come to think of it, the Daala group is looking to recreate the
development process they had for Opus.  Skype and Mumble, both VOIP
clients, were able to push updates to client software silently,
providing the Opus team with a live test-bed which they used to
benchmark their various tweaks.  They are also actively seeking "killer
features" for the codec.  H.26X designed to compress movies, not
low-motion talking heads and screencasts:
https://www.youtube.com/watch?v=u3Yc5Hn9bhQ

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 02 Apr 2014 01:40:10 +0200

>spec as performance improvements are made by the browser vendors.

I meant performance characteristics like, say, battery life per second
of decoded video at H.261 resolutions and GPRS bitrates, to throw in a
couple of metrics. Or something like "ORBX needs more CPU but produces
smaller files at the same quality level". Maybe ORBX is cheap to decode
but expensive to encode, while with H.261 both is cheap? Comparisons
along those lines supported by references would be useful.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Zach Lym <[email-masked]>
Date: Tue, 01 Apr 2014 18:39:58 -0700

I cannot find the exact analysis you appear to be looking for.  You can 
boot up some AMI instances and check for yourself, if you like.  
Brendan Eich apparently saw a demo of ORBX streaming to an iPhone 4s in 
Safari.

There is some discussion of performance on this thread:
https://brendaneich.com/2013/12/orbx-js-and-related-news/#div-comment-17627

And there is a whitepaper detailing ORBX2 here (although they have 
already moved on to ORBX3):
http://aws.otoy.com/docs/ORBX2_Whitepaper.pdf

However, the larger point Brendan Eich makes, and what I am trying to 
convey here, is that eschewing a fixed bit-stream format in favor of a 
living standard would make such comparisons moot.  Clients could even 
send the JS needed to decode their custom format to each other.  The 
web is based on specifying interfaces, think of this as an interface 
for codecs.

The only reason we are having this argument is because H.264 is etched 
into custom hardware.  The above stats show performance comparable with 
H.264 and they chalk it up to their ability to do everything on the 
GPU/WebGL, which you can't do with H.264 or VP8.  As long as we can 
offload to the GPU efficiently, there is no argument to be had.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 2 Apr 2014 09:24:22 -0700

> web is based on specifying interfaces, think of this as an interface

Unfortunately, a JS-implemented codec does not play well with isolated
sessions, otherwise we can't establish true private calling.  In that
context, we still need something that the browser can control.
Perhaps with the right sandbox it could work, but I think that it's a
little too early to be considering that option.  Soon, perhaps.

---

From: Zach Lym <[email-masked]>
Date: Wed, 2 Apr 2014 09:32:36 -0700

My apologies, I am rather ignorant of the gritty implementation details.

Maybe ORBX.js will be of some use, I guess we will see what the summer
brings.

I'll get back to my corner of expertise,
-Zach
On Wednesday, April 2, 2014, Martin Thomson <[email-masked]>
wrote:

> On 1 April 2014 18:39, Zach Lym <[email-masked] <javascript:;>>
> wrote:

-- 
Thank you,
-Zach Lym