Subject: [rtcweb] Discussion on codec choices from a developer who doesn't come to IETF

From: Dean Willis <[email-masked]>
Date: Fri, 4 May 2012 00:11:40 -0500

I asked a colleague who develops serious enterprise video stuff:

> Here's a question you can perhaps help me with. One of the big fights in the IETF right now is which codec to require as the "must implement" for real-time web communications. The leading candidates are H.264 and VP8, although I've proposed H.263 as a cheaper-to-license starting point.
>  
> How much of an issue has codec licensing been for you, and what would you prefer as the "standard" codec?
>  

And the (somewhat surprising to me) response (which I've expurgated a bit as I'm passing it on without explicit permission)  was:

> I believe the battle is over and H.264 has won. The question is ultimately where has the market gone in terms of playback mechanisms. The answer:
>  
> Flash: H.264 and (maybe)VP8
> Adobe Adaptive HDS: H.264
> QuickTime: H.264
> Apple Adaptive HLS – H.264
> Smooth Streaming: VC-1 and H.264
> DASH – All implementations are H.264
> Android Cell Phones: H.264
> Cell Phone hardware: H.264
> HTML5 on IE: H.264
> HTML5 on Safari: H.264
> HTML5 on Firefox: VP8 but recently announced support for H.264 see http://www.theregister.co.uk/2012/03/19/firefox_sucks_on_h264/
> HTML5 on Chrome: H.264 and VP8
> Video Conferencing World: H.264 supported universally
> Broadcast World – Moving from MPEG2 to H.264
> Every Set Top Box in the world – MPEG2 and H.264
>  
> Once the chip manufacturers started building H.264 into the hardware, it was game over. Even though Google was the big VP8 developer/supporter, it had to support H.264 in Chrome and Android phones since that’s all they had to work with on the phones.
>  
> Pretty obvious pattern.
>  
> Codec licensing is largely an issue of the past for us. We do license the H.264 encoders, but it is a miniscule part of our material cost. The decoder license is on a per-machine basis. Every PC/phone/pad that ships today has been licensed for H.264(and AAC) by the manufacturer of the hardware and/or the OS provider. Of course the main complaint on licensing of H.264 is from the content producers which is where the MPEG-LA members want to make money. I don’t have much comment on that, but in our customer base – enterprise – I have never once heard it raised as an issue during a buying decision.
>  
> Also – if VP8 really got popular it is likely that the MPEG-LA patent holders would come after them. Everything I have read indicates that it infringes lots of patents and is basically the same as H.264 in many ways. That is what happened to VC-1 (Windows Media Video). The patent holders won’t bother unless is appears to be a real competitor.
>  
> The html5 people originally wanted to insist on a royalty free codec and gave up based on the reality was that H.264 is the only essential codec. If IETF insists on VP8 or 363 as a “must implement” they will be ignored. It would be a huge deviation from their general principal of consensus and following the market. I cannot conceive of how it could happen.
>  
> We really like a standard codec since it eliminates one area of confusion. 

This seems like a fairly compelling argument for H.264 as a baseline must-implement in RTCWeb. I'm thinking of changing my position.

--
Dean

---

From: Lorenzo Miniero <[email-masked]>
Date: Fri, 4 May 2012 10:44:46 +0200

Il giorno Fri, 4 May 2012 00:11:40 -0500
Dean Willis <[email-masked]> ha scritto:

> > Here's a question you can perhaps help me with. One of the big
> > fights in the IETF right now is which codec to require as the "must
> > implement" for real-time web communications. The leading candidates
> > are H.264 and VP8, although I've proposed H.263 as a
> > cheaper-to-license starting point. How much of an issue has codec
> > licensing been for you, and what would you prefer as the "standard"
> > codec? 
> 
> 
> And the (somewhat surprising to me) response (which I've expurgated a
> bit as I'm passing it on without explicit permission)  was:
> 
> > I believe the battle is over and H.264 has won. The question is
> > ultimately where has the market gone in terms of playback
> > mechanisms. The answer: Flash: H.264 and (maybe)VP8
> > HTML5 on Firefox: VP8 but recently announced support for H.264 see
> > http://www.theregister.co.uk/2012/03/19/firefox_sucks_on_h264/
> > HTML5 on Chrome: H.264 and VP8 Video Conferencing World: H.264
> > supported universally Broadcast World – Moving from MPEG2 to H.264
> > Once the chip manufacturers started building H.264 into the
> > hardware, it was game over. Even though Google was the big VP8
> > developer/supporter, it had to support H.264 in Chrome and Android
> > phones since that’s all they had to work with on the phones. Pretty
> > obvious pattern. Codec licensing is largely an issue of the past
> > for us. We do license the H.264 encoders, but it is a miniscule
> > part of our material cost. The decoder license is on a per-machine
> > basis. Every PC/phone/pad that ships today has been licensed for
> > H.264(and AAC) by the manufacturer of the hardware and/or the OS
> > provider. Of course the main complaint on licensing of H.264 is
> > from the content producers which is where the MPEG-LA members want
> > to make money. I don’t have much comment on that, but in our
> > customer base – enterprise – I have never once heard it raised as
> > an issue during a buying decision. Also – if VP8 really got popular
> > it is likely that the MPEG-LA patent holders would come after them.
> > Everything I have read indicates that it infringes lots of patents
> > and is basically the same as H.264 in many ways. That is what
> > happened to VC-1 (Windows Media Video). The patent holders won’t
> > bother unless is appears to be a real competitor. The html5 people
> > originally wanted to insist on a royalty free codec and gave up
> > based on the reality was that H.264 is the only essential codec. If
> > IETF insists on VP8 or 363 as a “must implement” they will be
> > ignored. It would be a huge deviation from their general principal
> > of consensus and following the market. I cannot conceive of how it
> > could happen. We really like a standard codec since it eliminates
> > one area of confusion. 
> 
> 
> This seems like a fairly compelling argument for H.264 as a baseline
> must-implement in RTCWeb. I'm thinking of changing my position.
> 
> --
> Dean
> 

Good arguments but this WILL cut out a lot of content producers or
interested folks. As you pointed out, you asked someone "who develops
serious enterprise video stuff", not the student, the geeky guy or the
small startup who wants to try and make some business with a
"funny-hat-chat" or "send-your-friends-a-silly-postcard" application
based on RTCWEB, that is, what is supposed to be the real fuel behind
innovation in RTCWEB in the future. None of them are likely to be
able/willing to afford or be encumbered with the license fees and
limitations such a choice would impose (what is "minuscule" to
enterprises is not that tiny for the poor mob I'm part of).

That said, I currently have no strong opinion towards any particular
choice, but I really am not that comfortable with an H.264 mandatory
codec.

L.

-- 
Lorenzo Miniero, COB

Meetecho s.r.l.
Web Conferencing and Collaboration Tools
http://www.meetecho.com

---

From: Stephan Wenger <[email-masked]>
Date: Fri, 4 May 2012 09:36:20 +0000

If this is about the MPEG-LA license: it is my understanding that content
for conversational purposes is covered by that license without any charge.
 RTCWEB is about conversational.  We can worry about licensing for the
encoder and decoder (which are part of the web browser), but users should
be fine.
Of course it is fairly well known that MPEG-LA is not the only entity
which has the right to use patent claims against an H.264 implementation
or an H.264 bitstream.
Stephan

On 5.4.2012 10:44 , "Lorenzo Miniero" <[email-masked]> wrote:

>> > Apple Adaptive HLS  H.264
>> > DASH  All implementations are H.264
>> > supported universally Broadcast World  Moving from MPEG2 to H.264
>> > Every Set Top Box in the world  MPEG2 and H.264
>> >  
>> > phones since that¹s all they had to work with on the phones. Pretty
>> > to make money. I don¹t have much comment on that, but in our
>> > customer base  enterprise  I have never once heard it raised as
>> > an issue during a buying decision. Also  if VP8 really got popular
>> > happened to VC-1 (Windows Media Video). The patent holders won¹t
>> > IETF insists on VP8 or 363 as a ³must implement² they will be
>_______________________________________________
>rtcweb mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Dean Willis <[email-masked]>
Date: Fri, 4 May 2012 09:10:19 -0500

On May 4, 2012 3:45 AM, "Lorenzo Miniero" <[email-masked]> wrote:
> innovation in RTCWEB in the future.

I am more worried about the guy implementing a WebRTC security camera that
uses an embedded Linux kernel and a software video encoder. Each unit might
"produce" video 24x7. But there is no MPEG-LA licensed browser or OS or
encoder chip to fall back on. The whole product might sell for less than it
might cost him to license the codec.

---

From: Jim Gettys <[email-masked]>
Date: Fri, 04 May 2012 10:15:42 -0400

On 05/04/2012 10:10 AM, Dean Willis wrote:
>
>
> On May 4, 2012 3:45 AM, "Lorenzo Miniero" <[email-masked]
> <mailto:[email-masked]>> wrote:
> >
> I am more worried about the guy implementing a WebRTC security camera
> that uses an embedded Linux kernel and a software video encoder. Each
> unit might "produce" video 24x7. But there is no MPEG-LA licensed
> browser or OS or encoder chip to fall back on. The whole product might
> sell for less than it might cost him to license the codec.
>
+1

The number of embedded devices, smartphones, cameras, etc, are rapidly
outnumbering the number of conventional desktop/laptops, and are FAR
more cost sensitive than those uses, and not necessarily going to have
bought out the patent for conventional video playback.

So I don't find arguments that come from "Enterprise" environments by
themselves at all compelling, but merely one more factoid to join the
other factoids in the discussion.
                    - Jim

---

From: Stephan Wenger <[email-masked]>
Date: Fri, 4 May 2012 19:09:00 +0000

From: Dean Willis <[email-masked]<mailto:[email-masked]>>
Date: Friday, 4 May, 2012 16:10
[…]
[…]

I am more worried about the guy implementing a WebRTC security camera that uses an embedded Linux kernel and a software video encoder. Each unit might "produce" video 24x7. But there is no MPEG-LA licensed browser or OS or encoder chip to fall back on. The whole product might sell for less than it might cost him to license the codec.

Really?  The MPEG-LA license terms for one encoder/decoder: $0 for the first 100,000 units per year.  $0.20 for the next several millions, after that $0.10 until the cap is reached.  And, the MPEG-LA license is "take it or leave it" (at least for small fish like your hypothetical camera guy), so even the lawyer cost for review should be minimal.

No, I continue to believe that the MPEG-LA license is not a hurdle for anyone but those giving stuff away for free, or having a business model that disallows them to pay licensing fees.  It's the patents not covered by that license that you may worry about.

Stephan

---

From: Bernard Aboba <[email-masked]>
Date: Fri, 4 May 2012 15:56:55 -0700

> sell for less than it might cost him to license the codec.

[BA]  We are rapidly moving toward inclusion of hardware video encode/decode by default on battery-powered devices (mobile handsets, tablets, etc.). 

Since software encode/decode is processor (and battery) intensive, the latest generation of webcams have video encode/decode built-in.  For example, see:
http://www.ehomeupgrade.com/2011/06/02/skype-certified-webcams-tote-built-in-720p-h-264-video-encoders/

When handled purely in software, it is quite difficult to enable video sessions of substantial duration without draining the battery.   

While my assumption is that hardware support for encode/decode will become less codec-specific over time,  that is typically not the case today.

---

From: Justin Uberti <[email-masked]>
Date: Tue, 8 May 2012 10:54:27 -0400

This "SW encoding chews battery" claim is often mentioned, but in a video
chat, the power draw from the screen and network are often 10x the CPU
power draw, which is typically < 1W (for ARM). So there is very little
impact on talk time between HW and SW encode.

Mobile devices are often limited in terms of what resolutions they can
encode in software, because they have less overall compute power than PCs.
In my experience, that is the factor to consider, not battery life.

On Fri, May 4, 2012 at 6:56 PM, Bernard Aboba <[email-masked]>wrote:

> [BA]  We are rapidly moving toward inclusion of hardware video
> encode/decode by default on battery-powered devices (mobile handsets,
> tablets, etc.).
>
> Since software encode/decode is processor (and battery) intensive, the
> latest generation of webcams have video encode/decode built-in.  For
> example, see:
>
> When handled purely in software, it is quite difficult to enable video
> sessions of substantial duration without draining the battery.
>
> While my assumption is that hardware support for encode/decode will become
> less codec-specific over time,  that is typically not the case today.
>
>

---

From: "David Benham (dbenham)" <[email-masked]>
Date: Tue, 8 May 2012 09:36:54 -0700

+1

Many companies shipping in high volume pay the annual cap and have a
marginal cost at or below 10c each and the low volumes (<100K) are free
as noted.       That leaves non-profits (or have no workable freemium
model) shipping in the millions to be about the only segment with an
hurdle.

From: Stephan Wenger [mailto:[email-masked]] 
Sent: Friday, May 04, 2012 12:09 PM
To: Dean Willis; Lorenzo Miniero
Cc: [email-masked]
Subject: Re: [rtcweb] Discussion on codec choices from a developer who
doesn't come to IETF

From: Dean Willis <[email-masked]>
Date: Friday, 4 May, 2012 16:10 

[...]

[...]

	I am more worried about the guy implementing a WebRTC security
camera that uses an embedded Linux kernel and a software video encoder.
Each unit might "produce" video 24x7. But there is no MPEG-LA licensed
browser or OS or encoder chip to fall back on. The whole product might
sell for less than it might cost him to license the codec.

Really?  The MPEG-LA license terms for one encoder/decoder: $0 for the
first 100,000 units per year.  $0.20 for the next several millions,
after that $0.10 until the cap is reached.  And, the MPEG-LA license is
"take it or leave it" (at least for small fish like your hypothetical
camera guy), so even the lawyer cost for review should be minimal.

No, I continue to believe that the MPEG-LA license is not a hurdle for
anyone but those giving stuff away for free, or having a business model
that disallows them to pay licensing fees.  It's the patents not covered
by that license that you may worry about.

Stephan

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Tue, 08 May 2012 09:48:30 -0700

Justin Uberti wrote:
> This "SW encoding chews battery" claim is often mentioned, but in a
> video chat, the power draw from the screen and network are often 10x the
> CPU power draw, which is typically < 1W (for ARM). So there is very
> little impact on talk time between HW and SW encode.
>
> encode in software, because they have less overall compute power than
> PCs. In my experience, that is the factor to consider, not battery life.

Even if they could encode higher resolutions, they typically don't have 
the bandwidth to push out HD content at several megabits per second (or 
it would be prohibitively expensive if they did). So I think mobile 
videoconferencing resolutions around 360p are much more realistic, and 
perfectly suitable for software. At this point I'll just refer people to 
the guy who posted here from TI:
http://www.ietf.org/mail-archive/web/rtcweb/current/msg03785.html

---

From: "DRAGE, Keith (Keith)" <[email-masked]>
Date: Tue, 8 May 2012 20:35:52 +0200

I suspect the key question is being missed here to make the arguments look more attractive.

Most mobile devices (at least those on 3GPP standards) trying to do anything other than RTCWEB with their screens and cameras will need to support H.264 anyway.

So the discussion needs to be around what are the pros and cons of supporting both VP8 and H.264 on the same mobile, as opposed to only having to support H.264.

Regards

Keith

---

From: Ted Hardie <[email-masked]>
Date: Tue, 8 May 2012 14:40:45 -0700

> Most mobile devices (at least those on 3GPP standards) trying to do anything other than RTCWEB with their screens and cameras will need to support H.264 anyway.

My faulty memory is that the availability of hardware acceleration for
certain audio codecs used to be restricted and that VoIP clients on
3GPP devices had no access to them.  Is that situation cleared up?
Can any installed application now access hardware accelerated codec
support?

regards,

Ted

---

From: Randell Jesup <[email-masked]>
Date: Wed, 09 May 2012 01:45:02 -0400

> So the discussion needs to be around what are the pros and cons of supporting both VP8 and H.264 on the same mobile, as opposed to only having to support H.264.

This only has any real import if you're talking hardware 
implementations.  Hardware H.264 doesn't preclude software VP8 (or vice 
versa), and the software is available, and a trivial amount of flash 
space.  So I see nothing to discuss around "both VP8 and H.264 vs. just 
H.264 on mobile".

-- 
Randell Jesup
[email-masked]