Subject: Re: [video-codec] Comments on draft-maxwell-videocodec-requirements-00

From: "Timothy B. Terriberry" <[email-masked]>
Date: Tue, 04 Dec 2012 15:28:54 -0800

John Koleszar wrote:
> can pipeline the encode with the camera. Even if you only get a whole
> frame at the encoder, it could be valuable to be able to ship the data
> as soon as you can fill an RTP packet, rather than waiting until the
> whole frame is encoded. There are also pipelining considerations where

Also a format capable of subframe latency encoding generally has good 
cache coherency. I'm not sure how much software engineering effort I 
would want to expend into proving this out, though, as long as we ensure 
the format doesn't compromise this unnecessarily.

>> Section 4, fifth paragraph, bullet 2:
>>
>> This is great for storage but, I'm not convinced this is helpful for
>> networks. For network operations, lower average bitrates with higher
>> unpredictability may be worse than higher average bitrates with greater
>> predictability.
>
> This is more of an issue for VOD applications rather than
> teleconferencing. The rate variance you can accept is governed by how
> much you can prebuffer. Taking advantage of this where possible can
> get you substantial quality gains vs constant bitrate, for the same
> average bitrate.

An important point about quality: people will often judge quality of a 
segment by that of the _worst_ frame. So, while I understand that 
keeping a consistent flow bandwidth helps when competing in a link 
bottleneck somewhere, the value of the extra bits (the ones used to 
boost the quality of the other frames above that of the worst one) is 
actually fairly low, in terms of its effect on perception.

This is something I hope rmcat will explore in more detail, though I 
recognize it's hard to evaluate. One idea that occurs to me is actually 
using VBR video and padding it with FEC data to get something closer to 
CBR. That might actually be a better use of the bits.

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Tue, 4 Dec 2012 23:34:06 +0000

-----Original Message-----
From: "Timothy B. Terriberry" <[email-masked]>
Date: Tuesday, December 4, 2012 6:28 PM
To: "[email-masked]" <[email-masked]>
Subject: Re: [video-codec] Comments
on	draft-maxwell-videocodec-requirements-00

>CBR. That might actually be a better use of the bits.

I doubt it. If a frame (or slide) has less bits due to VBR encoding, that
means it is easier to encode it, which implies it is less important than
the one that has more bits. FEC'ing these frames will not make the stream
CBR.

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Tue, 04 Dec 2012 15:40:46 -0800

Kevin Gross wrote:
> There are 8 applications listed in section 3. VOD is the only one that
> clearly benefits here. How much emphasis do we want to put on that
> application?

I've seen estimates that at peak hours this constitutes 50% of the 
Internet bandwidth usage of the United States [1]. That might be worth 
emphasizing.

> Why? The motivation for bandwidth efficiency is never convincingly
> substantiated in the draft. I'm aware that bandwidth efficiency is a
> very respected metric among codec developers. I'm questioning how
> important it actually is for applications and users given that available
> network bandwidth is increasing much faster than required media
> bandwidth and the efficiency increases we expect to be able
> to achieve with a new codec are comparatively modest.

The users seem to think it is pretty darn important. See, for example, 
Chris DiBona's claims that switching to Theora would break the internet 
[2] (rebutted [3], for the record, but the quality bar has moved beyond 
where it was in 2009, and will have moved farther before we're done).

I'll cite our experience with Opus as another example: it always had 
excellent latency and scalability, but people did not start to get 
really excited about it until the quality started to exceed that of the 
best high-latency codecs (this is not quite true: I gave a presentation 
on CELT, one of the predecessors of Opus, at LCA 2009, and after 
explaining that bitrate efficiency was explicitly a non-goal, had a 
number of excited people come up to me and ask, "How long until this 
compresses better than Vorbis?"). If you want people to use your codec, 
this matters.

We've tried twice now to get people to use codecs that were 
substantially less complex than their competitors but which did not 
resoundingly beat them in bitrate efficiency (to put it generously). It 
hasn't worked.

I'll also point out that HEVC's target was 50% less bitrate than H.264 
High Profile at > HD resolutions. I don't have numbers on how close they 
got off-hand, but I think most users would agree that that is not a 
modest number. If we don't at least match this performance, I'm not 
going to say we've failed, but we should _certainly_ be doing better 
than the RF options we already have.

[1] http://www.sandvine.com/news/pr_detail.asp?ID=312
[2] 
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-June/020380.html
[3] http://people.xiph.org/~greg/video/ytcompare/comparison.html

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Tue, 04 Dec 2012 15:42:02 -0800

> the one that has more bits. FEC'ing these frames will not make the stream

Well, the idea would be to put the redundant information from the larger 
(more important) frames in space you didn't use in the smaller frames, 
to protect against lost packets in those frames.

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Tue, 4 Dec 2012 23:46:14 +0000

-----Original Message-----
From: "Timothy B. Terriberry" <[email-masked]>
Date: Tuesday, December 4, 2012 6:42 PM
To: "[email-masked]" <[email-masked]>
Subject: Re: [video-codec] Comments

>to protect against lost packets in those frames.

That is possible but that will introduce potentially long delays (which we
don't want in video-codec).

-acbegen

>_______________________________________________
>video-codec mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/video-codec

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Tue, 04 Dec 2012 15:51:08 -0800

> don't want in video-codec).

I wasn't thinking of going beyond the most recent frame. That 
restriction would limit your ability to make things truly CBR (or 
conversely to keep truly consistent quality), but I think it would be a 
step in the right direction. It might work very well with a 
hierarchical-P or P/B reference frame structure.

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Wed, 05 Dec 2012 01:01:43 +0100

> don't want in video-codec).
Redundant information is important for congestion control algorithms. 
For example the google draft:

http://tools.ietf.org/html/draft-alvestrand-rtcweb-congestion-03

Expects a packet lost rate >10% before reducing the video rate. 
Providing this redundancy in the codec instead
of using FEC could have some benefits.

Also, I think that for videoconferencing, what is most important is not 
CBR vs VBR but to be able to limit the
maximum size of an encoded frame (i.e. something like constrained VBR).

Best regards
Sergio

---

From: Kevin Gross <[email-masked]>
Date: Tue, 4 Dec 2012 20:50:57 -0700

Good stuff. Thanks for including the DiBona rebuttal - good points made
there, but, like you say, you have significant experience that indicates
the less optimized approach is not successful in the "market". We should
not ignore that.

We do need to decide how important VOD is. I'm not sure that wasn't ironic
understatement in your response. My understanding is that a significant
driver for this codec effort is RTCweb and there's not much VOD there.

Kevin Gross
+1-303-447-0517
Media Network Consultant
AVA Networks - www.AVAnw.com <http://www.avanw.com/>, www.X192.org

On Tue, Dec 4, 2012 at 4:40 PM, Timothy B. Terriberry <[email-masked]>wrote:

> Chris DiBona's claims that switching to Theora would break the internet [2]
> (rebutted [3], for the record, but the quality bar has moved beyond where
> it was in 2009, and will have moved farther before we're done).
>
> excellent latency and scalability, but people did not start to get really
> excited about it until the quality started to exceed that of the best
> high-latency codecs (this is not quite true: I gave a presentation on CELT,
> one of the predecessors of Opus, at LCA 2009, and after explaining that
> bitrate efficiency was explicitly a non-goal, had a number of excited
> people come up to me and ask, "How long until this compresses better than
> Vorbis?"). If you want people to use your codec, this matters.
>
> We've tried twice now to get people to use codecs that were substantially
> less complex than their competitors but which did not resoundingly beat
> them in bitrate efficiency (to put it generously). It hasn't worked.
>
> got off-hand, but I think most users would agree that that is not a modest
> number. If we don't at least match this performance, I'm not going to say
> we've failed, but we should _certainly_ be doing better than the RF options
> we already have.
>
> [1] http://www.sandvine.com/news/**pr_detail.asp?ID=312<http://www.sandvine.com/news/pr_detail.asp?ID=312>
> [2] http://lists.whatwg.org/htdig.**cgi/whatwg-whatwg.org/2009-**
> June/020380.html<http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-June/020380.html>
> [3] http://people.xiph.org/~greg/**video/ytcompare/comparison.**html<http://people.xiph.org/~greg/video/ytcompare/comparison.html>
>
> ______________________________**_________________
> video-codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/video-codec<https://www.ietf.org/mailman/listinfo/video-codec>
>

---

From: Basil Mohamed Gohar <[email-masked]>
Date: Wed, 05 Dec 2012 07:15:02 -0500

I think the value of including the use-case of VOD in conjunction with 
RTCWeb is that, in most cases, the application which will be playing VOD 
will be the same one used for VOD (e.g., the web browser).  Targeting 
both will reach a wider audience, and there are cases in the life cycle 
of RTCWeb content were it will begin as an RTCWeb session but then 
end-up as VOD.  An education might do a live session for students using 
RTCWeb for video conferencing to a wide audio, and then archive that 
same video to serve as VOD later.  Many "live" examples can fit this 
method of usage as well.  Being able to use the exact same bits, and not 
need to reencode, would add a lot of value to spec and open-up a lot 
more opportunities.

I think this doesn't subtract at all from the "significant driver" 
aspect, but to ignore it is to cut out an enormous segment of the 
potential usage and benefit of having a RF, standardized video codec.

On 12/04/2012 10:50 PM, Kevin Gross wrote:
> Good stuff. Thanks for including the DiBona rebuttal - good points 
> made there, but, like you say, you have significant experience that 
> indicates the less optimized approach is not successful in the 
> "market". We should not ignore that.
>
> We do need to decide how important VOD is. I'm not sure that wasn't 
> ironic understatement in your response. My understanding is that a 
> significant driver for this codec effort is RTCweb and there's not 
> much VOD there.
>
> Kevin Gross
> +1-303-447-0517
> Media Network Consultant
> <http://www.X192.org>
> On Tue, Dec 4, 2012 at 4:40 PM, Timothy B. Terriberry 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Kevin Gross wrote:
>
>     Internet bandwidth usage of the United States [1]. That might be
>     worth emphasizing.
>
>
>     The users seem to think it is pretty darn important. See, for
>     example, Chris DiBona's claims that switching to Theora would
>     break the internet [2] (rebutted [3], for the record, but the
>     quality bar has moved beyond where it was in 2009, and will have
>     moved farther before we're done).
>
>     I'll cite our experience with Opus as another example: it always
>     had excellent latency and scalability, but people did not start to
>     get really excited about it until the quality started to exceed
>     that of the best high-latency codecs (this is not quite true: I
>     gave a presentation on CELT, one of the predecessors of Opus, at
>     LCA 2009, and after explaining that bitrate efficiency was
>     explicitly a non-goal, had a number of excited people come up to
>     me and ask, "How long until this compresses better than Vorbis?").
>     If you want people to use your codec, this matters.
>
>     substantially less complex than their competitors but which did
>     not resoundingly beat them in bitrate efficiency (to put it
>     generously). It hasn't worked.
>
>     I'll also point out that HEVC's target was 50% less bitrate than
>     H.264 High Profile at > HD resolutions. I don't have numbers on
>     how close they got off-hand, but I think most users would agree
>     that that is not a modest number. If we don't at least match this
>     performance, I'm not going to say we've failed, but we should
>     _certainly_ be doing better than the RF options we already have.
>
>     <http://people.xiph.org/%7Egreg/video/ytcompare/comparison.html>
>
>     [email-masked] <mailto:[email-masked]>

-- 
Libre Video
http://librevideo.org

---

From: Ralph Giles <[email-masked]>
Date: Wed, 05 Dec 2012 13:25:43 -0800

On 12-12-04 7:50 PM, Kevin Gross wrote:

> significant driver for this codec effort is RTCweb and there's not much
> VOD there.

Personally I'm very interested in the VOD case. RTCweb's latency
requirements tend to be a tighter constraint, but I hope this group will
take advantages to improve performance in applications where latency
doesn't matter as well.

I do suspect that the desire to treat 'streaming' and 'downloading' as
distinct cases will lead to a lot of VOD services offering video over
RTCweb.

 -r