Subject: [video-codec] WGLC: draft-ietf-netvc-requirements

From: Adam Roach <[email-masked]>
Date: Thu, 4 Aug 2016 16:18:10 -0500

[as chair]

NETVC participants:

We would like to start a two-week working group last call on the codec 
requirements document:

https://tools.ietf.org/html/draft-ietf-netvc-requirements-02

All working group participants are encouraged to carefully read the 
document and comment on the mailing list -- even something simple like 
"I have reviewed the document and believe it is in good shape" is helpful.

I'll also note that the question as to whether we request publication of 
this document as an RFC is still open. As part of this last call, please 
indicate whether you believe such publication would be of value. If we 
do not request publication, we will instead announce a document freeze 
at some point after the end of WGLC (roughly, the amount of time 
typically between end of WGLC and publication request).

Thanks!

/a

---

From: Jonathan Lennox <[email-masked]>
Date: Fri, 5 Aug 2016 16:15:36 +0000

One thing I would like to see in the document (probably in the error resilience section, 3.2.5) is an explicit requirement to support independently-decodable subframes that can be MTU-targeted by an encoder.  This is very important for error resilience for RTP-packetized streams.  (See RFC 2736, to which an informative reference probably wouldn’t be out of order.)

> On Aug 4, 2016, at 5:18 PM, Adam Roach <[email-masked]> wrote:
> 
> [as chair]
> 
> NETVC participants:
> 
> We would like to start a two-week working group last call on the codec requirements document:
> 
> All working group participants are encouraged to carefully read the document and comment on the mailing list -- even something simple like "I have reviewed the document and believe it is in good shape" is helpful.
> 
> I'll also note that the question as to whether we request publication of this document as an RFC is still open. As part of this last call, please indicate whether you believe such publication would be of value. If we do not request publication, we will instead announce a document freeze at some point after the end of WGLC (roughly, the amount of time typically between end of WGLC and publication request).
> 
> Thanks!
> 
> /a
> 
> _______________________________________________
> video-codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/video-codec
>

---

From: Adam Roach <[email-masked]>
Date: Wed, 24 Aug 2016 11:07:37 -0500

[as chair]

As a reminder, the WGLC for this document ends tomorrow. If you have any 
interest in commenting on the requirements before we freeze them, please 
comment now.

Thanks.

/a

On 8/4/16 16:18, Adam Roach wrote:
> I'll also note that the question as to whether we request publication 
> of this document as an RFC is still open. As part of this last call, 
> please indicate whether you believe such publication would be of 
> value. If we do not request publication, we will instead announce a 
> document freeze at some point after the end of WGLC (roughly, the 
> amount of time typically between end of WGLC and publication request).
>
> Thanks!
>
> /a
>

---

From: Andrey Norkin <[email-masked]>
Date: Wed, 24 Aug 2016 15:25:39 -0700

I have several comments, which I discussed with Alexey and some other
participants during the Berlin meeting. The version cited above does not
address them since it was published before our discussion.

Technical comments:

1. Page 12. Section 3.1.2. It may be useful to explicitly mention that
levels definition may change in the future when we know better what
complexity the codec has.
2. Page 13. Section 3.2.1. "Support of arbitrary resolution for such
applications where a picture can have an arbitrary size (e.g., in
screencasting)"  -->  "Support of arbitrary resolution *according to level
constraints* for such applications where a picture can have an arbitrary
size (e.g., in screencasting)"
It is not practical for the codec to support completely arbitrary
resolutions (e.g. a one-macroblock wide and very tall picture does not need
to be supported).
3. Page 15. If we move Section 3.3.3 "Complexity" from optional to basic
requirements, it would likely address Jonathan's comment.
4. Page 15-16, Section 4.1. Compression performance evaluation.  It is
proposed to calculate the BD-rate based on 12 bitrates, where three groups
of four bitrates correspond to the low, medium, and high bitrate ranges. It
would be better to align QP's or quality levels of two codecs rather than
bitrates. The BD-rate savings are calculated on overlap in quality (e.g.
PSNR or MS-SSIM values) between the rate-distortion curves of two codecs.
If we align the bitrates then a big difference in codecs performance (i.e.
a big distance between the RD-curves) would translate to poor overlap in
quality ranges and less reliable BD-rate values. Another point is that the
same bitrate may correspond to a very high quality for an easy sequence and
a poor quality for a difficult sequence. Let's use alignment in QPs/quality
levels instead as the basis for generating the bitstreams.
5. Same section. We should not require a 20-25% improvement on each quality
range and each resolution. We could require instead 20-25% bitrate savings
(average of three quality ranges) on high resolutions and somewhat lower
(e.g. 15%) on lower resolutions. Finally, the requirements for improvements
on each quality range could be half of the requirements for the average of
three ranges. For example, we might not want to discard a proposed codec if
it gets less than 20% BD-rate reduction at a very high quality range, where
the codec is rarely used.
6. Page 16. Section 4.1. "To assess the quality of output (decoded)
sequences, two indexes, PSNR [3] and MS-SSIM [3,11], should be separately
calculated for each color plane." We should not calculate MS-SSIM for
chroma planes since it was developed for luminance signal and will likely
not reflect the visual quality (chroma is usually much smoother than luma).
We can and probably should calculate the chroma PSNR to ensure it does not
get a big drop due to a change in luma/chroma bit allocation. The question
is whether we should use it when evaluating the bitrate savings (in my
opinion, we do not have to but if we do, we should use some smaller weights
for chroma planes, such as 1/8th for each chroma plane and 6/8th for the
luma).

Editorial comments:

1. Page 6. End of Section 2.1 - Fix the bullet sign.
2. Page 13. Section 3.1.4. Any elementary stream -> An elementary stream.
3. Page 14. Section 3.2.2. "Note 1: end-to-end delay should be up to 320 ms
[2] but it's preferable value should be less than 100 ms [7]"  Change
"it's" to "its".

Best regards,
Andrey

On Wed, Aug 24, 2016 at 9:07 AM, Adam Roach <[email-masked]> wrote:

>> document and comment on the mailing list -- even something simple like "I
>> have reviewed the document and believe it is in good shape" is helpful.
>>
>> indicate whether you believe such publication would be of value. If we do
>> not request publication, we will instead announce a document freeze at some
>> point after the end of WGLC (roughly, the amount of time typically between
>> end of WGLC and publication request).
>>
>> Thanks!
>>
>> /a
>>

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Wed, 24 Aug 2016 15:35:34 -0700

> One thing I would like to see in the document (probably in the error resilience section, 3.2.5) is an explicit requirement to support independently-decodable subframes that can be MTU-targeted by a…

In general I support this idea, but I'm a little wary of writing this 
requirement in a way that specifies a solution (independently-decodable 
subframes) instead of the problem it solves. Any ideas for actual text 
we could use?

---

From: Jonathan Lennox <[email-masked]>
Date: Wed, 24 Aug 2016 22:59:32 +0000

> On Aug 24, 2016, at 6:35 PM, Timothy B. Terriberry <[email-masked]> wrote:
> 
> Jonathan Lennox wrote:
> In general I support this idea, but I'm a little wary of writing this requirement in a way that specifies a solution (independently-decodable subframes) instead of the problem it solves. Any ideas for actual text we could use?

I’m not sure how best to phrase it, maybe something like:

The codec should support mechanisms such that individual packets sent in common network protocols can be decoded even when other packets are lost.

But it’s not clear to me that a) it’s clear what this text means if you haven’t read this thread, or b) whether the text seems to imply that you have to be able to decode successfully even if you’re missing a reference frame, which clearly won’t be the case.

Any wordsmithing suggestions?

---

From: Jonathan Lennox <[email-masked]>
Date: Fri, 26 Aug 2016 17:14:16 +0000

> On Aug 24, 2016, at 6:59 PM, Jonathan Lennox <[email-masked]> wrote:
> 
> 

I am informed that the term I’m looking for (meaning when you can’t decode a packet if you’re missing a previous one) is “bitstream dependency”.

So perhaps" “The codec should support mechanisms that allow individual packets to be sent in common network protocols without bitstream dependency between the packets.”  Thoughts?

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Fri, 26 Aug 2016 15:43:01 -0700

> So perhaps" “The codec should support mechanisms that allow individual packets to be sent in common network protocols without bitstream dependency between the packets.”  Thoughts?

That feels like it's moving in the right direction. Perhaps the better 
question is, "What breaks if there is a bitstream dependency?" That 
seems like it could tell you what the actual requirement is.

---

From: Andrey Norkin <[email-masked]>
Date: Fri, 26 Aug 2016 16:32:27 -0700

The codec should support mechanisms that allow individual packets sent in
common network protocols to be independently decodable by the entropy
decoder (including packets related to the same picture/frame).

On Fri, Aug 26, 2016 at 3:43 PM, Timothy B. Terriberry <[email-masked]>
wrote:

> Jonathan Lennox wrote:
>
>> So perhaps" “The codec should support mechanisms that allow individual
>> packets to be sent in common network protocols without bitstream dependency
>> between the packets.”  Thoughts?
>>
>
> question is, "What breaks if there is a bitstream dependency?" That seems
> like it could tell you what the actual requirement is.
>
>

---

From: Alexandros Eleftheriadis <[email-masked]>
Date: Sat, 27 Aug 2016 11:58:54 +0300

Don’t forget the MTU targeting capability. 

What you get is ensuring your ability to decode the media in individual packets when other media packets may be lost, and also minimizing packetization overhead by sending as much media data as possible in a single packet. 

> On Aug 27, 2016, at 1:43 AM, Timothy B. Terriberry <[email-masked]> wrote:
> 
> Jonathan Lennox wrote:
> That feels like it's moving in the right direction. Perhaps the better question is, "What breaks if there is a bitstream dependency?" That seems like it could tell you what the actual requirement is.
>

---

From: Thomas Daede <[email-masked]>
Date: Wed, 31 Aug 2016 11:30:46 -0700

Apparently Andrey's comments got caught in gmail's filters (thanks,
DMARC), so make sure to check the archive if you didn't get them:

https://www.ietf.org/mail-archive/web/video-codec/current/msg00499.html

---

From: "Thomas Davies (thdavies)" <[email-masked]>
Date: Thu, 1 Sep 2016 02:58:40 +0000

I think the underlying requirement is to be able to decode as much as possible of a picture in the event that part of the data for that picture is lost. MTU targetting is one way to do that, but I'd be reluctant to mandate that specific mechanism, as it's a means to that end. 

It might also be technically hard to do for a modern codec, because of interactions of packetisation with tiles for parallelism and large blocks for coding efficiency, for example. The natural granularity of a bitstream with tiles is the tile, and that might be quite coarse for exact MTU targetting, but fine enough for reasonable error concealment. I'd also want to allow the codec to route around any IP issues that could turn up in trying to solve any of these problems.  

My suggestion would be something like: "the codec should support effective mechanisms for allowing decoding and reconstruction of significant parts of pictures in the event that parts of the picture data are lost in transmission". 

best regards

Thomas

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Wed, 31 Aug 2016 20:02:21 -0700

> My suggestion would be something like: "the codec should support effective mechanisms for allowing decoding and reconstruction of significant parts of pictures in the event that parts of the pictur…

Yes, that's much more like what a requirement should sound like to me.