Subject: [rtcweb] resolutions in draft-cbran-rtcweb-codec-01

From: "Roni Even" <[email-masked]>
Date: Thu, 17 Nov 2011 05:19:13 +0200

Hi,

The text in the draft on video codec is

o  MUST support a minimum resolution of 320X240

o  SHOULD support resolutions of 1280x720, 720x480, 1024x768,

      800x600, 640x480, 640 x 360 , 320x240

I propose adding 1920 x 1080 to the SHOULD.  This resolution are used by
video conferencing systems

I also assume that there will be a way to negotiate the video parameters (no
issue if we use offer/ answer RFC 3264)

Roni Even

---

From: "Cary Bran (Standards Mailer)" <[email-masked]>
Date: Thu, 17 Nov 2011 13:53:06 +0800

Hi Roni,

Sent from my iPad

On Nov 17, 2011, at 11:19 AM, "Roni Even" <[email-masked]> wrote:

> I propose adding 1920 x 1080 to the SHOULD.  This resolution are used by video conferencing systems
>  
I am ok with doing this and will add it to the 02 version of the document.

>  
> I also assume that there will be a way to negotiate the video parameters (no issue if we use offer/ answer RFC 3264)
>  
Agreed.

> Roni Even
>  
>  
>  
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 17 Nov 2011 16:50:54 +0800

> by video conferencing systems

I think I want to object to making this a SHOULD, but it's possible we
just don't understand the requirement the same way....

The RFC 2119 SHOULD definition says:

3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.

When designing an RTCWEB implementation for my mobile phone, with its
(comparatively) microscopic screen and anemic processor and battery, it
takes approximately zero seconds to decide that I won't waste energy
(literally) on decoding HD; I'll make sure I negotiate down to something
more sane for me if someone's unkind enough to suggest it to me.

I would be happy to write the requirement as

SHOULD support resolutions of 1920x1080, 1280x720, 720x480, 1024x768,

800x600, 640x480, 640 x 360 , 320x240 if these resolutions can be
displayed on the target device

but I would not be happy to just extend the requirement as it is
proposed today.

(Yes, I know, this means that I've turned down-negotiation of video
resolution into a MUST. I think it already is, but I think we haven't
talked explicitly about it.)

> I also assume that there will be a way to negotiate the video
> parameters (no issue if we use offer/ answer RFC 3264)
>
> Roni Even
>
>

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 17 Nov 2011 17:54:25 +0800

The highlighted part of Harald's suggestion below

>       800x600, 640x480, 640 x 360 , 320x240 *if these resolutions can be
> displayed on the target device*
> 
seems sensible.  However, I have another issue.  I don't believe that naming
specific resolutions is advisable.  In a system where the rendering side has
almost undefined picture properties because they are typically rendered in
"windows", it IMO does not make much sense to limit yourself to certain
aspect ratios, picture sizes etc., even if those correspond to today's
preferred sensor resolutions (or integer fractions thereof).  Resampling an
input signal is cheap nowadays (at least compared to the encoding itself),
and cropping is even cheaper.
I would instead recommend naming one single MUST resolution (as the draft
does) to ensure minimum interoperability, and otherwise negotiate a
processing rate in macro blocks per second (or equivalent), with fixed
constraints around the aspect ratio.  Just as what H.264 does in its level
specification.  In fact, if H>264 turns out to be our mandatory codec, then
I would adopt the level spec wholesale.
As for said mandatory resolution, I believe that 320x240 is sooo yesterday.
Even on something like an iPhone, you can do better today.  Who uses 4:3
ratio nowadays?  My vote would go towards something like WQVGA or even WVGA.
Stephan
From:  Harald Alvestrand <[email-masked]>
Date:  Thu, 17 Nov 2011 16:50:54 +0800
To:  <[email-masked]>
Subject:  [rtcweb] Video resolution SHOULDs (Re: resolutions in
draft-cbran-rtcweb-codec-01)

> I propose adding 1920 x 1080 to the SHOULD.  This resolution are used by video
 
 I think I want to object to making this a SHOULD, but it's possible we just
don't understand the requirement the same way....

 

 but I would not be happy to just extend the requirement as it is proposed
today.

(Yes, I know, this means that I've turned down-negotiation of video
resolution into a MUST. I think it already is, but I think we haven't talked
explicitly about it.)

> [email-masked]://www.ietf.org/mailman/listinfo/rtcweb
>  

_______________________________________________ rtcweb mailing list
[email-masked] https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 17 Nov 2011 18:10:12 +0800

On 11/17/2011 05:54 PM, Stephan Wenger wrote:
>           800x600, 640x480, 640 x 360 , 320x240* **if these
>     resolutions can be displayed on the target device*
>
>
> seems sensible.  However, I have another issue.  I don't believe that 
> naming specific resolutions is advisable.  In a system where the 
> rendering side has almost undefined picture properties because they 
> are typically rendered in "windows", it IMO does not make much sense 
> to limit yourself to certain aspect ratios, picture sizes etc., even 
> if those correspond to today's preferred sensor resolutions (or 
> integer fractions thereof).  Resampling an input signal is cheap 
> nowadays (at least compared to the encoding itself), and cropping is 
> even cheaper.
> I would instead recommend naming one single MUST resolution (as the 
> draft does) to ensure minimum interoperability, and otherwise 
> negotiate a processing rate in macro blocks per second (or 
> equivalent), with fixed constraints around the aspect ratio.
For requirements, I prefer to list specifics, because they're testable.

One of my colleagues had a failing test the other day because he'd 
specified a picture height of 94 pixels - he did not know that his 
encoders only supported pixel counts that were multiples of 8.

Of course a device will not fail to interoperate just because it 
supports decoding more resolutions than those that are required.

BTW, I believe 640x360 is a 16:9 aspect ratio, so handling both 16:9 and 
4:3 seems to be a SHOULD based on this resolution list. I assume that's 
intentional.

---

From: "Roni Even" <[email-masked]>
Date: Thu, 17 Nov 2011 13:03:41 +0200

HI,

I am not familiar with VP8 but for H.264 the specification says that a
complaint decoder must receive any stream from the encoder and crop/scale
for rendering.

In order to get over this limitation there are parameters  for the codec
that can limit the size starting from the level parameter. In the mobile
case since the BW is limited you will use a lower level that does not
support the HD resolution. For example level 1 is limited to 64 kbit/sec
with maximum resolution of 176x144, for the must support you need to have
192 kbit/sec level 1.1.

You can override the maximum resolution by lowering the maximum frame rate
and this are parameters specified in the H.264 payload specification.

There is also the SDP image attribute that allow the receiver to ask for
specific resolution and aspect ratio.

As for your suggested text, I am not sure why you need it since these are
requirement that the codec should support and not for using for send or
receive. It says that a candidate codec should have an HD mode, some older
codecs like H.261 did not have this mode.

 

Roni Even

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Harald Alvestrand
Sent: Thursday, November 17, 2011 12:10 PM
To: Stephan Wenger
Cc: [email-masked]
Subject: Re: [rtcweb] Video resolution SHOULDs (Re: resolutions in
draft-cbran-rtcweb-codec-01)

I would instead recommend naming one single MUST resolution (as the draft
does) to ensure minimum interoperability, and otherwise negotiate a
processing rate in macro blocks per second (or equivalent), with fixed
constraints around the aspect ratio.

One of my colleagues had a failing test the other day because he'd specified
a picture height of 94 pixels - he did not know that his encoders only
supported pixel counts that were multiples of 8.

Of course a device will not fail to interoperate just because it supports
decoding more resolutions than those that are required.

BTW, I believe 640x360 is a 16:9 aspect ratio, so handling both 16:9 and 4:3
seems to be a SHOULD based on this resolution list. I assume that's
intentional.

---

From: "Roni Even" <[email-masked]>
Date: Thu, 17 Nov 2011 13:11:27 +0200

Stephan,

I think that your point make sense, and I am not sure why to list
resolutions at all (having one that MUST be supported may not be the same
for all devices). Requirement for the max macroblocks per second number can
provide information about what is the maximum resolution that the codec can
support and I think that the other requirement should be that the receiver
must be able to ask for a specific resolution and aspect ratio which is
covered in current SDP.

Roni

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Stephan Wenger
Sent: Thursday, November 17, 2011 11:54 AM
To: Harald Alvestrand; [email-masked]
Subject: Re: [rtcweb] Video resolution SHOULDs (Re: resolutions in
draft-cbran-rtcweb-codec-01)

Stephan

Hi,

 

Roni Even

---

From: "James M. Polk" <[email-masked]>
Date: Thu, 17 Nov 2011 10:20:19 -0600

At 04:10 AM 11/17/2011, Harald Alvestrand wrote:
>>       800x600, 640x480, 640 x 360 , 320x240 *if these resolutions 
>> can be displayed on the target device*

I think Harald's suggestion included "Must support 320 x 240" too, 
which is listed as a MUST and SHOULD throughout this thread (which I 
believe is in error). If this resolution is specified, it needs to be 
either MUST or SHOULD, not both.

Roni has a good suggestion about the ability of a receiver to request 
a resolution in SDP, but I think we still should have a given 
resolution defined here, and understanding that each device could 
have wildly different capabilities, Harald's suggestion above covers 
the most cases IMO but only with the text between the * ... * included as well.

James

>>seems sensible.  However, I have another issue.  I don't believe 
>>that naming specific resolutions is advisable.  In a system where 
>>the rendering side has almost undefined picture properties because 
>>they are typically rendered in "windows", it IMO does not make much 
>>sense to limit yourself to certain aspect ratios, picture sizes 
>>etc., even if those correspond to today's preferred sensor 
>>resolutions (or integer fractions thereof).  Resampling an input 
>>signal is cheap nowadays (at least compared to the encoding 
>>itself), and cropping is even cheaper.
>BTW, I believe 640x360 is a 16:9 aspect ratio, so handling both 16:9 
>and 4:3 seems to be a SHOULD based on this resolution list. I assume 
>that's intentional.
>
>

---

From: Aron Rosenberg <[email-masked]>
Date: Thu, 17 Nov 2011 17:10:28 -0800

On Wed, Nov 16, 2011 at 7:19 PM, Roni Even <[email-masked]> wrote:

> Hi,****
>
> The text in the draft on video codec is****
>
> ** **
>
> o  MUST support a minimum resolution of 320X240****
>
> ** **
>
> o  SHOULD support resolutions of 1280x720, 720x480, 1024x768,****
>
>       800x600, 640x480, 640 x 360 , 320x240****
>
> ** **
>
>
> I would suggest dropping 720x480 as it is neither 16:9 or 4:3 ratio, but a
non-standard 3:2 ratio. From a camera perspective 1024x768 and 800x60 are
not well supported in video modes. Cameras, both integrated and external
generally support 640x480 and below with high quality, or support 1280x720
and below with high quality. I would also replace the intermediate 4:3
(1024x768 and 800x600) resolutions with 16:9 sizes such as 1024x576 and
864x480.

Aron Rosenberg
LifeSize, A Division of Logitech Inc.

---

From: Ralph Giles <[email-masked]>
Date: Thu, 17 Nov 2011 17:30:06 -0800

On 17 November 2011 17:10, Aron Rosenberg <[email-masked]> wrote:

> non-standard 3:2 ratio.

720x480 is a standard SD broadcast/DVD-Video resolution. With
non-square pixels and overscan, it can represent either 4:3 or 16:9.
Both proposed baseline codecs in this draft support non-square pixels,
since a lot of video is in this format.

I don't know how useful it is to have this as a SHOULD, but I would
ask why the corresponding PAL resolution (576 lines) isn't also
included if this one is considered useful.

> From a camera perspective 1024x768 and 800x60[0] are
> not well supported in video modes.

That's true, but these are common VGA screen resolutions. I think it's
useful to have them for presentations and screencasting. 1024x768 has
been a standard format for conference presentations for more than a
decade, and 800x600 is a useful fallback.

As to how this SHOULD should be read, I think the point here is that
while some devices have fixed formats due to hardware constraints,
others (like a desktop browser) are quite flexible, so it's helpful to
have a list of common resolutions implementers can choose to offer in
the absence of other constraints, and to test against.

 -r

---

From: "James M. Polk" <[email-masked]>
Date: Thu, 17 Nov 2011 23:48:30 -0600

At 07:10 PM 11/17/2011, Aron Rosenberg wrote:
>On Wed, Nov 16, 2011 at 7:19 PM, Roni Even 
><<mailto:[email-masked]>[email-masked]> wrote:
>
>Hi,
>

why is 320x240 listed as a MUST and as a SHOULD?

James

>I would suggest dropping 720x480 as it is neither 16:9 or 4:3 ratio, 
>but a non-standard 3:2 ratio. From a camera perspective 1024x768 and 
>800x60 are not well supported in video modes. Cameras, both 
>integrated and external generally support 640x480 and below with 
>high quality, or support 1280x720 and below with high quality. I 
>would also replace the intermediate 4:3 (1024x768 and 800x600) 
>resolutions with 16:9 sizes such as 1024x576 and 864x480.
>
>Aron Rosenberg

---

From: "Bran, Cary" <[email-masked]>
Date: Tue, 29 Nov 2011 09:57:29 -0800

Hi Ralph,

On 11/17/11 5:30 PM, "Ralph Giles" <[email-masked]> wrote:

>>but a

I have added this to the 02 version as a SHOULD.
>the absence of other constraints, and to test against.

Stole a bit of this verbiage and integrated it into the 02 version.

---

From: "Bran, Cary" <[email-masked]>
Date: Tue, 29 Nov 2011 09:59:05 -0800

>why is 320x240 listed as a MUST and as a SHOULD?

I got a little crazy with the cut and paste - for the 02 version and have
removed the duplicate resolution from the SHOULD section.

Thanks!

---

From: Ralph Giles <[email-masked]>
Date: Tue, 29 Nov 2011 10:01:47 -0800

Thanks!

> Stole a bit of this verbiage and integrated it into the 02 version.

Glad it was helpful, and thanks for fixing the duplicate 320x240.

 -r

---

From: "Bran, Cary" <[email-masked]>
Date: Tue, 29 Nov 2011 10:30:52 -0800

Hi,

I have made changes to the 02 version that reflect this thread and a
couple of other threads  the new text is as follows:

"The following feature list applies to all required video codecs.

   Required video codecs:
   o  MUST support at least 10 frames per second (fps) and SHOULD
      support 30 fps

   o  If VP8, then MUST support a the bilinear and none reconstruction
      filters

   o  OPTIONALLY offer support for additional color spaces

   WebRTC client implementations will span across a variety of devices.
   Some of the these devices have fixed formats due to hardware
   constraints, others like a desktop browser are quite flexible.
   Instead of placing implementation constraints on the supported
   resolutions, below is a list of suggested common resolutions for
   WebRTC client implementations.  The suggested resolutions should be
   considered only if they can be displayed on the target device.

   WebRTC client implementations SHOULD support:

   o  Resolutions of 1920X1080, 1280x720, 720x480,
      1024x768, 800x600, 640x480, 640 x 360

   o  A PAL resolution of 576 lines."

---

From: Ralph Giles <[email-masked]>
Date: Tue, 29 Nov 2011 10:49:59 -0800

>   o  If VP8, then MUST support a the bilinear and none reconstruction

s/ a//

>   o  A PAL resolution of 576 lines."

I would suggest just adding 720x576 to the list of resolutions.
There's no reason to single out PAL.

If the NTSC and PAL resolutions are confusing, you could try to say
something about pixel aspect ratios, interlacing, etc. Not sure if
that level detail is helpful or not.

 -r

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 29 Nov 2011 20:48:29 +0100

>     o  If VP8, then MUST support a the bilinear and none reconstruction
Can you get rid of this one?
The VP8 people explicitly want to discourage anything that looks like 
we're blessing partial implementation - we want all implementations to 
support everything defined in the RFC.
>     o  A PAL resolution of 576 lines."
This looks good to me. Is there a defined answer to "what's the 
horizontal resolution when the vertical resolution is 576 lines", or is 
it "it depends"?
>
>
>
>
> On 11/17/11 8:20 AM, "James M. Polk"<[email-masked]>  wrote:
>

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 29 Nov 2011 20:49:10 +0100

On 11/29/2011 07:49 PM, Ralph Giles wrote:
> On 29 November 2011 10:30, Bran, Cary<[email-masked]>  wrote:
>
Interlace? Just Say No!

---

From: Ralph Giles <[email-masked]>
Date: Tue, 29 Nov 2011 20:56:23 -0800

> Interlace? Just Say No!

Works for me. I'm an optimist.

 -r

---

From: "James M. Polk" <[email-masked]>
Date: Tue, 29 Nov 2011 23:18:02 -0600

>Works for me. I'm an optimist.

not meaning to open a can of worms, but what still requires 
interlaced video? Most new things convert I to P, but less of these 
convert P to I.

James

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 30 Nov 2011 08:25:34 +0100

On 11/30/2011 06:18 AM, James M. Polk wrote:
> not meaning to open a can of worms, but what still requires interlaced 
> video? Most new things convert I to P, but less of these convert P to I.
Just hit the same conversation here in Geneva at MPEG meeting. Funny 
what you learn!

Interlaced material requires quite extensive resources on the decoding 
end to do properly, and if you just ignore the interlace marker and 
display the half-frames on top of each other, the resulting display is 
clearly substandard.
But converting is a small-but-significant cost to the broadcasters with 
stored interlaced material, and the hardware guys who already have to 
implement it for some profile see no extra cost to having that support, 
so they don't push back much against it.

In contrast, WebRTC is much more greenfield, and has far less 
established interlace support than TVs.

If we can push interlaced material up the food chain into those devices 
that have to deal with it because it's in the archives, and get them to 
do the I-to-P conversion, the Web will be better off.
</rant>

                       Harald

---

From: Stephen Botzko <[email-masked]>
Date: Wed, 30 Nov 2011 10:23:03 +0100

-inline
Stephen Botzko

On Wed, Nov 30, 2011 at 8:25 AM, Harald Alvestrand <[email-masked]>wrote:

> Just hit the same conversation here in Geneva at MPEG meeting. Funny what
> you learn!
>
> Interlaced material requires quite extensive resources on the decoding end
> to do properly, and if you just ignore the interlace marker and display the
> half-frames on top of each other, the resulting display is clearly
> substandard.
> implement it for some profile see no extra cost to having that support, so
> they don't push back much against it.
>
> In contrast, WebRTC is much more greenfield, and has far less established
> interlace support than TVs.
>
> that have to deal with it because it's in the archives, and get them to do
> the I-to-P conversion, the Web will be better off.
> </rant>
>
>                      Harald
>
[SB]Interlaced encoding also adds delay, so it is not generally used for
interactive applications. So I agree that it is better to keep it out of
RTCWeb.

>
> ______________________________**_________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>

---

From: Ralph Giles <[email-masked]>
Date: Wed, 30 Nov 2011 12:21:07 -0800

On 29 November 2011 23:25, Harald Alvestrand <[email-masked]> wrote:

> If we can push interlaced material up the food chain into those devices that
> have to deal with it because it's in the archives, and get them to do the
> I-to-P conversion, the Web will be better off.

I entirely agree. I was snarking because we tried to do something
similar by making Theora fixed frame rate, and it feels like we've
gone backwards on that with WebM and WebRTC. Not a fair comparison, of
course, since cheap fixes to variable frame rate video don't look as
bad as cheap deinterlacing, and computers are natively more flexible
there.

 -r