Subject: [video-codec] BoF request: video-codec - Internet Video Codec

From: "Timothy B. Terriberry" <[email-masked]>
Date: Mon, 24 Sep 2012 13:56:20 -0700

The proposers would like to request a 2-hour slot for a BoF at IETF 85 
in Atlanta on the subject of standardizing a video codec.

Description
-----------

There are efforts underway by several groups to produce a 
next-generation, royalty-free (RF) video codec, including VPnext by 
Google [1,2] and Daala by Mozilla/Xiph.Org [3]. While far from complete, 
we hope that these can become the backbone of a truly universally 
deployable, interactive video codec standard that is competitive with 
royalty-bearing offerings. These projects are just the start. Recent 
efforts within the MPEG LA to create an RF license for the Restricted 
Baseline profile of H.264, while ultimately unsuccessful, showed that 
there are many consumer device manufacturers that would support RF 
licensing for a video codec. Not all will participate in development, 
but even providing review, as some have agreed to do, is useful. In the 
CODEC WG, it was the input and feature requests from a wide range of 
IETF participants that led to a codec that excelled at an extremely 
broad range of applications. This kind of review is much easier for 
codecs, where there are easily-understood, measurable objectives 
("everyone has two eyes"). The IETF is typically used to dealing with 
much thornier questions, such as, "Will this interoperate with legacy, 
possibly-broken implementations?" or "Is this protocol sufficiently 
extensible?"

The success of Opus from the CODEC WG has also shown that collaboration, 
based on the IETF's principals of open participation, can produce better 
results than competition between patented technologies. The IPR rules in 
BCP 78 and 79 are also critical for success. They impose a duty to 
disclose, and require exact patent or patent application numbers, in 
addition to basic licensing terms. This allows participants to evaluate 
the risk of infringement and, if appropriate, design work arounds, in 
any technology adopted, and assess the cost of adopting such technology. 
Because it does not force participants to agree to license their patents 
under RF terms, it helps to encourage participation even by those 
opposed to such terms (instead of guaranteeing they stay away). In 
addition to an environment which encourages third-party disclosures, 
this provides much better chances of success than SDOs which have a 
"patent-blind" process [4] or which require blanket RF grants.

[1] http://downloads.webmproject.org/ngov2012/pdf/04-ngov-project-update.pdf
[2] http://downloads.webmproject.org/ngov2012/wilkins/vpnext-results.html
[3] http://xiph.org/daala/
[4] http://lists.xiph.org/pipermail/theora/2010-April/003769.html

Details
-------

The proposed working group name is videoc-codec.

Conflicts to avoid: APPSAWG, AVTCORE, CLUE, CODEC, IRI, MMUSIC, PRECIS, 
RMCAT, RTCWEB, XMPP

Expected attendance: 80 persons

Does it require WebEX?: No
Does it require Meetecho?: Yes

BOF agenda
----------

Chair(s): Peter Saint-Andre (plus a possible TBD co-chair)

Area: RAI

Area directors: Gonzalo Camarillo and Robert J. Sparks

Draft agenda (subject to modifications):
- Note Well, logistics, agenda bash (Chairs, 5 min)
- Introduction and scoping of BoF (AD, 10 min)
- Goals (Chairs, 10 min)
- Engineering progress to date (Google, Mozilla 20 min)
- Is it reasonable to do this work at the IETF? (All, 45 min)
- Charter discussion (All, 25 min)
- Conclusions and next steps (Chairs/AD, 5 min)

Total timeslot: 2 hours

WG charter (proposed)
---------------------
Internet Video Codec (video-codec)

Status: Proposed Working Group
Last Updated: 2012-09-24

Chair(s): TBD

RAI Area Director(s):
  Robert J. Sparks ([email-masked])
  Gonzalo Camarillo ([email-masked])

RAI Area Advisor:
  Robert J. Sparks ([email-masked])

Mailing Lists: [email-masked]

Description of Working Group
----------------------------

Problem Statement

According to reports from developers of Internet video applications and 
operators of Internet video services, there is no standardized, 
high-quality video codec that meets all of the following three conditions:

1. Is optimized for use in interactive Internet applications.

2. Is published by a recognized standards development organization (SDO) 
and therefore subject to clear change control and IPR disclosure rules.

3. Can be widely implemented and easily distributed among application 
developers, service operators, and end users.

There exist codecs that provide high quality encoding of video 
information, but that are not optimized for the actual conditions of the 
Internet; according to reports, this mismatch between design and 
deployment has hindered interoperability of such codecs in interactive 
Internet applications.

There exist codecs that can be widely implemented, but were not 
developed under the IPR rules of any SDO; according to reports, the lack 
of clarity in their IPR status has hindered adoption of such codecs in 
Internet applications.

There exist codecs that are standardized, but that cannot be widely 
implemented and easily distributed; according to reports, the presence 
of various usage restrictions (e.g., in the form of requirements to pay 
royalty fees, obtain a license, enter into a business agreement, or meet 
other special conditions imposed by a patent holder) has hindered 
adoption of such codecs in Internet applications.

According to application developers and service operators, a video codec 
that meets all three of these conditions would: (1) enable protocol 
designers to more easily specify a mandatory-to-implement codec in their 
protocols and thus improve interoperability; (2) enable developers to 
more easily build innovative applications for the Internet; (3) enable 
service operators to more easily deploy affordable, high-quality video 
services on the Internet; and (4) enable end users of Internet 
applications and services to enjoy an improved user experience.

Objectives

The goal of this working group is to ensure the existence of a single 
high-quality video codec that can be widely implemented and easily 
distributed among application developers, service operators, and end 
users. At present it appears that ensuring the existence of such a codec 
will require a development effort within the working group.
The core technical considerations for such a codec include, but are not 
necessarily limited to, the following:

1. Designing for use in interactive applications (examples include, but 
are not limited to, point-to-point video calls, multi-party video 
conferencing, telepresence, teleoperation, and in-game video chat).

2. Addressing the real transport conditions of the Internet, such as the 
flexibility to rapidly respond to changing bandwidth availability and 
loss rates, or as otherwise identified and prioritized by the working group.

3. Ensuring interoperability and clean integration with the Real-time 
Transport Protocol (RTP), including secure transport via SRTP.

4. Ensuring interoperability with Internet signaling technologies such 
as Session Initiation Protocol (SIP), Session Description Protocol 
(SDP), and Extensible Messaging and Presence Protocol (XMPP); however, 
the result should not depend on the details of any particular signaling 
technology.

Optimizing for very low bit rates (typically below 64 kbps) is out of 
scope because such work might necessitate specialized optimizations.

In addition to the technical objectives, there is one process goal, which is

5. Ensuring the work is done under the IPR rules of the IETF.

Although a codec produced by this working group or another standards 
organization might be used as a mandatory-to-implement technology by 
designers of particular Internet protocols, it is explicitly not a goal 
of the working group to produce or select a codec that will be mandated 
for use across the entire IETF or Internet community nor would their be 
any expectation that this would be the only mandatory-to-implement codec.

Based on the working group's analysis of the design space, the working 
group might determine that it needs to produce more than one codec, or a 
codec with multiple modes; however, it is not the goal of working group 
to produce more than one codec, and to reduce confusion in the 
marketplace the working group shall endeavor to produce as few codecs as 
possible.

In completing its work, the working group should collaborate with other 
IETF working groups to complete particular tasks. These might include, 
but would not be limited to, the following:

- Within the AVT WG, define the codec's payload format for use with the 
Real-time Transport Protocol (RTP).

- Collaborate with RMCAT and any other appropriate working groups in the 
Transport Area to identify important aspects of packet transmission over 
the Internet.

- Collaborate with RMCAT to understand the degree of rate adaptation 
desirable, and to reflect that understanding in the design of a codec 
that can adjust its transmission in a way that minimizes disruption to 
the video.

- Collaborate with working groups in the RAI Area to ensure that 
information about and negotiation of the codec can be easily represented 
at the signaling layer.

- Collaborate with working groups in the RAI Area such as CLUE and 
RTCWEB to ensure the codec can satisfy all of their video-related use cases.

The working group will coordinate with the ITU-T (Study group 16) and 
ISO/IEC (JTC1/SC29 WG11), with the intent of submitting the completed 
codec RFC for co-publication if they find that appropriate. The working 
group will communicate a detailed description of the requirements and 
goals to other SDOs including the ITU-T and MPEG to help determine if 
existing codecs meet the requirements and goals. Information about 
codecs being standardized will be available to other SDOs in the form of 
Internet-Drafts and the working group welcomes technical feedback from 
other SDOs and experts from other organizations.

The Guidelines for Development of an Audio Codec within the IETF (RFC 
6569) will form the starting point for guidelines and requirements for 
achieving the forgoing objectives for video. The working group will 
modify them as necessary with lessons learned during that process, 
refining them into a new document in accordance with the usual IETF 
procedures once consensus has been achieved.

A codec that can be widely implemented and easily distributed among 
application developers, service operators, and end users is preferred. 
Many existing codecs that might fulfill some or most of the technical 
attributes listed above are encumbered in various ways. For example, 
patent holders might require that those wishing to implement the codec 
in software, deploy the codec in a service, or distribute the codec in 
software or hardware need to request a license, enter into a business 
agreement, pay licensing fees or royalties, or attempt to adhere to 
other special conditions or restrictions.

Because such encumbrances have made it difficult to widely implement and 
easily distribute high-quality video codecs across the entire Internet 
community, the working group prefers unencumbered technologies in a way 
that is consistent with BCP 78 and BCP 79. In particular, the working 
group shall heed the preference stated in BCP 79: "In general, IETF 
working groups prefer technologies with no known IPR claims or, for 
technologies with claims against them, an offer of royalty-free 
licensing." Although this preference cannot guarantee that the working 
group will produce an unencumbered codec, the working group shall follow 
BCP 79, and adhere to the spirit of BCP 79. The working group cannot 
explicitly rule out the possibility of adopting encumbered technologies; 
however, the working group will try to avoid encumbered technologies 
that require royalties or other encumbrances that would prevent such 
technologies from being easy to redistribute and use.

Deliverables

1. A set of Codec Standardization Guidelines that define the work 
processes of the working group. This document shall be Informational.

2. A set of technical Requirements. This document shall be Informational.

3. Specification of a codec that meets the agreed-upon requirements, in 
the form of an Internet-Draft that defines the codec algorithm along 
with source code for a reference implementation. The text description of 
the codec shall describe the mandatory, recommended, and optional 
portions of the encoder and decoder. It is envisioned that this document 
shall be a Proposed Standard document.

4. Specification of a storage format for non-interactive (HTTP) 
streaming or file transfer of the codec. It is envisioned that this 
document shall be a Proposed Standard document.

Goals and Milestones
TBD  WGLC on codec standardization guidelines
TBD  WGLC on requirements, liaise to other SDOs
TBD  Requirements to IESG (Informational)
TBD  Liaise requirements RFC to other SDOs
TBD  Codec standardization guidelines to IESG (Informational)
TBD  WGLC on codec specification, liaise to other SDOs
TBD  Submit codec specification to IESG (Standards Track)
TBD  WGLC on storage format specification
TBD  Submit storage format specification to IESG (Standards Track)
TBD  WGLC on Testing document
TBD  Testing document to IESG

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Mon, 24 Sep 2012 14:05:14 -0700

Just as an FYI for the list, the BoF request includes an updated 
charter, mostly incorporating off-list feedback on grammar and wording 
from Peter Saint-Andre and Ted Hardie. It now references the 
newly-formed RMCAT WG (thanks Ted), and mentions liaisons to both ITU-T 
and MPEG.

One item I forgot was to add a "Test results" document to the 
deliverables (there are milestones for it, but no entry in the 
deliverables list). I'll make sure that gets added to the next iteration.

---

From: Gonzalo Camarillo <[email-masked]>
Date: Tue, 25 Sep 2012 10:45:31 +0300

Hi,

thanks for the request, which was fairly close to the deadline :-)

Cheers,

Gonzalo

---

From: Robert Sparks <[email-masked]>
Date: Wed, 26 Sep 2012 12:31:21 -0500

This BoF is approved.

Due to tool issues, the acronym will need to be different - for now, 
I've asked the secretariat to use vidcodec
(no hyphen, <9 letters). I will work with them to see if we can use 
videocodec instead.

Please continue to refine the description and proposed charter on list.

RjS

On 9/24/12 3:56 PM, Timothy B. Terriberry wrote:
> Google [1,2] and Daala by Mozilla/Xiph.Org [3]. While far from 
> complete, we hope that these can become the backbone of a truly 
> universally deployable, interactive video codec standard that is 
> competitive with royalty-bearing offerings. These projects are just 
> the start. Recent efforts within the MPEG LA to create an RF license 
> for the Restricted Baseline profile of H.264, while ultimately 
> unsuccessful, showed that there are many consumer device manufacturers 
> that would support RF licensing for a video codec. Not all will 
> participate in development, but even providing review, as some have 
> agreed to do, is useful. In the CODEC WG, it was the input and feature 
> requests from a wide range of IETF participants that led to a codec 
> that excelled at an extremely broad range of applications. This kind 
> of review is much easier for codecs, where there are 
> easily-understood, measurable objectives ("everyone has two eyes"). 
> The IETF is typically used to dealing with much thornier questions, 
> such as, "Will this interoperate with legacy, possibly-broken 
> implementations?" or "Is this protocol sufficiently extensible?"
>
> The success of Opus from the CODEC WG has also shown that 
> collaboration, based on the IETF's principals of open participation, 
> can produce better results than competition between patented 
> technologies. The IPR rules in BCP 78 and 79 are also critical for 
> success. They impose a duty to disclose, and require exact patent or 
> patent application numbers, in addition to basic licensing terms. This 
> allows participants to evaluate the risk of infringement and, if 
> appropriate, design work arounds, in any technology adopted, and 
> assess the cost of adopting such technology. Because it does not force 
> participants to agree to license their patents under RF terms, it 
> helps to encourage participation even by those opposed to such terms 
> (instead of guaranteeing they stay away). In addition to an 
> environment which encourages third-party disclosures, this provides 
> much better chances of success than SDOs which have a "patent-blind" 
> process [4] or which require blanket RF grants.
>
> [1] 
> http://downloads.webmproject.org/ngov2012/pdf/04-ngov-project-update.pdf
> Conflicts to avoid: APPSAWG, AVTCORE, CLUE, CODEC, IRI, MMUSIC, 
> PRECIS, RMCAT, RTCWEB, XMPP
>
> According to reports from developers of Internet video applications 
> and operators of Internet video services, there is no standardized, 
> 2. Is published by a recognized standards development organization 
> (SDO) and therefore subject to clear change control and IPR disclosure 
> rules.
>
> information, but that are not optimized for the actual conditions of 
> the Internet; according to reports, this mismatch between design and 
> developed under the IPR rules of any SDO; according to reports, the 
> lack of clarity in their IPR status has hindered adoption of such 
> codecs in Internet applications.
>
> of various usage restrictions (e.g., in the form of requirements to 
> pay royalty fees, obtain a license, enter into a business agreement, 
> or meet other special conditions imposed by a patent holder) has 
> hindered adoption of such codecs in Internet applications.
>
> According to application developers and service operators, a video 
> codec that meets all three of these conditions would: (1) enable 
> protocol designers to more easily specify a mandatory-to-implement 
> codec in their protocols and thus improve interoperability; (2) enable 
> developers to more easily build innovative applications for the 
> Internet; (3) enable service operators to more easily deploy 
> affordable, high-quality video services on the Internet; and (4) 
> enable end users of Internet applications and services to enjoy an 
> improved user experience.
>
> Objectives
>
> users. At present it appears that ensuring the existence of such a 
> codec will require a development effort within the working group.
> The core technical considerations for such a codec include, but are 
> not necessarily limited to, the following:
>
> 1. Designing for use in interactive applications (examples include, 
> but are not limited to, point-to-point video calls, multi-party video 
> 2. Addressing the real transport conditions of the Internet, such as 
> the flexibility to rapidly respond to changing bandwidth availability 
> and loss rates, or as otherwise identified and prioritized by the 
> working group.
>
> the result should not depend on the details of any particular 
> signaling technology.
>
> designers of particular Internet protocols, it is explicitly not a 
> goal of the working group to produce or select a codec that will be 
> mandated for use across the entire IETF or Internet community nor 
> would their be any expectation that this would be the only 
> mandatory-to-implement codec.
>
> group might determine that it needs to produce more than one codec, or 
> a codec with multiple modes; however, it is not the goal of working 
> group to produce more than one codec, and to reduce confusion in the 
> marketplace the working group shall endeavor to produce as few codecs 
> as possible.
>
> In completing its work, the working group should collaborate with 
> other IETF working groups to complete particular tasks. These might 
> include, but would not be limited to, the following:
>
> - Within the AVT WG, define the codec's payload format for use with 
> the Real-time Transport Protocol (RTP).
>
> - Collaborate with RMCAT and any other appropriate working groups in 
> the Transport Area to identify important aspects of packet 
> transmission over the Internet.
>
> information about and negotiation of the codec can be easily 
> represented at the signaling layer.
>
> codec RFC for co-publication if they find that appropriate. The 
> working group will communicate a detailed description of the 
> requirements and goals to other SDOs including the ITU-T and MPEG to 
> help determine if existing codecs meet the requirements and goals. 
> Information about codecs being standardized will be available to other 
> SDOs in the form of Internet-Drafts and the working group welcomes 
> technical feedback from other SDOs and experts from other organizations.
>
> Because such encumbrances have made it difficult to widely implement 
> and easily distribute high-quality video codecs across the entire 
> Internet community, the working group prefers unencumbered 
> technologies in a way that is consistent with BCP 78 and BCP 79. In 
> particular, the working group shall heed the preference stated in BCP 
> 79: "In general, IETF working groups prefer technologies with no known 
> IPR claims or, for technologies with claims against them, an offer of 
> royalty-free licensing." Although this preference cannot guarantee 
> that the working group will produce an unencumbered codec, the working 
> group shall follow BCP 79, and adhere to the spirit of BCP 79. The 
> working group cannot explicitly rule out the possibility of adopting 
> encumbered technologies; however, the working group will try to avoid 
> encumbered technologies that require royalties or other encumbrances 
> that would prevent such technologies from being easy to redistribute 
> and use.
>
> Deliverables
>
> 3. Specification of a codec that meets the agreed-upon requirements, 
> in the form of an Internet-Draft that defines the codec algorithm 
> along with source code for a reference implementation. The text 
> description of the codec shall describe the mandatory, recommended, 
> and optional portions of the encoder and decoder. It is envisioned 
> that this document shall be a Proposed Standard document.
>