Subject: [codec] WG Review: Internet Wideband Audio Codec (codec)

From: IESG Secretary <[email-masked]>
Date: Wed, 23 Dec 2009 09:15:01 -0800 (PST)

A new IETF working group has been proposed in the Real-time Applications
and Infrastructure Area.  The IESG has not made any determination as yet.
The following draft charter was submitted, and is provided for
informational purposes only.  Please send your comments to the IESG
mailing list ([email-masked]) by January 20, 2010.

Internet Wideband Audio Codec (codec)
-------------------------------------------------------------------------
Last Modified: 2009-12-17

Proposed Chair(s):
 * TBD
 
Real-time Applications and Infrastructure Area Director(s):
 * Robert Sparks <[email-masked]>
 * Cullen Jennings <[email-masked]>

Real-time Applications and Infrastructure Area Advisor:
 * Cullen Jennings <[email-masked]>

Mailing Lists:
General Discussion: [email-masked]
To Subscribe: [email-masked]
In Body: subscribe
Archive: https://www.ietf.org/mailman/listinfo/codec

Description of Working Group
Problem Statement

According to reports from developers of Internet audio applications and
operators of Internet audio services, there are no standardized,
high-quality audio codecs that meet all of the following three
conditions:

1. Are optimized for use in interactive Internet applications.

2. Are published by a recognized standards development organization
(SDO) and therefore subject to clear change control.

3. Can be widely implemented and easily distributed among application
developers, service operators, and end users.

There exist codecs that provide high quality encoding of audio
information, but that are not optimized for the actual conditions of the
Internet; according to reports, this mismatch between design and
deployment has hindered adoption of such codecs in interactive Internet
applications.

There exist codecs that can be widely implemented and easily
distributed, but that are not standardized through any SDO; according to
reports, this lack of standardization and clear change control has
hindered adoption of such codecs in interactive Internet applications.

There exist codecs that are standardized, but that cannot be widely
implemented and easily distributed; according to reports, the presence
of various usage restrictions (e.g., in the form of requirements to pay
royalty fees, obtain a license, enter into a business agreement, or meet
other special conditions imposed by a patent holder) has hindered
adoptions of such codecs in interactive Internet applications.

According to application developers and service operators, an audio
 codec that meets all three of these would: (1) enable protocol
 designers to more easily specify a mandatory-to-implement codec in
 their protocols and thus improve interoperability; (2) enable
 developers to more easily easily build innovative, interactive
 applications for the Internet; (3) enable service operators to more
 easily deploy affordable, high-quality audio services on the Internet;
 and (4) enable end users of Internet applications and services to enjoy
 an improved user experience.

Objectives

The goal of this working group is to develop a single high-quality audio
codec that is optimized for use over the Internet and that can be widely
implemented and easily distributed among application developers, service
operators, and end users.  Core technical considerations include, but
are not necessarily limited to, the following:

1. Designing for use in interactive applications (examples include, but
are not limited to, point-to-point voice calls, multi-party voice
conferencing, telepresence, teleoperation, in-game voice chat, and live
music performance)

2. Addressing the real transport conditions of the Internet as
identified and prioritized by the working group

3. Ensuring interoperability with the Real-time Transport Protocol
(RTP), including secure transport via SRTP

4. Ensuring interoperability with Internet signaling technologies such
as Session Initiation Protocol (SIP), Session Description Protocol
(SDP), and Extensible Messaging and Presence Protocol (XMPP); however,
the result should not depend on the details of any particular signaling
technology

Optimizing for very low bit rates (typically below 2.4 kbps) and for
non-interactive audio is out of scope because such work might
necessitate specialized optimizations.

Although the codec produced by the working group might be used as a
mandatory-to-implement technology by designers of particular Internet
protocols, it is explicitly not a goal of the working group to produce a
codec that will be mandated for use across the entire IETF or Internet
community nor would their be any expectation that this would be the only
mandatory-to-implement codec.

The goal of the working group is to produce only one codec.  Based on
the working group's analysis of the design space, the working group
might determine that it needs to produce more than one codec, or a codec
with multiple modes; however, it is not the goal of working group to
produce more than one codec, and to reduce confusion in the marketplace
the working group shall endeavor to produce as few codecs as possible.

In completing its work, the working group should collaborate with other
IETF working groups to complete particular tasks.  These might include,
but would not be limited to, the following:

- Within the AVT WG, define the codec's payload format for use with the
  Real-time Transport Protocol (RTP).

- Collaborate with working groups in the Transport Area to identify
  important aspects of packet transmission over the Internet.

- Collaborate with working groups in the Transport Area to understand
  the degree of rate adaptation desirable, and to reflect that
  understanding in the design of a codec that can adjust its
  transmission in a way that minimizes disruption to the audio.

- Collaborate with working groups in the RAI Area to ensure that
  information about and negotiation of the codec can be easily
  represented at the signaling layer.

The working group will inform the ITU-T (Study group 16) of each new
revision of working group drafts, with the intent of submitting the
completed codec RFC for co-publication by the ITU-T if the ITU-T finds
that appropriate. The working group will communicate detailed
description of the requirements and goals to other SDOs including the
ITU-T, 3GPP, and MPEG to help determine if existing codecs meet the
requirements and would therefore enable co-publication of an existing
standard at the IETF. The working group will also continue to discuss
with other standards bodies to determine if it becomes possible to
satisfy the IETF requirements through a new or revised standard at other
bodies.

Suggested Codec Standardization Guidelines and Requirements for
achieving the foregoing objectives are provisionally outlined in
draft-valin-codec-guidelines and draft-valin-codec-requirements
respectively; these documents will form the starting point for working
toward consensus and, if accepted as work items of the working group,
will be refined by the working group in accordance with the usual IETF
procedures.

A codec that can be widely implemented and easily distributed among
application developers, service operators, and end users is preferred.
Many existing codecs that might fulfill some or most of the technical
attributes listed above are encumbered in various ways.  For example,
patent holders might require that those wishing to implement the codec
in software, deploy the codec in a service, or distribute the codec in
software or hardware need to request a license, enter into a business
agreement, pay licensing fees or royalties, or attempt to adhere to
other special conditions or restrictions.

Because such encumbrances have made it difficult to widely implement and
easily distribute high-quality audio codecs across the entire Internet
community, the working group prefers unencumbered technologies in a way
that is consistent with BCP 78 and BCP 79.  In particular, the working
group shall heed the preference stated in BCP 79: "In general, IETF
working groups prefer technologies with no known IPR claims or, for
technologies with claims against them, an offer of royalty-free
licensing."  Although this preference cannot guarantee that the working
group will produce an unencumbered codec, the working group shall
attempt to adhere to the spirit of BCP 79.  This preference does not
explicitly rule out the possibility of adapting encumbered technologies;
such decisions will be made in accordance with the rough consensus of
the working group.

Deliverables

1. A set of Codec Standardization Guidelines that define the work
processes of the working group. This document shall be Informational.

2. A set of technical Requirements. This document shall be
Informational.

3. Specification of a codec that meets the agreed-upon requirements, in
the form of an Internet-Draft that defines the codec algorithm along
with source code for a reference implementation.  The text description
of the codec shall indicate which components of the encoder and decoder
are mandatory, recommended, and optional.  It is envisioned that this
document shall be a Proposed Standard document.

Milestones

Mar-2010: WGLC on Codec Standardization Guidelines
May-2010: Codec Standardization Guidelines to IESG (Informational)
May-2010: WGLC on Requirements
Jul-2010: Requirements to IESG (Informational)
Dec-2010: Freeze codec structure
Jun-2011: Finalize codec parameters
Jul-2011: WGLC on codec specification
Oct-2011: Submit codec specification to IESG (Standards Track)

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 23 Dec 2009 12:58:47 -0700

Begin forwarded message:

> From: IESG Secretary <[email-masked]>
> Date: December 23, 2009 10:15:01 AM MST
> To: [email-masked]
> Cc: [email-masked]
> Subject: [codec] WG Review: Internet Wideband Audio Codec (codec)
> Reply-To: [email-masked]
> 
> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec

---

From: "Roni Even" <[email-masked]>
Date: Thu, 24 Dec 2009 08:50:30 +0200

Hi,
I am not sure but are you suggesting that the IETF will define the
requirements, metric and quality assessment requirements and all proposed
codecs should provide the results and then the WG will choose the best codec
bases without discussing the codec itself. This is what I would call a
selection process (at least in ITU terms).
The problem is that the IETF process allows anyone to contribute to existing
work hopefully leading to a better the end result. 
What about the change control, does it stay with the original contributor or
can the IETF modify the codec based on input from other parties, which means
that the codec may change by the IETF anyhow. 
Thanks
Roni Even

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Robert Elz
> Sent: Wednesday, December 23, 2009 10:40 PM
> To: Tschofenig, Hannes (NSN - FI/Espoo)
> Cc: [email-masked]; [email-masked]
> Subject: Re: WG Review: Internet Wideband Audio Codec (codec)
> 
>     Date:        Wed, 23 Dec 2009 21:48:18 +0200
>     From:        "Tschofenig, Hannes (NSN - FI/Espoo)"
> <[email-masked]>
>     Message-ID:
> <[email-masked]>
> 
>   | That's something for the working group to figure out.
>   | My experience: things are typically more complicated than they
> initially
>   | look like.
> 
> Yes, of course, but the proposed charter goes to great lengths to point
> out why solutions from the first and third of the three categories of
> existing codecs are no good, but it more or less ignored the middle
> category - then, it seemed to me, more or less demanded that a new
> codec
> (or perhaps codecs) be developed.
> 
> That's the wrong approach, the emphasis should be on adopting something
> that exists, if at all possible, and only inventing something new if
> there really is no other choice.
> 
> That's why I'd prefer the charter to be revised with that in mind.
> 
>   | WG charters are also written for those who have not followed the
> history
>   | of the work very closely. These folks typically need a bit more
>   | background information.
> 
> Yes, but no-one needs that much ... (no need to delete all of that
> stuff about encumbered technology, just most of it)
> 
> kre
> 
> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf

---

From: Robert Elz <[email-masked]>
Date: Thu, 24 Dec 2009 16:47:16 +0700

Date:        Thu, 24 Dec 2009 08:50:30 +0200
    From:        "Roni Even" <[email-masked]>
    Message-ID:  <[email-masked]>

  | I am not sure but are you suggesting that the IETF will define the
  | requirements, metric and quality assessment requirements and all proposed
  | codecs should provide the results and then the WG will choose the best codec
  | bases without discussing the codec itself. This is what I would call a
  | selection process (at least in ITU terms).

The WG can decide how it wants to go about the process, I'd just prefer that
the charter not (effectively) rule out selection of something that already
exists with an assumption that something entirely new will be created.

  | The problem is that the IETF process allows anyone to contribute to existing
  | work hopefully leading to a better the end result.

Of course, but also be aware that there's no one definition of "better".
Something that can be defined quickly and used immediately might be much
better than something it takes 5 years to create and more to implement,
even if the invented one saves a little bandwidth or has better loss
recovery characteristics.

  | What about the change control, does it stay with the original contributor or
  | can the IETF modify the codec based on input from other parties, which means
  | that the codec may change by the IETF anyhow. 

The IETF will have change control over its protocol, of course, which may
cause it to diverge from that upon which it was originally based.  And yes,
everything changes with time.

kre

---

From: "Roni Even" <[email-masked]>
Date: Thu, 24 Dec 2009 15:56:17 +0200

Hi,
In line
Roni Even

> From: [email-masked] [mailto:[email-masked]]
> Sent: Thursday, December 24, 2009 11:47 AM
> To: Roni Even
> Cc: 'Tschofenig, Hannes (NSN - FI/Espoo)'; [email-masked];
> [email-masked]; [email-masked]

This is the IETF process for better or worse, I asked similar questions and
the response is that the IETF decide what is better is based on rough
consensus. 
BTW: my personal view is that your suggestion is in line with the process at
the ITU when doing codec selection, but there are people who prefer doing it
at the IETF using the IETF procedures for other reasons.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 24 Dec 2009 22:26:07 -0500

I don't think it is a very good idea to attempt this type of work in
the IETF. We have enough difficulty doing crypto algorithms and that
is an area where we have tens of people with decades worth of
expertise who pretty much mostly agree on the algorithms to use in any
case.

An unencumbered CODEC would be very useful, but any new CODEC that was
developed would be subject to attack by patent trolls. So the group
would be pretty much limited to reviewing existing technologies and
attempting to select one that is out of patent.

Looking at technologies that are out-of-patent or soon to be out of
patent, well DVD came out in 1995 and the patent licensing terms are
reasonably well defined. MP3 and AC3 are the existing industry
standards. If we know when the patents drop dead, I can't see how IETF
imprimatur is going to add or detract anything there. Its not as if
the IETF can stand behind the spec and say that it is definitively
unencumbered. So the most we are going to have is a document that
brings together all the relevant information and allows people to
quickly come to a degree of confidence that the technology will be
inencumbered on a certain date.

-- 
-- 
New Website: http://hallambaker.com/
View Quantum of Stupid podcasts, Tuesday and Thursday each week,
http://quantumofstupid.com/

---

From: Steve Underwood <[email-masked]>
Date: Fri, 25 Dec 2009 13:22:25 +0800

> inencumbered on a certain date.
So, you think the IETF should stop all work, because there's not a 
single thing they define that can't be subject to the same patent issues.

Steve

---

From: Koen Vos <[email-masked]>
Date: Thu, 24 Dec 2009 23:55:30 -0800

Quoting Phillip Hallam-Baker:
> MP3 and AC3 are the existing industry standards.

These codecs are rarely used for real-time communications, mostly  
because of their high bitrates/poor quality for voice signals.

> So the most we are going to have is a document that

Any difference with other standards?

koen.

---

From: Steve Underwood <[email-masked]>
Date: Fri, 25 Dec 2009 16:22:00 +0800

> because of their high bitrates/poor quality for voice signals.
I would have thought their latency was the key factor for interactive 
communication, which is the main focus here.

Regards,
Steve

---

From: Koen Vos <[email-masked]>
Date: Fri, 25 Dec 2009 00:52:42 -0800

Quoting Steve Underwood:

> I would have thought their latency was the key factor for  
> interactive communication

That's a factor too.

MP3 has 54 ms algorithmic delay (at 48 kHz sampling rate) - around 11  
ms more than G.718.

AC3 seems to have < 10 ms algorithmic delay.

koen.