Subject: [codec] Welcome to Greg

From: Cullen Jennings <[email-masked]>
Date: Mon, 13 Jul 2009 14:23:14 -0700

Greg Lebovitz has kindly volunteered to be the IAB shepherd for this  
BOF. Greg is on the IAB and has plenty of experience in "complicated"  
topics. I also believe that he brings a very neutral view to the topic  
of codecs. Just wanted to let folks know that Greg would be helping  
and tell him thanks.

Cullen

PS. Greg, I assume you will be sporting a large grey beard by IETF  
75 :-)

---

From: "Gregory M. Lebovitz" <[email-masked]>
Date: Tue, 14 Jul 2009 21:21:02 -0700

>of codecs. Just wanted to let folks know that Greg would be helping

Glad to join the effort. This will be a fun and challenging topic for 
the IETF, and I'm looking forward to both aspects. Thx for the intro Cullen.

>PS. Greg, I assume you will be sporting a large grey beard by IETF

A brown goatie, and that's my final offer. ;-)

---

From: Henry Sinnreich <[email-masked]>
Date: Wed, 15 Jul 2009 11:02:22 -0700

Just to make clear my company's position:

We're supportive of the efforts to arrive at a selection of a preferred codec for voice over the Internet, as proposed for the Voice Codec BOF.
To accommodate the broadest possible community, we support the selection of a voice codec which

 *   has a broad range of audio bandwidths, with no more than 30ms and preferably lower delay,
 *   can be implemented efficiently,
 *   is available for use without patent license fees or any other known IPR constraints,
 *   has an open source reference implementation,
 *   works over the Internet.

We think several voice codecs have been proposed which seem to meet those requirements, and hope that the working group can produce the standard for an Internet voice codec along these lines.

Henry Sinnreich
Adobe Systems, Inc.

---

From: Brian West <[email-masked]>
Date: Wed, 15 Jul 2009 13:21:18 -0500

Comments in line.

On Jul 15, 2009, at 1:02 PM, Henry Sinnreich wrote:

> has a broad range of audio bandwidths, with no more than 30ms and  
> preferably lower delay,
I can agree with this but it should be noted that higher ptimes are  
preferred in some cases... performance gains (network and CPU) of 30ms  
vs 60ms can be seen very clearly in our testing.  You have to think of  
the server side that has to scale the codec to 1000's of channels per  
instance.
> can be implemented efficiently,
Do you mean low CPU requirements?
> is available for use without patent license fees or any other known  
> IPR constraints,
I think this is a requirement also.
> has an open source reference implementation,
The reference implementation licensing should be as broad as possible  
in my opinion so its compatible with as many open source licenses as  
possible.
> works over the Internet.
> We think several voice codecs have been proposed which seem to meet  
> those requirements, and hope that the working group can produce the  
> standard for an Internet voice codec along these lines.
>
> Henry Sinnreich
> Adobe Systems, Inc.
>
> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec

---

From: Stephan Wenger <[email-masked]>
Date: Wed, 15 Jul 2009 11:36:21 -0700

Hi Henry,
I would object to any charter proposal that requires an ³open source
reference implementation², because it unfairly excludes from IETF¹s standard
process all those companies which, on principle, do not contribute to open
source. 
You may also wish to clarify the sentence ³is available for use without
patent license fees or any other known IPR constraints, ².  Isn¹t this what
you really want to say: ³a voice codec whose known IPR is available under
terms compatible with all major open source licenses²?
Regards,
Stephan

On 7/15/09 11:02 AM, "Henry Sinnreich" <[email-masked]> wrote:

> Just to make clear my company¹s position:
> 
> We¹re supportive of the efforts to arrive at a selection of a preferred codec
> for voice over the Internet, as proposed for the Voice Codec BOF.
>> We think several voice codecs have been proposed which seem to meet those
>> requirements, and hope that the working group can produce the standard for an
>> Internet voice codec along these lines.
>> 
>> Henry Sinnreich
>> Adobe Systems, Inc.
>> 
> 
>

---

From: "Slava Borilin" <[email-masked]>
Date: Wed, 15 Jul 2009 23:00:02 +0400

Stephan,

The goals (of WG and codec) should be driven by customer demand, not by
potential contributors - so Henry got the point that customers want to
have open-source, if feasible. If there are means to get open-source and
satisfy other goals of the codec, then it does not matter then "many did
not contribute because they do not do open source".

We should remove "open source" only in the case when it is NOT feasible
to achieve other codec performance targets by any open-source solutions
(current or to be developed).

best regards,

Slava Borilin

________________________________

From: [email-masked] [mailto:[email-masked]] On Behalf
Of Stephan Wenger
Sent: Wednesday, July 15, 2009 10:36 PM
To: Henry Sinnreich; [email-masked]
Subject: Re: [codec] useful for Internet applications with voice

Hi Henry,
I would object to any charter proposal that requires an "open source
reference implementation", because it unfairly excludes from IETF's
standard process all those companies which, on principle, do not
contribute to open source. 
You may also wish to clarify the sentence "is available for use without
patent license fees or any other known IPR constraints, ".  Isn't this
what you really want to say: "a voice codec whose known IPR is available
under terms compatible with all major open source licenses"?
Regards,
Stephan

We're supportive of the efforts to arrive at a selection of a preferred
codec for voice over the Internet, as proposed for the Voice Codec BOF. 
To accommodate the broadest possible community, we support the selection
of a voice codec which 

*	has a broad range of audio bandwidths, with no more than 30ms
and preferably lower delay, 
*	can be implemented efficiently, 
*	is available for use without patent license fees or any other
known IPR constraints, 
*	has an open source reference implementation, 
*	works over the Internet. 

	We think several voice codecs have been proposed which seem to
meet those requirements, and hope that the working group can produce the
standard for an Internet voice codec along these lines.
	
	Henry Sinnreich
	Adobe Systems, Inc.

________________________________

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 15 Jul 2009 15:12:39 -0400

A standard is not an implementation.  I know that reference 
implementations are often useful.  However, the IETF does not generally 
identify or mandate reference implementations.
We look for implementations.  We want interoperable implementations.
(In fact, there is an argument that a reference implementation may even 
undermine the quality of the specification, as we are much less likely 
to see two completely independent implementations from the 
specifications, which is our preferred way of judging the quality of our 
documents.)

I am quite sympathetic to the goal of having a standard that can be 
implemented by anyone who wants to, without payment or negotiation with 
anyone else.  (My opinion on the likelihood of the outcome using the 
IETF process has been stated clearly elsewhere.)

It is quite a leap from there to requiring the existence of an open 
source reference implementation.  If one exists, that's nice.  But I do 
not see why that should be a decision criterion for the working group. 
(Note that the presence of absence of such an implementation does not 
tell us anything about the actual IPR situation, so it does not help 
with the claimed concerns.)  Returning to my comment above, if the spec 
can not be interoperably implemented from the specification, then we 
need a better document before it becomes an RFC.  So that value of a 
"reference implementation" seems doubtful to me.

Yours,
Joel M. Halpern

Slava Borilin wrote:
> potential contributors – so Henry got the point that customers want to 
> satisfy other goals of the codec, then it does not matter then “many did 
> not contribute because they do not do open source”.
> 
>  
> 
> We should remove “open source” only in the case when it is NOT feasible 
> * From: * [email-masked] [mailto:[email-masked]] *On 
> Behalf Of *Stephan Wenger
> *Sent:* Wednesday, July 15, 2009 10:36 PM
> *To:* Henry Sinnreich; [email-masked]
> *Subject:* Re: [codec] useful for Internet applications with voice
> 
>  
> 
> Hi Henry,
> I would object to any charter proposal that requires an “open source 
> reference implementation”, because it unfairly excludes from IETF’s 
> You may also wish to clarify the sentence “is available for use without 
> patent license fees or any other known IPR constraints, ”.  Isn’t this 
> what you really want to say: “a voice codec whose known IPR is available 
> under terms compatible with all major open source licenses”?
> Regards,
> Stephan
> 
> 
> Just to make clear my company’s position:
> 
> We’re supportive of the efforts to arrive at a selection of a preferred 
> ------------------------------------------------------------------------
>

---

From: Henry Sinnreich <[email-masked]>
Date: Wed, 15 Jul 2009 12:21:53 -0700

Brian,

>Remember you have to also think about how this works on the server side.  And don't get me wrong I run at 20ms but its not feasible to run sub 10ms if you wish to scale on a server application.

There is a long argument about client-side implementations being faster than server-side ones.
But this another topic.

Henry

On 7/15/09 2:16 PM, "Brian West" <[email-masked]> wrote:

On Jul 15, 2009, at 2:07 PM, Henry Sinnreich wrote:

>has a broad range of audio bandwidths, with no more than 30ms and preferably lower delay,
>I can agree with this but it should be noted that higher ptimes are preferred in some cases...

(My individual contributor's hat on)

Codec delay less than 30 ms is still desirable in certain interactive scenarios, especially where the network delay is small (locality).

 *   Financial, stock and commodity trading desks,
 *   Games with voice,
 *   Digital living room and a heated discussion is going on :-)

fortunately the end user will not hear any perceived delay with anything less than about 100-200ms in the first place.   Using anything less than 10ms over the public internet is worthless... I have personally tried CELT at 1ms, 2ms and 5ms over the public internet.  You do not gain anything from doing this in fact you get very messed up audio unless you put a jitter buffer on the far end also which then again more delay added.  The implementation we did in FreeSWITCH with CELT can do 2,4,6,8 and 10ms but in reality anything below 10 doesn't work out well without a jitter buffer over the public internet.  Local network we did 1ms with no problem.

I will argue that less than 10ms is a waste of bandwidth, CPU and context switches.  You're copying sub 10ms chunks of audio in and out of kernel space and that will never scale.  Remember you have to also think about how this works on the server side.  And don't get me wrong I run at 20ms but its not feasible to run sub 10ms if you wish to scale on a server application.

/b

Any extra delay may add irritation to a discussion.
This should also count for conferencing/telepresence.
Besides, one of the proposed codecs has <10 ms delay.

Henry

---

From: Brian West <[email-masked]>
Date: Wed, 15 Jul 2009 14:16:40 -0500

On Jul 15, 2009, at 2:07 PM, Henry Sinnreich wrote:

> preferred in some cases...
>
> (My individual contributor’s hat on)
>
> Codec delay less than 30 ms is still desirable in certain  
> interactive scenarios, especially where the network delay is small  
> (locality).
> Financial, stock and commodity trading desks,
> Games with voice,
> Digital living room and a heated discussion is going on :-)
fortunately the end user will not hear any perceived delay with  
anything less than about 100-200ms in the first place.   Using  
anything less than 10ms over the public internet is worthless... I  
have personally tried CELT at 1ms, 2ms and 5ms over the public  
internet.  You do not gain anything from doing this in fact you get  
very messed up audio unless you put a jitter buffer on the far end  
also which then again more delay added.  The implementation we did in  
FreeSWITCH with CELT can do 2,4,6,8 and 10ms but in reality anything  
below 10 doesn't work out well without a jitter buffer over the public  
internet.  Local network we did 1ms with no problem.

I will argue that less than 10ms is a waste of bandwidth, CPU and  
context switches.  You're copying sub 10ms chunks of audio in and out  
of kernel space and that will never scale.  Remember you have to also  
think about how this works on the server side.  And don't get me wrong  
I run at 20ms but its not feasible to run sub 10ms if you wish to  
scale on a server application.

/b

---

From: Stephan Wenger <[email-masked]>
Date: Wed, 15 Jul 2009 12:43:39 -0700

Hi Henry,
Antitrust is all about business decisions, albeit the wrong ones...
Stephan

On 7/15/09 12:14 PM, "Henry Sinnreich" <[email-masked]> wrote:

>> reference implementation²,
>> >because it unfairly excludes from IETF¹s standard process all those
>> companies which, on principle, do not contribute to open source.
>  
> (my individual contributor hat on)
> This is their business decision.
> Other companies chose to support and contribute to open source; also a
> business decision.
> 
> The real value from an engineering point of view is that written code is so
> much more precise and useful than endless verbiage in documents and mailing
> lists IMO. If in doubt, let¹s count our emails today on various WG lists...
> 
> Henry
> 
> 
> On 7/15/09 1:36 PM, "Stephan Wenger" <[email-masked]> wrote:
> 
>> Hi Henry,
>>> We¹re supportive of the efforts to arrive at a selection of a preferred
>>>> requirements, and hope that the working group can produce the standard for
>>>> an Internet voice codec along these lines.
>>>> 
>>>> Henry Sinnreich
>>>> Adobe Systems, Inc.
>>>> 
>>> 
>>>

---

From: stephen botzko <[email-masked]>
Date: Wed, 15 Jul 2009 15:46:00 -0400

I'm with Stephan on this.

If working groups are allowed to add constraints that prevent reasonable
companies currently involved in the IETF from participating in the group,
then that is a HUGE problem.

Participation has to be inclusive.

Stephen Botzko
Polycom

On Wed, Jul 15, 2009 at 3:00 PM, Slava Borilin <[email-masked]>wrote:

> potential contributors – so Henry got the point that customers want to have
> open-source, if feasible. If there are means to get open-source and satisfy
> other goals of the codec, then it does not matter then “many did not
> contribute because they do not do open source”.
>
>
>
> We should remove “open source” only in the case when it is NOT feasible to
> achieve other codec performance targets by any open-source solutions
>   ------------------------------
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Stephan Wenger
> reference implementation”, because it unfairly excludes from IETF’s standard
> patent license fees or any other known IPR constraints, ”.  Isn’t this what
> you really want to say: “a voice codec whose known IPR is available under
> terms compatible with all major open source licenses”?
> Regards,
> Stephan
>
>
>    - has a broad range of audio bandwidths, with no more than 30ms and
>    preferably lower delay,
>    - can be implemented efficiently,
>    - is available for use without patent license fees or any other known
>    IPR constraints,
>    - has an open source reference implementation,
>    - works over the Internet.
>
>
>
>  ------------------------------
>

---

From: Stephan Wenger <[email-masked]>
Date: Wed, 15 Jul 2009 12:50:19 -0700

Hi Henry,

I think I need to row back a bit regarding the open source license stuff
(and advance my bigger case at the same time).  Since just recently (RFC
5378), the RFC  editor attaches a BSD-style license (undoubtly an open
source license) to any piece of code that appears in an IETF RFC.  This is
known to any contributor, and not a WG-specific.  The main application for
this, and, based on my recollection, the intention of the IPR WG at the
time, has been to allow open source projects to use small code snippets of
code that hardly could contain a patent right.  But that¹s beside the point,
I fear.  Now, BSD-style licenses are believed at least by some to include an
implicit grant to patent rights owned by the Contributor of the code.  Let
me just assume that these folks are correct.  The result could well be that
any major contribution towards an RFC whose normative content  lies in the
reference code would include the implicit grant of patent rights.

In this light, I cannot reasonably hold up my previous objection against a
principle of requiring an open-source licensed implementation, and argue at
the same time (as I have before) that specifying a bit-exact codec is best
done through source code.  At the same time, I don¹t believe that we are on
safe ground when excluding all those folks that are not prepared to
contribute to a reference implementation, but are prepared to contribute to
the IETF¹s standardization process as it stands.

What it boils down to (for me), once again, is that the IETF is the wrong
organization for this work.

Regards,
Stephan

P.s.: (I have no idea what would happen in a courtroom if one would
contribute to something made available, as part of an IETF RFC, under a BSD
style license and, at the same time, explicitly announces that patent rights
are available only under RAND terms.  But that¹s a side discussion not
necessarily appropriate for this forum.)

---

From: "Slava Borilin" <[email-masked]>
Date: Wed, 15 Jul 2009 23:48:01 +0400

Any company can make a business decision to participate in open-source-
nobody but them limit their freedom here

best regards,

Slava Borilin

From: stephen botzko [mailto:[email-masked]] 
Sent: Wednesday, July 15, 2009 11:46 PM
To: Slava Borilin
Cc: Stephan Wenger; Henry Sinnreich; [email-masked]
Subject: Re: [codec] useful for Internet applications with voice

If working groups are allowed to add constraints that prevent reasonable
companies currently involved in the IETF from participating in the
group, then that is a HUGE problem.

Stephen Botzko
Polycom

On Wed, Jul 15, 2009 at 3:00 PM, Slava Borilin <[email-masked]>
wrote:

Stephan,

best regards,

Slava Borilin

Hi Henry,

	
	Henry Sinnreich
	Adobe Systems, Inc.

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 15 Jul 2009 13:45:36 -0600

> contribute to open source.

Your argument is false -- the existence of an open-source reference
implementation in no way excludes or prevents the existence of other
implementations, whether open-source or closed-source.

Your conclusion, however, might be valid, along the lines of what Joel
Halpern has posted.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Brian West <[email-masked]>
Date: Wed, 15 Jul 2009 14:19:57 -0500

Also I think we are confusing coding delay vs ptime in the packet on  
the wire.

/b

---

From: Henry Sinnreich <[email-masked]>
Date: Wed, 15 Jul 2009 12:07:21 -0700

Henry

On 7/15/09 1:21 PM, "Brian West" <[email-masked]> wrote:

Comments in line.

I can agree with this but it should be noted that higher ptimes are preferred in some cases... performance gains (network and CPU) of 30ms vs 60ms can be seen very clearly in our testing.  You have to think of the server side that has to scale the codec to 1000's of channels per instance.

The reference implementation licensing should be as broad as possible in my opinion so its compatible with as many open source licenses as possible.

 Henry Sinnreich
 Adobe Systems, Inc.

---

From: Gregory Maxwell <[email-masked]>
Date: Wed, 15 Jul 2009 13:11:31 -0700

Brian West [[email-masked]] wrote:
[snip]
> fortunately the end user will not hear any perceived delay with anything less than about 100-200ms in the first place.

It depends on your application.  100-200ms is noticeable in pure speech applications and is getting into the realm where active echo cancellation is required (with computation and quality implications).

For interactive musical applications, per the best research available to me it looks like 25ms total system one way delay is the bound of perception (https://www.itm.uni-luebeck.de/users/carot/TMT08.pdf). This meshes up well with existing practices regarding seating distances and use of a conductor in orchestras, so I believe it to be accurate.

With a 5ms frame size codec you can achieve 25ms total one way delay (hardware, drivers, network, jitter-buffers, etc) over a public regional network. 10ms very likely not, and 20ms is right out.

Once you've admitted that there exists a desirable delay limit every ms shaved off the codec algorithmic delay makes that limit easier to achieve. If nothing else shaving a millisecond off the codec allows increases network diameter for a given target delay by about 200km. 

> Using anything less than 10ms over the public internet is worthless... I have personally tried CELT at 1ms, 2ms and 5ms 

For strict telephony applications, probably. Wideband audio over the internet is more than telephony.  

It would also be a mistake to assume that the Internet will continue to experience the same absolute levels of jitter.  Low delay / low jitter is already a market differentiator for VPN services, the routing hardware can now deliver very low delay and jitter, and some improvement is guaranteed from decreased serialization delays as core link speeds increase. 

> over the public internet. You do not gain anything from doing this in fact you get very messed up audio unless you put a jitter buffer on the far end also which then again more delay added.

You should have a jitter buffer for any real-time application. If you don't ... well.

Since packet based codecs operate with frame-size granularity a smaller frame size can lead to delay which is closer to jitter limited, or alternatively can achieve the same delay with lower jitter induced losses.

> The implementation we did in FreeSWITCH with CELT can do 2,4,6,8 and 10ms but in reality anything below 10 doesn't work out well without a jitter buffer over the public internet. Local network we did 1ms with no problem.

Interesting, when I last looked at the FreeSWITCH code all that was there was 10ms, and there were a lot of assumptions around Nx10ms packetization, it didn't seem easy to change.

I think it would be more accurate to say that below 10ms is not very useful if one of the end-points is windows, since currently windows's software mixer forces applications to a 10ms frame rate. Even on other platforms some care is required to make sure that system software isn't really making your delay >>10ms when you think you're operating at low delay. 

> I will argue that less than 10ms is a waste of bandwidth, CPU and context switches. You're copying sub 10ms chunks of audio in and out of kernel space and that will never scale.

A codec mandating 5ms intervals wouldn't make sense for the great number of applications where delay that low isn't relevant, but it does not follow from that that it isn't desirable for the applications where lower delay is relevant.  Different things have different requirements. 

A modern desktop system can handle conferencing a full orchestra's worth of 5ms users. Many applications strictly are peer to peer. Etc.

Operating codecs with low delay has consequences, if nothing else IP+UDP+RTP overhead can become rather insane,  but low delay does have advantages. This is a trade off like any other parameter, and different applications will weigh the different factors in different ways.  This is why frame size (and thus delay) is parametrizable in CELT (At 48kHz from ~2-20ms, now).

I don't personally think that the working group should require CELT's level of delay flexibility or delays as low as 2ms, but delay is a real consideration. All other things equal I think there would be a clear preference for a codec with 20ms algorithmic delay over something with 40ms of delay as well as 10ms over 20ms even for straight telephony applications. Below that, lower delay likely becomes undesirable for straight telephony but can still be interesting for other applications.

---

From: "Christian Hoene" <[email-masked]>
Date: Wed, 15 Jul 2009 22:14:33 +0200

I had the spam filter. My last try…

http://tinyurl.com/codecbof

--------------------------------------------------------
Dr.-Ing. Christian Hoene
Interactive Communication Systems (ICS), University of Tübingen
Sand 13, 72076 Tübingen, Germany, Phone +49 7071 2970532

 <http://www.net.uni-tuebingen.de/> http://www.net.uni-tuebingen.de/

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Brian West
Sent: Wednesday, July 15, 2009 9:20 PM
To: Henry Sinnreich
Cc: [email-masked]
Subject: Re: [codec] useful for Internet applications with voice

Also I think we are confusing coding delay vs ptime in the packet on the
wire.

/b

Codec delay less than 30 ms is still desirable in certain interactive
scenarios, especially where the network delay is small (locality).

Henry

---

From: "Christian Hoene" <[email-masked]>
Date: Wed, 15 Jul 2009 22:15:28 +0200

I hate the spam filter. My last try…

 

/b

Henry

---

From: Christian Hoene <[email-masked]>
Date: Wed, 15 Jul 2009 20:20:32 +0200

I have read again all the millions of emails that have been send on this
mailing list during the last weeks.

To avoid that arguments are repeated and to have a summary of the
discussions, I have compiled the following few web pages:
https://groupware1.cs.uni-tuebingen.de/wiki/codec_bof/index.php/Main_Page
It is not all 100% neutral (of course), but feel free to correct the
Wiki articles or to add comments.

Christian

---

From: Henry Sinnreich <[email-masked]>
Date: Wed, 15 Jul 2009 12:14:17 -0700

>I would object to any charter proposal that requires an "open source reference implementation",
>because it unfairly excludes from IETF's standard process all those companies which, on principle, do not contribute to open source.

(my individual contributor hat on)
This is their business decision.
Other companies chose to support and contribute to open source; also a business decision.

The real value from an engineering point of view is that written code is so much more precise and useful than endless verbiage in documents and mailing lists IMO. If in doubt, let's count our emails today on various WG lists...

Henry

Hi Henry,
I would object to any charter proposal that requires an "open source reference implementation", because it unfairly excludes from IETF's standard process all those companies which, on principle, do not contribute to open source.
You may also wish to clarify the sentence "is available for use without patent license fees or any other known IPR constraints, ".  Isn't this what you really want to say: "a voice codec whose known IPR is available under terms compatible with all major open source licenses"?
Regards,
Stephan

Henry Sinnreich
Adobe Systems, Inc.