Subject: Re: [codec] Codec BOF approved, much work needed

From: "Ingemar Johansson S" <[email-masked]>
Date: Tue, 23 Jun 2009 19:02:51 +0200

Hi

I support Stephanes opinion on this matter, I fail to see how this WG (if formed) would add any substantial value. The proposed work clearly overlaps with already existing work in other SDOs and I see a big risk with this. Is the IETF willing to take the risk of stepping into other SDOs areas?, I see this not only as a RAI issue. This should be discussed at the BoF.
Also, I see that the main driver behind this WG is the strive for a royalty free codec. Reality has however shown that this is much more complicated to achive than what is believed in the beginning, the BoF should discuss this issue as well. 
I would like the BoF chair(s) to provide with ample time to discuss if this WG should actually be formed, to be honest I don't really see a need to discuss codec alternatives at the BoF. We should formulate the problem first, a solution comes later (if needed).

Regards
Ingemar Johansson
 
>ITU-T and 3GPP have already specified a wide portfolio of speech & audio codecs including Royalty Free codecs >(for narrowband, wideband, superwideband...) with purpose to answer the needs of all conversational services >(including new multimedia services over IP) and ensure end to end interoperability. Many of these codecs are 
>already deployed and successfully used to answer the requirements of most of all voice services all over the 
>world (over Circuit Switched, IP and Internet networks...). New extensions of these codecs have been or are 
>being standardized to extend the quality towards very high fullband/stereo quality.

>With respect to the end to end interoperability issue, this codec portfolio is even already too large 
>considering the need to reduce networks additional costs and quality degradations due to transcoding 
>operations.

>Adding new IETF standardized codecs to this (with probably little or no coordination with other SDOs) would 
>make no sense: those codecs would obviously compete with the ones from ITU-T or 3GPP and any other legacy 
>codecs already widely deployed which would confuse the market, degrade the overall end to end voice&audio 
>interoperability and quality and increase the transcoding costs in networks. 

>Furthermore, IETF is not the place where sufficient codec expertise is gathered to standardize state of the 
>art codecs (as in Technical Groups SA4 of 3GPP or WP3/SG16 of ITU-T) and duplicating the works within multiple >Standard Bodies would not help with creating quality codecs; this concerns not only the expertise and works 
>needed for codec design but also all what is related to voice & audio quality assessment.

>Best regards

>Stéphane Proust

---

From: Henry Sinnreich <[email-masked]>
Date: Tue, 23 Jun 2009 10:23:24 -0700

Sorry to confess, it is now over 10 years when we had similar discussions about forming a SIP WG.
H.323 was doing good and the ITU-T took care of multimedia communications.

Henry

On 6/23/09 12:02 PM, "Ingemar Johansson S" <[email-masked]> wrote:

Hi

Regards
Ingemar Johansson

>Stéphane Proust

_______________________________________________
codec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/codec

---

From: Jean-Marc Valin <[email-masked]>
Date: Tue, 23 Jun 2009 14:27:36 -0400

Hi,

Ingemar Johansson S wrote:
> I support Stephanes opinion on this matter, I fail to see how this WG
> (if formed) would add any substantial value. The proposed work clearly
> overlaps with already existing work in other SDOs and I see a big risk
> with this. Is the IETF willing to take the risk of stepping into other
> SDOs areas?, 

Care to elaborate on the risks involved in stepping into other SDO areas? I 
wrote Speex in 2002 and have yet to see the ITU police knocking on my door ;-)

Seriously, we would be standardising something different from what the 
ITU-T and MPEG currently standardise.

> I see this not only as a RAI issue. This should be
> discussed at the BoF. Also, I see that the main driver behind this WG is
> the strive for a royalty free codec. 

> Reality has however shown that this
> is much more complicated to achive than what is believed in the
> beginning,

Again, I'd be interested if you could elaborate on this. The CELT codec is 
Xiph.Org's 5th codec (4th audio codec) and it has worked well so far. It's 
not a trivial task to do, but then patents are a reality for all software, 
not just codecs.

Cheers,

	Jean-Marc

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Wed, 24 Jun 2009 09:35:35 +0200

Hi

Comments inline 

Regards
Ingemar

> -----Original Message-----
> From: Jean-Marc Valin [mailto:[email-masked]] 
> Sent: den 23 juni 2009 20:28
> To: Ingemar Johansson S
> Cc: [email-masked]; [email-masked]
> Subject: Re: [codec] Codec BOF approved, much work needed
> 
> Hi,
> 
> Care to elaborate on the risks involved in stepping into 
> other SDO areas? I wrote Speex in 2002 and have yet to see 
> the ITU police knocking on my door ;-)
IMHO IETF relies on a good relationship with other SDOs, as an example
ITU and IETF has had somewhat conflicting interests in the
standardization of MPLS, today I believe an agreement has been reached
where both parties are satisfied (I hope), I am not an MPLS expert so I
must leave the details to others. My point is that if we decide to step
into other SDOs areas so clearly as this BoF suggests, the benefits must
clearly outweight the risk that work in other IETF areas is disturbed.
This should be discussed at the BoF.

> 
> Seriously, we would be standardising something different from 
> what the ITU-T and MPEG currently standardise.
I don't agree, the EVS codec being specified in 3GPP clearly adresses
packet switched access at various bitrates and sampling rates. Also,
earlier work has shown that it is perfectly OK to tailor excisting
codecs originally designed for circuit-switched access to work well in a
packet-switched environment, see for instance
http://www.ind.rwth-aachen.de/fileadmin/publications/mertz03.pdf (if you
have access to the IEEE library I recommend a look at reference [3]).

> 
> > I see this not only as a RAI issue. This should be discussed at the 
> > BoF. Also, I see that the main driver behind this WG is the 
> strive for 
> > a royalty free codec.
> 
> 
> Again, I'd be interested if you could elaborate on this. The 
> CELT codec is Xiph.Org's 5th codec (4th audio codec) and it 
> has worked well so far. It's not a trivial task to do, but 
> then patents are a reality for all software, not just codecs.
OK... With H.264 the original intention was to make the baseline
royaly-free, this however failed in the end. What makes you believe that
IETF would work better here ?. The IPR situatuion has already been
discussed by people more skilled in that area than me so I wont
elaborate on this part. 
In addition, reading your comment above, I don't really understand why
you need IETF for the codec standardization?, why not just standardize
your codecs in Xiph.org and propose a payload format in AVT?. This gives
me the impression that the IETF involvement is just rubberstamping.

---

From: "Christian Hoene" <[email-masked]>
Date: Wed, 24 Jun 2009 11:21:04 +0200

Dear Ingemar Johannsson,

> My point is that if we decide to step
> ... the EVS codec being specified in 3GPP clearly adresses

To my opinion, it is the greatest mistake of the traditional codec designs
that they thought "Let's make a codec for circuit switched networks. It will
work anyhow with packet sizes networks." In the days of NGN  we do not need
any support for circuit switched networks anymore. Why shall a codec be
designed for a networking technology, which losses its relevance and
importance at a high speed?

One clear benefit of this BoF is that codec experts and Internet guys can
come together to discuss on a codec intended only for the Internet.

In my opinion, the requirements between a circuit switched codec design and
a codec for the Internet differs, for example
- The Internet codec should be royalty free because also most of the other
Internet protocols are free of charge, too. It is a matter of fairness...
- It should support packet loss concealment because it happens do often in
the Internet. On the other hand, bit errors are quite seldom.
- This BoF should think about a jitter compensation including modern time
stretching and time concealment algorithms. Jitter is the norm not a
"special" case. Codec designer should not ignore it.
- The Internet codec should take advantage of a feedback loop for variable
rate/frame control in order to allow a wide range of operation regardless of
the current available bandwidth. Congestion control should a must for all
Internet applications. BTW equally important to coding rate is the frame
rate of a codec. All efficient ITU codecs that I am aware go, to not support
variable frame rates.
- It is fine if the internet codec has not perfect but fairly well audio
quality. Most of the bandwidth is wasted for Internet packet headers anyhow.
Does it make sense to optimize the coding rate, when Ethernet, IP, UDP, and
RTP protocols are wasting bandwidth without any concerns?
- Also, if no royalties and big money are involved, it does not make any
difference, which of multiple nearly equally well designed codecs is
selected. If the selection process is not as perfect as in ITU, a lot of
money can be saved, which otherwise would be spend on professional codec
testing.
- Also, ITU, MPEG and 3GPP has failed yet to develop a codec supporting
ultra low latency important for new kind of applications like ensemble
performances over the net..

I am not a codec experts, but all these all requirements are quite different
from the approach that the ITU follows. Also, the stockholders in the IETF
are different. IETFs contribution are driven by individuals not by larger
companies like in the ITU. Thus, hopefully the interests of users and the
entire society will be considered to larger extend.

> me the impression that the IETF involvement is just rubberstamping.

If Jean-Marc wants to rubberstamp his codec, he has the option of submitting
an independent RFC. Then, there is not be a need of a BoF... I do not
believe this is the case.

Greetings
 Christian Hoene

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Wed, 24 Jun 2009 02:36:31 -0700

> performances over the net..

I would like to add support for layered coding in the codec in the
list of requirements.  This is the kind of requirement that is AFAIK
specific to the Internet (multicast) and so it is an argument in
favor of developing the codec in the IETF.

(we did a bit of work at my previous employer to add layered coding
to Speex, but that would be great to have the codec designed for
layered coding from the beginning).

-- 
Marc Petit-Huguenin
Home: [email-masked]
Work: [email-masked]

---

From: "Roni Even" <[email-masked]>
Date: Wed, 24 Jun 2009 13:08:30 +0300

Hi,
The list you gave is not a codec requirement but a system requirement
including all layers, this is not codec work, and maybe it shows why the
IETF does not have currently the expertise to do such work. 
A small note, there is a contradiction between ensemble performance and low
quality speech codec and by the way this application is already being used
in distance video conferencing applications. It also require good support
for open microphone.

I will not go through your list but the information you gave about ITU codec
not designed for the Internet but only for circuit switch is not correct
please check the requirements that were used and the tests that were done
for ITU codecs including support for packet loss.

As for the comment " If Jean-Marc wants to rubberstamp his codec, he has the
option of submitting an independent RFC". I think this is not true since my
understanding is that the IETF currently will not look at such work even for
informational RFC since it felt that there is no knowledge at the IETF to
evaluate it.

Roni Even

> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Christian Hoene
> Sent: Wednesday, June 24, 2009 12:21 PM
> To: 'Ingemar Johansson S'; 'Jean-Marc Valin'

---

From: Herve Taddei <[email-masked]>
Date: Wed, 24 Jun 2009 12:19:10 +0200

Dear Christian,

> One clear benefit of this BoF is that codec experts and Internet guys 

> can

> come together to discuss on a codec intended only for the Internet.

There has been quite a bit of concerns raised that this is not going to be
the case. I would be interested also in understanding what you call "the
Internet". Is that a network, a protocol, an application? ITU-T worked on
codec for VoIP, 3GPP is looking to PS networks codecs and we hear companies
speaking about all IP network. Is that what you call Internet? Then there is
nothing really different to what is looking out by other SDOs.

> Internet protocols are free of charge, too. It is a matter of fairness...

We don't speak about protocols here but codecs. 

> variable frame rates.

What about scalable codecs defined in ITU-T as G.718, G.729.1 or even G.722?
They can well address congestion control. And I imagined you heard about
G.722.2 (aka AMR-WB) that is an adaptive multi rate codec whose bitrate can
be changed during the communication.

> quality. Most of the bandwidth is wasted for Internet packet headers
anyhow.

Header compression can be used to reduce overhead. What is fairly well audio
quality?

> RTP protocols are wasting bandwidth without any concerns?

Then why not transmitting non compressed audio signal?

> testing.

The money you save on one side will be lost on the other side when you will
have to install and discuss mechanisms to interoperate with existing codecs.

> performances over the net..

G.711, G.711.1, G.722? At least those are low latency to my opinion. 

How do you define ultra low latency? BTW, for VoIP, the lower the delay the
higher the overhead. That is why many VoIP codecs are rather looking to
10/20 and even 30 ms frame size.

> I am not a codec experts, 

I think that is why many people would prefer that codec work is conducted
where audio experts are, in other SDOs as ITU-T and 3GPP.

> companies like in the ITU. Thus, hopefully the interests of users and the

BTW, you could give a look to the ITU-T list of companies, there are some
very small companies registered, sometime with less than 10 employees. So
this statement is not really correct.

Best regards

Herve

---

From: "Gibbs Jon-CJG019" <[email-masked]>
Date: Wed, 24 Jun 2009 11:24:24 +0100

Dear All,
There seem to be a few misconceptions about the codec development work
of the ITU-T which I feel ought to be pointed out...

> To my opinion, it is the greatest mistake of the traditional codec 
> designs that they thought "Let's make a codec for circuit switched 
> networks. It will work anyhow with packet sizes networks." 

> I would like to add support for layered coding in the codec in the
list of requirements.  This > is the kind of requirement that is AFAIK
specific to the Internet (multicast) and so it is an > argument in favor
of developing the codec in the IETF.

Please take a look at the features of the relatively recent ITU-T Codecs
and, in particular, Rec ITU-T G.718
(http://www.itu.int/rec/T-REC-G.718/en &
http://en.wikipedia.org/wiki/G.718). G.718 is optimized for packet
transmission and highly resistant to packet loss. It provides a layered
solution starting at 8kb/s for wideband speech transmission and there
are also superwideband and stereo enhancement layers in development.

So, I think that it's fair to say that the ITU-T does develop codecs
which have relevance to the work of IETF. 

Regards
Jon Gibbs

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Marc Petit-Huguenin
Sent: 24 June 2009 10:37
To: Christian Hoene
Cc: 'Ingemar Johansson S'; [email-masked]; [email-masked]
Subject: Re: [codec] Codec BOF approved, much work needed

Christian Hoene wrote:
> Dear Ingemar Johannsson,
> 
>> My point is that if we decide to step into other SDOs areas so 
>> clearly as this BoF suggests, the benefits must clearly outweight the

>> risk that work in other IETF areas is disturbed.
>> ... the EVS codec being specified in 3GPP clearly adresses packet 
>> switched access at various bitrates and sampling rates. Also, earlier

>> work has shown that it is perfectly OK to tailor excisting codecs 
>> originally designed for circuit-switched access to work well in a 
>> http://www.ind.rwth-aachen.de/fileadmin/publications/mertz03.pdf (if 
>> you have access to the IEEE library I recommend a look at reference
[3]).
> 
> To my opinion, it is the greatest mistake of the traditional codec 
> designs that they thought "Let's make a codec for circuit switched 
> networks. It will work anyhow with packet sizes networks." In the days

> of NGN  we do not need any support for circuit switched networks 
> anymore. Why shall a codec be designed for a networking technology, 
> which losses its relevance and importance at a high speed?
> 
> can come together to discuss on a codec intended only for the
Internet.
> 
> In my opinion, the requirements between a circuit switched codec 
> design and a codec for the Internet differs, for example
> - The Internet codec should be royalty free because also most of the 
> other Internet protocols are free of charge, too. It is a matter of
fairness...
> - It should support packet loss concealment because it happens do 
> often in the Internet. On the other hand, bit errors are quite seldom.
> - This BoF should think about a jitter compensation including modern 
> time stretching and time concealment algorithms. Jitter is the norm 
> not a "special" case. Codec designer should not ignore it.
> - The Internet codec should take advantage of a feedback loop for 
> variable rate/frame control in order to allow a wide range of 
> operation regardless of the current available bandwidth. Congestion 
> control should a must for all Internet applications. BTW equally 
> important to coding rate is the frame rate of a codec. All efficient 
> ITU codecs that I am aware go, to not support variable frame rates.
> - It is fine if the internet codec has not perfect but fairly well 
> audio quality. Most of the bandwidth is wasted for Internet packet
headers anyhow.
> Does it make sense to optimize the coding rate, when Ethernet, IP, 
> UDP, and RTP protocols are wasting bandwidth without any concerns?
> - Also, if no royalties and big money are involved, it does not make 
> any difference, which of multiple nearly equally well designed codecs 
> is selected. If the selection process is not as perfect as in ITU, a 
> lot of money can be saved, which otherwise would be spend on 
> professional codec testing.
> - Also, ITU, MPEG and 3GPP has failed yet to develop a codec 
> supporting ultra low latency important for new kind of applications 
> like ensemble performances over the net..

I would like to add support for layered coding in the codec in the list
of requirements.  This is the kind of requirement that is AFAIK specific
to the Internet (multicast) and so it is an argument in favor of
developing the codec in the IETF.

(we did a bit of work at my previous employer to add layered coding to
Speex, but that would be great to have the codec designed for layered
coding from the beginning).

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Wed, 24 Jun 2009 04:21:53 -0700

> are also superwideband and stereo enhancement layers in development.

I missed this draft, but yes, this is exactly what I am talking
about. All RTP payload (and hopefully all future IETF codecs) should
be designed like this.

Too bad there is no open source implementation and an IPR statement
on this draft.

---

From: "Gibbs Jon-CJG019" <[email-masked]>
Date: Wed, 24 Jun 2009 12:50:36 +0100

Marc Petit-Huguenin wrote:
>Too bad there is no open source implementation and an IPR statement on
this draft.

FYI,
The fixed point source code is freely available here
(http://www.itu.int/rec/T-REC-G.718/en) already, all 60,000 lines of it,
and the floating point code is coming soon as you'll see.

The declared IPR patent and copyright statements may be found here
http://www.itu.int/ipr/IPRSearch.aspx - just enter "G.718" for
Recommendation No.

...but it seems that you've already decided that it's not for you, or
the IETF...

...and so you want to re-invent the wheel...

...paradoxically to save some money... 

..."Too bad", indeed!

Jon Gibbs

-----Original Message-----
From: Marc Petit-Huguenin [mailto:[email-masked]] 
Sent: 24 June 2009 12:22
To: Gibbs Jon-CJG019
Cc: [email-masked]
Subject: Re: [codec] Codec BOF approved, much work needed

> specific to the Internet (multicast) and so it is an > argument in 
> Please take a look at the features of the relatively recent ITU-T 
> Codecs and, in particular, Rec ITU-T G.718 
> transmission and highly resistant to packet loss. It provides a 
> layered solution starting at 8kb/s for wideband speech transmission 
> and there are also superwideband and stereo enhancement layers in
development.

I missed this draft, but yes, this is exactly what I am talking about.
All RTP payload (and hopefully all future IETF codecs) should be
designed like this.

Too bad there is no open source implementation and an IPR statement on
this draft.

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 24 Jun 2009 08:27:24 -0400

> and the floating point code is coming soon as you'll see.

"source code freely available" and "open source implementation" are
totally different things. I suggest you have a look at (first Google
result) http://www.opensource.org/docs/definition.php

> ...but it seems that you've already decided that it's not for you, or

It is actually the other way around. Someone has decided that G.718 was
not for us. I am not familiar with all the technical details of G.718,
but I have no problem imagining that it fits the telco's needs very well
-- even for IP transport. The problem is that its licensing makes it
impossible to use for many applications because of the patents it
involves. How good is a codec which you cannot use. If my VoIP
application has to be freely downloadable, or embedded in a web page as
a Java applet, or if I want to link it with open source software... then
a patent-encumbered codec cannot be legally used. What are you
suggesting we should do. So far, I see three possible options:
1) Break the law by infringing on patents
2) Use codecs on which patents are expired (G.711 and GSM-FR are still
popular)
3) Design a new codec that will solve the problem

I choose 3).

	Jean-Marc

---

From: "Christian Hoene" <[email-masked]>
Date: Wed, 24 Jun 2009 13:55:58 +0200

Hi Roni,

> IETF does not have currently the expertise to do such work.

But should the codec be aware of the surrounding system? Does it makes sense
to have a perfect codec, if the rest of the system is in bad shape or the
interfaces between system and codec are not defined rightly?

The last ten years of research have shown that a joint optimization of codec
and the rest of the system can yield high performance gains. For example, if
the time concealment, the controlling of the frame rate, or the amount of
FEC/redundancy are controlled jointly. To support these enhancements, the
old functionality split between codecs and the rest of the world cannot be
maintained.

> for ITU codecs including support for packet loss.

Please, Roni, convince me with technical arguments and references to
documents that I am wrong. Then--I promise--I will stop to discuss these
issues immediately.

Christian

---

From: stephen botzko <[email-masked]>
Date: Wed, 24 Jun 2009 08:55:24 -0400

No one is interested in circuit-switch codecs anymore.
The focus is wireless, IP networks, and distribution of TV and music.

Stephen Botzko
Polycom

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Wed, 24 Jun 2009 14:41:14 +0200

Hi

What stikes me is that there seems to be some kind of misconception that
other SDOs that do speech coding standardization still live in a
circuit-switched world where bit-errors are the key design constraint. 
This was maybe true 10 years ago but time changes and so does other
SDOs. 
But... speech codec standardization also has to recognize the fact that
today > 3 billion users are talking through legacy handset using circuit
switched technology where bit-errors are a key factor. 
We cannot assume that all these users will switch to VoIP all of a
sudden, making a call must be as simple as 1-2-3 even when the
underlying technology changes. 99.9999% of the users don't care about
the codec and transport, it is the quality, cost, ease of use and
reliability that counts. 
This may give people the idea that e.g 3GPP sticks to "old technology"
but nothing could be more wrong.

My comments to comments below, I believe I have made my point for now,
look forward to discuss this more at the BoF.

Regards
Ingemar

> From: Christian Hoene [mailto:[email-masked]] 
> Sent: den 24 juni 2009 11:21
> To: Ingemar Johansson S; 'Jean-Marc Valin'
> Cc: [email-masked]; [email-masked]
> Subject: RE: [codec] Codec BOF approved, much work needed
> 
> Dear Ingemar Johannsson,
> 
> > into other SDOs areas so clearly as this BoF suggests, the benefits 
> > must clearly outweight the risk that work in other IETF 
> areas is disturbed.
> > you have access to the IEEE library I recommend a look at 
> reference [3]).
> 
> To my opinion, it is the greatest mistake of the traditional 
> codec designs that they thought "Let's make a codec for 
> circuit switched networks. It will work anyhow with packet 
> sizes networks." In the days of NGN  we do not need any 
> support for circuit switched networks anymore. Why shall a 
> codec be designed for a networking technology, which losses 
> its relevance and importance at a high speed?
Wrong interpretation of work done in other SDOs, see above why.

> 
> One clear benefit of this BoF is that codec experts and 
> Internet guys can come together to discuss on a codec 
> intended only for the Internet.
Like said above, other SDOs does it already.

> 
> In my opinion, the requirements between a circuit switched 
> codec design and a codec for the Internet differs, for example
> - The Internet codec should be royalty free because also most 
> of the other Internet protocols are free of charge, too. It 
> is a matter of fairness...
The ambitions are fine but if you look at the other discussion on this
list there is raised a number of reasons why this may well fall flat in
the end.

> - It should support packet loss concealment because it 
> happens do often in the Internet. On the other hand, bit 
> errors are quite seldom.
This is fact is clearly recognized in other SDOs as well.

> - This BoF should think about a jitter compensation including 
> modern time stretching and time concealment algorithms. 
> Jitter is the norm not a "special" case. Codec designer 
> should not ignore it.
Also recognized by other SDOs.

> - The Internet codec should take advantage of a feedback loop 
> for variable rate/frame control in order to allow a wide 
> range of operation regardless of the current available 
> bandwidth. Congestion control should a must for all Internet 
> applications. BTW equally important to coding rate is the 
> frame rate of a codec. All efficient ITU codecs that I am 
> aware go, to not support variable frame rates.
This is also recognized by other SDOs, have a look for instance at 3GPP
TS26.114
http://www.3gpp.org/ftp/Specs/archive/26_series/26.114/26114-830.zip
The busy reader may want to read sections 9, 10 and annex C

> - It is fine if the internet codec has not perfect but fairly 
> well audio quality. Most of the bandwidth is wasted for 
> Internet packet headers anyhow.
> Does it make sense to optimize the coding rate, when 
> Ethernet, IP, UDP, and RTP protocols are wasting bandwidth 
> without any concerns?
Header compression can be used, which means that this argument is not
valid.

> - Also, if no royalties and big money are involved, it does 
> not make any difference, which of multiple nearly equally 
> well designed codecs is selected. If the selection process is 
> not as perfect as in ITU, a lot of money can be saved, which 
> otherwise would be spend on professional codec testing.
Codec testing is not only about selecting the best codec, it is also
about finding bugs before standards are frozen. Lack of testing likely
means contly bugfixes/patches later on. 

> supporting ultra low latency important for new kind of 
> applications like ensemble performances over the net..
Does not mean that this has not been considered.

> 
> I am not a codec experts, but all these all requirements are 
> quite different from the approach that the ITU follows. Also, 
> the stockholders in the IETF are different. IETFs 
> contribution are driven by individuals not by larger 
> companies like in the ITU. Thus, hopefully the interests of 
> users and the entire society will be considered to larger extend.
Please then feel free to dig more into the work done in e.g ITU and
3GPP, you will find out that lots of the technical issues that are
brought forward to motivate a Codec WG are already addressed in other
SDOs. The only one left is possibly the "royalty-free" motivation but
this is a very shaky argument IMHO.

> > your codecs in Xiph.org and propose a payload format in AVT?. This 
> > gives me the impression that the IETF involvement is just 
> rubberstamping.
> 
> If Jean-Marc wants to rubberstamp his codec, he has the 
> option of submitting an independent RFC. Then, there is not 
> be a need of a BoF... I do not believe this is the case.
> 
> Greetings
>  Christian Hoene
> 
>

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Wed, 24 Jun 2009 05:22:44 -0700

> ...but it seems that you've already decided that it's not for you, or

I do not decide for the IETF, only for me.

> ...paradoxically to save some money... 

No open source implementation is not the problem - I can write code,
it's just nice to have one.

A RAND IPR is definitively a problem (for me).

---

From: "Roni Even" <[email-masked]>
Date: Wed, 24 Jun 2009 15:59:26 +0300

What you describe is different scope than codec because some of it is codec
independent.
For ITU codecs look at John Gibbs email and at G.718
Roni

> Sent: Wednesday, June 24, 2009 2:56 PM
> To: 'Roni Even'
> Cc: [email-masked]
> > I will not go through your list but the information you gave about
> ITU
> codec

---

From: "Roni Even" <[email-masked]>
Date: Wed, 24 Jun 2009 16:04:48 +0300

Hi,

G.718 is a new codec and I am not aware that some entity decided that it is
not for us humans.
As for your third choice you forgot to say "hopefully" solve the problem
since like was discussed in separate thread you cannot guarantee it and you
will not provide any protection against it. 

Roni

> Of Jean-Marc Valin
> Sent: Wednesday, June 24, 2009 3:27 PM
> To: Gibbs Jon-CJG019
> Cc: [email-masked]
> on

---

From: "Gibbs Jon-CJG019" <[email-masked]>
Date: Wed, 24 Jun 2009 14:31:30 +0100

Thanks Roni - I couldn't have put it better myself.

Jon 

-----Original Message-----
From: Roni Even [mailto:[email-masked]] 
Sent: 24 June 2009 14:05
To: 'Jean-Marc Valin'; Gibbs Jon-CJG019
Cc: [email-masked]
Subject: RE: [codec] Codec BOF approved, much work needed

Hi,

G.718 is a new codec and I am not aware that some entity decided that it is not for us humans.
As for your third choice you forgot to say "hopefully" solve the problem since like was discussed in separate thread you cannot guarantee it and you will not provide any protection against it. 

Roni

> > ...but it seems that you've already decided that it's not for you, 
> > or the IETF...
> 
> It is actually the other way around. Someone has decided that G.718 
> was not for us. I am not familiar with all the technical details of 
> G.718, but I have no problem imagining that it fits the telco's needs 
> very well
> application has to be freely downloadable, or embedded in a web page 
> as a Java applet, or if I want to link it with open source software...
> then

---

From: Koen Vos <[email-masked]>
Date: Wed, 24 Jun 2009 12:36:06 -0700

> will not provide any protection against it.

Good point.  Wouldn't the IETF WG be especially useful for solving the  
problem with a larger degree of confidence?  We want to identify  
patents that may be infringed upon as early as possible - when we can  
still modify the codec without disrupting a large install base.   
Having more experts involved increases the likelihood of that.  As a  
bonus it will instill confidence in adopters of the codec that the IP  
risk is low.

koen.

---

From: Stephan Wenger <[email-masked]>
Date: Wed, 24 Jun 2009 21:40:08 -0400

> bonus it will instill confidence in adopters of the codec that the IP

Hi Koen,

I thought I would drop out of this discussion, but since this is something
where I think I can educate without necessarily being opinionated in the
creation of a Codec WG, I decided to write anyway.  FYI, I have followed the
IETF IPR policy process for many years, and influencing IPR policies in some
30 SDOs was my day job during my Nokia tenure.

The IETF's patent policy can be found in RFC3979.  Compared to the IPR
policies of some other SDOs, this is a reasonably accessible document, and I
encourage study.  Also very relevant are are the "Note Well" statement and
RFC 2026.  A number of other documents complete the framework; you can find
references to those, for example, here:
http://www.ietf.org/IESG/content/procdocs.html .

To summarize the parts relevant for my argumentation below in my own words,
one has to "participate" in the creation of a document in order to be
obligated to make IPR disclosures.  Common understanding is that if you go
to the mike during the WG meeting, or post something to a relevant mailing
list, or co-author an I-D, you have a disclosure obligation.  However, if
you sit in a meeting and/or are subscribed to a mailing list, and you are
careful not to influence the consensus process (and therefore do not
"participate"), you do not have a disclosure obligation.  See section 7 of
RFC 3978.  This situation, I can assure you, is very well known and
understood in companies who are actively playing in the IPR playground.

Now, if you were from one of these companies which think that they have
valuable IPR in this field to protect, would you participate?  Probably
not---even if you would wish so personally---because you may well be under
orders by the lawyers.

However, it is completely acceptable, by IETF policy and practice, to sit in
those meetings, take notes, perhaps improve the codec back home and file
applications of your improving inventions like crazy.  Those may well
"interpolate" the possible outcome.  You just must never say a thing.

Beyond that, while those who open their mouth do have a disclosure
obligation, the policy is silent about minimum licensing terms that need to
be offered.  There's not even a requirement for RAND terms---though those
(and non-assert terms) are most common in the IETF.  This is in contrast to,
for example, the ITU---where you do have to license your stuff under RAND
terms.  This doesn't help opens source, but---all in all---has worked well
enough in the rest of the industry.  In W3C, to go to the other end of the
spectrum, when you are a member of a WG, you must license under Royalty-Free
(though not necessarily open-source compatible) terms, or face consequences.

The IETF disclosure process is designed to work well when most (or all) of
the companies in the room have the same goal of creating a successful IETF
RFC to solve a certain problem.  It even works when competing proposals are
in the picture.  However, it's not my reading that some of the powerhouses
will likely participate in hypothetical IETF codec work.  In that case, the
IETF's IPR policy IMO does not work very well for your goal of an accessible
standard.

There is also one other thing that should be kept in mind: the possibility
of gaming.   If you would have an (unwritten, I assume) WG-local policy of a
codec whose IPR is available under open-source compatible terms, it takes
only one reckless---or, uninformed, or erroneous---disclosure to derail the
whole process.  This is FUD at its best.  Possibly, the work will come to a
screeching halt for more than a year, until the disclosed application is
published.  Even if there is a published application, someone would have to
offer a claim chart or another analysis of the patent or application.
That's lawyer work and not likely to happen, due to the possible
consequences (willful infringement etc.).  Organizations such as the W3C
have a well defined conflict resolution mechanism for this situation; the
IETF has not.   IMHO, IPR-related gaming (or something that comes awfully
close to it) has happened before in our organization---you will have to
allow me to skip expressing my suspicions.

That is why I do not see the IETF as a good organization to entertain a
contentious project like this.

Regards,
Stephan

---

From: Randell Jesup <[email-masked]>
Date: Thu, 25 Jun 2009 00:42:12 -0400

>G.718 is a new codec and I am not aware that some entity decided that it is

What Jean-Marc meant (I think) was that G.718 is (quite) encumbered, and
thus not useful to people who can't license it for the application they
want to persue (open-source, downloadable, etc - see below).  I couldn't
even find a unified licensing agency like MPEG-LA (though I didn't look
hard either).

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

---

From: Randell Jesup <[email-masked]>
Date: Thu, 25 Jun 2009 00:49:17 -0400

>Header compression can be used to reduce overhead. What is fairly well audio

Header compression may not be usable in a lot of (most?) situations.

>10/20 and even 30 ms frame size.

G.718 for example has over 40ms of algorithmic latency (add that to
packetization period, jitter-buffering delay, other fixed delays, and
transit time, and you have issues.  Usable: sure, for some network types
and applications, but limited due to that >40 ms delay.

>this statement is not really correct.

"Being in the list" and "driving development" (especially day-to-day)
are very different things.  I checked; ITU membership is WAY to
expensive for me.

---

From: Koen Vos <[email-masked]>
Date: Wed, 24 Jun 2009 22:27:59 -0700

Stephan,

You may have missed my point - I meant the following:

People who write speech codecs typically have a decent grasp of what  
is already patented and what not - even for patents not owned by  
themselves or their employers. Being part of the Codec WG, some of  
these codec developers will feel inclined to notify that a codec  
proposal may infringe on a certain patent, because doing so is in the  
interest of achieving the Royalty Free goal. Patents that are found to  
be infringed on can usually be worked around. As a result, the IETF  
codec standardization process results in a higher degree of confidence  
that no patents are infringed than if the codec were made available  
without the IETF process.

> not---even if you would wish so personally---because you may well be under

That's not an argument against my thesis: I'm talking about other  
people, who will participate.

> "interpolate" the possible outcome.  You just must never say a thing.

This approach to gaming the system seems like a long (and expensive)  
shot. But if we're really paranoid about it we could decide to only  
make modifications for which prior art exists. In that case these  
modifications can not have been patented in the mean time. Would that  
alleviate your worries?

koen.

---

From: stephen botzko <[email-masked]>
Date: Thu, 25 Jun 2009 07:52:53 -0400

I'd be interested in Stephan's response as well, but I do have a couple of
comments inline.

Stephen Botzko
Polycom

On Thu, Jun 25, 2009 at 1:27 AM, Koen Vos <[email-masked]> wrote:

> People who write speech codecs typically have a decent grasp of what is
> already patented and what not - even for patents not owned by themselves or
> their employers. Being part of the Codec WG, some of these codec developers
> will feel inclined to notify that a codec proposal may infringe on a certain
> patent, because doing so is in the interest of achieving the Royalty Free
> goal. Patents that are found to be infringed on can usually be worked
> around. As a result, the IETF codec standardization process results in a
> higher degree of confidence that no patents are infringed than if the codec
> were made available without the IETF process.
>
I think you will find in practice that the knowledge of what's patented is
much less complete than you are thinking.  There are lots of countries where
patents can be filed, translations of applications are not always available,
and understanding of claim scope requires an attorney's guidance and access
to the prosecution history. Design-arounds require a legal analysis of the
claim scope.

Also, many patents are never asserted by the original assignee, and are
"under the radar" until they are purchased (and asserted) by a patent
"troll".  Codec designers may have some idea of what other codec
technologies need to be licensed, but I think this is the just the tip of
the iceberg.

> That's not an argument against my thesis: I'm talking about other people,
> who will participate.
>

I think Stephan is using "participate" in limited definition of RFC
3978/3979.  They might very well attend the working group meetings, and
might even make some contributions (perhaps to other codecs than the one
they hold IPR on).

> This approach to gaming the system seems like a long (and expensive) shot.
> But if we're really paranoid about it we could decide to only make
> modifications for which prior art exists. In that case these modifications
> can not have been patented in the mean time. Would that alleviate your
> worries?

US applications do not need to be published, and can take 5 or more years to
issue.  It would be difficult to apply your prior art rule unless you are
thinking about using very old prior art.  Also, the expense of this approach
is not prohibitive if your organization has a goal of obtaining revenue from
their patents.  This is not just a commercial activity either, since some
universities also have signficant patent portfolios.

I am concerned that a rule like this would greatly diminish the value of the
collaboration you are seeking.  Since codec quality matters, not just IPR
terms, I'd say we need to be very careful here.

---

From: Koen Vos <[email-masked]>
Date: Thu, 25 Jun 2009 12:37:37 -0700

> universities also have signficant patent portfolios.

If the modifications consists of a method (let's say from a journal  
publication) that already existed before today, then that modification  
cannot be patented anymore.  So someone trying to "sneakily" file for  
a patent on an improvement that he sees coming by "interpolating"  
events at IETF could indeed be stopped.

> terms, I'd say we need to be very careful here.

Not sure. A good approach to building a royalty free codec is by  
starting out with prior art. Preferably from the 70s and 80s. I'd say  
at least 80% of the technology in any modern speech codec already  
existed in 1990. You get a long way by just combining existing ideas.

Otherwise I agree with you that we'll never be sure that a codec does  
not infringe on any patents. That's a risk with any standard, and in  
fact any piece of technology. But I maintain that a codec that has  
gone through IETF standardization will be perceived as having less  
risk than if the same codec had not gone through standardization.

koen.

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 25 Jun 2009 15:59:20 -0400

Hi Koen,
See a few remarks inline.
Stephan

On 6/25/09 3:37 PM, "Koen Vos" <[email-masked]> wrote:

> cannot be patented anymore.

To be precise, it depends on whether the journal article precedes the
priority date of the patent application in question.  The priority "locks
in" the specification, but not necessarily the claims.

> So someone trying to "sneakily" file for
> existed in 1990. 

That's correct, IMHO.

> You get a long way by just combining existing ideas.

A novel combination of Prior Art "ideas" can be patentable in its own right.

> risk than if the same codec had not gone through standardization.

Now this is the real problem.  Yes, it probably will be PERCEIVED has having
less IPR risk.  But what good does that do, except damaging the IETF's image
as an organization that knows what it's doing---in the case the perception
is wrong?

If the IETF were to engage in an effort that specifically requires an RF IPR
property (or, worse from a requirements viewpoint, an open-source
compatibility property---I'm not even talking about what some people in the
Boston area call "free" software), then the IETF must be able to stand
behind that claim.  I don't see a way how the IETF can do this, neither from
a policy, nor from a practical viewpoint.

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 25 Jun 2009 15:24:01 -0400

Hi Stephen, hi Koen,

Another long email: sorry.

A couple of points on the third party studies:
1. Relying on third party opinions, rendered by engineers without a legal
background (and, I fear, with limited or no legal support) will hardly
provide anyone with an environment where one can, with reasonable certainty,
argue that patents were avoided.  The end result may well be that large
companies will make their own assessment, and act on it, by arguing to avoid
any endorsement of the new standard; that is, killing it after it¹s
finished.    
2. Providing a third party opinion of a patent is a risky business.  In
doing so, you show that you have studied the patent.  In the US, if you were
later found infringing on the patent, you cold be in for triple damages.
This is one reason among many why most larger companies are very careful to
retain an external law firm for this kind of study, especially if there is
the intention of making the results public.  Sometimes, even standardization
organizations do so; see, for example, some of the PAGs of W3C.
3. Because of point 2, large companies are careful in letting their
engineers study other company¹s patents.  Let alone talk about the results.
Engineers of the larger companies, in my experience, have often much more
experience with patents than, say, open source hackers or academics.

Two generic points:
4. In most legislations, as long as there is still one application open in a
given patent family, one can amend claims and thereby adjust the scope of
protection long after the application publication date.  That is, one can do
so as long as the specification supports the amended claims.  If the
lawyer/agent who drafted the specification has not completely screwed up,
you can quite likely find yourself in a situation where you thought you have
avoided the claims of a published application, and didn¹t, because you
didn¹t review the prosecution history and its fine-print.  Just grab a
random US patent application, download the ³File Wrapper² in USPTO¹s Public
PAIR, and look for claims and claim amendments for examples.  Chances are,
if the application is long enough down the prosecution path, you will find
several instances of claim amendments.  Some may be triggered by the
examiner, but others may stem from the above mentioned mechanism.
This is the key reason why it is basically impossible to ³design around²
well written patent applications in a closely related field.  The fact that
you don¹t have visibility into the third party filings for 18 months---more,
if provisionals are involved---is just the icing on the cake.
5. The IPR game is always a question of risks and benefits.  In the codec
world, big industry operates by licensing and cross-licensing patents and
dishing out licensing fees if they have to, and occasionally entertaining
legal disputes that typically end up in settlements, that make the lawyers
rich and no one else.  There are provisions that make this environment quite
survivable for smaller players, and that¹s IMHO not going away as the
antitrust efforts in most developed countries are really getting teeth.  One
clear problem that the Open Source community (not the world, I insist!) has,
is that they are so inflexible in their own legal framework that they cannot
adapt to this business model.  A few open source projects do against the
spirit of their own policies AFAIK, by boldly ignoring known royalty bearing
IPR.  So far, I have not heard of anyone going after them.  They play the
risk/benefit equation in their own way, taking into account that they have
not enough flesh on their bones to be a valuable target for the
rightholders.

All that said, it¹s not unheard of that a reasonably sized pool of experts,
with sufficient legal support, comes together to create a meaningful patent
landscape study.  Especially if the field is as limited as speech coding.
Arguably, we did so during the early H.264 royalty-free baseline
discussions.  There have been a few more attempts since then; less in public
view.  Without discussing the success/failure of any of these exercises, it
should be clear that serious money was invested by those interested in the
RF baseline.  Serious in the sense that it was more than most individuals
would be interested or willing to pay, even if they were very emotionally
invested in the project and determined to make it a success.  One key factor
that¹s different from the hypothetical IETF speech codec exercise has been
that a large part of the industry was behind the effort.  That is, a large
percentage of the existing and future right holders.  The reasoning behind
is that a large group is more likely to be able to use business tools to
lure in the rest of the industry.

I would be personally interested to be involved in such an effort once more,
although I do not know right now whether I would have the time.  However, a
necessary condition is that at least a good percentage of the existing
powerhouses would be interested in the outcome.  If they were, the effort
could as well take place in traditional SDOs; there have always been ways in
those organizations to overcome logistic troubles such as membership fees.

Regards,
Stephan

On 6/25/09 7:52 AM, "stephen botzko" <[email-masked]> wrote:

>> goal. Patents that are found to be infringed on can usually be worked around.
>> As a result, the IETF codec standardization process results in a higher
>> degree of confidence that no patents are infringed than if the codec were
>> made available without the IETF process.
>> 
> and understanding of claim scope requires an attorney's guidance and access to
> the prosecution history. Design-arounds require a legal analysis of the claim
> scope.
> 
> Also, many patents are never asserted by the original assignee, and are "under
> the radar" until they are purchased (and asserted) by a patent "troll".  Codec
> designers may have some idea of what other codec technologies need to be
> licensed, but I think this is the just the tip of the iceberg.
>  
>> 
>> 
>> That's not an argument against my thesis: I'm talking about other people, who
>> will participate.
> 
> 3978/3979.  They might very well attend the working group meetings, and might
> even make some contributions (perhaps to other codecs than the one they hold
> IPR on).
>  
>> 
>> 
>>

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 25 Jun 2009 17:08:17 -0400

(sorry if this ends up as a dupe, more mail problems)

> as an organization that knows what it's doing---in the case the perception

Just curious, how does the ITU protect itself from patent trolls?

    Jean-Marc

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 25 Jun 2009 16:39:30 -0400

Jean-Marc

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 25 Jun 2009 17:26:25 -0400

Hi Jean Marc,

On 6/25/09 4:39 PM, "Jean-Marc Valin" <[email-masked]> wrote:

[...]
> Just curious, how does the ITU protect itself from patent trolls?
Not sure that this is the right forum, but nevertheless:

To take your question verbatim: the ITU does not need to fear trolls, as it
hardly can be sued over patent infringement as it is not practicing the
inventions in its standards.  The ITU also has no real image problem, as its
statements related to possible encumbrances of its standards are quite
clear.  You can easily find those by downloading any ITU Recommendation.
Packed in one sentence, they say: you are on your own in determining how
much, and to whom, you need to pay to practice a standard.

To answer the implied question (hope I got that one right):
If one is practicing potentially patented technologies, there is no real
mechanism to protect oneself from a troll, standard or not.  The main
mitigation the ITU has implemented to benefit its members is the RAND
licensing commitment requirement, which is bound to the ITU membership.
Historically, most companies interested in, and with the resources for,
standardization in the Telco market have been ITU members, and were
therefore bound to the RAND commitment.  Arguably, that's still mostly the
case.

This RAND commitment is, on one hand, liberal enough to drag in most
companies which see IPR royalty revenues as their business model as well as
those who don't.  On the other hand, it has also historically been strict
enough to avoid the worst excesses on royalty charging.

Stephan

---

From: stephen botzko <[email-masked]>
Date: Thu, 25 Jun 2009 16:33:27 -0400

If the modifications consists of a method (let's say from a journal
publication) that already existed before today, then that modification
cannot be patented anymore.  So someone trying to "sneakily" file for a
patent on an improvement that he sees coming by "interpolating" events at
IETF could indeed be stopped.

A patent application could have been filed prior to the publication.  Also,
a US patent application can be filed up to one year after publication.  Or
specific modification could be claimed by a broader earler patent that the
inventor was not even aware of.

Regards,
Stephen Botzko
Polycom

On Thu, Jun 25, 2009 at 3:37 PM, Koen Vos <[email-masked]> wrote:

> cannot be patented anymore.  So someone trying to "sneakily" file for a
> patent on an improvement that he sees coming by "interpolating" events at
> IETF could indeed be stopped.
>
> Not sure. A good approach to building a royalty free codec is by starting
> out with prior art. Preferably from the 70s and 80s. I'd say at least 80% of
> the technology in any modern speech codec already existed in 1990. You get a
> long way by just combining existing ideas.
>
> Otherwise I agree with you that we'll never be sure that a codec does not
> infringe on any patents. That's a risk with any standard, and in fact any
> piece of technology. But I maintain that a codec that has gone through IETF
> standardization will be perceived as having less risk than if the same codec
> had not gone through standardization.
>
>
> koen.

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 25 Jun 2009 17:43:06 -0400

Stephan Wenger wrote:
> 5. The IPR game is always a question of risks and benefits.  In the 
> codec world, big industry operates by licensing and cross-licensing 
> patents and dishing out licensing fees if they have to, and occasionally 
> entertaining legal disputes that typically end up in settlements, that 
> make the lawyers rich and no one else.  There are provisions that make 
> this environment quite survivable for smaller players, and that’s IMHO 
> not going away as the antitrust efforts in most developed countries are 
> really getting teeth.  One clear problem that the Open Source community 
> (not the world, I insist!) has, is that they are so inflexible in their 
> own legal framework that they cannot adapt to this business model.  

There's a lot more than open source involved in here. While I agree that 
the environment (i.e. having to pay $x of royalties per channel) may be 
survivable for even small players in the standard "carrier market", it is 
not viable for many other applications. I can probably start a new phone 
company, pay the royalties for some ITU codec and still make a profit. On 
the other hand, in other markets, even the *big* players cannot pay the 
per-channel royalties. Why do you think players as big as Google (Talk), 
Adobe (Flash) and Skype use Speex and/or iLBC and not G.729 or AMR-* ? It's 
not about the size, but about the business model. The current way the ITU 
works just does not support many of the new business models created by the 
Internet. When we talk about an "Internet audio codec", it's a lot more 
than merely transporting the bits over an IP network.

> A few open source projects do against the spirit of their own policies 
> AFAIK, by boldly ignoring known royalty bearing IPR.  So far, I have not 
> heard of anyone going after them.  They play the risk/benefit equation 
> in their own way, taking into account that they have not enough flesh on 
> their bones to be a valuable target for the rightholders.

I totally agree that this is certainly not something that should be encouraged.

	Jean-Marc

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 25 Jun 2009 19:38:21 -0400

Jean-Marc,

Do you know that these players have not been approached by a possible
rightholder?  Are you sure they will not?  Do you know the business
arrangements that have in place?  Have they picked up a license, perhaps?
Subject to a very broad cross-licensing deal?  Or are the leads of their
licensing group golf buddies and decided over a game not to go at each other
right now?

Concluding from the current situation (no litigation around speex that I'm
aware of, no licensing deal I have heard of) that there are really no
patents related to that technology, is not really a clever move.

However, I concede that the googles of the world have a very different
business model compared to traditional telco operators or vendors (not so
certain about skype, but let's skip that discussion :-).  You have not
convinced me that those players necessarily need a royalty-free codec, but
if they would, they could make their voices easily heard in the traditional
venues.  Everyone listens to google, and everyone smart in the Telco
industry listens to Skype, considering their respective successes.

Regards,
Stephan

On 6/25/09 5:43 PM, "Jean-Marc Valin" <[email-masked]> wrote:

>> this environment quite survivable for smaller players, and that¹s IMHO

---

From: "Roni Even" <[email-masked]>
Date: Fri, 26 Jun 2009 07:24:45 +0300

Hi,
I would like to add that I am not sure why the "googles of the world"
require a standard codec since they do not worry about interoperability
issue. I am also wondering why should the IETF who care about
interoperability worry about non-interoperable solutions.

Roni Even

> Of Stephan Wenger
> Sent: Friday, June 26, 2009 2:38 AM
> To: Jean-Marc Valin
> Cc: [email-masked]; stephen botzko

---

From: Koen Vos <[email-masked]>
Date: Fri, 26 Jun 2009 01:13:10 -0700

> inventor was not even aware of.

Just to clarify: we were talking about the situation that Stefan  
mentioned where someone would quietly listen in on the IETF WG and  
predict/interpolate a future modification, and then files a patent on  
that modification before it was actually verbalized in the WG.

You're right that we'd have to limit the modifications to prior art  
that's at least a year old (and preferably older).  And also require  
the combination of the prior art and the codec proposal to be an  
obvious one. (Because non-obvious combinations may also be patentable,  
as Stefan has pointed out.)

koen.

---

From: Jean-Marc Valin <[email-masked]>
Date: Fri, 26 Jun 2009 08:06:03 -0400

> interoperability worry about non-interoperable solutions.

Why do you think that supporting SIP and XMPP for signalling and codecs
such as G.711 and Speex isn't enough? What else is required for
interoperability?

	Jean-Marc
	(not related to Google in any way)

---

From: Koen Vos <[email-masked]>
Date: Fri, 26 Jun 2009 11:43:55 -0700

> patents related to that technology, is not really a clever move.

When big companies adopt open technology, they do due diligence to  
evaluate the risk of being sued for infringement. Sometimes they may  
indeed find patents that are their own, or which they cross-license  
already, and are thus no threat to them. Other patents they may  
overlook. But when enough large companies adopt an open technology,  
that (1) reduces the patent risk, and (2) that creates a powerful body  
to fight against frivolous patent claims (as in the SCO case).

Stephan, no new(ish) technology can ever be guaranteed to avoid  
patents. You sound as if open source and IETF standardization are  
fundamentally flawed in this regard. But they've been working well so  
far, and I just don't see a better alternative to creating free,  
interoperable technology.

> certain about skype, but let's skip that discussion :-).

Skype has 100s of millions of downloads per year, the vast majority of  
which never brings Skype a cent in revenue.

> You have not

Are you suggesting that ITU could create a royalty free standard? Many  
ITU stakeholders have big interests in avoiding exactly that. They  
have invested lots of R&D in this area and are now earning this back  
through their current standard codecs. Some of them even depend on  
codec royalties for their existence. And bear in mind that the ITU  
process is very expensive - who is going to pay for it if it only  
means less revenues for the average ITU member? But most importantly,  
even if ITU would try to standardize a royalty free codec, why would  
that codec have any less risk of infringing patents? Surely all the  
stakeholders who are today earning royalties would not participate,  
and therefore have no obligation to disclose related patents (just  
like with IETF).
http://www.itu.int/ITU-T/dbase/patent/patent-policy.html

best,
koen.

---

From: stephen botzko <[email-masked]>
Date: Sat, 27 Jun 2009 06:38:43 -0400

I am not intenting to preempt Stephan's reply

But they've been working well so far, and I just don't see a better
alternative to creating free, interoperable technology.

(a) generally speaking, there seems to be a lot less interest in patenting
(or at least asserting patents) on signaling and transport specifications
than for codecs.  This applies both to the IETF (SIP for instance), as well
as the ITU (H.323).  So I sdon't think the "free" component of the above
statement is a result of the IETF process differences.

If you remove the audio and video codec groups from SG16 (ITU), then you'd
find that there is little if any difference in the need to license / pay
royalties on the standards produced by the two organizations.

Surely all the stakeholders who are today earning royalties would not
participate, and therefore have no obligation to disclose related patents
(just like with IETF).
(b) Source code is disclosed very late in the ITU audio standardization
process, making it much more difficult for non-participant attendees to know
exactly how the codec functions.  So if it unlikely that someone can
research/patent improvements ahead of the process.

I agree that the obligations to disclose are similar in both organizations,
so the obligation to disclose is not an advantage (nor a disadvantage) of
the IETF when compared with the ITU.

Are you suggesting that ITU could create a royalty free standard?
I believe Stephan is saying that Skype (and others) can create a standard in
the ITU for which the proposing companines do not charge royalties.  G.722.1
is one proof point that this can be done in the ITU,

I believe he is also saying (in other posts) that developing a codec which
is guaranteed not to infringe any exisitng patents is not possible in either
organization.  I would concur with both statements.  I would be particularly
concerned about a working group that requires participants to research other
organization's patents - as Stephan indicates, that is against the policies
of many corporations because it can result in liability for triple damages
in the US.

And bear in mind that the ITU process is very expensive - who is going to
pay for it if it only means less revenues for the average ITU member?
(d) I don't understand what expense you are talking about here.  Audio codec
standardization requires the proposers of the each codec candidate to pay
for the listening tests.  ITU membership fees do not depend on what workiing
groups you participate in (that is, your membership fees do not go up if you
standardize codecs).  Are you somehow thinking that ITU members share the
codec revemues?  The ITU is not a patent pool, and does not license
anything.  (Neither does the IETF of course).

There's been no discussion about who bears the expense of codec
qualitifcation tests in the proposed IETF working group, other than some
folks expressing a hope that the testimg might be cost less.

Stephen Botzko
Polycom

On Fri, Jun 26, 2009 at 2:43 PM, Koen Vos <[email-masked]> wrote:

> When big companies adopt open technology, they do due diligence to evaluate
> the risk of being sued for infringement. Sometimes they may indeed find
> patents that are their own, or which they cross-license already, and are
> thus no threat to them. Other patents they may overlook. But when enough
> large companies adopt an open technology, that (1) reduces the patent risk,
> and (2) that creates a powerful body to fight against frivolous patent
> claims (as in the SCO case).
>
> Stephan, no new(ish) technology can ever be guaranteed to avoid patents.
> You sound as if open source and IETF standardization are fundamentally
> flawed in this regard. But they've been working well so far, and I just
> don't see a better alternative to creating free, interoperable technology.
>
>
> Are you suggesting that ITU could create a royalty free standard? Many ITU
> stakeholders have big interests in avoiding exactly that. They have invested
> lots of R&D in this area and are now earning this back through their current
> standard codecs. Some of them even depend on codec royalties for their
> existence. And bear in mind that the ITU process is very expensive - who is
> going to pay for it if it only means less revenues for the average ITU
> member? But most importantly, even if ITU would try to standardize a royalty
> free codec, why would that codec have any less risk of infringing patents?
> Surely all the stakeholders who are today earning royalties would not
> participate, and therefore have no obligation to disclose related patents
> (just like with IETF).

---

From: Randell Jesup <[email-masked]>
Date: Sat, 27 Jun 2009 14:41:09 -0400

>is one proof point that this can be done in the ITU,

Sure - but that's not what's being proposed/discussed here.  That's been
done in IETF as well with iLBC.

>of many corporations because it can result in liability for triple damages

Who would do that in ITU, or would it simply be assumed that the
proposing entity had dealt with it (if needed), and the RAND requirement
covered their patents?  Wouldn't the issues be pretty much the same?
Even if IETF has not RAND requirement, there are IPR disclosure
requirements, and the (likely) royalty/licensing status can be part of
the evaluation/selection/development.

>anything.  (Neither does the IETF of course).

"proposer pays" makes sense for an all-industry, royalty-generating
standards organization where it's a selection group, but it does no
group "development" or testing.  The proposer stands to benefit
monetarily in most cases.

In an organization where individuals often participate, and proposals
may be group efforts or incorporate input from many sources, this may
not be a reasonable way to proceed.

As for the cost, as I said before, I'm not able/willing to pay the cost
of participating in ITU groups as an individual - even if one of the
groups you have to go through would allow me in.  IETF participation is
much more free-form, individual, and without hurdles to participation.

>folks expressing a hope that the testimg might be cost less.

Well worth discussing, not just costs but also what's needed to
accomplish the goals.

---

From: Koen Vos <[email-masked]>
Date: Sat, 27 Jun 2009 14:10:50 -0700

> royalties on the standards produced by the two organizations.

The last time ITU came up with a free speech/audio codec was in 1972  
with G.711. Since then, all codecs, including G.722 and G.722.1, were  
standardized as non-free, royalty-bearing codecs. G.722.1 was only  
made available free of royalties 7 years after being standardized.

> research/patent improvements ahead of the process.

The ITU process has a natural tendency to come up with technology that  
is covered by patents from many of the stakeholders. This lubricates  
the process of reaching agreement. Take for example G.723.1, which is  
an inelegant combination of MPC-MLQ 6.3 kbps and ACELP 5.3 kbps. The  
ACELP mode is so much inferior to the MPC-MLQ mode that it's never  
used. The only reason it's in there is that the patent owners of ACELP  
wouldn't agree to the standard unless their technology was part of it.

best,
koen.

---

From: "Roni Even" <[email-masked]>
Date: Sun, 28 Jun 2009 01:04:49 +0300

Koen,
Standard bodies make decision by consensus, so such compromise like the
G.723.1 may happen in the IETF as well  if you would want a standard track
document (example in AVT is SVC payload specification). This is why I
suggested that we need to discuss in the BOF if the result will be an
Informational RFC or Standard track RFC.

A document can be blocked in the ITU and in the IETF and in this cases the
ADs are trying to get some compromise and not to force a solution.

A famous quote from Abba Eban on consensus -
"A consensus means that everyone agrees to say collectively what no one
believes individually."

Roni Even

> Of Koen Vos
> Sent: Sunday, June 28, 2009 12:11 AM
> To: [email-masked]

---

From: Brian West <[email-masked]>
Date: Sat, 27 Jun 2009 18:21:05 -0500

As someone that has gone thru the paperwork for G722.1/1C ... If  
you're going to offer a codec royalty free at the very least drop the  
paperwork requirement.  I had to do the whole process 3 times.   Now  
that we have G722.1/1C in FreeSWITCH its a moot point but anyone else  
wanting to use either of these codecs would need to go thru the  
paperwork mess. (which the lib is public domain you can use it if you  
like its in our SVN tree.)

/b

On Jun 27, 2009, at 4:10 PM, Koen Vos wrote:

> with G.711. Since then, all codecs, including G.722 and G.722.1,  
> were standardized as non-free, royalty-bearing codecs. G.722.1 was  
> only made available free of royalties 7 years after being  
> standardized.

---

From: David Rowe <[email-masked]>
Date: Sun, 28 Jun 2009 09:36:18 +0930

Hello list,

My name is David Rowe, I have a PhD in speech codecs and 20 years
experience in design and real time implementation of speech codecs and
other DSP algorithms.  I helped found Speex and have been a minor
contributor over the years. I have also run small businesses that uses
speech codecs and experienced first hand the pain of trying to use
codecs like g729 with $40k license fees.  

My experience from as both a codec designer and a business entity using
codecs is that they are a useless tax on business and telecommunication.
They license fees benefit no-one but the people receiving the royalties.
Why can I get a first class operating system like Linux for free but
have to pay some one to use a tiny bit of software on it like g729?

We have the skills in the open source community to build better codecs
using open source techniques.  The world will be a better place for it.

Patent free, competitive DSP algorithms can and should be developed, and
there are precedents:

1/ Speex and the various other open codecs.

2/ A patent free, royalty free line echo canceller (Oslec) which is
successfully replacing expensive royalty based solutions.  I recall much
of the same FUD about patents when developing this algorithm.  And yet
it works, pushing out the royalty-based code in many many thousands of
cases, and it's now part of the Linux kernel (try doing that with closed
code).

What has held Speex adoption back is lack of standardisation - people
have been forced to use royalty based codecs like g729.  Well, it looks
like we have a chance to fix that.

Speex has shown it's possible to build a patent free codec.  A lot of
the algorithms involved in codecs are just well known math, e.g
transforms, vector and scalar quantisation.  There are many ways to
achieve a certain math operation (e.g. time and frequency duality) to
get a certain technique done if an annoying patent is in the way.  It
really is no big deal.  Much of this technology was published in the
60's and 70's.

Where the patents come in is that some one gets a good quality codec
working using some clever technique in 1% of algorithm, then they patent
that 1%.  I would imagine they explicitly search for a novel technique
to lock up their codec. Then when that codec is standardised we are
stuck paying them royalties for bit-exact interoperability reasons.  

g729 does not hold patents on linear prediction, or vector quantisation,
or CELP, or Line Spectrum Pairs, or pitch prediction.  All these
techniques are also in Speex which sounds about the same at 8 kbit/s.

It's easy to replace that 1% early in the codec design process - we
simply make royalty free a priority rather than maximising royalties.
In other worlds design the codec to help as many people as possible,
rather than designing it to make a small number of people wealthy.
Isn't that what the IETF is all about?

I think it's a great ideas to release source code from day 1.  Open
source development has been shown to be superior to closed, so I am sure
we can develop a better codec faster with open source, peer reviewed
code.

Re funding an open development effort and travel to meetings the $
involved are trivial compared to the cumulative costs of license fees
down the track for the closed approach.  So it's a great business
decision to support an open codec.

Thanks,

David

-- 
Free Telephony Project
open embedded IP-PBX hardware and software
http://www.rowetel.com/ucasterisk

---

From: Brian West <[email-masked]>
Date: Sat, 27 Jun 2009 20:39:18 -0500

Its actually 20k to get started.  In either case its silly.  Speex is  
a great codec as is CELT.  With the two you have the complete range 8,  
16, 32 and 48kHz.

/b

---

From: Henry Sinnreich <[email-masked]>
Date: Sat, 27 Jun 2009 18:47:00 -0700

>Isn't that what the IETF is all about?

There is large volume of papers in the IETF and ACM on Engineering Ethics that mirror this position.
One can do a search or start at http://en.wikipedia.org/wiki/Engineering_ethics

Henry

On 6/27/09 7:06 PM, "David Rowe" <[email-masked]> wrote:

---

From: Koen Vos <[email-masked]>
Date: Sat, 27 Jun 2009 19:19:39 -0700

Quoting Roni Even:
> G.723.1 may happen in the IETF as well

Roni,

the ugly G.723.1 compromise came about solely to protect royalties  
from a party that would otherwise block the standard. Therefore, such  
a compromise could *not* happen if the IETF WG were to set out to  
create a royalty-free codec.

ITU serves to protect and generate royalties for its members.

koen.

---

From: stephen botzko <[email-masked]>
Date: Sun, 28 Jun 2009 07:17:34 -0400

On Sat, Jun 27, 2009 at 5:10 PM, Koen Vos <[email-masked]> wrote:

> The last time ITU came up with a free speech/audio codec was in 1972 with
> G.711. Since then, all codecs, including G.722 and G.722.1, were
> standardized as non-free, royalty-bearing codecs. G.722.1 was only made
> available free of royalties 7 years after being standardized.

G722.1 Annex C (which is a complete superwideband codec) has always been
royalty free, as I indicated in previous posts, G,722,1 (wideband) was made
available royalty free shortly afterwards.

Though I think you missed the main point of the comment, which was the
relative lack of patent burden on the IETF RFCs is partly att least partly a
result of their subject matter, and not the process, or that the
participants are somehow different.

> The ITU process has a natural tendency to come up with technology that is
> covered by patents from many of the stakeholders. This lubricates the
> process of reaching agreement. Take for example G.723.1, which is an
> inelegant combination of MPC-MLQ 6.3 kbps and ACELP 5.3 kbps. The ACELP mode
> is so much inferior to the MPC-MLQ mode that it's never used. The only
> reason it's in there is that the patent owners of ACELP wouldn't agree to
> the standard unless their technology was part of it.

I wasn't in those meetings, so I can't speak to "the only reason its in
there".  The current audio ITU process does *not* allow someone to object to
a standard because their technology is not in it.  Getting concensus might
require that the terms of reference include both 6.3 kpbs and 5.3 kbps
operating modes, however the terms of reference would not indicate the
technology used by those operatoring modes.

Obvously most ITU audio codecs require royalties.  However, that is not a
natural tendency of its process.  Candidate codecs can be submitted without
regard to IPR, after a competitive selection process the best codec is
standardized, and the contributors set the terms.  There is nothing I see
that is inherent in this process that "naturally tends" to result in
royalty-bearing codecs. I'd say the result is due to the companies that have
chosen to contribute there, and not the process that is used.

Also, I thought the goal was royalty-free, not patent free.  Am I mistaken
here?

---

From: stephen botzko <[email-masked]>
Date: Sun, 28 Jun 2009 07:46:14 -0400

> done in IETF as well with iLBC.

iLBC is an experimental RFC, I am not proposing/discussing that here. What I
was indicating is that royalty-free audio codecs can be standardized in
other SDOs, including the ITU, which I think is quite relevant to the
question of whether the IETF should also get into that business.  I agree
that there are not many of these, however that could change if other
companies and universities chose to participate in those venues.

I agree that individuats would find the IETF more affordable than the ITU,
though I am not sure if that is a sufficient reason to form a working group.

> the evaluation/selection/development.

The IPR disclosure processs in both organizations only requires that
participants disclose if they have IPR.  Neither the IETF nor the ITU
requires the participants to determine if any non-participants have IPR, or
require participants to attempt to construe patent claims.  I think its
important to keep i that way.

Since many of the proposals here are essentially proposing creating a new
IPR process for the proposed WG, I think both the codec standardization
question itself and the IPR process need to be discussed at the BoF.

> Well worth discussing, not just costs but also what's needed to

Agreed.

---

From: "Christer Holmberg" <[email-masked]>
Date: Sun, 28 Jun 2009 15:34:16 +0200

Hi,

>a great codec as is CELT.  With the two you have the complete range 8,  

What, in your opinion, is it then that we do NOT have?

(Henry gave some examples of applications for which he thinks something new is needed.)

Regards,

Christer

/b

---

From: Koen Vos <[email-masked]>
Date: Sun, 28 Jun 2009 14:53:38 -0700

Quoting stephen botzko:

> Getting concensus might require that the terms of reference include  
> both 6.3 kpbs and 5.3 kbps operating modes, however the terms of  
> reference would not indicate the technology used by those  
> operatoring modes.

The 5.3 kbps mode is rarely used in practice because of the poor  
quality. But it needs to be supported to be compliant to the standard,  
which increases memory usage and creates a burden when implementing  
and porting the codec.

> standardized, and the contributors set the terms.

Even if it were true that ITU standardizes the best codec, this  
process differs from what the IETF WG is considering. The IETF WG aims  
to "as much as possible choose technology that does not require paying  
patent royalties." ITU has never had such a goal when standardizing a  
codec, to my knowledge.

If fact, the G.723.1 example suggests that generating extra royalties  
has outweighed technical merit in the ITU process. We want the exact  
opposite: generating *less* royalties should outweigh technical merit.

It may be true that annex C of G.722.1 was standardized with the  
understanding of being royalty free, but G.722.1.C was never a threat  
to the royalties generated by other ITU standards.

koen.

---

From: Brian West <[email-masked]>
Date: Sun, 28 Jun 2009 17:24:24 -0500

On Jun 28, 2009, at 4:53 PM, Koen Vos wrote:

> quality. But it needs to be supported to be compliant to the  
> standard, which increases memory usage and creates a burden when  
> implementing and porting the codec.

I have never heard a single problem with 5.3 kbps 723.1... maybe its  
subjective.

> process differs from what the IETF WG is considering. The IETF WG  
> aims to "as much as possible choose technology that does not require  
> paying patent royalties." ITU has never had such a goal when  
> standardizing a codec, to my knowledge.
>
> If fact, the G.723.1 example suggests that generating extra  
> royalties has outweighed technical merit in the ITU process. We want  
> the exact opposite: generating *less* royalties should outweigh  
> technical merit.

Doesn't all the patents expire on g723.1 shortly?

> understanding of being royalty free, but G.722.1.C was never a  
> threat to the royalties generated by other ITU standards.
>
> koen.

---

From: stephen botzko <[email-masked]>
Date: Sun, 28 Jun 2009 19:32:18 -0400

Comments in-line

Stephen Botzko
Polycom

On Sun, Jun 28, 2009 at 5:53 PM, Koen Vos <[email-masked]> wrote:

> Quoting stephen botzko:
>
>  Getting concensus might require that the terms of reference include both
>> 6.3 kpbs and 5.3 kbps operating modes, however the terms of reference would
>> not indicate the technology used by those operatoring modes.
>>
>
> The 5.3 kbps mode is rarely used in practice because of the poor quality.
> But it needs to be supported to be compliant to the standard, which
> increases memory usage and creates a burden when implementing and porting
> the codec.
>
I really don't know where you are going here.  First you say that someone
threatened to block the standardization "because their technology wan't in
it".  I addressed that point, saying that that objection wouldn't be
possible under the current ITU audio groups procedures.

I agree that the 5.3 kbps mode is poor.  Though to my ears, both modes are
poor.

> Even if it were true that ITU standardizes the best codec, this process
> differs from what the IETF WG is considering. The IETF WG aims to "as much
> as possible choose technology that does not require paying patent
> royalties." ITU has never had such a goal when standardizing a codec, to my
> knowledge.
>
> If fact, the G.723.1 example suggests that generating extra royalties has
> outweighed technical merit in the ITU process. We want the exact opposite:
> generating *less* royalties should outweigh technical merit.

The fact that one operating mode of one rather old codec sucks does not
really suggest anything about royalties.  At most it suggests that the ITU-T
made a mistake in standardizing that particular mode.

> understanding of being royalty free, but G.722.1.C was never a threat to the
> royalties generated by other ITU standards.

Now we are moving into something I can speak to with some authority, since
Polycom standardized this.  No matter how G.722.1C was perceived by the
other participants , the decision to make it royalty free was Polycom's
alone.  Polycom was under no obligation to inform the ITU ahead of time that
it was planning to make it royalty-free, it is sufficient to say that the
terms are in accordance with the ITU RAND policy.  RAND can include free of
course.

Again, I am not sure where you are going (if anywhere).

It is totally possible to standardize something that is royalty-free in
existing SDOs, including the ITU-T.  That is a simple fact.  The truth as
long as you comply with the ITU IPR policy you can charge (or not charge)
any royalties you like.  The "catch" if you want to call it that, is that
you need to (a) get consensus on the terms of reference (the codec
requirements), and (b) win the competitive selection process.  These things
are not easy to do.  But it is possible.

A second (also simple) fact is that almost ITU-T codecs do involve royalty
payments.  I am not disputing that.  But I am saying that it is *not* a
result of their process.  It is only because most of the present players
there are choosing to charge royalties.

The *process* is literally agnostic to the IPR terms.  You don't even need
to disclose that your RAND terms are free until after the standardization is
completed.  There is no conspiracy plot here, the ITU-T is not out to get
you.

The conclusion I have drawn is that the only thing anyone cares about in the
proposed IETF WG is the "free" word.  As long as there is a guarantee that
all resulting RFCs are royalty free, the group will be a success, even if we
end up with G.711. I think this is regrettable.

Stephen Botzko
Polycom

---

From: Koen Vos <[email-masked]>
Date: Sun, 28 Jun 2009 23:37:24 -0700

> end up with G.711. I think this is regrettable.

I respectfully disagree. I think we've reached the point where "the  
best codec at any cost" is no longer much superior to "the best free  
codec".

Commoditization is unavoidable, and a good thing.

koen.

---

From: Randell Jesup <[email-masked]>
Date: Mon, 29 Jun 2009 10:22:53 -0400

>chosen to contribute there, and not the process that is used.

I disagree strongly - the process you describe, while not focused on
royalties, is almost certain to produce specs that generate royalties
for the proposer.  (They don't *have* to end up that way, if the
proposer has some strategic goal that includes bypassing royalties - but
that's an outside consideration.)  Ironically, the ignoring of IPR other
than RAND requirements, when combined with the costs involved (testing,
participation, etc) and that it's supposed to be a pure runoff will
rarely result in a non-royalty codec.

Though I don't know the details of how it happened, look at H.264 baseline.

>Also, I thought the goal was royalty-free, not patent free.  Am I mistaken

Yes, at least to some sub-set of the people on this list.  Several
people have articulated that they want a codec spec that no one has to
worry about licenses or IPR status (not just no royalties owed).  That
is a point for discussion, since that's not universally agreed to here,
and it's a fundamental "what is the goal" question.

---

From: Randell Jesup <[email-masked]>
Date: Mon, 29 Jun 2009 10:40:26 -0400

>possible under the current ITU audio groups procedures.

Ah, so you're saying the rules for codec standardization in the ITU
changed then? (since g.723.1?)

>terms are in accordance with the ITU RAND policy.  RAND can include free of

That's exactly the point of the problem - the ITU truly didn't care
(beyond RAND) about the licensing status.  That's what makes it likely
(not given) that any random ITU standard will be subject to royalties,
unless the company(s) that submitted it has a strategic reason to pass
on them - they may well still hold the patents and require no-cost
licenses or put other restrictions on use.

>completed.  There is no conspiracy plot here, the ITU-T is not out to get

Correct - it was simply created under the assumption that proposers
would be taking royalties.  RAND was put in to make the royalty terms
reasonable to other players in the industry, so someone couldn't
standardize something and then use the licensing as a club (I assume).

I believe (without having taken part) the ITU scheme also has the
side-effect of discouraging group participation and development, and
discouraging melding of ideas during the process - it's more of a
"select among company proposals" setup.  I'd guess this tendency is less
extreme, and in some of the joint groups I think it's been more
collaberative than is typical for ITU (i.e. JVC/H.264) from what I hear.
But this is all guesswork based on what I'm hearing - I'd guess I'd
never be involved in ITU directly due to the monetary and hoop-jumping
requirements.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 29 Jun 2009 12:48:47 -0400

I would agree that agreeing on the goal before starting work.
In this case however, I am not sure it matters.
Based on experience, and on the known properties of the IETF, it is 
extremely unlikely that either goal (patent free, or merely royalty 
free) can be achieved here.  Thus, the question seems to become, not 
"which goal do we want?" but rather "does coming close to, but missing, 
one or the other of these goals help?"

And the collateral question would seem to be "is the IETF process a good 
way to approach or achieve these goals?"

Yours,
Joel M. Halpern

> stephen botzko <[email-masked]> writes:
...

---

From: Jean-Marc Valin <[email-masked]>
Date: Mon, 29 Jun 2009 12:59:32 -0400

> one or the other of these goals help?"

Considering that these goals have already been achieved for codecs outside 
of standards organizations (and once at the IETF with iLBC), what makes you 
think it isn't achievable at the IETF?

Cheers,

	Jean-Marc

---

From: "Roni Even" <[email-masked]>
Date: Mon, 29 Jun 2009 20:35:18 +0300

Jean-Marc,
RFC 3951 iLBC is not a standard track RFC. It is not an Internet Standard in
any sense. See http://www.ietf.org/u/ietfchair/info-exp.html

Roni Even

> Sent: Monday, June 29, 2009 8:00 PM
> To: Joel M. Halpern
> Cc: [email-masked]

---

From: stephen botzko <[email-masked]>
Date: Mon, 29 Jun 2009 13:39:00 -0400

Comments in line

Stephen Botzko
Polycom

On Mon, Jun 29, 2009 at 10:22 AM, Randell Jesup <[email-masked]> wrote:

> is

I would say that the proponents here do have a strategic goal of bypassing
royalties, and several have the resources to pay for the testing and
membership fees.  For many (not all), up-front costs to do the
standardization would be acceptable, and we've found that there is quite a
bit of value in doing scientifically controlled, reproducible, statistically
valid listening tests in multiple labs, using native speakers and multiple
languages.  At this point it isn't obvious how the costs will compare (other
than that the membership fees will be lower), since there has not been much
discussion on the testing process, let alone consensus on it.

If we were talking about a *video* codec group, I would agree that the ITU-T
video process could not result in a royalty-free codec. The video group in
the ITU uses a totally different process from the audio group.  It is a
fully collaborative process, where any sufficiently useful compression tool
is encorporated into a shared source reference model. In the case of H.264,
it was done by JVT,(ITU-T in combination with MPEG/ISO), using MPEG rules as
well as ITU-T rules.

It is true that the royalty-free H.264 baseline goal failed (Polycom was a
strong proponent of that goal BTW).  That's because (given the ITU-T video
process), any single contributor can destroy the royalty free goal.  The
entire video group is contributing to the final codec, so there are a lot
more contributors.  That is quite different from the audio standardization
process, where the contributors for each candidate codec control the
technology used in their candidate, and can get private agreement on the
terms before they collaborate.

> and it's a fundamental "what is the goal" question.

I think its a quite important point.  Patent-free is difficult for many
companies to establish (most large companies actually don't know their
patent portfolios well enough to know exactly what they cover).  Also many
companies do not want to risk "disclaiming" what their patents cover.

Current IPR reporting in the IETF is focused on terms for the IPR you hold,
and is not designed for positive confirmation that you don't hold IPR.  If
the goal is patent free, then the group would need to infer that from the
absense of an IPR declaration, which seems a bit roundabout.

Why exclude patented technology if the patent holder is willing to offer it
on acceptable royalty-free terms?

In my opinion, the ideal would be royalty free, and conditional on
reciprocity.  That means if someone else sues you, that you could countersue
with your patent, but you could not initiate any lawsuit.  That permits
companies to build a defensive-only patent portfolio, and reduces the risk
that some third party (who practices the standard) would initiate a suit.

---

From: stephen botzko <[email-masked]>
Date: Mon, 29 Jun 2009 13:58:23 -0400

At this point in the discussion, I would agree with your comment.  Though
I'd suggest that patent free is less likely than royalty free.

I definitely think these goals are easier to achieve in an non-IETF
consortium, where all participants agree up front to the IPR rules in play,
open-source, etc.  In the IETF, the only up front agreement is to the IETF
procedures, not the constrained (unwritten?) rules that this proposed group
would need.  It will be harder in the IETF.

Stephen Botzko
Polycom

On Mon, Jun 29, 2009 at 12:48 PM, Joel M. Halpern <[email-masked]>wrote:

> extremely unlikely that either goal (patent free, or merely royalty free)
> can be achieved here.  Thus, the question seems to become, not "which goal
> do we want?" but rather "does coming close to, but missing, one or the other
> of these goals help?"
>

---

From: stephen botzko <[email-masked]>
Date: Mon, 29 Jun 2009 14:05:54 -0400

No.  The process could have changed, but another possibility is that Koen is
mistaken on the history.  I wasn't in those meetings (13-15 years ago); I
will not speculate about what happened and why.

Regards,
Stephen Botzko
Polycom

On Mon, Jun 29, 2009 at 10:40 AM, Randell Jesup <[email-masked]> wrote:

> quality.

---

From: Christopher Montgomery <[email-masked]>
Date: Mon, 29 Jun 2009 14:24:20 -0400

> I'd suggest that patent free is less likely than royalty free.

Not so much 'harder' as suffering from a halting problem.  There is no
equivalent of summary judgement for patents, and as such every product
released to the world every day from any company or any group can be
litigated against.  In most of the world with a US-style patent
system, barratry is not in itself illegal, and as such a troll does
not even need to have a glimmer of hope if simply lobbing a grenade is
itself strategically advantageous.

This is true for every piece of software on earth.  I for one am not
prepared to give up, go home, and become a farmer.

First let's seperate the patent threats.

There are the 'legitimate' IPR holders, such as the large players in
the ITU.  Do we expect these players are gaming the system with such
recklessness that they will withold discovered knowledge of
infringement for years at a time in order to maximize the damages
recovered in eventual litigation?  Even in the US, this is becoming
deeply frowned upon in the courts.  So, whether these entities are in
a WG that compels disclosure or not, I would expect these entities to
be 'good' players, or at least keep within the rules.

Then there are the trolls.  Trolls are of fully equal danger to
IPR-encumbered as well as any attempts at fully unencumbered
technology.  I'd actually say less... because it makes less sense to
sue where money isn't involved.  This is a threat entirely outside any
groups' control.

So the 'main threat' is the large, legitimate players.  It's been
voiced several times that an open, transparent working group would
suffer from IP 'theft'; that is people watching the process and going
off to patent immediate improvements.  This is an easy one:   Publish
everything.   Anything published is removed from 'theft' risk.
Subsequent improvements are still exposed to patents, but this is a
legitimate use of the patent system.

Monty

---

From: Marshall Eubanks <[email-masked]>
Date: Mon, 29 Jun 2009 18:25:29 -0400

On Jun 29, 2009, at 1:39 PM, stephen botzko wrote:

> I would say that the proponents here do have a strategic goal of  
> bypassing royalties, and several have the resources to pay for the  
> testing and membership fees.  For many (not all), up-front costs to  
> do the standardization would be acceptable, and we've found that  
> there is quite a bit of value in doing scientifically controlled,  
> reproducible, statistically valid listening tests in multiple labs,  
> using native speakers and multiple languages.  At this point it  
> isn't obvious how the costs will compare (other than that the  
> membership fees will be lower), since there has not been much  
> If we were talking about a video codec group, I would agree that the  
> ITU-T video process could not result in a royalty-free codec. The  
> video group in the ITU uses a totally different process from the  
> audio group.  It is a fully collaborative process, where any  
> sufficiently useful compression tool is encorporated into a shared  
> source reference model. In the case of H.264, it was done by JVT, 
> (ITU-T in combination with MPEG/ISO), using MPEG rules as well as  
> ITU-T rules.
>
> It is true that the royalty-free H.264 baseline goal failed (Polycom  
> was a strong proponent of that goal BTW).  That's because (given the  
> ITU-T video process), any single contributor can destroy the royalty  
> free goal.

Just for my curiosity, would a different SDO approach (say, the IETF's  
or some other) have resulted in a different
outcome vis-a-vis royalties ?

Regards
Marshall

>   The entire video group is contributing to the final codec, so  
> there are a lot more contributors.  That is quite different from the  
> audio standardization process, where the contributors for each  
> candidate codec control the technology used in their candidate, and  
> can get private agreement on the terms before they collaborate.
>
>
>
>
> I think its a quite important point.  Patent-free is difficult for  
> many companies to establish (most large companies actually don't  
> know their patent portfolios well enough to know exactly what they  
> cover).  Also many companies do not want to risk "disclaiming" what  
> their patents cover.
>
> Current IPR reporting in the IETF is focused on terms for the IPR  
> you hold, and is not designed for positive confirmation that you  
> don't hold IPR.  If the goal is patent free, then the group would  
> need to infer that from the absense of an IPR declaration, which  
> seems a bit roundabout.
>
> Why exclude patented technology if the patent holder is willing to  
> offer it on acceptable royalty-free terms?
>
> reciprocity.  That means if someone else sues you, that you could  
> countersue with your patent, but you could not initiate any  
> lawsuit.  That permits companies to build a defensive-only patent  
> portfolio, and reduces the risk that some third party (who practices  
> the standard) would initiate a suit.
>
> --

Regards
Marshall Eubanks
CEO / AmericaFree.TV