Subject: [codec] comparitive quality testing

From: Gregory Maxwell <[email-masked]>
Date: Thu, 14 Apr 2011 07:25:27 -0700

Roni Even [[email-masked]] wrote:
> I do not mind if the WG will decide to remove the quality claim and continue
> with developing  a royalty free codec with "good enough" quality not saying
> it is better than other codecs.
> I just think that it should be clear from the charter and requirements what
> is the purpose of the work.

It's funny how we can argue and argue, only to later realize that it comes
down to a simple mutual misunderstanding.

I thought everyone was already on the same page with respect to the 
goals: it's good to be as good as possible, but the chartered purpose
of the WG was only to do a "good quality" codec that was suited
to the listed applications and deployments.

As a developer I know that quality testing is important, and of course
we've done a lot of it of various types.  I strongly believe in scientific
testing, so of course my first instinct would have been to do it here,
but perhaps the reality of the consensus process makes that less
reasonable—as others have pointed out, most other WGs don't really
do anything comparable to quality testing.

Likewise, making sure the outcome is as legally unencumbered as I can
is also very important to me,  but because of the vulgarities of the
process and the law, this isn't something that the working group itself
makes promises about.

So, perhaps it makes sense for the working group to not make any quality
promises in the same way it makes no promises about patents.

It seems clear enough to me now that we can much more easily come to
consensus about achieving good-enough status than about formal testing
gates and requirements.

We should accept your suggestion—drop all the comparative quality
requirements from the requirements draft, and stop discussing comparative
quality here—and then make some progress on technology, rather than
continue bickering about details where we are not going to come to
consensus.

The market can figure out the comparative quality question on its own.

---

From: Roman Shpount <[email-masked]>
Date: Thu, 14 Apr 2011 14:21:18 -0400

I think part of the confusion comes from the fact that there are two
purposes for the comparative testing. One is to validate that the codec
meets the WG requirements. Another is to show how new codec compares to the
industry dominant codecs. For me, the second goal is more important then the
first one. I think if we care about the adoption of Opus, we should consider
making the comparative test results a deliverable for the working group. It
is very hard for a real company in the open market to justify doing
something, like adapting a new codec without a compelling reason. Knowing
how this codec compares to other existing codecs is a big part of providing
such a reason. If we look at the tests from this point of view, we need to
see how Opus compares to G.729 and AMR in narrow band, and AMR-WB and G.722
in wideband, Since there are no existing deployments of a meaningful size
(apart from a closed proprietary systems, like Skype) for UWB and FB, we can
compare Opus with industry leaders, such as G.719.

One can argue that we should also compare Opus with patent free codecs,
which adds iLBC and Speex to the list, but I personally see this as less of
a requirement. iLBC never managed to get market traction outside of the open
source world, and even in the open source world nobody bothered to write
even moderately optimized version of it. Speex is known for audio quality
problems so that it would be an easy target to beat. On the other hand this
would probably not be much of a milestone and will not tell anybody a lot
about the Opus quality.

There were several tests that compared Opus with non-interactive codecs, but
once again this is not something which would affect choosing Opus vs other
codecs, since other codecs are clearly inappropriate for Opus purposes.

We can argue about adding more codecs to the list, but I am not sure this
will make the difference. We only need to compare with a very few to give
everybody a clear idea about the Opus quality. As far as defining the
criteria for the codec being acceptable for standardization, all we really
need is a comparable quality (not worse then the other codecs by some
defined margin). This is not a competition where Opus needs to win every
race to be successful.

The whole reason why I am interested in formal comparative testing of Opus
is because I am impressed by its quality. I think having well documented
test results which were cross-checked by multiple people might make a
critical difference in Opus adoption, and as a result the success of this
working group.

No hats, just my two cents...
_____________
Roman Shpount

On Thu, Apr 14, 2011 at 10:25 AM, Gregory Maxwell <[email-masked]>wrote:

> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec
>

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 14 Apr 2011 18:55:32 -0600

On Apr 14, 2011, at 12:21 PM, Roman Shpount wrote:

> I think part of the confusion comes from the fact that there are two purposes for the comparative testing. One is to validate that the codec meets the WG requirements. Another is to show how new codec compares to the industry dominant codecs. For me, the second goal is more important then the first one. I think if we care about the adoption of Opus, we should consider making the comparative test results a deliverable for the working group. It is very hard for a real company in the open market to justify doing something, like adapting a new codec without a compelling reason. Knowing how this codec compares to other existing codecs is a big part of providing such a reason. If we look at the tests from this point of view, we need to see how Opus compares to G.729 and AMR in narrow band, and AMR-WB and G.722 in wideband, Since there are no existing deployments of a meaningful size (apart from a closed proprietary systems, like Skype) for UWB and FB, we can compare Opus with industry leaders, such as G.719. 
> 
> One can argue that we should also compare Opus with patent free codecs, which adds iLBC and Speex to the list, but I personally see this as less of a requirement. iLBC never managed to get market traction outside of the open source world,

Sort of curios how you came to that conclusion about iLBC's market traction?

---

From: Roman Shpount <[email-masked]>
Date: Thu, 14 Apr 2011 21:20:37 -0400

I know that iLBC was picked by PacketCable and is supported by quite a few
gateways. On the other hand, iLBC standard itself looks as if the codec was
contributed by one company with almost nobody investing any time in its
actual standardization. Quality comparisons and resource requirements seemed
to be coming from the same company and accepted on their say so. Most of the
deployed versions of iLBC are a floating point reference implementation,
which is coded in a manner as inefficient as a codec implementation can
possible be. There are no open source fixed-point implementations, even
though creating one is trivial. Reference vectors consist of a single audio
file recorded by a single voice. Encoder, despite being a bit exact
reference, produces different results based on optimization options or
compiler. To summarize, I assumed that there is no interest in this code
since there are a lot of problems with it that would have been easy to fix
if anybody bothered. It looks like a proof of concept and not a finished
product. Finally, amount of voice traffic carried by iLBC is only a small
fraction of G.729.
_____________
Roman Shpount

On Thu, Apr 14, 2011 at 8:55 PM, Cullen Jennings <[email-masked]> wrote:

> source world,

---

From: David Virette <[email-masked]>
Date: Fri, 15 Apr 2011 16:46:58 +0200

Dear Roman,

Would you have some specific conditions you would like to see tested for
G.729: clean speech and/or noisy speech experiments, error conditions,
levels? It is just to have an idea of what has to be added/modified in the
test plan.

Best regards,

David

David Virette
HUAWEI TECHNOLOGIES CO.,LTD. huawei_logo

Building C
Riesstrasse 25
80992 Munich, Germany
Tel: +49 89 158834 4148
Fax: +49 89 158834 4447
Mobile: +49 1622047469
E-mail: [email-masked]
www.huawei.com
----------------------------------------------------------------------------
---------------------------------------------------------
This e-mail and its attachments contain confidential information from
HUAWEI, which 
is intended only for the person or entity whose address is listed above. Any
use of the 
information contained herein in any way (including, but not limited to,
total or partial 
disclosure, reproduction, or dissemination) by persons other than the
intended 
recipient(s) is prohibited. If you receive this e-mail in error, please
notify the sender by 
phone or email immediately and delete it!

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Roman Shpount
Sent: jeudi 14 avril 2011 20:21
To: Gregory Maxwell
Cc: [email-masked]
Subject: Re: [codec] comparitive quality testing

On Thu, Apr 14, 2011 at 10:25 AM, Gregory Maxwell <[email-masked]>
wrote:

reasonable-as others have pointed out, most other WGs don't really

We should accept your suggestion-drop all the comparative quality
requirements from the requirements draft, and stop discussing comparative
quality here-and then make some progress on technology, rather than
continue bickering about details where we are not going to come to
consensus.

---

From: Roman Shpount <[email-masked]>
Date: Fri, 15 Apr 2011 13:01:04 -0400

David,

We should compare with standard G.729 (not A or B) at 8 kbit/s, 0 and 5%
packet loss, clean speech and speech mixed with at least one noise sample.
We can use ITU-T audio library for both speech and noise samples.
_____________
Roman Shpount

On Fri, Apr 15, 2011 at 10:46 AM, David Virette <[email-masked]>wrote:

> HUAWEI TECHNOLOGIES CO.,LTD. 
>
>
>
> Building C
> Riesstrasse 25
> 80992 Munich, Germany
> Tel: +49 89 158834 4148
> Fax: +49 89 158834 4447
> Mobile: +49 1622047469
> -------------------------------------------------------------------------------------------------------------------------------------
> is intended only for the person or entity whose address is listed above.
> Any use of the
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Roman Shpount
> *Sent:* jeudi 14 avril 2011 20:21
> *To:* Gregory Maxwell
> *Cc:* [email-masked]
> *Subject:* Re: [codec] comparitive quality testing
>
>
>
> There were several tests that compared Opus with non-interactive codecs,
> but once again this is not something which would affect choosing Opus vs
> other codecs, since other codecs are clearly inappropriate for Opus
> purposes.
>

---

From: Steve Underwood <[email-masked]>
Date: Sat, 16 Apr 2011 01:14:25 +0800

Hi Roman,

On 04/16/2011 01:01 AM, Roman Shpount wrote:
> David,
>
> We should compare with standard G.729 (not A or B) at 8 kbit/s, 0 and 
> 5% packet loss, clean speech and speech mixed with at least one noise 
> sample. We can use ITU-T audio library for both speech and noise samples.
> _____________
> Roman Shpount
I fully agree that G.729 should be tested without the B annex, but I 
think your other choice is questionable. G.729 may sound distinctly 
better than G.729A, but G.729A is the dominant codec in the real world. 
Basic G.729 is actually quite rare. Are you trying to test against what 
people are familiar with, or what a pretty good current generation 8kbps 
codec can achieve?

Steve

---

From: Roman Shpount <[email-masked]>
Date: Fri, 15 Apr 2011 14:13:51 -0400

Steve,

I was actually not 100% certain which flavor of G.729 should be selected. I
had exactly the same reasoning as you that G.729 is better quality, but
G.729A is more commonly used. Since my primary goal is to compare with
something that most people find acceptable quality for a voice call, testing
against G.729A is probably more appropriate.
_____________
Roman Shpount

On Fri, Apr 15, 2011 at 1:14 PM, Steve Underwood <[email-masked]> wrote:

> I fully agree that G.729 should be tested without the B annex, but I think
> your other choice is questionable. G.729 may sound distinctly better than
> G.729A, but G.729A is the dominant codec in the real world. Basic G.729 is
> actually quite rare. Are you trying to test against what people are familiar
> with, or what a pretty good current generation 8kbps codec can achieve?
>
> Steve
>

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 15 Apr 2011 15:13:44 -0600

Just a side note about iLBC... Cisco ships iLBC on most the voice platform it supports - this covers a wide range of optimized fixed point implementations. It is a used by many of customers. Some service provider terminate far more PSTN traffic with iLBC than the do with G.711 and G.729 combined.  Many other vendors also have widespread support for iLBC. Much of the data about how many minutes of each codec is considered confidential but if you have access to some of that data, have a look. 

Cullen <with my cisco employee hat on >

On Apr 14, 2011, at 7:20 PM, Roman Shpount wrote:

> I know that iLBC was picked by PacketCable and is supported by quite a few gateways. On the other hand, iLBC standard itself looks as if the codec was contributed by one company with almost nobody investing any time in its actual standardization. Quality comparisons and resource requirements seemed to be coming from the same company and accepted on their say so. Most of the deployed versions of iLBC are a floating point reference implementation, which is coded in a manner as inefficient as a codec implementation can possible be. There are no open source fixed-point implementations, even though creating one is trivial. Reference vectors consist of a single audio file recorded by a single voice. Encoder, despite being a bit exact reference, produces different results based on optimization options or compiler. To summarize, I assumed that there is no interest in this code since there are a lot of problems with it that would have been easy to fix if anybody bothered. It looks like a proof of concept and not a finished product. Finally, amount of voice traffic carried by iLBC is only a small fraction of G.729.
> _____________
> Roman Shpount
> 
>

---

From: Alan Duric <[email-masked]>
Date: Sat, 16 Apr 2011 12:07:30 +0200

Dear Roman,

despite the fact that I was not able to follow work of the group for quite some time, I was informed about Your post form colleagues and I needed to react, since there is quite a number of disinformations that You are placing (though not sure if by purpose or just by not knowing better)!

Comments are inline

On Apr 15, 2011, at 3:20 AM, Roman Shpount wrote:

I know that iLBC was picked by PacketCable and is supported by quite a few gateways. On the other hand, iLBC standard itself looks as if the codec was contributed by one company with almost nobody investing any time in its actual standardization.

A number of people that were involved in the verification were and are unarguably among the authorites in the speech coding world and have been heavily involved in the previous standardization efforts by number of standardization bodies. Their reputation is a way to big to endorse something what they do not fully stand behind. In the past a number of standards brought up by ITU, ETSI and TIA have been primarily or solely contributed by one company and it does not make those standards less valuable.

Quality comparisons and resource requirements seemed to be coming from the same company and accepted on their say so.

Please see above.As well, Dynastat (at that point of time one of the best independent labs) have done assessment of the codec in comparison to other existing standards. Results are available through IETF archive.

Most of the deployed versions of iLBC are a floating point reference implementation, which is coded in a manner as inefficient as a codec implementation can possible be.

Where do You get this information??? This is prime example of  "say so"! Information like this are confusing those that are following mailing list and I find it: either malicious or - You do not know much of VOIP deployments in general and thus should not be raising Your voice in front of this many people! CalbeLabs and Skype are terminating with iLBC more minutes towards PSTN then with any other codec combined. My company also happens to terminate billions of voice minutes traffic since 2004 and through very close cooperation with voice (or media) gateway manufacturers and by being advisor to a number of major carriers, I trust I have pretty good insight on implementations of various codecs and minutes that are carried with them.

There are no open source fixed-point implementations, even though creating one is trivial. Reference vectors consist of a single audio file recorded by a single voice. Encoder, despite being a bit exact reference, produces different results based on optimization options or compiler. To summarize, I assumed that there is no interest in this code since there are a lot of problems with it that would have been easy to fix if anybody bothered. It looks like a proof of concept and not a finished product. Finally, amount of voice traffic carried by iLBC is only a small fraction of G.729.

Please do not post stuff like this to the mailing list before doing any due diligence in advance, since people may not take seriously Your other comments that are valuable. G.729 is not much deployed at all, I suppose You are referring to G.729A implementations, which by the number of carried VoIP minutes is dwarfed by iLBC at this point of time. My company carries over 60 pct of all residential international traffic from Norway and cooperates with world leading carriers and we all know the issues of terminating G.729A carried calls on GSM based systems. That is the reason why You will find "premium" quality minutes from number of carriers conveyed by iLBC, but never with G.729A!

Let's rather use (deployment) facts then "say so" when bringing something as important to communications world as OPUS is!

Best regards,
Alan

Alan Duric
CTO and Cofounder
Telio Holding ASA | Oslo exchange: TELIO

twitter:  alanduric
linkedin.com/in/alanduric<http://linkedin.com/in/alanduric>

On Thu, Apr 14, 2011 at 8:55 PM, Cullen Jennings <[email-masked]<mailto:[email-masked]>> wrote:

<ATT00001..txt>

---

From: Anisse Taleb <[email-masked]>
Date: Mon, 18 Apr 2011 23:40:34 +0000

> The market can figure out the comparative quality question on its own.

What is even more funny, is that now we see more arguments against having a standard at all.

Let the market decide !

> -----Original Message-----
> Gregory Maxwell
> Sent: Thursday, April 14, 2011 4:25 PM
> To: Roni Even; [email-masked]
> Subject: [codec] comparitive quality testing
>

---

From: Monty Montgomery <[email-masked]>
Date: Tue, 19 Apr 2011 04:05:37 -0400

> What is even more funny, is that now we see more arguments against having a standard at all.

Non-standards are not even part of the market.

Monty

---

From: Roman Shpount <[email-masked]>
Date: Tue, 19 Apr 2011 19:08:33 -0400

Dear Alan,

This discussion is a bit unrelated to the overall discussion in the group,
but I would still like to clarify some of my points.

As far as iLBC market penetration is concerned, what I know (and please
correct me if I am wrong), none of the major US VoIP carriers (AT&T,
Verizon, Qwest, Level 3, XO, GlobalCorssing) offer iLBC termination to their
customers. It is a choice of G.711 or G.729 A or B only. This is either
caused by the luck of market demand, or by them using Sonus gateways which,
as far as I know, do not support iLBC.

Once again, as far as I know Skype is not using iLBC for anything. It is
using Silk, G.711. G.729A, and ISAC.

As far as I know none of the cable VoIP operators in US use iLBC, it is all
G.711.

As far as hardware vendors are concerned, Cisco does support iLBC on its
gateways a quite a few (but I do not think all) of its phones. Sonus only
supports iLBC in its SBC, but not in gateways. Polycom, Snom, Aastra do not
seem to support iLBC in their phones.

Where iLBC seemed to thrive is in soft-phones and various types of desktop
based software VoIP products. In such products the benefits of not having a
per instance license are probably the greatest. iLBC is also universally
supported by open source projects such as Asterisk, Freeswitch, PJ-SIP,
OpenH323 descendents, and so on. iLBC is also present in Android and iOS.
What is interesting about all of those products (and I did check pretty much
all the open source projects I can find) that the iLBC implementation in
these products are essentially the reference code from the RFC. This code is
extremely inefficient. What I mean by this it consumes more then 10 times
the CPU resources of Voiceage or IPP optimized G.729A. With Freeswitch or
Asterisk you cannot get more the 50-100 channels of iLBC per modern dual CPU
server. On the other hand running over 500 channels of G.729A is fairly
normal. What was interesting for me that no one spent any time optimizing
any of the open source versions of the codec even though this is fairly
trivial. On the other hand, such high CPU requirements on server and mobile
platforms make iLBC almost unusable. If anyone on this list knows the open
source optimized or fixed point implementation, please point it out to me.

There is also a question of ILBC Freeware license. Based on IETF rules, all
the code released as a part of IETF RFC should be covered by BSD license.
The reasoning and applicability of iLBC Freeware license in this context is
unclear.

IPR disclosure for iLBC RFC was not updated when RFC was finalized and only
has provisional patent applications, not the final granted patent numbers.

Finally, I would like to switch to something that has direct applicability
to the current working group. iLBC claims to be the same computation
complexity as G.729A. From what I know, ILBC reference implementation is
about 3-4 times the complexity of G.729A. It is possible to create an iLBC
implementation of the same complexity as G.729A, but this will, from what
I've seen, cause the quality degradation similar to degradation between
G.729 and G.729A. I did not measure this quality degradation, but I know
that, for instance, reducing the number of computation points in LPC to LSF
conversion, to bring it inline with G.729A, will produce different encoded
audio. All the quality tests for iLBC, from what I've seen, were done using
the reference code. I assume (I have no good way of testing this), that the
optimized code in Cisco and other production gateways will produce different
encoded audio. I have never seen the quality test for this optimized code.
In case of testing codecs where implementation varies for the same
bitstream, it is very important to compare quality and computation
complexity of the same implementation. Otherwise you will get results that
are misleading.

Furthermore, reference code in the RFC, based on the compiler used and
optimization options will produce different encoded bitstream on the same
input audio. In some cases encoded bitsream will match the reference vector
provided on iLBCfreeware.org, but will produce different results on other
audio files. Some differences are expected in decoder, since it is a
floating point implementation, but I am not sure encoded audio differences
are expected according even to iLBC Freeware site.

If we plan to run comparison test with iLBC we need to specify which
implementation we are planning to use. Additionally we need to specify
platform, compiler version, and compilation settings. Finally, we need to
specify if enhancer should be used in decoder. If we will not do this, the
test input will not be identical across the test sites.
_____________
Roman Shpount

On Sat, Apr 16, 2011 at 6:07 AM, Alan Duric <[email-masked]> wrote:

> Dear Roman,
>
> despite the fact that I was not able to follow work of the group for quite
> some time, I was informed about Your post form colleagues and I needed to
> react, since there is quite a number of disinformations that You are placing
> (though not sure if by purpose or just by not knowing better)!
>
> Comments are inline
>
>
> actual standardization.
> A number of people that were involved in the verification were and are
> unarguably among the authorites in the speech coding world and have been
> heavily involved in the previous standardization efforts by number of
> standardization bodies. Their reputation is a way to big to endorse
> something what they do not fully stand behind. In the past a number of
> standards brought up by ITU, ETSI and TIA have been primarily or solely
> contributed by one company and it does not make those standards less
> valuable.
>
> Quality comparisons and resource requirements seemed to be coming from the
> same company and accepted on their say so.
>
>
> Please see above.As well, Dynastat (at that point of time one of the best
> independent labs) have done assessment of the codec in comparison to other
> existing standards. Results are available through IETF archive.
>
>
> Most of the deployed versions of iLBC are a floating point reference
> implementation, which is coded in a manner as inefficient as a codec
> implementation can possible be.
>
>
> Where do You get this information??? This is prime example of  "say so"!
> Information like this are confusing those that are following mailing list
> and I find it: either malicious or - You do not know much of VOIP
> deployments in general and thus should not be raising Your voice in front of
> this many people! CalbeLabs and Skype are terminating with iLBC more minutes
> towards PSTN then with any other codec combined. My company also happens to
> terminate billions of voice minutes traffic since 2004 and through very
> close cooperation with voice (or media) gateway manufacturers and by being
> advisor to a number of major carriers, I trust I have pretty good insight on
> implementations of various codecs and minutes that are carried with them.
>
> There are no open source fixed-point implementations, even though creating
> one is trivial. Reference vectors consist of a single audio file recorded by
> a single voice. Encoder, despite being a bit exact reference, produces
> different results based on optimization options or compiler. To summarize, I
> assumed that there is no interest in this code since there are a lot of
> problems with it that would have been easy to fix if anybody bothered. It
> looks like a proof of concept and not a finished product. Finally, amount of
> voice traffic carried by iLBC is only a small fraction of G.729.
>
>
> Please do not post stuff like this to the mailing list before doing any due
> diligence in advance, since people may not take seriously Your other
> comments that are valuable. G.729 is not much deployed at all, I suppose You
> are referring to G.729A implementations, which by the number of carried VoIP
> minutes is dwarfed by iLBC at this point of time. My company carries over 60
> pct of all residential international traffic from Norway and cooperates with
> world leading carriers and we all know the issues of terminating G.729A
> carried calls on GSM based systems. That is the reason why You will find
> "premium" quality minutes from number of carriers conveyed by iLBC, but
> never with G.729A!
>
> Let's rather use (deployment) facts then "say so" when bringing something
> as important to communications world as OPUS is!
>
>
>
> Best regards,
> Alan
>
>
> Alan Duric
> CTO and Cofounder
> linkedin.com/in/alanduric
>
>
>
>

---

From: Stephan Wenger <[email-masked]>
Date: Tue, 19 Apr 2011 17:45:40 -0700

Interesting information.  Two comments inline (after having removed a lot of
valuable info).
Stephan

From:  Roman Shpount <[email-masked]>
Date:  Tue, 19 Apr 2011 19:08:33 -0400
To:  Alan Duric <[email-masked]>
Cc:  "[email-masked]" <[email-masked]>
Subject:  Re: [codec] comparitive quality testing

[Š]

iLBC, and the drafts leading to the RFC, pre-date the requirement for code
in RFCs to use the BSD style license.  Therefore, the authors had the right
to license their code even in the RFC text under any license compliant with
the then in force IPR documents, which were 2026 (overall procedures), 3667
(for copyright), and 3688 (for patent rights).  I believe the RFC is in
compliance with these documents.

Further, no one requires an author to provide the IETF trust with an
exclusive license for code portions of an RFC.  Therefore, anyone is free to
make available code contributed to an I-D or RFC (and, thereby and for the
purpose of this discussion, BSD-style licensed) also under licenses other
than the BSD style license.  They just have to use a different medium than
the RFC text.

I'm not aware of the nature of the patent rights, or their numbers, that
were disclosed by GlobalIPSound.  I can't find such information in the
disclosure.  That said, those with a bit of patent experience, access to the
right tools, and and a few hours of time, can find the relevant patents
covered by the disclosure with limited effort and reasonable certainty.

Even today the IETF does not require "updates" of IPR disclosures beyond
what is mandated in the rather generic language of the policy documents.
Some rightholders go at great length in keeping their disclosures updated,
others don't.

The really interesting question in this context is whether google finds
itself bound to the promises made by GIPS at the time.  The policy documents
provide no guidance on this topic, and this area of law is currently
litigated in several venues, with (so far) rather inconclusive results.

[Š]

Stephan

---

From: Steve Underwood <[email-masked]>
Date: Wed, 20 Apr 2011 09:12:53 +0800

On 04/20/2011 07:08 AM, Roman Shpount wrote:
> [...]
>
> Where iLBC seemed to thrive is in soft-phones and various types of 
> desktop based software VoIP products. In such products the benefits of 
> not having a per instance license are probably the greatest. iLBC is 
> also universally supported by open source projects such as Asterisk, 
> Freeswitch, PJ-SIP, OpenH323 descendents, and so on. iLBC is also 
> present in Android and iOS. What is interesting about all of those 
> products (and I did check pretty much all the open source projects I 
> can find) that the iLBC implementation in these products are 
> essentially the reference code from the RFC. This code is extremely 
> inefficient. What I mean by this it consumes more then 10 times the 
> CPU resources of Voiceage or IPP optimized G.729A. With Freeswitch or 
> Asterisk you cannot get more the 50-100 channels of iLBC per modern 
> dual CPU server. On the other hand running over 500 channels of G.729A 
> is fairly normal. What was interesting for me that no one spent any 
> time optimizing any of the open source versions of the codec even 
> though this is fairly trivial. On the other hand, such high CPU 
> requirements on server and mobile platforms make iLBC almost unusable. 
> If anyone on this list knows the open source optimized or fixed point 
> implementation, please point it out to me.
As someone who adapted the iLBC reference code into a packaged library 
that it easy to use, I can comment on this. I produced a library with a 
nice clean API, consistent with other codecs I have implemented, as a 
first step. This went into projects like FreeSwitch, and I waited for 
the demand for a faster version for servers, and a fixed point version 
for embedded users. The call has yet to come. I have no idea how much 
faster you can make iLBC. I haven't even been pushed to do that 
analysis. There are certainly people using iLBC with the various open 
source projects, but its mostly small scale users. Most big users either 
go for G.711 where they have sufficient bandwidth, or G.729A where they 
don't. Because of the points below, I am somewhat happy about this.
>
> There is also a question of ILBC Freeware license. Based on IETF 
> rules, all the code released as a part of IETF RFC should be covered 
> by BSD license. The reasoning and applicability of iLBC Freeware 
> license in this context is unclear.
Correct. The copyright situation for the floating point reference code 
is vague. The right to adapt that code to highly optimised floating 
point, fixed point code, or other implementations is vaguer. Nobody 
appears interested in clarifying things. The development of RFC3951 
seems quite unlike most IETF work.
>
> IPR disclosure for iLBC RFC was not updated when RFC was finalized and 
> only has provisional patent applications, not the final granted patent 
> numbers.
Correct. We have little idea which GIPS patents might apply, and nobody 
ever seems to have seriously considered who else might have relevant 
patents.

iLBC isn't a great codec, but it didn't have to be. Pretty good and 
unencumbered is clearly a winning combination these days - look at how 
much wideband VoIP is G.722 vs anything more modern, more advanced, and 
more encumbered. With a clear legal position it might have been a 
winner, but today its an also ran.

Steve

---

From: Roman Shpount <[email-masked]>
Date: Tue, 19 Apr 2011 21:17:23 -0400

One more interesting point is that RFC 3551 or the related IPR
releases do not include any mention of the GIPS freeware license. The
only mention is on iLBCFreeware.org website, which technically has
nothing to do with RFC and cannot be used to impose additional
licensing restrictions on the code. The only copyright notice present
in the RFC is to IETF which I assume means BSD, but might need some
historic clarification.
_____________
Roman Shpount

On Tue, Apr 19, 2011 at 8:45 PM, Stephan Wenger <[email-masked]> wrote:
> […]
> […]

---

From: Roman Shpount <[email-masked]>
Date: Tue, 19 Apr 2011 21:29:59 -0400

Steve,

I've seen your version of this and other codec. Great work!

I have actually done the optimization of iLBC for Intel platform and
can testify that you can get about 10x performance boost. I guess the
ironic part is that I actually support couple of sizable projects that
use iLBC. This being said, I would love to have something better
quality, less CPU intensive to replace iLBC.

As far as patents are concerned, I guess we are not suppose to discuss
them here, but I have some concerns about iLBC that I would be happy
to discuss offline.
_____________
Roman Shpount

On Tue, Apr 19, 2011 at 9:12 PM, Steve Underwood <[email-masked]> wrote:
> As someone who adapted the iLBC reference code into a packaged library that
> it easy to use, I can comment on this. I produced a library with a nice
> clean API, consistent with other codecs I have implemented, as a first step.
> This went into projects like FreeSwitch, and I waited for the demand for a
> faster version for servers, and a fixed point version for embedded users.
> The call has yet to come. I have no idea how much faster you can make iLBC.
> I haven't even been pushed to do that analysis. There are certainly people
> using iLBC with the various open source projects, but its mostly small scale
> users. Most big users either go for G.711 where they have sufficient
> bandwidth, or G.729A where they don't. Because of the points below, I am
> somewhat happy about this.
>>
>> There is also a question of ILBC Freeware license. Based on IETF rules,
>> all the code released as a part of IETF RFC should be covered by BSD
>> license. The reasoning and applicability of iLBC Freeware license in this
>> context is unclear.
>
> Correct. The copyright situation for the floating point reference code is
> vague. The right to adapt that code to highly optimised floating point,
> fixed point code, or other implementations is vaguer. Nobody appears
> interested in clarifying things. The development of RFC3951 seems quite
> unlike most IETF work.
>>
> Correct. We have little idea which GIPS patents might apply, and nobody ever
> seems to have seriously considered who else might have relevant patents.
>
> unencumbered is clearly a winning combination these days - look at how much
> wideband VoIP is G.722 vs anything more modern, more advanced, and more
> encumbered. With a clear legal position it might have been a winner, but
> today its an also ran.
>
> Steve
>

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 26 Apr 2011 11:04:02 -0600

On Apr 19, 2011, at 5:08 PM, Roman Shpount wrote:

> Dear Alan,
> 
> This discussion is a bit unrelated to the overall discussion in the group, but I would still like to clarify some of my points.
> 
> As far as iLBC market penetration is concerned, what I know (and please correct me if I am wrong), none of the major US VoIP carriers (AT&T, Verizon, Qwest, Level 3, XO, GlobalCorssing) offer iLBC termination to their customers.

Hmm, at least some of the above are willing to sell iLBC termination to PSTN.

> It is a choice of G.711 or G.729 A or B only. This is either caused by the luck of market demand, or by them using Sonus gateways which, as far as I know, do not support iLBC.

Most the call flows I see to a Sonus box have an ACME Packet SBC in front of them. ACME fixes everything including the codec :-) I may be mistaken but I was under the impression that some Sonus equipment does support iLBC. 

> 
> Once again, as far as I know Skype is not using iLBC for anything. It is using Silk, G.711. G.729A, and ISAC.
> 
> As far as I know none of the cable VoIP operators in US use iLBC, it is all G.711.
> 
> As far as hardware vendors are concerned, Cisco does support iLBC on its gateways a quite a few (but I do not think all) of its phones.

> Sonus only supports iLBC in its SBC, but not in gateways. Polycom, Snom, Aastra do not seem to support iLBC in their phones.

Cisco has it on most of it's phones. I just did a grep of some call flows that I had from various phones.  At least some phones from snom support iLBC, and some phones from aastra, and some from polycom, and a boat load over other vendors that matched my grep.  I can't tell you how prevalent it is across other vendors product lines but the three you mention do support it in at least some of the phones. 

> 
> Where iLBC seemed to thrive is in soft-phones and various types of desktop based software VoIP products. In such products the benefits of not having a per instance license are probably the greatest. iLBC is also universally supported by open source projects such as Asterisk, Freeswitch, PJ-SIP, OpenH323 descendents, and so on. iLBC is also present in Android and iOS. What is interesting about all of those products (and I did check pretty much all the open source projects I can find) that the iLBC implementation in these products are essentially the reference code from the RFC. This code is extremely inefficient. What I mean by this it consumes more then 10 times the CPU resources of Voiceage or IPP optimized G.729A. With Freeswitch or Asterisk you cannot get more the 50-100 channels of iLBC per modern dual CPU server. On the other hand running over 500 channels of G.729A is fairly normal. What was interesting for me that no one spent any time optimizing any of the open source versions of the codec even though this is fairly trivial. On the other hand, such high CPU requirements on server and mobile platforms make iLBC almost unusable.

> If anyone on this list knows the open source optimized or fixed point implementation, please point it out to me.
> 
> There is also a question of ILBC Freeware license. Based on IETF rules, all the code released as a part of IETF RFC should be covered by BSD license. The reasoning and applicability of iLBC Freeware license in this context is unclear.
> 
> IPR disclosure for iLBC RFC was not updated when RFC was finalized and only has provisional patent applications, not the final granted patent numbers.
> 
> Finally, I would like to switch to something that has direct applicability to the current working group. iLBC claims to be the same computation complexity as G.729A. From what I know, ILBC reference implementation is about 3-4 times the complexity of G.729A. It is possible to create an iLBC implementation of the same complexity as G.729A, but this will, from what I've seen, cause the quality degradation similar to degradation between G.729 and G.729A. I did not measure this quality degradation, but I know that, for instance, reducing the number of computation points in LPC to LSF conversion, to bring it inline with G.729A, will produce different encoded audio. All the quality tests for iLBC, from what I've seen, were done using the reference code. I assume (I have no good way of testing this), that the optimized code in Cisco and other production gateways will produce different encoded audio. I have never seen the quality test for this optimized code. In case of testing codecs where implementation varies for the same bitstream, it is very important to compare quality and computation complexity of the same implementation. Otherwise you will get results that are misleading.
> 
> Furthermore, reference code in the RFC, based on the compiler used and optimization options will produce different encoded bitstream on the same input audio. In some cases encoded bitsream will match the reference vector provided on iLBCfreeware.org, but will produce different results on other audio files. Some differences are expected in decoder, since it is a floating point implementation, but I am not sure encoded audio differences are expected according even to iLBC Freeware site.
> 
> If we plan to run comparison test with iLBC we need to specify which implementation we are planning to use. Additionally we need to specify platform, compiler version, and compilation settings. Finally, we need to specify if enhancer should be used in decoder. If we will not do this, the test input will not be identical across the test sites.
> _____________
> Roman Shpount
> 
>

---

From: Alan Duric <[email-masked]>
Date: Tue, 26 Apr 2011 23:14:56 +0200

Dear Roman,

unfortunately, You are continuing with "say so" statements ...  Please read my postings before You reply and use facts and sources, rather then throwing FUD!

Basically, You have expanded our initial conversation about iLBC deployments with 2 additional topics. I would like to focus on iLBC deployments and close on that subject on this thread. As per other newly introduced topics:
1) iLBC implementations and quality
- my response to You is that it depends on how good work is done with the implementation. There is a number of very good and excellent implemenations (e.g by silicon & gateway vendors and a number of ATA/IP Phone vendors), which run in a similar manner as implemented and "advertised" by GIPS at the time.

2) iLBC license
- this new introduced topic I propose that we take as separate thread and discussion point, not confuse it with this thread. However, there is so far no charges pressed to anyone, years after it has been deployed. As per G.729A, I am also very glad that it has been released on royalty free basis to the community, which was also pretty much happened due to availability of iLBC.

As per iLBC deployments, please find response inline:

> As far as iLBC market penetration is concerned, what I know (and please correct me if I am wrong), none of the major US VoIP carriers (AT&T, Verizon, Qwest, Level 3, XO, GlobalCorssing) offer iLBC termination to their customers. It is a choice of G.711 or G.729 A or B only.

Yes, You are wrong! One of the listed carriers (which is present in Europe) is offering it. According to Sonus there is 8 of major US carriers using iLBC!

> This is either caused by the luck of market demand, or by them using Sonus gateways which, as far as I know, do not support iLBC.

Where do You get this info??? Sonus supports iLBC on all of their gateways since GSX 6.5.0R0 (to be more exact under a feature PCR 111). This has been available for a number of years - the best illustration is that this software version is now going out of life ... Source is Sonus and my company, we are Sonus customers since end of 2005 and running billions of minutes of traffic on Sonus.
> Once again, as far as I know Skype is not using iLBC for anything. It is using Silk, G.711. G.729A, and ISAC.

And again ... where do You get all this info from??? A number of wrong statements in only one sentence (BTW, SILK has phased out pretty much completely ISAC a long time ago).
> As far as I know none of the cable VoIP operators in US use iLBC, it is all G.711.

This is also not true, i trust that someone from CableLabs that follows the list will react on this (providing us with first hand info).

> 
> As far as hardware vendors are concerned, Cisco does support iLBC on its gateways a quite a few (but I do not think all) of its phones. Sonus only supports iLBC in its SBC, but not in gateways. Polycom, Snom, Aastra do not seem to support iLBC in their phones.

As Cullen mentioned it is supported by Cisco, some Polycoms and also a whole range of Asian vendors that by now own quite a bit of the market around the world. I know that SNOM 870 also supports iLBC and I trust a number of their phones  ... Where do You get all this??

As mentioned in my previous email, if You disagree please provide us with facts and sources, otherwise it is just "say so" and should be treated as such ...

Best regards,
Alan

Alan Duric
CTO and Cofounder

twitter: 	alanduric

---

From: Roman Shpount <[email-masked]>
Date: Tue, 26 Apr 2011 18:12:02 -0400

Alan,

1. As far as iLBC implementation is concerned, encoder is not specified to
be bit exact. This means you can produce any encoder that you, as an
implementer, would consider are close enough and claim that they satisfy the
standard. You can take shortcuts, and the resulting audio quality will vary
based on that, as will the required computation power. My point is, that the
reference implementation, which is part of RFC is much more computationally
expensive then G.729A. You can build a version of the codec, that has the
same computation requirements as G.729A, but it will probably come at a cost
of quality degradation. For instance you will have to change how LPC to LSF
conversion is done, since reference implementation computes polynomial
values in a lot smaller intervals then G.729A. I am sure there are good iLBC
implementations on the market, but those are not the ones measured for
quality, and not the ones with which OPUS is going to be compared with. Once
again, and this is the main point of this entire discussion, if we choose to
compare with iLBC, we need to very clearly define what we are comparing with
and keep in mind that this is probably not the version running in production
gateways or phones from Cisco or other manufacturers.

2. I agree on iLBC license.

3. I said, none of those carriers OFFER iLBC to their customers. I never
said that it is not used anywhere within their network, even though I would
be very interested to know what it is used for. In the carrier service
agreements you have a choice of PCMU or G.729A. You can probably get iLBC
from those carriers, but you need to be big enough and convincing enough to
get it. Carrier SBCs would definitely be able to convert audio to iLBC if
this is something required by the customer, so those carriers definitely
have the capability to offer iLBC. The point is, as far as I know, it is not
offered.

4. The statement about Sonus gateways comes from the Sonus web site, where
iLBC is not mentioned as supported codec for their gateways, and from
talking to the carriers I use (pretty much the whole list I mentioned in my
previous email), that said that Sonus gateways do not support it. They did
mention that they are using GSX 6.4, so this information about Sonus can be
outdated or the sales engineers can be misinformed.

5. Can you tell me when Skype is using iLBC? Is there a way for me to place
a call using Skype and see iLBC as a codec used in the technical info? ISAC
is still used when calling old (ancient to be precise) Skype clients.

6. I would love to hear form somebody from CableLabs or a from a cable
operator to tell us how they use iLBC. As a side note, cable operators have
no motivation of using low bitrate codec since they have plenty of capacity
to the customer.

7. My information about phones comes from the manufacturer web sites and the
specifications for the phones there. For instance, SNOM 870 datasheet (
http://www.snom.com/en/products/ip-phones/snom-870/) or datasheets for any
other SNOM phones never mention iLBC.

I am not trying to spread FUD. I am telling you what I've seen from my
personal experience and I clearly state this. If you can show me the
examples of the opposite, I would love to see them.

Regards,
_____________
Roman Shpount

On Tue, Apr 26, 2011 at 5:14 PM, Alan Duric <[email-masked]> wrote:

> Dear Roman,
>
> unfortunately, You are continuing with "say so" statements ...  Please read
> my postings before You reply and use facts and sources, rather then throwing
> FUD!
>
> Basically, You have expanded our initial conversation about iLBC
> deployments with 2 additional topics. I would like to focus on iLBC
> deployments and close on that subject on this thread. As per other newly
> introduced topics:
> - my response to You is that it depends on how good work is done with the
> implementation. There is a number of very good and excellent implemenations
> (e.g by silicon & gateway vendors and a number of ATA/IP Phone vendors),
> which run in a similar manner as implemented and "advertised" by GIPS at the
> time.
>
> 2) iLBC license
> - this new introduced topic I propose that we take as separate thread and
> discussion point, not confuse it with this thread. However, there is so far
> no charges pressed to anyone, years after it has been deployed. As per
> G.729A, I am also very glad that it has been released on royalty free basis
> to the community, which was also pretty much happened due to availability of
> iLBC.
>
> > This discussion is a bit unrelated to the overall discussion in the
> group, but I would still like to clarify some of my points.
> >
> customers. It is a choice of G.711 or G.729 A or B only.
>
> Yes, You are wrong! One of the listed carriers (which is present in Europe)
> is offering it. According to Sonus there is 8 of major US carriers using
> iLBC!
>
> > This is either caused by the luck of market demand, or by them using
> Sonus gateways which, as far as I know, do not support iLBC.
>
> Where do You get this info??? Sonus supports iLBC on all of their gateways
> since GSX 6.5.0R0 (to be more exact under a feature PCR 111). This has been
> available for a number of years - the best illustration is that this
> software version is now going out of life ... Source is Sonus and my
> company, we are Sonus customers since end of 2005 and running billions of
> minutes of traffic on Sonus.
> >
> And again ... where do You get all this info from??? A number of wrong
> statements in only one sentence (BTW, SILK has phased out pretty much
> completely ISAC a long time ago).
> >
> This is also not true, i trust that someone from CableLabs that follows the
> list will react on this (providing us with first hand info).
>
> >
> As Cullen mentioned it is supported by Cisco, some Polycoms and also a
> whole range of Asian vendors that by now own quite a bit of the market
> around the world. I know that SNOM 870 also supports iLBC and I trust a
> number of their phones  ... Where do You get all this??
>
> As mentioned in my previous email, if You disagree please provide us with
> facts and sources, otherwise it is just "say so" and should be treated as
> such ...
>
> Best regards,
> Alan
>
>
> Alan Duric
> CTO and Cofounder

---

From: Roman Shpount <[email-masked]>
Date: Tue, 26 Apr 2011 18:24:32 -0400

Correction to my previous email: Snom m3 DECT phone supports iLBC according
to its data sheet, all the other Snom phones do not mention iLBC. Once
again, it is possible that all of their data sheets on the web site are
outdated.
_____________
Roman Shpount

On Tue, Apr 26, 2011 at 6:12 PM, Roman Shpount <[email-masked]> wrote:

> 7. My information about phones comes from the manufacturer web sites and
> the specifications for the phones there. For instance, SNOM 870 datasheet (
>> unfortunately, You are continuing with "say so" statements ...  Please
>> read my postings before You reply and use facts and sources, rather then
>> throwing FUD!
>>
>> Yes, You are wrong! One of the listed carriers (which is present in
>> Europe) is offering it. According to Sonus there is 8 of major US carriers
>> using iLBC!
>>
>> This is also not true, i trust that someone from CableLabs that follows
>> the list will react on this (providing us with first hand info).
>>
>> >

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 27 Apr 2011 12:36:53 +0200

In order to keep the sanity of those of us who know something about 
codec deployment and other relevant topics, but less about comparative 
quality testing, could people please try to use a subject line that 
reflects the conversation's content?

                     Harald

On 04/26/11 23:14, Alan Duric wrote:
> - my response to You is that it depends on how good work is done with the implementation. There is a number of very good and excellent implemenations (e.g by silicon&  gateway vendors and a number of ATA/IP Phone vendors), which run in a similar manner as implemented and "advertised" by GIPS at the time.
>
> 2) iLBC license

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 27 Apr 2011 09:30:24 -0600

Roman - I understand your pain on this. It is really hard to figure out what codecs a company's devices support by looking at their website. I'd love to give you an exact list of exactly what codecs every cisco product supports but I can't figure that out and I work there. Even when you can figure it out for a product, it's hard to know if that product is widely used of barely deployed at all. Even harder to know is how carriers actually configure and use things in their networks. Statistics from sipit are highly skewed towards products early in their development cycle while deployments tend to consist of products late that have been around for awhile. Most of the actually usage statistics in deployments are considered confidential. From my point of view, most major vendors support iLBC in many of their products that do narrowband voip across the internet and they do so because there is a large demand for it in the market place. On one side note about license agreements. In the acquisitions I have been involved with, the acquiring company typically continues to honor all the commitments made by the company they acquired. Given Google's general goals on widespread interoperable codecs, I'd be simply shocked to see them start suing people over iLBC IPR. 

On Apr 26, 2011, at 4:24 PM, Roman Shpount wrote:

> Correction to my previous email: Snom m3 DECT phone supports iLBC according to its data sheet, all the other Snom phones do not mention iLBC. Once again, it is possible that all of their data sheets on the web site are outdated.
> _____________
> Roman Shpount
> 
> 
> 1. As far as iLBC implementation is concerned, encoder is not specified to be bit exact. This means you can produce any encoder that you, as an implementer, would consider are close enough and claim that they satisfy the standard. You can take shortcuts, and the resulting audio quality will vary based on that, as will the required computation power. My point is, that the reference implementation, which is part of RFC is much more computationally expensive then G.729A. You can build a version of the codec, that has the same computation requirements as G.729A, but it will probably come at a cost of quality degradation. For instance you will have to change how LPC to LSF conversion is done, since reference implementation computes polynomial values in a lot smaller intervals then G.729A. I am sure there are good iLBC implementations on the market, but those are not the ones measured for quality, and not the ones with which OPUS is going to be compared with. Once again, and this is the main point of this entire discussion, if we choose to compare with iLBC, we need to very clearly define what we are comparing with and keep in mind that this is probably not the version running in production gateways or phones from Cisco or other manufacturers.  
> 
> 3. I said, none of those carriers OFFER iLBC to their customers. I never said that it is not used anywhere within their network, even though I would be very interested to know what it is used for. In the carrier service agreements you have a choice of PCMU or G.729A. You can probably get iLBC from those carriers, but you need to be big enough and convincing enough to get it. Carrier SBCs would definitely be able to convert audio to iLBC if this is something required by the customer, so those carriers definitely have the capability to offer iLBC. The point is, as far as I know, it is not offered.
> 
> 4. The statement about Sonus gateways comes from the Sonus web site, where iLBC is not mentioned as supported codec for their gateways, and from talking to the carriers I use (pretty much the whole list I mentioned in my previous email), that said that Sonus gateways do not support it. They did mention that they are using GSX 6.4, so this information about Sonus can be outdated or the sales engineers can be misinformed.
> 
> 5. Can you tell me when Skype is using iLBC? Is there a way for me to place a call using Skype and see iLBC as a codec used in the technical info? ISAC is still used when calling old (ancient to be precise) Skype clients.
> 
> 6. I would love to hear form somebody from CableLabs or a from a cable operator to tell us how they use iLBC. As a side note, cable operators have no motivation of using low bitrate codec since they have plenty of capacity to the customer.
> 
> 7. My information about phones comes from the manufacturer web sites and the specifications for the phones there. For instance, SNOM 870 datasheet (http://www.snom.com/en/products/ip-phones/snom-870/) or datasheets for any other SNOM phones never mention iLBC.
> 
> I am not trying to spread FUD. I am telling you what I've seen from my personal experience and I clearly state this. If you can show me the examples of the opposite, I would love to see them.
> 
> Regards,
> _____________
> Roman Shpount
> 
> 
>

---

From: Jean-Francois Mule <[email-masked]>
Date: Fri, 29 Apr 2011 04:06:45 -0600

Hi, Alan and Roman,

   I am not sure what this thread leads the working group but I'd like to
say a few points about iLBC.

Roman wrote:
> which adds iLBC and Speex to the list, but I personally see this as
> less of a requirement. iLBC never managed to get market traction
> outside of the open source world,
Doing comparative quality testing against G.711, iLBC, G.729, G.722, and
G.722.2 AMR-WB provides considerable values to many vendor companies that
have implemented these codecs and to numerous service providers that have
deployed them (or are considering to).  I'd like to see iLBC added to the
list.

My team at CableLabs did quite a bit of work on iLBC and other codecs
almost 10 years ago.
Around 2002, we launched a project to find a royalty-free, narrowband
codec that could be implemented in PacketCable(tm) networks and that could
be submitted to our IPR pool including codec implementation references.
All we had when we started was G.711.  I'll skip the multi-year process...
iLBC was selected by CableLabs after some extensive testing outsourced to
two well-known and independent testing labs staffed with what I considered
at the time worldwide codec experts having worked on many codecs and
implementations (including some well-known standard codecs).  iLBC became
part of our PacketCable 1.1 now called PacketCable 1.5 specifications
prior to the codec becoming an IETF Experimental RFC.
In terms of voice codec deployments in cable networks, the fact is that it
is mostly G.711 today in the field - no questions there. Each cable
operator makes its own decision. The reasons for this lack of iLBC
deployments in cable networks are numerous and for the most part, outside
codec considerations.

That said, iLBC has been implemented by many as noted on the list.
I'm not sure if this was mentioned but my wireshark capture of an Apple
Facetime call shows a SIP invite with support for iLBC in addition to
X-AAC_ELD, X-G722 and PCMU.
--- Facetime call capture done in Oct 2010
User-Agent: Viceroy 1.4.1/GK
        Content-Type: application/sdp
        Content-Length: 587
    Message Body
        Session Description Protocol
            Session Description Protocol Version (v): 0
            Owner/Creator, Session Id (o): GKVoiceChatService 0 0 IN IP4
192.160.73.209
                Owner Username: GKVoiceChatService
                Session ID: 0
                Session Version: 0
 
...

Media Attribute (a): rtpmap:124 iLBC/8000

Jean-Francois.
[email-masked]

On 04/26/2011 23:14, "Alan Duric" <[email-masked]> wrote:

>implementation. There is a number of very good and excellent
>implemenations (e.g by silicon & gateway vendors and a number of ATA/IP
>Phone vendors), which run in a similar manner as implemented and
>"advertised" by GIPS at the time.
>
>2) iLBC license
>discussion point, not confuse it with this thread. However, there is so
>far no charges pressed to anyone, years after it has been deployed. As
>per G.729A, I am also very glad that it has been released on royalty free
>basis to the community, which was also pretty much happened due to
>availability of iLBC.
>
>>Verizon, Qwest, Level 3, XO, GlobalCorssing) offer iLBC termination to
>>their customers. It is a choice of G.711 or G.729 A or B only.
>
>Europe) is offering it. According to Sonus there is 8 of major US
>carriers using iLBC!
>
>Where do You get this info??? Sonus supports iLBC on all of their
>gateways since GSX 6.5.0R0 (to be more exact under a feature PCR 111).
>This has been available for a number of years - the best illustration is
>that this software version is now going out of life ... Source is Sonus
>and my company, we are Sonus customers since end of 2005 and running
>billions of minutes of traffic on Sonus.
>> 
>> As far as hardware vendors are concerned, Cisco does support iLBC on
>>its gateways a quite a few (but I do not think all) of its phones. Sonus
>>only supports iLBC in its SBC, but not in gateways. Polycom, Snom,
>>Aastra do not seem to support iLBC in their phones.
>

---

From: Roman Shpount <[email-masked]>
Date: Fri, 29 Apr 2011 19:14:55 -0400

Jean-Francois,

My argument against iLBC essentially comes down to two points:

1. It is argued that we should compare with iLBC since it is IPR free and
unencumbered CODEC. What I am not sure about is the current license for the
iLBC source code. RFC 3951 specifies that the code belongs to IETF.
iLBCFreeware.org states that the code is covered by "GIPS Royalty Free
License", which is strange since you normally should mention this in the
code to be binding. At the same time "GIPS Royalty Free License" gives no
rights to use GIPS IP which is part of iLBC. It only covers the code. IPR
release specifies that GIPS will provide a license for the codec IP upon
written request, but this license is not publicly available. GIPS is now
Google, and even if Google will still provide a royalty free license per
IPR, this license can have other terms in it that might restrict iLBC use.
It would be very interesting to see this license. Second problem are IPR
disclosures. None of them have been updated when RFC was accepted and all of
them only mention the provisional patents. It would be nice to have actual
patent numbers if the patents where granted or if they still apply to the
final RFC. Finally, based on the latest Qualcomm disclosure in regard to
Opus, it is unclear why similar disclosures weren't filed against Silk and
iLBC. It is unclear if this because the patents do not apply or is this due
to some other reason. My assumption was that iLBC was not extremely
important to the market so that no one followed up to clarify the above
mentioned points. In any case, the reasoning for this is not important, but
the resulting iLBC licensing status is.

2. iLBC is not a bit exact codec. You can have multiple standard compliant
implementations that would produce audio of different quality. Reference
implementation is a lot more computationally demanding then G.729A. There
are proprietary implementations on the market that are comparable in their
computational requirements to G.729A, but producing different encoded audio
then the reference. No open source fixed point or otherwise optimized iLBC
implementations are available, so there is nothing that we can use to
estimate the audio quality of an optimized iLBC implementation. The
reasoning for not having such implementation is unclear, but I assumed this
was due to the luck of interest in the market. Once again, the reasoning is
irrelevant. What this means, that if we are testing against reference iLBC
implementation built from the RFC code even though it might be producing a
better quality audio then the optimized implementations. If we are comparing
against the reference we need to specify compiler and compilation options
used to build the codec, since codec output changes based on it. These
changes might not affect the quality, but would produce an unreproducible
test. In some sense, stating in the requirements that Opus should be better
then iLBC is the same as stating it should be better then MP3 without
specifying the encode or the options used.

>From my point of view we can spend time and clarify these two issues (or at
least the second issue). Alternatively we can remove the requirement to test
against iLBC, and spend our limited resource testing Opus against other
codecs.
_____________
Roman Shpount