Subject: Re: [codec] questions to be answered

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Mon, 2 Nov 2009 08:41:44 -0500

All,

I strongly agree with Roni here, re: "I think that the if the charter
describes a list of applications it need to have a separate requirement
for each application on which this proposed group is going to develop a
separate codec."

Let's consider two SIMPLE examples.

Example 1 - Let's assume I have an application that requires a certain
linearity within a certain BW ranges (need > 25 dB of waveform linearity
in the band 0 - 4k, > 20 db from 4k - 8k, etc). Perhaps this application
uses adaptive signal processing techniques and linearity is important.
Perhaps it is a far-end echo cancellation application, or a microphone
beam array processing application that processes multiple streams at the
receiver end. I could go on with any number of interesting examples.
Note that such an applications would generate specific REQUIREMENTS to
meet these use cases. The candidate codec would probably be
waveform-based (at least in the bands having the linearity
requirements).

Example 2 - Many voice codecs use spectral techniques (such as spectral
band replication, SBR) above 4k. Such codecs, while potentially
capturing the spectral content well enough, will NOT preserve the phase
(as the phase information is not in the compressed representation). Such
a codec would NOT meet the simple > 20 dB linearity requirement (of the
4k - 8k band) of the previous paragraph. Will such a codec SOUND GOOD to
a human - perhaps, as the human auditory system is not as sensitive to
phase in this region. Additionally, many of these codecs are speech
model based - and typically have < 20 dB of linearity on unvoiced speech
even in 0 - 4k region. (You could throw in an entropy coding stage after
the model to fix this - at the cost of BW - but I digress).

One might ask why codec manufacturers use techniques such as SBR?
Answer: because they are generally more efficient (precisely because
they don't require as much phase information in the compressed domain)
and sound good to humans.

I could go on and on ... but I sincerely doubt that ONE MAGICAL codec
can satisfy both examples above ... because both APPLICATIONS would
result in different REQUIREMENTS.

I think the rush to a "single-codec" requirements here is premature ...
I agree with Roni ... one size does not fit all.

Given the simple examples above - isn't it clear that we need to define
applications (one might call them use cases) - and then requirements for
them - before we determine how a given candidate satisfies the
requirements?

At a minimum, I see two legitimate codec classes:

CLASS 1) codecs that are bandwidth efficient and sound good to humans
(and perhaps satisfy tendeming or tone pass-through requirements), and

CLASS 2) codecs whose output may later be processed by sophisticated
signal processing functions (high-quality mixing, many
classification-based applications such as speaker/sound/intrusion
identification) that have SNR/waveform/distortion criteria generally
exceeding CLASS 1.

Granted, we should make each use case as broad as possible so as not to
have too many of them. Once one codec is standardized for a particular
application type - a subsequent candidate codec/application would need
to prove it is "sufficiently different" from the existing codec(s) to go
further. The WG chairs need to exert leadership to ensure the desired
result (of the smallest number of codecs spanning the greatest
application space).

IF
a given codec meets the requirements of its use case
AND
that use case is valid
THEN
I (for one) don't have an issue with the IETF "rubber stamping" of it.

If you have a specific use case - then get together with other similarly
minded folks and generate a "use case" draft and then a requirements
draft for that use case. Don't try to solve world hunger in one step.

Off Soapbox,

Michael Ramalho, Ph.D.

[Flames expected ... but please try to disprove my debate above when
doing so ... I dislike flames that do not disprove a thesis ;-) ]

PS - I think CLASS 1 codecs should be addressed first by the WG ... and
later consider other classes as people advocate for their use case.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Roni Even
Sent: Friday, October 30, 2009 6:34 PM
To: 'Peter Saint-Andre'; [email-masked]
Subject: Re: [codec] questions to be answered

Hi Peter,
I am not sure that there is a consensus about what the group need to
deliver. I do not feel that the issue of the number of codecs and how to
decide which codecs are in scope for this working group is agreed. My
view
that this issue is not addressed since I assume that all the parties
that
submitted candidate codecs would like their codecs to be approved at the
IETF which will make this group a rubber stamping WG.
I think that the if the charter describes a list of applications it need
to
have a separate requirement for each application on which this proposed
group is going to develop a separate codec. If there is one requirement
document for all apllications there should be one codec to address it.

Thanks
Roni Even

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Peter Saint-Andre
> Sent: Friday, October 30, 2009 10:12 PM
> To: [email-masked]
> Subject: [codec] questions to be answered
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> RFC 5434 has a list of questions that need to be answered before a WG
> is
> formed, and I think we have fairly clear answers to these questions:
> 
> 1. Is there a problem that needs solving?
> 
> (Yes, see problem statement in charter.)
> 
> 2. Is the scope of the problem well defined and understood?
> 
> (Yes, see objectives in charter, guidelines I-D, and requirements
I-D.)
> 
> 3. Is there agreement on what the WG needs to deliver?
> 
> (Yes, see deliverables in charter, guidelines I-D, and requirements I-
> D.)
> 
> 4. Are there enough IETF participants to do the work?
> 
> (Yes, see codec I-Ds, poll at Stockholm BoF, and list discussion.)
> 
> 5. Does the WG have a reasonable probability of success?
> 
> (Yes, see codec I-Ds, guidelines I-D, and list discussion, with an
> emphasis on "reasonable" because we won't know until we try.)
> 
> 6. Is the IETF is the right place to do the work?
> 
> (Maybe. I think this is the major item to clarify at the BoF.)
> 
> Peter
> 
> - --
> Peter Saint-Andre
> https://stpeter.im/
> 
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.8 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> 
> iEYEARECAAYFAkrrSIIACgkQNL8k5A2w/vxzmQCeINOY/RWKd924HGWH8p4BJmUz
> lHIAnAwhO+Htb6Cxne9b9w/1WEnaRlgP
> =9vPK
> -----END PGP SIGNATURE-----
> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec

_______________________________________________
codec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/codec

---

From: Jean-Marc Valin <[email-masked]>
Date: Mon, 02 Nov 2009 09:24:03 -0500

Hi Michael,

Defining the requirements for each set of applications is exactly what 
we've been trying to do in section 2 of the requirements draft. Maybe we 
just need to expand this section with more detailed requirements.

> receiver end.

Well, right now there is no application that requires waveform similarity 
in the draft, so if that is needed, we should add a section to the 
requirements draft.

> I agree with Roni ... one size does not fit all.

Indeed, I am now thinking that we really shouldn't set a maximum number of 
codecs for the proposed WG until we have a complete list of target 
applications and requirements.

> them - before we determine how a given candidate satisfies the

As I said, this is really the intent of Section 2 of the requirements 
draft. Though I agree it can probably be improved with more requirements 
and more details on the existing ones.

> (and perhaps satisfy tendeming or tone pass-through requirements), and

It seems like all the applications we have so far belong to this class.

> identification) that have SNR/waveform/distortion criteria generally

Can you formulate uses cases for that one?

My personal opinion is that with some work it should possible to satisfy 
both classes you list above with a single codec (possibly with different 
encoding options), but of course that remains to be verified in practice.

> result (of the smallest number of codecs spanning the greatest

I don't think mapping applications to different codecs is something that 
can be done in advance. In fact, I wouldn't be too surprised if we ended up 
  with some classes of applications needing two codecs and/or having 
several applications covered by the same codec.

> I (for one) don't have an issue with the IETF "rubber stamping" of it.

Wouldn't that lead to more codecs than necessary?

> later consider other classes as people advocate for their use case.

Well, your two classes above represents a split along one possible 
dimension, which is waveform vs perceptual-only approach. There are many 
more possible splits, for instance: layered vs non-layered, algorithmic 
delay, VBR vs CBR, speech vs music, fullband vs narrowband, ...

Having one codec for each is neither desirable, nor necessary. We will need 
to take a more global approach to split things up. Otherwise, one can argue 
that all the already-submitted codecs will fall into a different class a 
can be rubberstamped on its own. And if we do that, Roni will not be happy 
:-) Nor will I.

Cheers,

	Jean-Marc

---

From: "Roni Even" <[email-masked]>
Date: Mon, 2 Nov 2009 17:11:10 +0200

Hi Jean-Marc,
I looked at section 2 and it does not provide enough information why you
need multiple codecs
I would like to start with editorial comment. If you want to show
requirements you should start with the parameters by which you define the
codec like bit rate , sampling rate, hands free support,... and build a
table that will include all the values for each application you think should
be addressed. This will enable to justify the need for different codecs and
to be able to qualify a codec for application. I think that having such
information deserves a separate requirement document for each application.
The current text has some values for some of the examples and some do not
have any values. 

Regards
Roni Even

> From: Jean-Marc Valin [mailto:[email-masked]]
> Sent: Monday, November 02, 2009 4:24 PM
> To: Michael Ramalho (mramalho)
> Cc: Roni Even; Peter Saint-Andre; [email-masked]

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 02 Nov 2009 10:35:42 -0700

> The current text has some values for some of the examples and some do not

Roni and all, this is a great discussion and just the kind of work that
a Codec WG should be doing. So let's charter the WG and get moving. :)

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Koen Vos <[email-masked]>
Date: Mon, 02 Nov 2009 09:53:03 -0800

Can we be more clear about what "multiple codecs" really means. After  
all, two codecs can always be merged to create a new single codec  
(with two modes). One question is: should any implementation of the  
codec always support all modes? Probably not.

We do need to make sure that having different modes is no problem with  
respect to signaling. Things like: can we indicate during call  
negotiation what modes are available, and can we signal mode switches  
during a call? (I just don't know if these are straightforward or not.)

koen.

---

From: Gregory Maxwell <[email-masked]>
Date: Mon, 2 Nov 2009 10:02:51 -0800

> codec always support all modes? Probably not.

"should any implementation of the codec always support all modes"
Is a key question.

My basic view on the matter is that if a mode is something that we wouldn't
expect to make mandatory for all conforming implementations *and* taking
it out or implementing only that mode substantially reduces implementation
complexity,  then we should have two codecs instead.

If, on the other hand, there is some kind of mode which will either be so
widely useful that it would be mandatory or doesn't have much impact on
implementation complexity,  then we should just have one codec which
signals the mode in use in-band.

---

From: "Roni Even" <[email-masked]>
Date: Mon, 2 Nov 2009 21:47:26 +0200

Peter,
This has to do with the charter, there need to be a decision how many codec are developed for one requirement document.
It looks to me based on the discussion that there is no consensus between the people who propose this charter since all have similar codecs they want to approve and are not looking at collaboration.
Roni Even

> From: Peter Saint-Andre [mailto:[email-masked]]
> Sent: Monday, November 02, 2009 7:36 PM
> To: Roni Even
> Cc: 'Jean-Marc Valin'; 'Michael Ramalho (mramalho)'; [email-masked]
> iEYEARECAAYFAkrvGG4ACgkQNL8k5A2w/vxRZQCfSodq7Q0OrswnkjAGncSD1hdz
> 9HQAoNjwao8FF8Pwem5WGWOtGFvOgvDO
> =5EdO

---

From: "Roni Even" <[email-masked]>
Date: Mon, 2 Nov 2009 21:52:50 +0200

Hi,
I thought that the group will not make mandatory requirements for
implementation so having a codec with extensions that can be negotiated is
the preferred direction.
Roni

> From: Gregory Maxwell [mailto:[email-masked]]
> Sent: Monday, November 02, 2009 8:03 PM
> To: Koen Vos; Roni Even
> Cc: [email-masked]
> Subject: RE: [codec] questions to be answered
>

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 02 Nov 2009 12:56:27 -0700

On 11/2/09 12:47 PM, Roni Even wrote:
> Peter, This has to do with the charter, there need to be a decision
> how many codec are developed for one requirement document. It looks
> to me based on the discussion that there is no consensus between the
> people who propose this charter since all have similar codecs they
> want to approve and are not looking at collaboration.

I see no evidence for that statement.

All of the individuals and teams who have submitted codecs have done so
with the understanding that each contribution will probably not be
approved, and even if approved would undergo modification (i.e., even a
codec that functioned as a "starting point" codec would be *adapted*,
not *adopted*). This is consistent with the following statement in
draft-valin-codec-guidelines-02:

   Furthermore, contributors need to be
   aware that any codec that results from work within the IETF is likely
   to be different from any existing codec that was contributed to the
   Internet Standards Process.

All of the speakers I heard at the BoF in Stockholm made it clear that
they expect to and are eager to collaborate on a codec that meets the
requirements, and that they do not expect to simply get their own codecs
approved as-is.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: "Roni Even" <[email-masked]>
Date: Mon, 2 Nov 2009 22:08:28 +0200

Peter,
So if there is no problem have the charter reflect how many codecs there will be. 
I suggest one for each requirement document
Roni Even

> Sent: Monday, November 02, 2009 9:56 PM
> To: Roni Even
> iEYEARECAAYFAkrvOWoACgkQNL8k5A2w/vyTagCbBEaXVzB2i5W+eY13RuEI3dPR
> /2cAoIl3k1ih6Wb05EtVeEzlWIQ6g9lm
> =Cswk

---

From: "Christian Hoene" <[email-masked]>
Date: Tue, 3 Nov 2009 10:30:03 +0100

Hello Michael,

> signal processing functions (high-quality mixing, 

High-quality mixing? That do you have in mind? Stereo or binaural (3D sound)
mixing?

> many

You may add distributed speech recognition to the second class. However, to
my understanding the current offered codecs have not been designed with the
second class in mind. Also, class 2 might not fall in the interactive
category (which implies that communication takes place between humans).
By the way, are you training your classification-based applications while
taking the distortions caused by the transmission system into consideration?

You forgot: We need to add CLASS 3 for video and CLASS 4 for general
content, too.

> result (of the smallest number of codecs spanning the greatest

Yes, actually Noll and Jayant have proposed to use µ-Law to support speech,
audio AND video compression. But to my understanding it is common consensus
to concentrate on a limited (e.g. class 1) scenario and finished this work
first before setting up another construction site.

Christian

---

From: "Christian Hoene" <[email-masked]>
Date: Tue, 3 Nov 2009 10:30:03 +0100

Hello Peter,

> requirements, and that they do not expect to simply get their own codecs

This all might be true. However, at the moment we are discussing the charter
and not the drafts nor any oral statements, which all can be updated or
revoked at any time. Thus, it would be beneficial having some high level
goals in the charter that help to decide on the number of codecs.

These could be for e.g. "Different codecs must have unmergable, orthogonal
requirements" or "One codec and a low number of optional extensions that
address additional requirements".

With best regards,

 Christian

---

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Tue, 3 Nov 2009 10:33:56 -0500

Hi Christian,

My replies are in-line below (w/ MAR:).

Thanks for your consideration.

Michael Ramalho

-----Original Message-----
From: Christian Hoene [mailto:[email-masked]] 
Sent: Tuesday, November 03, 2009 4:30 AM
To: Michael Ramalho (mramalho); [email-masked]
Subject: RE: [codec] questions to be answered

Hello Michael,

MAR: I continued in the email to say that I could potentially think of lots of other examples that require some bounds on the linear distortion because of the signal processing desired at the receiver. But yes, one could conceive of mixing to reproduce the effect of where spatially the sound came from in the sending location using only the signals rendered at the receiver.

second class in mind.

MAR: Precisely. I am purposely not limiting my thinking to the current offered codecs.

MAR: Please note that I am not proposing Class 2 work at this time ... my point is that some organization in the future could advocate such in the future and NOT burden the present "Class 1" codec work with (what I believe are) unnecessary requirements.

MAR: Personally, I think there will be a need for Class 2 codecs soon ... but I think it is most important NOT to burden the present work with "low-runner" or "special-application" requirements. The whole point of my dissertation was that one codec cannot fit "all Internet applications". We should focus first on what the WG feels is the most pressing application - a bandwidth efficient, Internet-friendly, human-consumable codec. This codec is likely to be model-based. If someone has a "pet requirement" outside the general Class 1 codec consensus - let them propose a use case (which would include a market need statement) and let the WG consider whether that "pet requirement" should be included in our first work or left for another day.

Also, class 2 might not fall in the interactive
category (which implies that communication takes place between humans).
By the way, are you training your classification-based applications while
taking the distortions caused by the transmission system into consideration?

MAR: If the application is not real-time interaction ... we would send via reliable protocols ... and such a codec need not be as robust to packet loss.

MAR: Again I am not advocating a Class 2 codec at this time ... but apparently you are knowledgeable about matched training/testing conditions for ASR systems and the like.

MAR: No, I didn't. I'll let someone who has convinced the WG chairs of a market need, a use case, proposed requirements for any additional classes to be considered. I would hope that a very limited number of codecs are developed (see next paragraph).

audio

MAR: This is patently false if you consider audio to have content above 4 kHz. All natural signals with finite energy (a guitar string pluck, a glottal speech pulse) have a spectral roll-off with increasing frequency (e.g., speech has approximately a 6 dB roll-off per octave). G.711 has a flat distortion characteristic with frequency. This means if you go out far enough in frequency (depends on the signal) your SNR turns negative at some point. I have personally interacted with Dr. Jayant at Bell Labs Murray Hill and I can assure you that he would NOT advocate G.711 for other than a narrowband speech audio application.

AND video compression.

MAR: Definitely false ... but would require too long an explanation (luminance related) ;-). I can't tell if you sarcastic-mode is set to on.

But to my understanding it is common consensus
to concentrate on a limited (e.g. class 1) scenario and finished this work
first before setting up another construction site.

MAR: And that is, in essence, my proposal - except I have a tweak. If you have a "pet requirement", I (personally) don't want it to bugger-up the process of getting to a good a bandwidth efficient, Internet-friendly, human-consumable codec.

Christian

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 05 Nov 2009 10:25:28 -0700

> address additional requirements".

These are good topics to discuss in the BoF.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Michael Knappe <[email-masked]>
Date: Mon, 9 Nov 2009 12:40:03 -0800

In looking at building the table illustrating codec applications vs. criteria, the problem space appears to be parameterized by three major criteria:

A) end-to-end latency: objectively, achieving the least amount of latency possible is better for real-time communications. Psychoacoustically, achieving end-to-end delays less than 150 ms has diminishing quality returns for human speech communication, while the specific application of live music interaction requires less than 50 ms end-to-end delay (and ideally 25 ms or less). Clearly, the live music interaction requirement requires not only a low delay codec, but also a carefully controlled, minimal jitter and very low propagation delay network. The 150 ms requirement, on the other hand, has more leeway in allocating latency budget between codec algorithmic delays and other sources of delay in the network. Bottom line: live music interaction appears to be an outlier relative to other codec applications with respect to end-to-end latency.

B) number of channels: objectively, recreating as complete a representation of the original source sound field is better for real-time communications. For two-party speech communications, source material from a single microphone can obviously be transmitted with a single audio channel, with the decision to present the signal to one or more audio outputs made at the receiving end (e.g.  Single output telephone handset receiver, dual output binaural headphones). However, even for two-party speech communications with a single talker at each end, speakerphone communications can be improved by transmitting two channels of audio from two microphones on the speakerphone, that when listened to with stereo headphones at the receiving end, the 'barrel effect' common to mono microphone speakerphones is greatly reduced. This concept of transmission of soundfield along with speech (or music) becomes more important as more complex acoustic environments are involved (including the creation of virtual acoustic environments with multiple talker sources mixed into a common soundfield). Thus, the need for more transmission channels should increase with the application's desire to produce a more immersive environment. Bottom line: although single party source material in two-party communications could benefit from more than one transmission channel, applications like advanced conferencing / telepresence,  gaming and live music interaction are likely to benefit the most from multi-channel environments.

C) frequency bandwidth: objectively, reproducing the widest possible audio frequency bandwidth to cover the frequency content of the original source material is better for real-time communications. Human speech has been transmitted with intelligibility limitations for decades over bandwidths as narrow as 3 kHz. Speech intelligibility improves significantly as bandwidth is increased to 6 or 7 kHz, with diminishing intelligibility improvements as bandwidth is increased past this, even though the subjective measure of 'naturalness' may continue to improve up to the limits of human hearing. This concept of 'intelligibility' and 'naturalness' also applies to music - an example of intelligibility here could be discerning between an oboe and bassoon in a symphony, while naturalness is more a measure of how 'real' that singled-out oboe sounds. It would appear to follow that the more complex a sound source and soundfield that we wish to transmit, the wider the bandwidth we should encode. In the high fidelity and professional recording domains, this quest for 'naturalness' has extended beyond what we typically feel the human limitation of 20 kHz auditory perception is, to sample rates up to 192 kHz and beyond. Bottom line:  human speech in simple two-party interactions will receive the most benefit in transmitting the first 7 kHz of audio bandwidth, but more complex source material will continue to benefit from wider audio bandwidths, possibly extending beyond the traditional 20 kHz 'auditory limit'.

I have not included in the above criteria the concept of fundamental codec type - many previous telephony compression algorithms rely on parameterizations of the human mechanics of the generation of speech (e.g. CELP based codecs) which do not translate well to parameterization of other source material such as music, whereas auditory receiving end perceptual audio parameterizations tend to work well for both music and speech, but historically have had latency limitations. My thought here, for this proposed codec, is that we should not be looking for a codec where a priori knowledge of the type of audio material is required (although certainly the codec could be designed to optimize its performance dynamically for varying types of source material).

Also not included above are fundamental concepts of distortion and noise - for me, this is a secondary requirement that once A,B and C are established, that sufficient signal-to-noise ratios be maintained to not undermine intelligibility and naturalness goals of the transmission. This may mean a natural progression to higher SNR targets for wider bandwidth, more immersive environments.

Thoughts?

Mike

On 11/2/09 5:41 AM, "Michael Ramalho (mramalho)" <[email-masked]> wrote:

All,

IF

Off Soapbox,

Michael Ramalho, Ph.D.

-----Original Message-----

Hi Peter,

Thanks
Roni Even

I-D.)

---

From: Koen Vos <[email-masked]>
Date: Mon, 09 Nov 2009 19:29:24 -0800

These are good requirements at the high end, e.g. for virtual presence  
and interactive music.

At the low end, for basic voice, I think also -and sometimes more-  
important are:
- Bitrate
- Robustness to packet loss
- Complexity
- Real-time scalability (the ability to adjust parameters)
These aspects determine how well communications work in challenging  
scenarios like over wireless networks and on embedded devices.

koen.

Quoting Michael Knappe:
> In looking at building the table illustrating codec applications vs.  
> criteria, the problem space appears to be parameterized by three  
> major criteria:
>
> A) end-to-end latency: objectively, achieving the least amount of  
> latency possible is better for real-time communications.  
> Psychoacoustically, achieving end-to-end delays less than 150 ms has  
> diminishing quality returns for human speech communication, while  
> the specific application of live music interaction requires less  
> than 50 ms end-to-end delay (and ideally 25 ms or less). Clearly,  
> the live music interaction requirement requires not only a low delay  
> codec, but also a carefully controlled, minimal jitter and very low  
> propagation delay network. The 150 ms requirement, on the other  
> hand, has more leeway in allocating latency budget between codec  
> algorithmic delays and other sources of delay in the network. Bottom  
> line: live music interaction appears to be an outlier relative to  
> other codec applications with respect to end-to-end latency.
>
> B) number of channels: objectively, recreating as complete a  
> representation of the original source sound field is better for  
> real-time communications. For two-party speech communications,  
> source material from a single microphone can obviously be  
> transmitted with a single audio channel, with the decision to  
> present the signal to one or more audio outputs made at the  
> receiving end (e.g.  Single output telephone handset receiver, dual  
> output binaural headphones). However, even for two-party speech  
> communications with a single talker at each end, speakerphone  
> communications can be improved by transmitting two channels of audio  
> from two microphones on the speakerphone, that when listened to with  
> stereo headphones at the receiving end, the 'barrel effect' common  
> to mono microphone speakerphones is greatly reduced. This concept of  
> transmission of soundfield along with speech (or music) becomes more  
> important as more complex acoustic environments are involved  
> (including the creation
>  of virtual acoustic environments with multiple talker sources mixed  
> into a common soundfield). Thus, the need for more transmission  
> channels should increase with the application's desire to produce a  
> more immersive environment. Bottom line: although single party  
> source material in two-party communications could benefit from more  
> than one transmission channel, applications like advanced  
> conferencing / telepresence,  gaming and live music interaction are  
> likely to benefit the most from multi-channel environments.
>
> C) frequency bandwidth: objectively, reproducing the widest possible  
> audio frequency bandwidth to cover the frequency content of the  
> original source material is better for real-time communications.  
> Human speech has been transmitted with intelligibility limitations  
> for decades over bandwidths as narrow as 3 kHz. Speech  
> intelligibility improves significantly as bandwidth is increased to  
> 6 or 7 kHz, with diminishing intelligibility improvements as  
> bandwidth is increased past this, even though the subjective measure  
> of 'naturalness' may continue to improve up to the limits of human  
> hearing. This concept of 'intelligibility' and 'naturalness' also  
> applies to music - an example of intelligibility here could be  
> discerning between an oboe and bassoon in a symphony, while  
> naturalness is more a measure of how 'real' that singled-out oboe  
> sounds. It would appear to follow that the more complex a sound  
> source and soundfield that we wish to transmit, the wider the  
> bandwidth we should enco
>  de. In the high fidelity and professional recording domains, this  
> quest for 'naturalness' has extended beyond what we typically feel  
> the human limitation of 20 kHz auditory perception is, to sample  
> rates up to 192 kHz and beyond. Bottom line:  human speech in simple  
> two-party interactions will receive the most benefit in transmitting  
> the first 7 kHz of audio bandwidth, but more complex source material  
> will continue to benefit from wider audio bandwidths, possibly  
> extending beyond the traditional 20 kHz 'auditory limit'.
>
> I have not included in the above criteria the concept of fundamental  
> codec type - many previous telephony compression algorithms rely on  
> parameterizations of the human mechanics of the generation of speech  
> (e.g. CELP based codecs) which do not translate well to  
> parameterization of other source material such as music, whereas  
> auditory receiving end perceptual audio parameterizations tend to  
> work well for both music and speech, but historically have had  
> latency limitations. My thought here, for this proposed codec, is  
> that we should not be looking for a codec where a priori knowledge  
> of the type of audio material is required (although certainly the  
> codec could be designed to optimize its performance dynamically for  
> varying types of source material).
>
> Also not included above are fundamental concepts of distortion and  
> noise - for me, this is a secondary requirement that once A,B and C  
> are established, that sufficient signal-to-noise ratios be  
> maintained to not undermine intelligibility and naturalness goals of  
> the transmission. This may mean a natural progression to higher SNR  
> targets for wider bandwidth, more immersive environments.
>
> Thoughts?
>
> Mike
>

---

From: Henry Sinnreich <[email-masked]>
Date: Mon, 9 Nov 2009 20:46:19 -0800

>- Robustness to packet loss

Are you thinking of burst packet loss or random distributed packet loss?
And what would be the metrics and values for either?

If we have no metrics and no credible values, I suggest to forget this rat hole.
Broadband Internet has IMO and negligible packet loss, except for congestion events where the codec cannot do anything anyway.

Packet loss was a big deal in old analog modem times and pre-3G/4G wireless networks :-)

My two cents,

Henry

On 11/10/09 12:29 PM, "Koen Vos" <[email-masked]> wrote:

koen.

---

From: Michael Knappe <[email-masked]>
Date: Mon, 9 Nov 2009 20:49:45 -0800

Koen,

Agreed that bitrate, robustness, computational complexity (implementability and cost) and RT scalability are absolutely all important aspects, and not just for the low end. In particular, robustness to packet loss is important for any proposed codec. Bitrate, possibly coupled with scalability, help determine how far the codec can reach - poorly analogous to whether certain blood cells can make it all the way to capillaries or not. I'm thinking that this last concept become a fourth axis of 'codec reach', in addition to latency, channels, and bandwidth, thanks for highlighting this.

Computational complexity is an interesting thought process in itself - since we're looking at something that possibly won't be fully deployed for several years, how much leeway should we allow here for economical deployments? Should we allow bit 'compatible' versions in addition to bit exact qualifications to allow for flexibility in computational limitations?

Mike

On 11/9/09 7:29 PM, "Koen Vos" <[email-masked]> wrote:

koen.

---

From: Michael Knappe <[email-masked]>
Date: Mon, 9 Nov 2009 20:53:20 -0800

Agreed that pure network packet loss (or even packet reordering) is not usually the issue, but sudden jumps in jitter that cause temporary receiving end packet starvation or overflow while the adaptive jitter buffer works its magic would still be an issue in many deployment scenarios.

Mike

My two cents,

Henry

koen.

---

From: "Gregory M. Lebovitz" <[email-masked]>
Date: Mon, 09 Nov 2009 21:35:17 -0800

speaking w/ my individual contributor hat here...

>scenarios like over wireless networks and on embedded devices.

In the beginning, I'm more interested in seeing us solve basic voice 
over a concatenation of various types of networks, with excellent 
quality, and do so in a focused way.

I'd be more interested and eager to address the interactive music 
type applications in a 2nd wave, re-chartering phase.

Just mho,
Gregory.

---

From: Stefan Sayer <[email-masked]>
Date: Tue, 10 Nov 2009 06:37:25 +0100

o Henry Sinnreich [11/10/2009 05:46 AM]:
> Broadband Internet has IMO and negligible packet loss, except for 
> congestion events where the codec cannot do anything anyway.
> 
and it is still a big issue in multi-hop wireless networks.

Stefan

---

From: Michael Knappe <[email-masked]>
Date: Mon, 9 Nov 2009 21:52:26 -0800

Gregory,

Being a bit careful on terminology .... 'Basic' can mean a lot of different things...even the description of the codec BoF in the IETF agenda is "Internet Wideband Audio Codec" - wideband would not be considered 'basic' to a lot of voice transmission people.  'Excellent quality' also means a lot of different things to different people, which is another good reason to try and develop a common parameter and criteria language so we can all be on the same page when we discuss relative merits.

The underlying purpose of this ongoing application parameterization exercise is to build an 80/20 grouping of criteria to help narrow down the most effective focus of the proposed codec. For instance, just from the latency requirement alone it would appear interactive music is an outlier application that might cause undue limitations on a proposed codec for other codec applications, and I would agree that this might be something either to address as a second phase or not at all within the context of the proposed WG.

Cheers,

Mike

On 11/9/09 9:35 PM, "Gregory M. Lebovitz" <[email-masked]> wrote:
That
speaking w/ my individual contributor hat here...

Just mho,
Gregory.

---

From: Jean-Marc Valin <[email-masked]>
Date: Tue, 10 Nov 2009 15:00:30 +0900

Gregory M. Lebovitz a écrit :
> I'd be more interested and eager to address the interactive music type
> applications in a 2nd wave, re-chartering phase.

So far, I don't see a reason why all the requirements we currently have
cannot be addressed at the same time and -- ideally -- with the same
codec. But the best way to answer the question is to get the technical
work started and see the outcome. At this point, I have a very hard time
predicting what the outcome of the technology merge (of the 4 codecs
"submitted" already) would look like in terms of capabilities, so I
think it would be a bad idea to make assumptions about that in the
initial charter.

Otherwise, I totally agree with Koen about the bitrate, robustness,
complexity and scalability being very important aspects.

Cheers,

	Jean-Marc

---

From: Jean-Marc Valin <[email-masked]>
Date: Tue, 10 Nov 2009 15:08:20 +0900

> The underlying purpose of this ongoing application parameterization
> exercise is to build an 80/20 grouping of criteria to help narrow
> down the most effective focus of the proposed codec. For instance,
> just from the latency requirement alone it would appear interactive
> music is an outlier application that might cause undue limitations on
> a proposed codec for other codec applications, and I would agree that
> this might be something either to address as a second phase or not at
> all within the context of the proposed WG.

Again, this is implicitly assuming the outcome of the technical work. I
could very well imagine that the "natural split" in terms of
applications could be between "medium quality" (narrowband/wideband)
speech-only codec using an LP-based source model and very high quality
super-wideband transform-based codec. Of course, it's reasonable to
ensure that addressing the requirements of one application not cause a
degradation of quality for other applications.

	Jean-Marc

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 10 Nov 2009 08:01:47 +0100 (CET)

> If we have no metrics and no credible values, I suggest to forget this rat hole.

I *strongly* disagree to this line of thinking. This thinking is what is 
causing a lot of the current system implementations (codec and transport 
together) to not work properly on "the Internet". See my previous email on 
this list on this issue.

I can come up with metrics that are realistic, that I have seen in real 
life. I might not have the exact numbers, but I have scenarios where they 
occur.

> Packet loss was a big deal in old analog modem times and pre-3G/4G wireless networks :-)

This is still a problem (and no, it wasn't a problem on analog modems with 
ARQ), depending on where in the world you are. I can come up with multiple 
scenarios where I want this "Internet codec/transport" to work that 
current implementations do not work in, where for instance you trade 
latency for pps for instance. In some cases it's better to send 5 pps with 
500 byte packets to get the packets thru, than to send 50pps of smaller 
packets because they do not go thru. Yes, latency suffers, but it *works*.

I'd rather have a 2 second end-to-end latency for my call than no call at 
all.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Henry Sinnreich <[email-masked]>
Date: Mon, 9 Nov 2009 23:11:52 -0800

>I can come up with metrics that are realistic,
>that I have seen in real life.

Metrics and values/numbers would be very interesting if you can share them, to better understand the requirements.
Also to make sure in what usage scenarios they may occur.

Thanks, Henry

On 11/10/09 4:01 PM, "Mikael Abrahamsson" <[email-masked]> wrote:

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 10 Nov 2009 09:03:41 +0100 (CET)

On Mon, 9 Nov 2009, Henry Sinnreich wrote:

> Metrics and values/numbers would be very interesting if you can share 
> them, to better understand the requirements. Also to make sure in what 
> usage scenarios they may occur.

2 seconds jitter and 10% continous random packet loss should definitely be 
handled gracefully (during the tsunami where several fiber optic cables 
were severed in asia, packet loss from the region to the rest of the world 
was continous 5-7% due to remaining paths being extremely congested). 2 
seconds is for multiple links in a row being full (typically Cisco 12000 
routers can buffer 600ms of data when congestion occurs). There are 
numerous cases (ADSL2+ without interleave, high speed links with CRC 
errors, ethernet duplex errors) where random packet loss can be seen. 
Typically this is less than 1%, otherwise people tend to fix the problem 
because TCP works very badly.

We then have the 3G/wifi network issue, where jitter can also be at least 
2 seconds, plus packet loss as well (some networks seem to buffer X 
kilobytes, some Y number of packets, after that they tail drop. I've seen 
both behaviours).

Burst loss, where when you have a network re-route, you might lose 
typically 0.05 - 2 seconds of data, and you might get re-ordering during 
the convergence. Since this is a transient event, I don't think much can 
be done about it but try to play data as they come in and just ignore out 
of order packets. Perhaps jitter buffer should be increased when 
out-of-order packets is seen, but when things calm down again it needs to 
be reduced again.

I've also seen networks which re-order packets constantly, I don't know 
what the load-sharing algorithm used is, but I think the jitter buffer 
size should be adapted and handle out of order packets so they're 
presented to the codec correctly.

When running voice over GSM GPRS, typically there is very low bandwidth 
and high delay. Here bw should be kept as low as possible and the IP 
header is a big overhead and pps should thus be kept at a minimum, for 
instance 3-5 pps and the very lowest quality setting should be used.

So, the SYSTEM (I'm not just talking codec here), as far as I can see it, 
needs to gracefully handle at least 10% packet loss, out of order packets 
(analyse the time they might be re-ordered and adapt jitter buffer 
accordingly), or just plain jitter up to 2s.

My reasoning here is mainly focused on voice over Internet, but as far as 
I can see this applies to all sound, even though if it's not interactive I 
guess the focus can be switched from low latency (it might be better for a 
voice call to have lower latency than actually getting perfect sound) to 
reliable sound delivery with higher latency (by increasing jitter buffer 
and putting sound information in multiple packets so that single packet 
loss doesn't mean silence in that interval). Basically I'd like to see 
that as soon as jitter buffer is increased, the resiliancy to packet loss 
should be increased as well by telling the other end that it can start to 
do packet loss concealment information in the packets as well (whatever 
it's called, I'm not a codec guy). If sender knows what the receiver 
jitter buffer is, then this can be used to increase resiliency (I guess).

Does this help?

---

From: "Gregory M. Lebovitz" <[email-masked]>
Date: Tue, 10 Nov 2009 00:30:24 -0800

At 12:03 AM 11/10/2009, Mikael Abrahamsson wrote:
>>Metrics and values/numbers would be very interesting if you can 
>>share them, to better understand the requirements. Also to make 
>>sure in what usage scenarios they may occur.
>
>2 seconds jitter and 10% continous random packet loss should 
>definitely be handled gracefully (during the tsunami where several 
>fiber optic cables were severed in asia, packet loss from the region 
>to the rest of the world was continous 5-7% due to remaining paths 
>being extremely congested). 2 seconds is for multiple links in a row 
>being full (typically Cisco 12000 routers can buffer 600ms of data 
>when congestion occurs). There are numerous cases (ADSL2+ without 
>interleave, high speed links with CRC errors, ethernet duplex 
>errors) where random packet loss can be seen. Typically this is less 
>than 1%, otherwise people tend to fix the problem because TCP works very badly.
>
>We then have the 3G/wifi network issue, where jitter can also be at 
>least 2 seconds, plus packet loss as well (some networks seem to 
>buffer X kilobytes, some Y number of packets, after that they tail 
>drop. I've seen both behaviours).
>
>typically 0.05 - 2 seconds of data, and you might get re-ordering 
>during the convergence. Since this is a transient event, I don't 
>think much can be done about it but try to play data as they come in 
>and just ignore out of order packets. Perhaps jitter buffer should 
>be increased when out-of-order packets is seen, but when things calm 
>down again it needs to be reduced again.
>
>I've also seen networks which re-order packets constantly, I don't 
>know what the load-sharing algorithm used is, but I think the jitter 
>buffer size should be adapted and handle out of order packets so 
>they're presented to the codec correctly.
>
>When running voice over GSM GPRS, typically there is very low 
>bandwidth and high delay. Here bw should be kept as low as possible 
>and the IP header is a big overhead and pps should thus be kept at a 
>minimum, for instance 3-5 pps and the very lowest quality setting 
>should be used.
>
>So, the SYSTEM (I'm not just talking codec here), as far as I can 
>see it, needs to gracefully handle at least 10% packet loss, out of 
>order packets (analyse the time they might be re-ordered and adapt 
>jitter buffer accordingly), or just plain jitter up to 2s.
>
>My reasoning here is mainly focused on voice over Internet, but as 
>far as I can see this applies to all sound, even though if it's not 
>interactive I guess the focus can be switched from low latency (it 
>might be better for a voice call to have lower latency than actually 
>getting perfect sound) to reliable sound delivery with higher 
>latency (by increasing jitter buffer and putting sound information 
>in multiple packets so that single packet loss doesn't mean silence 
>in that interval). Basically I'd like to see that as soon as jitter 
>buffer is increased, the resiliancy to packet loss should be 
>increased as well by telling the other end that it can start to do 
>packet loss concealment information in the packets as well (whatever 

I'm not a codec designer, or expert in this area at all, but that 
seems VERY practical and tangible. Those are observable metrics for 
behavior, and you provided a clear explanation of WHY you offered 
those metrics, which make decent sense to me.

So, yes, it helps.

Gregory.

---

From: "Gregory M. Lebovitz" <[email-masked]>
Date: Tue, 10 Nov 2009 00:54:45 -0800

At 11:01 PM 11/9/2009, Mikael Abrahamsson wrote:
>I *strongly* disagree to this line of thinking. This thinking is 
>what is causing a lot of the current system implementations (codec 
>and transport together) to not work properly on "the Internet". See 
>my previous email on this list on this issue.
>
>I can come up with metrics that are realistic, that I have seen in 
>real life. I might not have the exact numbers, but I have scenarios 
>where they occur.
>

Well, I'm sitting here in the CONEX BoF where several cable operators 
have presented on why they think there is a problem with congestion 
and the associated service degradation and packet loss. It's 
anecdotal input, yes, but it's compelling.

>>Packet loss was a big deal in old analog modem times and pre-3G/4G 

Again, listen to the archived audio stream of CONEX, and the jabber 
archive too, which held a lot of discussion on the observed loss in networks.

Gregory.

>This is still a problem (and no, it wasn't a problem on analog 
>modems with ARQ), depending on where in the world you are. I can 
>come up with multiple scenarios where I want this "Internet 
>codec/transport" to work that current implementations do not work 
>in, where for instance you trade latency for pps for instance. In 
>some cases it's better to send 5 pps with 500 byte packets to get 
>the packets thru, than to send 50pps of smaller packets because they 
>do not go thru. Yes, latency suffers, but it *works*.
>
>I'd rather have a 2 second end-to-end latency for my call than no call at all.
>
>--

---

From: Michael Knappe <[email-masked]>
Date: Tue, 10 Nov 2009 01:02:01 -0800

Mikael,

Thanks for the very useful industry examples and metrics. Handling of these examples fall either into jitter buffer design (which so far in the BoF email correspondence, other than noting the need for some form of adaptive jitter buffer algorithm, is being treated as largely outside the scope of the proposed WG) or packet loss concealment (which IS often tied to the codec's design). As you've noted, other than doing the best it can to maintain a balance between the impairment due to end-to-end latency and impairment due to periods of packet loss concealment, there is really no crystal ball that can be employed to recover large numbers of consecutively lost packets during wildly varying jitter conditions.

As you mention in your email, it is possible to more tightly couple PLC and the jitter buffer. For instance, a proposed codec's decode operation could provide an intelligent feedback loop to the jitter buffer at the receiving end to let it know how well it's packet loss concealment algorithm is currently operating - for instance telling the jitter buffer to go ahead and rapidly increase its size if the PLC algorithm is simply falling apart under enormous packet loss. There are also examples of time varying / frequency invariant 'stretch' jitter buffers that are an integral component of the codec design, and we should treat them as a whole while evaluating any such proposed codecs.

Mike

On 11/10/09 12:03 AM, "Mikael Abrahamsson" <[email-masked]> wrote:

Does this help?

---

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Tue, 10 Nov 2009 09:29:49 -0500

Comments in-line with "MAR:".

Michael Ramalho

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Mikael Abrahamsson
Sent: Tuesday, November 10, 2009 3:04 AM
To: [email-masked]
Subject: Re: [codec] questions to be answered

> usage scenarios they may occur.

2 seconds jitter and 10% continous random packet loss should definitely
be 
handled gracefully (during the tsunami where several fiber optic cables 
were severed in asia, packet loss from the region to the rest of the
world 
was continous 5-7% due to remaining paths being extremely congested). 2 
seconds is for multiple links in a row being full (typically Cisco 12000

MAR: Let's consider ... 10% RANDOM packet loss ... and ptime = 20ms.

MAR: Probability of more than 2 consecutive losses (40ms conceal) is
lower bounded at 1% (once every 2 seconds at 50 pps).

MAR: Probability of more than 3 consecutive losses is bounded at .1%
(once every 20 seconds).

MAR: Probability of more than 4 consecutive losses is bounded at 0.01%
(once every 200 seconds).

MAR: Isolated packet losses are typically concealed well (even more so
if they occur less often).

MAR: A doublet loss (2 consecutive losses) are also typically concealed
well for long lasting vowels (and silence). So these too are TYPICALLY
concealed well - the exception being short lasting phonemes (such as a
"t"). But in the example above, this loss event only happens once every
2 seconds - what is the probability of the 40ms loss coming during the
short-lived phoneme? Still a very low occurrence. Maybe once every 5
minutes.

MAR: Extend analysis for > 3+ consecutive RANDOM losses.

MAR: Consecutive losses exceeding 200ms (10 consecutive losses) are
almost guaranteed to lose significant phonetic content. Most people hear
200ms drops consistently. But the time between such events - assuming
RANDOM LOSSES - is indeed very long (do the math ;-) )!

MAR: Is there any wonder that even 10% RANDOM packet loss is still
highly intelligible? No. But the key is the RANDOM (assuming i.i.d.)
loss assumption.

MAR: I assure you that i.i.d. random loss isn't the challenging case
(assuming a good PLC).

We then have the 3G/wifi network issue, where jitter can also be at
least 
2 seconds, plus packet loss as well (some networks seem to buffer X 
kilobytes, some Y number of packets, after that they tail drop. I've
seen 
both behaviours).

be done about it but try to play data as they come in and just ignore
out 
of order packets. Perhaps jitter buffer should be increased when 
out-of-order packets is seen, but when things calm down again it needs
to 
be reduced again.

MAR: OK .. now you jump from RANDOM loss to DROP OUTs. These occur
during such things as route changes. Yes, you will typically hear these
if they take more than about 100ms ~ 200ms to converge. If they occur
once .. and then not again for > 4 minutes or so .. the listener forgets
about them (just like bad cell phone reception).

MAR: But you jumped from RANDOM LOSS to DROP OUTs - there is a vast
middle ground where the characteristic of the losses ISN'T RANDOM. I
(and others) usually use the term "burst loss" to cover these cases that
are in between RANDOM losses and DROP OUTs.

So, the SYSTEM (I'm not just talking codec here), as far as I can see
it, 
needs to gracefully handle at least 10% packet loss, out of order
packets 
(analyse the time they might be re-ordered and adapt jitter buffer 
accordingly), or just plain jitter up to 2s.

MAR: This cannot be your conclusion EXCEPT if the 10% loss is nearer to
a i.i.d. random distribution than to any particularly (auditorially
nasty) burst characteristic.

MAR: As an aside, another person responding to this thread (not you)
talked about wireless (e.g., 802.11) characteristics. Well they can have
auditorially nasty burst loss characteristics when the endpoint is
nearly out-of-range. Layer 2 tries to get the VoIP packet across, but
"gives up" after a while ... it does this consistently because it is
nearly out-of-range ... and it gets some across.

MAR: The bottom line for anyone still reading is IF the loss event WIPES
OUT a particular phoneme completely (the "t" example above) NO AMOUNT OF
REAL-TIME CODEC MAGIC PIXIE DUST can resurrect the phoneme. The only way
around this is to add redundant information in other packets and to
trade off delay.

[I say real-time .. because you can run ASR and guess what phoneme(s)
was/were missing ... and then insert it .. but that isn't a real time
activity. I digress once again .. sorry about that.]

My reasoning here is mainly focused on voice over Internet, but as far
as 
I can see this applies to all sound, even though if it's not interactive
I 
guess the focus can be switched from low latency (it might be better for
a 
voice call to have lower latency than actually getting perfect sound) to

and putting sound information in multiple packets so that single packet 
loss doesn't mean silence in that interval).

MAR: If you don't include redundant information so that you can recover
"missing content", I don't agree with your conclusion.

MAR: All of this points to Henry's request. Please bound the problem. I
(for one) are really tired of people talking about the codec handling X%
RANDOM packet loss. VoIP streams will typically (assuming good PLC) be
highly intelligible until the probability of > 1 consecutive loss
becomes significant (up to about 20% ... again ... do the math at your
ptime to find out).

Basically I'd like to see 
that as soon as jitter buffer is increased, the resiliancy to packet
loss 
should be increased as well by telling the other end that it can start
to 
do packet loss concealment information in the packets as well (whatever 
it's called, I'm not a codec guy). If sender knows what the receiver 
jitter buffer is, then this can be used to increase resiliency (I
guess).

Does this help?

MAR: Anyone still listening can flame ... but please disprove the
analysis before you do so. Thanks in advance.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 10 Nov 2009 17:16:21 +0100 (CET)

> Comments in-line with "MAR:".

Please consider using the generally accepted "Internet style" way of 
citation outlined in: 
<http://en.wikipedia.org/wiki/Posting_style#Inline_replying>.

> if they occur less often).

Unless they cause crackling sound when this happens, so I guess it depends 
on the codec. I've had VOIP phone calls where it was very annoying.

> are in between RANDOM losses and DROP OUTs.

"jumped" sounds here like you're accusing me of being unstructured, or 
what are you trying to say?

> around this is to add redundant information in other packets and to

I thought I made this point numerous time in my email. What is the term to 
use so people like you understand that I want information regarding the 
audio spread out over several packets (interleaving?) so if some packet(s) 
are lost, at least some of the sound can be presented albeit at much lower 
quality? Every time jitter buffer is increased, I see no downside in using 
this technique to make the transfer of sound more robust?

> "missing content", I don't agree with your conclusion.

As stated before, my meaning was exactly to include redundant/partial 
information in multiple packets.

> analysis before you do so. Thanks in advance.

What is your suggestion as to what should be done? Do nothing at all?

---

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Tue, 10 Nov 2009 12:08:49 -0500

>on the codec. I've had VOIP phone calls where it was very annoying.

Your issue could be the codec (for the case that it does it's own PLC)
or the PLC. For reasonable ptimes (30ms or less), longer lived phonemes
and isolated losses - shame on your system for generating annoying
sounds. Sounds like you have a simple "repeat last frame" PLC. So circa
1970.

...

>What is your suggestion as to what should be done? Do nothing at all?

Quite the contrary, Mikael. I was simply trying to highlight two (highly
interrelated) points:

1 - That the burst lost characteristic is an important criteria -
designing for i.i.d. random losses where the losses are < 10% is
relatively easy if your goal is intelligibility (not transparency).
Again, shame on ANY 1990 ~ 2009 vintage codec for not concealing most
ISOLATED losses without auditory detriment.

2 - That Henry's suggestion (to bound the problem, define metrics) is a
good one.

Subsequent emails have continued w.r.t. the intelligibility/transparency
issue.

Thanks for your commentary. I didn't mean to frustrate, but rather to
enlighten.

Michael Ramalho