Subject: [codec] codec tandeming/codec-requirements

From: Michael Knappe <[email-masked]>
Date: Thu, 22 Oct 2009 10:56:59 -0700

One thing we might want to add to 'draft-valin-codec-requirements' is a brief discussion of quality expectations in a tandemed codec transmission path. For example, a potential scenario for the proposed codec, at least early on, may have the proposed codec tandemed between potentially two disparate cell codecs:

Cell phone 1 codec type 'A' -> transcode to internet codec -> transcode to cell phone 2 codec type 'B'

As we develop a quality methodology, it will be worthwhile to specify some typical tandem scenarios to evaluate voice quality in mixed codec environments, including tandeming of the proposed codec with itself (e.g. Two IP PBX's that get connected to each other over via TDM). Evaluations of different LBR codecs in the past have shown that under the same tandem scenario, two candidate codecs may get markedly different tandem MOS scores even though the two might have very similar single encode/decode scores.

Thoughts on that?

Michael Knappe
Juniper Networks

---

From: Gregory Maxwell <[email-masked]>
Date: Thu, 22 Oct 2009 11:53:55 -0700

Michael Knappe wrote:
> One thing we might want to add to 'draft-valin-codec-requirements' is a brief discussion of quality expectations in a tandemed codec transmission path.
[snip]
> two candidate codecs may get markedly different tandem MOS scores even though the two might have very similar single encode/decode scores.

In the single codec case it can also be important to distinguish 'synchronous' and 'asynchronous' tandeming: I.e. is the sample the same position in a frame in all cases or has there been some uncompensated offset.  This can make a fairly large difference (for example, tandeming is currently almost nearly in CELT (in terms of PQevalaudio ODG loss[1], at least) if you're careful to retain alignment— but the quality degrades with tandeming cycles if you do not).  I expect that since frame alignment can't always be easily preserved (i.e. an TDM stage in between) that both cases are worth profiling. 

I'm not sure that this could easily be converted into a clear requirement— but I think no one would consider it acceptable for a single stage to cause an enormous quality drop. 

[1] On a 64kbit/sec CBR encode of a random music clip I get a PEAQ ODG of -0.346 for one pass through the codec; -0.362 for 1000 passes. This is somewhat less than the loss from a 3kbit reduction in bitrate at  -0.369)

---

From: "Christian Hoene" <[email-masked]>
Date: Thu, 22 Oct 2009 21:32:46 +0200

Dear Michael Knappe,

> cell phone 2 codec type 'B'

Honestly, in this case I would recommend you to use ITU codecs because they
have been designed for these use cases. 

With best regards,

 Christian

---

From: "Slava Borilin" <[email-masked]>
Date: Thu, 22 Oct 2009 23:40:10 +0400

Michael,

If you are thinking about core network to carry voice in IP between BSCs
- why should you care about any of internet codecs- as this is managed
network normally?

Or I misunderstand you goal?

best regards,
Slava Borilin

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Christian Hoene
Sent: Thursday, October 22, 2009 11:33 PM
To: 'Michael Knappe'
Cc: [email-masked]
Subject: Re: [codec] codec tandeming/codec-requirements

Dear Michael Knappe,

> Cell phone 1 codec type 'A' -> transcode to internet codec ->
transcode to
> cell phone 2 codec type 'B'

Honestly, in this case I would recommend you to use ITU codecs because
they
have been designed for these use cases. 

With best regards,

 Christian

_______________________________________________
codec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/codec

---

From: Stefan Sayer <[email-masked]>
Date: Thu, 22 Oct 2009 22:08:33 +0200

> have been designed for these use cases. 

I don't think so. IMO bringing applications to cellular networks by good 
quality tandeming with wideband from cellular networks will be one of 
the major application areas for IWAC. Things like

Cell phone 1 codec type 'A' -> transcode to IWAC -> media/application 
server in the internet.

Cell phone 2 codec type 'B' -> transcode to IWAC -> same 
media/application server in the internet.

VoIP phone 3 on IWAC/internet -> same media/application server in the 
internet.

etc.

Stefan

-- 
Stefan Sayer
VoIP Services

[email-masked]
www.iptego.com

IPTEGO GmbH
Wittenbergplatz 1
10789 Berlin
Germany

Amtsgericht Charlottenburg, HRB 101010
Geschaeftsfuehrer: Alexander Hoffmann

---

From: Michael Knappe <[email-masked]>
Date: Thu, 22 Oct 2009 14:54:20 -0700

The rather extreme example I was thinking was a cell phone coming in via TDM to an IP PBX based branch office, then, being forwarded via SIP trunk using the proposed internet codec to another IP PBX branch where it happens to get routed off the PBX via TDM and the PSTN to another cell network and phone. This is a particularly problematic multiple tandem scenario with several passes through G.711, but even a double tandem scenario of a cell phone codec transcoded through G.711 and TDM then transcoded again to reach an IP network endpoint on a fabric using the proposed codec would be relatively common in the near term.

>From a 40,000 ft level, this all becomes a multi-variate quality balance of the different non-linear human perception impact thresholds (one-way delay, distortion, noise, residual echo, etc). It's difficult from the quality perspective to separate a single codec out of the end-to-end quality equation due to the non-linearity of the human perception curves. For example, the perceived quality impact of one-way network delay stays relatively low and constant below 150 ms, but rises dramatically by the time you hit 250 ms). In the case of one-way delay, you can have two codecs that by themselves easily meet the 150 ms quality criteria on their own, but once in-situ and tandemed, they cross the knee of the non-linear quality impact curve with a noticeable impact on quality. This can easily be heard on many cell-phone to cell-phone calls across disparate cell networks.

That said, we most definitely should have specific quality goals and measurement methodologies specified for the codec on its own. Just recommending from my own experience that we define a few more complex topologies where we look at end-to-end quality with the new codec in-situ.

Mike

On 10/22/09 12:40 PM, "Slava Borilin" <[email-masked]> wrote:

Michael,

best regards,
Slava Borilin

-----Original Message-----

Dear Michael Knappe,

transcode to

With best regards,

 Christian

---

From: Henry Sinnreich <[email-masked]>
Date: Thu, 22 Oct 2009 17:07:03 -0700

Mike,

>The rather extreme example I was thinking was a cell phone coming in via TDM to an IP PBX based branch >office, then, being forwarded via SIP trunk using the proposed internet codec to another IP PBX branch where >it happens to get routed off the PBX via TDM and the PSTN to another cell network and phone. This is a >particularly problematic multiple tandem scenario with several passes through G.711, but even a double >tandem scenario of a cell phone codec transcoded through G.711 and TDM then transcoded again to reach an
>IP network endpoint on a fabric using the proposed codec would be relatively common in the near term.

How about using the Internet e2e topology?

Henry

On 10/22/09 4:54 PM, "Michael Knappe" <[email-masked]> wrote:

Mike

Michael,

best regards,
Slava Borilin

-----Original Message-----

Dear Michael Knappe,

transcode to

With best regards,

 Christian

---

From: "Slava Borilin" <[email-masked]>
Date: Fri, 23 Oct 2009 11:24:16 +0400

Henry, e2e (or p2p) is cool but what about Legal interception for
example in this case, which every cellco has to implemenet?

best regards,

Slava Borilin

________________________________

From: Henry Sinnreich [mailto:[email-masked]] 
Sent: Friday, October 23, 2009 4:07 AM
To: Michael Knappe; Slava Borilin; Christian Hoene
Cc: [email-masked]
Subject: Re: [codec] codec tandeming/codec-requirements

Mike,

>The rather extreme example I was thinking was a cell phone coming in
via TDM to an IP PBX based branch >office, then, being forwarded via SIP
trunk using the proposed internet codec to another IP PBX branch where
>it happens to get routed off the PBX via TDM and the PSTN to another
cell network and phone. This is a >particularly problematic multiple
tandem scenario with several passes through G.711, but even a double
>tandem scenario of a cell phone codec transcoded through G.711 and TDM
then transcoded again to reach an 

Henry

The rather extreme example I was thinking was a cell phone coming in via
TDM to an IP PBX based branch office, then, being forwarded via SIP
trunk using the proposed internet codec to another IP PBX branch where
it happens to get routed off the PBX via TDM and the PSTN to another
cell network and phone. This is a particularly problematic multiple
tandem scenario with several passes through G.711, but even a double
tandem scenario of a cell phone codec transcoded through G.711 and TDM
then transcoded again to reach an IP network endpoint on a fabric using
the proposed codec would be relatively common in the near term.

>From a 40,000 ft level, this all becomes a multi-variate quality balance
of the different non-linear human perception impact thresholds (one-way
delay, distortion, noise, residual echo, etc). It's difficult from the
quality perspective to separate a single codec out of the end-to-end
quality equation due to the non-linearity of the human perception
curves. For example, the perceived quality impact of one-way network
delay stays relatively low and constant below 150 ms, but rises
dramatically by the time you hit 250 ms). In the case of one-way delay,
you can have two codecs that by themselves easily meet the 150 ms
quality criteria on their own, but once in-situ and tandemed, they cross
the knee of the non-linear quality impact curve with a noticeable impact
on quality. This can easily be heard on many cell-phone to cell-phone
calls across disparate cell networks.

That said, we most definitely should have specific quality goals and
measurement methodologies specified for the codec on its own. Just
recommending from my own experience that we define a few more complex
topologies where we look at end-to-end quality with the new codec
in-situ.

Mike

Michael,

best regards,
Slava Borilin

-----Original Message-----

Dear Michael Knappe,

transcode to

With best regards,

 Christian

---

From: Ron <[email-masked]>
Date: Fri, 23 Oct 2009 20:18:36 +1030

On Thu, Oct 22, 2009 at 10:08:33PM +0200, Stefan Sayer wrote:
> I don't think so. IMO bringing applications to cellular networks by
> good quality tandeming with wideband from cellular networks will be
> one of the major application areas for IWAC. Things like
> 
> Cell phone 1 codec type 'A' -> transcode to IWAC ->
> VoIP phone 3 on IWAC/internet -> same media/application server in
> the internet.

By the time this codec is widely deployed will there be (m)any
mobile devices without technology to do IP directly?

If "codec type 'A'" is a terrible codec for this purpose, how far
should we compromise to get acceptable results with things that
are "off the internet"?

I agree this raises interesting issues to consider, but requiring
compatibility with particular legacy codecs seems more like a SHOULD
than a MUST.  I think I'd rather change the string between my tin
cans than have this codec constrained to its current best passband.

 Ron

---

From: "Benjamin M. Schwartz" <[email-masked]>
Date: Fri, 23 Oct 2009 09:29:31 -0400

> example in this case, which every cellco has to implemenet?

Whether or not we want to encourage it, interception of VoIP traffic does
not require a re-encode.  The encoded packets can just be copied to the
additional receiver.

To address the "tandeming" issue, I think the following text should be
added to the "Basic Requirements" or "Additional Considerations":

"""
Audio Corruption Robustness

The systems providing input to the encoder and output from the decoder are
likely to be far from ideal in actual use.  Input and output audio streams
may be corrupted by artifacts from analog hardware and digital processing.
 The codecs to be developed should be tested to ensure that they degrade
gracefully under adverse audio conditions.  Types of digital corruption
that may be tested include tandem encoding with similar and different
codecs, low-quality resampling, and digital clipping.  Types of analog
corruption that may be tested include microphones with substantial
background noise, analog clipping, and loudspeaker distortion.
"""

---

From: Henry Sinnreich <[email-masked]>
Date: Fri, 23 Oct 2009 07:18:17 -0700

>Whether or not we want to encourage it, interception of VoIP traffic does not require a re-encode.
>The encoded packets can just be copied to the additional receiver.

This is correct IMO.
There is also traffic analysis and many other tools - not the right topic for this list.
Henry

On 10/23/09 8:29 AM, "Benjamin M. Schwartz" <[email-masked]> wrote:

"""

---

From: Michael Knappe <[email-masked]>
Date: Mon, 26 Oct 2009 13:47:12 -0700

Benjamin,

Well worded paragraph, thanks. I would add just some minor edits and a scoping sentence at the end. My proposed changes are in italics.

Cheers,

Mike

The systems providing input to the encoder and s//receiving/ output from the decoder s/are
likely to/may/ be far from ideal in actual use.  Input and output audio streams
may be corrupted by s//compounding non-linear/ artifacts from analog hardware and digital processing. The codecs to be developed should be tested to ensure that they degrade
gracefully under adverse audio conditions.  Types of digital corruption
that may be tested include tandem encoding with similar and different
codecs, low-quality resampling, and digital clipping.  Types of analog
corruption that may be tested include microphones with substantial
background noise, analog clipping, and loudspeaker distortion. s//No specific end-to-end quality requirements are mandated for use with the proposed codec. It is advisable, however, that several typical in-situ environments/processing chains be specified for the purpose of benchmarking end-to-end quality with the proposed codec./
"""

"

On 10/23/09 6:29 AM, "Benjamin M. Schwartz" <[email-masked]> wrote:

"""

---

From: Michael Knappe <[email-masked]>
Date: Mon, 26 Oct 2009 13:58:22 -0700

For ease of reading, incorporating the edits it would read as:

The systems providing input to the encoder and receiving output from the decoder may be far from ideal in actual use.  Input and output audio streams may be corrupted by compounding non-linear artifacts from analog hardware and digital processing. The codecs to be developed should be tested to ensure that they degrade gracefully under adverse audio conditions.  Types of digital corruption that may be tested include tandem encoding with similar and different codecs, low-quality resampling, and digital clipping.  Types of analog corruption that may be tested include microphones with substantial background noise, analog clipping, and loudspeaker distortion. No specific end-to-end quality requirements are mandated for use with the proposed codec. It is advisable, however, that several typical in-situ environments/processing chains be specified for the purpose of benchmarking end-to-end quality with the proposed codec.

On 10/26/09 1:47 PM, "Michael Knappe" <[email-masked]> wrote:

Benjamin,

Cheers,

Mike

"

"""