Subject: [codec] Suggested summary...

From: "Christian Hoene" <[email-masked]>
Date: Fri, 2 Jul 2010 12:09:13 +0200

Hello,

taking Cullen advise, I would like to suggest the following summary.

> 1) low delay mode

The codec shall be able to operated at a mode having an algorithmic delay of 8ms or less while having a frame duration of 5 1/3 ms or less. This is require to support ensemble performances over the Internet and other highly interactive conversational tasks.

> 2) low complexity mode (whatever this means)

The codec shall be able to operate at a low complexity mode while requiring less computational resources than a AMR-WB codec 
(< 38 WMOPS if measured with ITU-T STL2005 BaseOP (ITU-T G.192)).

> 3) technical understanding on how latency sums up on different platforms

Standard ITU-T G.114 (05/00 and 05/03) describes how different system components contribute to the one-way transmission delay. It states that the processing time of the codec contributes with an additional delay as large as the frame duration.

However, it is common consensus that plenty computational resources will be available most of the time. Then, the codec processing will be much faster than one frame duration. Typical values are range from a factor faster of 100 (smart phones) to 1000 (PCs). A device working at full load is a rare case.

Any suggestion to improve it?

With best regards,

 Christian 

---------------------------------------------------------------
Dr.-Ing. Christian Hoene
Interactive Communication Systems (ICS), University of Tübingen 
Sand 13, 72076 Tübingen, Germany, Phone +49 7071 2970532 
http://www.net.uni-tuebingen.de/

-----Original Message-----
From: Cullen Jennings [mailto:[email-masked]] 
Sent: Monday, June 21, 2010 7:21 PM
To: Christian Hoene
Cc: [email-masked]
Subject: Re: [codec] #16: Multicast?

On May 27, 2010, at 9:48 AM, Christian Hoene wrote:

> So, we have consensus on 
> 1) low delay mode
> 2) low complexity mode (whatever this means)
> 3) technical understanding on how latency sums up on different platforms

>From a Chair point of view, I don't think the Chairs could summarize or call consensus on these three - however, I'm not sure that matters. If you think a key piece of consensus has come out of this conversation and that it needs to captured in the archive, can you summarize what you think it is folks agree with and then the chairs can make some sort of consensus call.

Thanks, Cullen <with my chair hat on>
 =

---

From: stephen botzko <[email-masked]>
Date: Fri, 2 Jul 2010 07:13:37 -0400

Maybe I somehow missed it, but i do not see any discussions on the list that
support your consensus claim.  While perhaps they are reasonable suggestions
that we can discuss further, for me consensus means that the specifics were
expressly discussed and generally agreed.

There seem to be details here that I do not find on the list.  For instance,
a general support for a complexity ceiling of AMR-WB, and a 5 1/3 ceiling on
low-delay frame size.

Regards
Stephen Botzko

2010/7/2 Christian Hoene <[email-masked]>

> The codec shall be able to operated at a mode having an algorithmic delay
> of 8ms or less while having a frame duration of 5 1/3 ms or less. This is
> require to support ensemble performances over the Internet and other highly
> interactive conversational tasks.
>
> The codec shall be able to operate at a low complexity mode while requiring
> less computational resources than a AMR-WB codec
> Standard ITU-T G.114 (05/00 and 05/03) describes how different system
> components contribute to the one-way transmission delay. It states that the
> processing time of the codec contributes with an additional delay as large
> as the frame duration.
>
> However, it is common consensus that plenty computational resources will be
> available most of the time. Then, the codec processing will be much faster
> than one frame duration. Typical values are range from a factor faster of
> 100 (smart phones) to 1000 (PCs). A device working at full load is a rare
> case.
>
> From a Chair point of view, I don't think the Chairs could summarize or
> call consensus on these three - however, I'm not sure that matters. If you
> think a key piece of consensus has come out of this conversation and that it
> needs to captured in the archive, can you summarize what you think it is
> folks agree with and then the chairs can make some sort of consensus call.
>
> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec
>

---

From: Herve Taddei <[email-masked]>
Date: Fri, 02 Jul 2010 14:18:08 +0200

I think I have the same opinion as Stephen, this does not really represent a
consensus as it never came on the table. This is exactly what I was asking
in my email by end of May when you wrote there was consensus, what is the
consensus? 

The 5 1/3 ms is quite precise, why this value? The AMR-WB codec complexity
as reference, for which reason? 

Kind regards

Herve Taddei

  _____  

From: [email-masked] [mailto:[email-masked]] On Behalf Of
stephen botzko
Sent: Friday, July 02, 2010 1:14 PM
To: Christian Hoene
Cc: [email-masked]
Subject: Re: [codec] Suggested summary...

Regards
Stephen Botzko

Hello,

> 1) low delay mode

The codec shall be able to operated at a mode having an algorithmic delay of
8ms or less while having a frame duration of 5 1/3 ms or less. This is
require to support ensemble performances over the Internet and other highly
interactive conversational tasks.

With best regards,

 Christian

-----Original Message-----
From: Cullen Jennings [mailto:[email-masked]]
Sent: Monday, June 21, 2010 7:21 PM
To: Christian Hoene
Cc: [email-masked]
Subject: Re: [codec] #16: Multicast?

>From a Chair point of view, I don't think the Chairs could summarize or call
consensus on these three - however, I'm not sure that matters. If you think
a key piece of consensus has come out of this conversation and that it needs
to captured in the archive, can you summarize what you think it is folks
agree with and then the chairs can make some sort of consensus call.

---

From: "Christian Hoene" <[email-masked]>
Date: Fri, 2 Jul 2010 14:47:52 +0200

So, I am not saying that there is a consensus on the claims I made.

I am just saying that this might be first starting point to discuss it further.

To explain the figures:

a)      5+1/3 ms was selected because at a sampling rate of 48000 Hz, it is has a size of 256 samples. 256 samples is good for sound cards and good for DFFT. In addition, it covers the codec contribution of Broadcom and Xiph.org.

b)      The complexity of AMR-WB is mentioned because both the Broadcom and the Skype codecs are faster.

If you want other figures, please bring them on the table.

Christian

Dr.-Ing. Christian Hoene

From: Herve Taddei [mailto:[email-masked]] 
Sent: Friday, July 02, 2010 2:18 PM
To: 'stephen botzko'; 'Christian Hoene'
Cc: [email-masked]
Subject: RE: [codec] Suggested summary...

I think I have the same opinion as Stephen, this does not really represent a consensus as it never came on the table. This is exactly what I was asking in my email by end of May when you wrote there was consensus, what is the consensus? 

The 5 1/3 ms is quite precise, why this value? The AMR-WB codec complexity as reference, for which reason? 

Kind regards

Herve Taddei

  _____  

From: [email-masked] [mailto:[email-masked]] On Behalf Of stephen botzko
Sent: Friday, July 02, 2010 1:14 PM
To: Christian Hoene
Cc: [email-masked]
Subject: Re: [codec] Suggested summary...

Maybe I somehow missed it, but i do not see any discussions on the list that support your consensus claim.  While perhaps they are reasonable suggestions that we can discuss further, for me consensus means that the specifics were expressly discussed and generally agreed.  

There seem to be details here that I do not find on the list.  For instance, a general support for a complexity ceiling of AMR-WB, and a 5 1/3 ceiling on low-delay frame size.

Regards
Stephen Botzko

Hello,

With best regards,

 Christian

---

From: Michael Knappe <[email-masked]>
Date: Fri, 2 Jul 2010 05:54:18 -0700

On codec complexity benchmarking, there is certainly industry precedence for using an existing codec as a rough relative complexity guide (G.729A comes to mind for narrowband codec comparisons), but some form of generalized fixed point / floating point instruction count rating would hopefully be less algorithm specific. Herve, Stephen, any suggestion(s) for codec complexity ratings?

Mike

On 7/2/10 5:18 AM, "Herve Taddei" <[email-masked]> wrote:

Kind regards

Herve Taddei

________________________________

Regards
Stephen Botzko

With best regards,

 Christian

---

From: Herve Taddei <[email-masked]>
Date: Fri, 02 Jul 2010 15:16:47 +0200

Hi Mike,

Actually if you ask me to name a low complexity codec (by the way at which
bandwidth?) I don't think I would name AMR-WB. G.722.1 is a good example of
a low complexity codec for WB services. 

Best regards

Herve 

> From: Michael Knappe [mailto:[email-masked]]
> Sent: Friday, July 02, 2010 2:54 PM
> To: Herve Taddei; 'stephen botzko'; 'Christian Hoene'
> Cc: [email-masked]
> On codec complexity benchmarking, there is certainly industry precedence
for
> using an existing codec as a rough relative complexity guide (G.729A comes
to
> mind for narrowband codec comparisons), but some form of generalized fixed
point
> / floating point instruction count rating would hopefully be less
algorithm specific.
> Herve, Stephen, any suggestion(s) for codec complexity ratings?
> 
> Mike
> 
> 
in my
> email by end of May when you wrote there was consensus, what is the
consensus?
> The 5 1/3 ms is quite precise, why this value? The AMR-WB codec complexity
as
> reference, for which reason?
> 
> Kind regards
> 
> 
> Herve Taddei
> 
> 
> 
> 
> There seem to be details here that I do not find on the list.  For
instance, a general
> support for a complexity ceiling of AMR-WB, and a 5 1/3 ceiling on
low-delay frame
> size.
of 8ms
> or less while having a frame duration of 5 1/3 ms or less. This is require
to support
> ensemble performances over the Internet and other highly interactive
> conversational tasks.
> 
> The codec shall be able to operate at a low complexity mode while
requiring less
> computational resources than a AMR-WB codec
as the
> frame duration.
than one
> frame duration. Typical values are range from a factor faster of 100
(smart phones)
> to 1000 (PCs). A device working at full load is a rare case.
> 
call
> consensus on these three - however, I'm not sure that matters. If you
think a key
> piece of consensus has come out of this conversation and that it needs to
captured
> in the archive, can you summarize what you think it is folks agree with
and then the
> chairs can make some sort of consensus call.
>

---

From: stephen botzko <[email-masked]>
Date: Fri, 2 Jul 2010 09:25:40 -0400

I am ok with identifying starting points for discussion.  From the thread
context it (the reference to Cullen, etc) it sounded like you were
attempting to summarize consensus.  Sorry if I misunderstood.

I understand the 256 sample size (I did the math also!), though it is
perfectly straightforward to adapt transforms to other sizes.  G.719 for
instance uses a 960 sample DCT transform for 20 ms frame size.  There are
many other examples that could be named.  I don't think the sound card
argument is all that relevant, since that is a normal place to put a buffer
anyway, and there is no reason why that buffer needs to be related to the
frame size.

Perhaps more importantly, I do not think it is appropriate to work backwards
from technology contributions to create the requirements.  The normal way in
the IETF is to work forward from use cases and problem statements.  IMHO
working forward is much more likely to get good results, and it is fair to
all (past and future) contributors.

BTW, personally I do not place a very high priority on distributed internet
music ensemble performances.  While I am ok if that happens to fall out of
the design, it possibly has a lot of other implications beyond delay.  I
would happily trade outstanding packet loss performance for music ensembles
if it came down to it, and I suspect many other VOIP folks would concur.  Is
there a more compelling  (maybe less fun) use case you can reference in the
low-delay requirement?

Stephen Botzko

> a)      5+1/3 ms was selected because at a sampling rate of 48000 Hz, it
> is has a size of 256 samples. 256 samples is good for sound cards and good
> for DFFT. In addition, it covers the codec contribution of Broadcom and
> Xiph.org.
>
> *From:* Herve Taddei [mailto:[email-masked]]
> *Sent:* Friday, July 02, 2010 2:18 PM
> *To:* 'stephen botzko'; 'Christian Hoene'
> *Cc:* [email-masked]
> *Subject:* RE: [codec] Suggested summary...
>
>
>
> I think I have the same opinion as Stephen, this does not really represent
> a consensus as it never came on the table. This is exactly what I was asking
>   ------------------------------
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *stephen botzko
> *Sent:* Friday, July 02, 2010 1:14 PM
> *To:* Christian Hoene
> *Cc:* [email-masked]
> *Subject:* Re: [codec] Suggested summary...
>
>
>
> Maybe I somehow missed it, but i do not see any discussions on the list
> that support your consensus claim.  While perhaps they are reasonable
> suggestions that we can discuss further, for me consensus means that the
> specifics were expressly discussed and generally agreed.
>
> instance, a general support for a complexity ceiling of AMR-WB, and a 5 1/3
> ceiling on low-delay frame size.
>
> Regards
> Stephen Botzko
>

---

From: "Christian Hoene" <[email-masked]>
Date: Fri, 2 Jul 2010 18:01:09 +0200

Hello Herve,

If I understand you correctly, you are not suggesting G.722.1 as reference. You just say that AMR-WB is not known as low-complexity.

So, a better wording would say

"The IWAC codec MUST be able to operate at a mode that requires less computational resources than the AMR-WB codec at 12.65 kbit/s. 
Thus, the complexity of the IWAC codec MUST be less than 38 WMOPS if measured with the ITU-T STL2005 BaseOP library."
 
Fine?

 Christian

-----Original Message-----
From: Herve Taddei [mailto:[email-masked]] 
Sent: Friday, July 02, 2010 3:17 PM
To: 'Michael Knappe'; 'stephen botzko'; 'Christian Hoene'
Cc: [email-masked]
Subject: RE: [codec] Suggested summary...

Hi Mike,

Best regards

Herve 

for
to
point
algorithm specific.
> I think I have the same opinion as Stephen, this does not really represent
a
in my
consensus?
as
> reference, for which reason?
that
instance, a general
low-delay frame
of 8ms
to support
requiring less
as the
than one
(smart phones)
call
think a key
captured
and then the

---

From: "Christian Hoene" <[email-masked]>
Date: Fri, 2 Jul 2010 18:24:22 +0200

Hi Stephen,

From: stephen botzko [mailto:[email-masked]] 
Sent: Friday, July 02, 2010 3:26 PM
To: Christian Hoene
Cc: Herve Taddei; [email-masked]
Subject: Re: [codec] Suggested summary...

I understand the 256 sample size (I did the math also!), though it is perfectly straightforward to adapt transforms to other sizes.  G.719 for instance uses a 960 sample DCT transform for 20 ms frame size.  There are many other examples that could be named.  I don't think the sound card argument is all that relevant, since that is a normal place to put a buffer anyway, and there is no reason why that buffer needs to be related to the frame size.

[Christian Hoene] If it comes to ultra low delay, the buffering at the sound card becomes important and must be optimized. Some sound cards (and their DMA controller) have restrictions on the size of blocks (called periods in ALSA). Having “strange” frame sizes will increase the latency because of re-framing.

Perhaps more importantly, I do not think it is appropriate to work backwards from technology contributions to create the requirements.  

[Christian Hoene] You are right. However, the requirements have to be realistic. Looking at existing technologies helps you to know what is possible at the moment.

The normal way in the IETF is to work forward from use cases and problem statements.  IMHO working forward is much more likely to get good results, and it is fair to all (past and future) contributors.

BTW, personally I do not place a very high priority on distributed internet music ensemble performances.  While I am ok if that happens to fall out of the design, it possibly has a lot of other implications beyond delay.  I would happily trade outstanding packet loss performance for music ensembles if it came down to it, and I suspect many other VOIP folks would concur.  Is there a more compelling  (maybe less fun) use case you can reference in the low-delay requirement?

[Christian Hoene] Musical telepresence is a good use case because it is the upper limit of quality. A monaural audio transmissions can hardly be better. It is also a good marketing argument: The IWAC codec provides the best audio quality at lowest latency.  Also, this use-case is unique because  here is not no other standardized audio codec out here supporting it.

Please have a look on the following PhD to understand the requirements of musical telepresence better:
http://www.itm.uni-luebeck.de/users/carot/Docs/dissertation_AC.pdf

Christian

---

From: stephen botzko <[email-masked]>
Date: Fri, 2 Jul 2010 14:43:21 -0400

Hi Christian

You are not mandating a 256 sample multiple on frame size, you are proposing
it as a maximum for the low delay mode.  So smaller "strange" frame sizes
are still allowed, and re-framing might be needed for some receivers.
Locking the frame size to a specific sound card's quirks does not make sense
to me, esp. since there are lots of sound cards out there, and they
certainly will have different quirks.

I agree that musical telepresence is fun to talk about, and would be fun to
build and use.  That is different from saying it is an essential application
for CODEC to work on.  In my opinion it is not. The delay bound alone limits
its use severely over the wide-area internet.  There is room for other
opinions, but I'd like to register mine.

Stephen Botzko

>  Hi Stephen,
>
>
>
>
>
> *From:* stephen botzko [mailto:[email-masked]]
> *Sent:* Friday, July 02, 2010 3:26 PM
> *To:* Christian Hoene
> *Cc:* Herve Taddei; [email-masked]
>
> *[Christian Hoene] If it comes to ultra low delay, the buffering at the
> sound card becomes important and must be optimized. Some sound cards (and
> their DMA controller) have restrictions on the size of blocks (called
> periods in ALSA). Having “strange” frame sizes will increase the latency
> because of re-framing.*
>
> Perhaps more importantly, I do not think it is appropriate to work
> backwards from technology contributions to create the requirements.
>
> *[Christian Hoene] You are right. However, the requirements have to be
> realistic. Looking at existing technologies helps you to know what is
> possible at the moment.*
>
> The normal way in the IETF is to work forward from use cases and problem
> statements.  IMHO working forward is much more likely to get good results,
> and it is fair to all (past and future) contributors.
>
> **
> *[Christian Hoene] Musical telepresence is a good use case because it is
> the upper limit of quality. A monaural audio transmissions can hardly be
> better. It is also a good marketing argument: The IWAC codec provides the
> best audio quality at lowest latency.  Also, this use-case is unique because
>  here is not no other standardized audio codec out here supporting it.*
>
> *Please have a look on the following PhD to understand the requirements of
> musical telepresence better:
> http://www.itm.uni-luebeck.de/users/carot/Docs/dissertation_AC.pdf*
>
> *Christian*
>

---

From: "Raymond (Juin-Hwey) Chen" <[email-masked]>
Date: Fri, 2 Jul 2010 18:34:32 -0700

Hi Christian,

Thanks for your summary.  Some comments on 2) and 3) below.

2) low complexity mode: 
All else being equal or comparable, the lower the complexity, the 
better.  Besides MIPS or WMOPS, the memory footprint is another 
important aspect of complexity that should be considered.  
Furthermore, if we want to specify a complexity target, then in 
addition to a target for the full-band 48 kHz sampling rate, it would 
be useful to specify also the complexity targets for lower sampling 
rates such as 16 and 8 kHz, since the codec may operate at these 
lower sampling rates in some important voice-centric applications.

3) How latency sums up: 
Thanks for mentioning the ITU-T G.114, which has a good discussion of 
the codec-related one-way delay along the line of what we discussed 
in the emails. However, I disagree with the statement that "Typical 
values are range from a factor faster of 100 (smart phones) to 1000 
(PCs). A device working at full load is a rare case." It is not at 
all a rare case to find devices running at or close to full load.  A 
good example is VoIP gateways.  Also, in numerous occasions I have 
seen engineers trying very hard to cut the complexity of algorithms 
to make them fit the processing power of existing DSPs or host 
processors. That means the resulting implementations would have the 
processor essentially fully loaded.  

The fastest processors in current smart phones and PCs are about 1 
GHz and slightly more than 3 GHz, respectively.  A factor of 100 and 
1000 faster would require that the codec complexity be less than 
about 10 MHz and 3 MHz, respectively.  Most contemporary codecs today 
have higher complexity than that.  Even if the codec complexity were 
lower than these numbers, to achieve a delay reduction factor of 100 
and 1000, the encoding and decoding operations would each have to be 
the only real-time task or the highest-priority task so the processor 
will get to it right away without any delay.  This is not the case in 
general, since having a full-duplex channel will requires at least a 
real-time encoding task and a real-time decoding task running at the 
same time. Both cannot be the only real-time task or the highest-
priority real-time task at the same time; furthermore, there are 
almost always some other real-time tasks in the system.  As I 
discussed in length in my previous email about Real-Time Scheduling 
(RTS) delay, most audio/video communication systems have many 
real-time tasks running at the same time; substantial delay needs 
to be added to ensure that none of such real-time tasks will run out 
of real time.  This often leads to an RTS delay of one frame.

Best Regards,

Raymond

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christian Hoene
Sent: Friday, July 02, 2010 3:09 AM
To: 'Cullen Jennings'
Cc: [email-masked]
Subject: [codec] Suggested summary...

Hello,

With best regards,

 Christian

---

From: "Christian Hoene" <[email-masked]>
Date: Sun, 4 Jul 2010 15:33:59 +0200

Hi Raymond,

Inline some comments of mine are inline

Christian

-----Original Message-----
From: Raymond (Juin-Hwey) Chen [mailto:[email-masked]] 

Hi Christian,

2) low complexity mode: 

[Christian Hoene] Any suggested minimal and median values? 

[Christian Hoene] Upps, the minimal complexity values were meant regardless of the sampling rate

3) How latency sums up: 
in the emails. 

[Christian Hoene] Yes, but I forgot to mention that is was industry consensus ten years ago. Things have changed.

However, I disagree with the statement that "Typical 
values are range from a factor faster of 100 (smart phones) to 1000 
(PCs). A device working at full load is a rare case." It is not at 
all a rare case to find devices running at or close to full load.  A 
good example is VoIP gateways.  

[Christian Hoene] Sorry, a telecommunication network overrating at full load is a rare case. For example, around  New Year's Eve networks tend to become overloaded: 2h per one year. Typically, the network shall be loaded somewhere between 5 and 20% or less. Even if you assume that all traffic is between end device and gateway (which is not the case - end to end IP is a more reasonable scenario), then the gateway is fully loaded less than 1<% of the time.

Also, in numerous occasions I have 

have higher complexity than that.  

[Christian Hoene] Then take the value 10 and 100. It still does not matter.

Even if the codec complexity were 

[Christian Hoene] Let me cite from TI specs of the TNETV3020 carrier grade voice gateway: "Processing delay in the Telogy Software solution is minimized by a staggered processing schedule". TI engineers do not have the same technical problems as Broadcom engineers. This paper (or any other) on the topic of staggered scheduling might be helpful http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.143.7213&rep=rep1&type=pdf 

Best Regards,

Raymond

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christian Hoene
Sent: Friday, July 02, 2010 3:09 AM
To: 'Cullen Jennings'
Cc: [email-masked]
Subject: [codec] Suggested summary...

Hello,

With best regards,

 Christian

---

From: Herve Taddei <[email-masked]>
Date: Mon, 05 Jul 2010 17:45:15 +0200

Hi Christian,

The wording is fine if everybody agrees that AMR-WB should be the reference
complexity for an internet codec. Why is AMR-WB the anchor? What is then the
complexity of the so called low computational mode? Is it half, one third,
10% of it? I don't see the wording low complexity anymore. This complexity
would be the one of the full codec at all bitrates/bandwidth, correct? I am
also fine with adding lower complexity values for lower sampling frequency
(e.g. 8-16 kHz) as proposed by Raymond as well as taking care of the memory
footprint. 

In some sense, the way proposed by Stephen is fine, "work forward from use
cases and problem statements" to figure out the requirements. In that case,
it would be needed to define where and how this codec is going to be used.
Shouldn't it be in the draft documents you were working on?

Best regards

Herve 

> From: Christian Hoene [mailto:[email-masked]]
> Sent: Friday, July 02, 2010 6:01 PM
> To: 'Herve Taddei'; 'Michael Knappe'; 'stephen botzko'
> Cc: [email-masked]
> If I understand you correctly, you are not suggesting G.722.1 as
reference. You just
> say that AMR-WB is not known as low-complexity.
> 
> "The IWAC codec MUST be able to operate at a mode that requires less
> computational resources than the AMR-WB codec at 12.65 kbit/s.
> Thus, the complexity of the IWAC codec MUST be less than 38 WMOPS if
> measured with the ITU-T STL2005 BaseOP library."
> 
> Fine?
> 
>  Christian
> 
> 
> 
> 
>

---

From: "Raymond (Juin-Hwey) Chen" <[email-masked]>
Date: Fri, 9 Jul 2010 18:39:34 -0700

Hi Christian,

Comments in-line, with my new comments preceded by ">>>[Raymond]:" to make it easier to identify.

Raymond

-----Original Message-----
From: Christian Hoene [mailto:[email-masked]] 
Sent: Sunday, July 04, 2010 6:34 AM
To: Raymond (Juin-Hwey) Chen
Cc: [email-masked]
Subject: RE: [codec] Suggested summary...

Hi Raymond,

Christian

-----Original Message-----

Hi Christian,

2) low complexity mode: 

>>>[Raymond]: I would suggest taking a look at the memory footprint 
numbers of many contemporary codecs in some codec comparison 
tables such as those in Annexes C and D of PacketCable 2.0 codec 
and media spec for narrowband and wideband codecs, respectively, at  
http://www.cablelabs.com/specifications/PKT-SP-CODEC-MEDIA-I09-100527.pdf
Whether the complexity is "low" is in a relative sense.  I think it 
makes sense to call a codec's complexity low if it is below the 
median value of the complexity of contemporary codecs at the same 
sampling rate.  Some may even argue that it has to be far below the 
median to be called "low".  Using the Annex C above and excluding 
very old codecs such as G.711 and G.726, the median RAM size is about 
2.6 kwords and the median ROM size is about 12 to 14 kwords for 
narrowband (8 kHz sampling) codecs.  Similarly, using Annex D gives
a median RAM size of 4.6 to 5.3 kwords and a median total memory 
footprint of 19.4 to 23 kwords for wideband (16 kHz sampling) codecs.

>>>[Raymond]: Using a single complexity target for all sampling rates 
doesn't make sense to me, as it could be too tight a constraint for high
sampling rates and too loose a constraint for low sampling rates.  It makes 
more sense to have separate targets for at least 8, 16, and 48 kHz sampling 
rates.  If we follow the same logic as in memory footprint above, then the 
median complexity is 20 MIPS for 8 kHz sampling and 17.5 MIPS to 38 WMOPS 
for 16 kHz sampling.  I know 17.5 to 38 is a wide range. That's an inherent
problem with taking the median of an even number of entries.  If we take 
the average in this case, it is 25.3 MIPS/WMOPS.  I know MIPS is not the 
same as WMOPS (usually slightly higher than WMOPS), but that was how the 
complexity data were collected, and since we don't have access to all 
complexity data using only MIPS or only WMOPS, I was forced to treat the 
two as roughly equivalent for the purpose of deriving median or average.

3) How latency sums up: 

>>>[Raymond]: Things certainly have changed, especially the processor 
speeds, but I am not sure that the industry consensus has changed much.  
If it did, you would expect that the ITU-T would probably update G.114.  
I think the reality is that most of the voice communications today (all 
the conventional land-line phones, cell phones, IP phones, etc.) are 
still handled by embedded processors in special-purpose hardware 
devices, which don't have the luxury of having a DSP that is 100 times
or even 10 times faster than the speed required by all real-time tasks 
of the system combined.  Thus, I would say the delay analysis in G.114 
is still relevant today.  In fact, one could even argue that some 
values of the delay components G.114 are on the low side as the wait 
time in RTS scheduling does not seem to be taken into account there. 
Hence, it seems to me that the delay values presented in G.114 are 
more like lower bounds of the delay values likely to be encountered 
in the real world.

>>>[Raymond]: Even if the overall network traffic load is relatively 
light, you still cannot rule out the possibility that some gateways may 
be significantly more loaded than others. More importantly, the delay 
depends on how the real-time scheduling system is designed and may not 
necessarily depend on the traffic load.  If you take a conservative 
approach and design your RTS system for the worst-case processing load, 
then even when the traffic load is lighter, the delay through the system 
remains the same.  If you try to design the RTS system to reduce the 
delay aggressively when the traffic load is lighter, then certain 
real-time tasks may run out of real time if you are not careful, 
resulting in audio quality degradation.  In any case, VoIP gateways 
were just given as an example. How about other devices with embedded 
processors such as IP phones? Their processor load doesn't depend on the 
traffic load and can be nearly fully loaded, so I would say a device 
working at (nearly) full load is not a rare case.

>>>[Raymond]: My real point is that even if the processor MHz rating 
is 10 or 100 times the codec complexity MHz requirement, it doesn't 
mean that your codec processing delay will be 1/10 or 1/100 of the 
frame size.  You can achieve such a processing delay of 1/10 or 1/100 
of the frame size only if the processor is immediately available to 
do the encoder task and decoder task as soon as the last audio sample 
of the frame and the last bit of the compressed frame arrives, 
respectively.  We know that this is generally not true because there 
is almost always some other real-time tasks in the system which can 
be running at those particular time instants and the encoder and 
decoder tasks just have to wait for their turns.  My original 
paragraph below in my last email discussed this point.

>>>[Raymond]: First, I checked the product bulletin of TNETV3020, and other than the sentence "Processing delay in the Telogy Software solution is minimized by a staggered processing schedule" that you quoted above, it doesn't say anything about how low the latency is or the relationship between the processing delay and the codec frame size.  This generic statement does not prove or disprove anything related to what we are discussing, and it does not say anything about the technical problems facing TI engineers and Broadcom engineers.
Second, I read the Holman and Anderson staggered model paper that your link points to, but I am not sure it is relevant to what we are discussing here.  If I understand it correctly, that paper is talking about scheduling the processors of a symmetric multiprocessors (SMP) system in a staggered manner to reduce the wait time due to bus contention.  That's very different from what we are talking about here, which is how to schedule multiple real-time tasks in a single-core processor without letting any of the tasks running out of real-time.  

Best Regards,

Raymond

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christian Hoene
Sent: Friday, July 02, 2010 3:09 AM
To: 'Cullen Jennings'
Cc: [email-masked]
Subject: [codec] Suggested summary...

Hello,

With best regards,

 Christian