Subject: Re: [codec] #5: Mention DTMF in requirements

From: Michael Knappe <[email-masked]>
Date: Fri, 2 Apr 2010 07:19:49 -0700

Agreed.

Mike

________________________________
From: [email-masked] <[email-masked]>
To: Peter Saint-Andre <[email-masked]>
Cc: [email-masked] <[email-masked]>
Sent: Fri Apr 02 10:01:42 2010
Subject: Re: [codec] #5: Mention DTMF in requirements

I heard no decision as to DTMF tone encoding.

As far as I am concerned, the question of whether the codec MUST encode DTMF tones accurately enough to be detected at the decoder output (or SHOULD or non-requirement) is still open.

That question clearly is in-scope, and has nothing to do with signaling.

Stephen Botzko

On Fri, Apr 2, 2010 at 9:57 AM, Peter Saint-Andre <[email-masked]<mailto:[email-masked]>> wrote:
How is the never-ending debate among DTMF signalling *methods* in-scope
for the Codec WG? I think that Henning brought this up in Anaheim only
to make sure that we test some DTMF tones. The signalling method is out
of scope for the codec itself.

On 4/2/10 7:53 AM, stephen botzko wrote:
> Are you two suggesting that in-band DTMF is a MUST?  Or alternatively a
> SHOULD?
>
> Stephen Botzko
>
> On Fri, Apr 2, 2010 at 9:48 AM, James Rafferty
> <[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>> wrote:
>
>     I'd agree with Steve that are still many deployments which do not
>     use RFC 2833 or RFC 4733. In our gateways, we've had to support
>     interworking variations of tone support such as INFO and in-band, in
>     addition to the RFC 2833 / RFC 4733.
>
>     James
>     -----Original Message-----
>     From: [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>
>     [mailto:[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>] On
>     Behalf Of Steve Underwood
>     Sent: Friday, April 02, 2010 2:27 AM
>     To: [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>
>     Subject: Re: [codec] #5: Mention DTMF in requirements
>
>     On 03/29/2010 02:22 AM, Marc Petit-Huguenin wrote:
>     > On 03/28/2010 11:00 AM, stephen botzko wrote:
>     >
>     >> I would agree with this if I saw reasonable evidence that a
>     >> preponderance of gateways and sending systems provide the
>     signaling in
>     >> these RFCs.
>     >>
>     >> Since I am not sure that this is the case, I am unconvinced that
>     we can
>     >> totally remove the requirement.
>     >>
>     >> I'd also say that an encoder that detects the DTMF tones and
>     outputs the
>     >> RFC 4733/34 events would fully meet the requirement.
>     >>
>     > As former CTO of a VoIP provider, I never saw a PSTN provider not
>     supporting at
>     > least RFC 2833 (even if one of them did not declare it in its SDP)
>     >
>     > Perhaps the question can be asked at the next SIPit event.
>     >
>     Its true that RFC2833 is widely deployed. Its even true that many
>     systems have updated to RFC4733. Sadly, its also true that there are
>     still many quirky implementations widely deployed, and a lot of people
>     still need to interwork with audio DTMF.
>
>     Steve

_______________________________________________
codec mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/codec

---

From: "Christian Hoene" <[email-masked]>
Date: Fri, 2 Apr 2010 16:26:59 +0200

I would vote for a
 
The DTMF transmission performance MUST be tested and characterized (via an automatic open source testing tool) 
but it MUST NOT work well under all operational conditions.  Especially at low coding rates under the presence of packet losses it might not work reliable.
 
Christian
 
---------------------------------------------------------------
Dr.-Ing. Christian Hoene
Interactive Communication Systems (ICS), University of Tübingen 
Sand 13, 72076 Tübingen, Germany, Phone +49 7071 2970532 
 <http://www.net.uni-tuebingen.de/> http://www.net.uni-tuebingen.de/
 
From: [email-masked] [mailto:[email-masked]] On Behalf Of Michael Knappe
Sent: Friday, April 02, 2010 4:20 PM
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: Re: [codec] #5: Mention DTMF in requirements
 
Agreed.

Mike
  _____  

Stephen Botzko
On Fri, Apr 2, 2010 at 9:57 AM, Peter Saint-Andre <[email-masked]> wrote:

On 4/2/10 7:53 AM, stephen botzko wrote:
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     From: [email-masked] <mailto:[email-masked]>
>     [mailto:[email-masked] <mailto:[email-masked]>] On
>     To: [email-masked] <mailto:[email-masked]>

[email-masked]

---

From: stephen botzko <[email-masked]>
Date: Fri, 2 Apr 2010 10:49:35 -0400

(a) "MUST NOT" means that it must fail.  Perhaps you meant "need not" or
"might not"?

(b) SHOULD is commonly used in establishing requirements, and it certainly
was used in MARTINI and other working groups at IETF77 in requirements
presentations with no confusion. It has a clear meaning in the requirements
context (desirable but not essential) and I see no reason to avoid its use
at this phase.  There will certainly be other things that are "nice to
have", and it is appropriate to track them and consider them in the
selection/standardization process.

I agree that language about DTMF might not need to be in the final RFC.
Though if DTMF quality is known to be inadequate, it probably makes sense to
tell people that.

I also agree to the obvious comment that DTMF detection methods are out of
scope.

But there is no point in testing DTMF at all unless there is a MUST or
SHOULD requirement.  The fact that folks suggested that it be tested
indicates that they believe there *is* a requirement.  And some others are
saying that explicitly.

Perhaps they are wrong.  But I see no justification for making the
discussion off limits to this list, it is clearly in scope.

Stephen Botzko

On Fri, Apr 2, 2010 at 10:26 AM, Christian Hoene <[email-masked]>wrote:

> but it MUST NOT work well under all operational conditions.  Especially at
> low coding rates under the presence of packet losses it might not work
> reliable.
>
>
>
> Christian
>
>
>
> http://www.net.uni-tuebingen.de/
>
>
>
> *From**:* [email-masked] [mailto:[email-masked]] *On
> Behalf Of *Michael Knappe
> *Sent:* Friday, April 02, 2010 4:20 PM
> *To:* [email-masked]; [email-masked]
> *Cc:* [email-masked]
>
> *Subject:* Re: [codec] #5: Mention DTMF in requirements
>
>
>
> Agreed.
>
> Mike
>  ------------------------------
>
> *From*: [email-masked] <[email-masked]>
> *To*: Peter Saint-Andre <[email-masked]>
> *Cc*: [email-masked] <[email-masked]>
> *Sent*: Fri Apr 02 10:01:42 2010
> *Subject*: Re: [codec] #5: Mention DTMF in requirements
>
> As far as I am concerned, the question of whether the codec MUST encode
> DTMF tones accurately enough to be detected at the decoder output (or SHOULD
> or non-requirement) is still open.
>

---

From: "Christian Hoene" <[email-masked]>
Date: Fri, 2 Apr 2010 19:34:39 +0200

Hi Stephan,

Comments inline:

From: stephen botzko [mailto:[email-masked]] 
Sent: Friday, April 02, 2010 4:50 PM
To: Christian Hoene
Cc: Michael Knappe; [email-masked]; [email-masked]
Subject: Re: [codec] #5: Mention DTMF in requirements

(a) "MUST NOT" means that it must fail.  Perhaps you meant "need not" or "might not"?

CH: Yes, I meant need not (or the German „muss nicht“)

(b) SHOULD is commonly used in establishing requirements, and it certainly was used in MARTINI and other working groups at IETF77 in
requirements presentations with no confusion. It has a clear meaning in the requirements context (desirable but not essential) and I
see no reason to avoid its use at this phase.  There will certainly be other things that are "nice to have", and it is appropriate
to track them and consider them in the selection/standardization process.

CH: Better? "The codec SHOULD support the transmission DTMF at most transmission conditions."

I agree that language about DTMF might not need to be in the final RFC.  Though if DTMF quality is known to be inadequate, it
probably makes sense to tell people that.

CH: Agreed. The limits of the codec shall be clearly stated.

I also agree to the obvious comment that DTMF detection methods are out of scope.

CH: Agreed.  

CH: Isn't time to include the MUST requirement for DTMF testing and SHOULD requirement for transmission support into the
requirements document. 
Then, we could close issue #5 and continue with other things.

 Christian

---

From: stephen botzko <[email-masked]>
Date: Fri, 2 Apr 2010 14:18:19 -0400

I am fine with your suggestions.

Stephen Botzko

On Fri, Apr 2, 2010 at 1:34 PM, Christian Hoene <[email-masked]>wrote:

> was used in MARTINI and other working groups at IETF77 in
> requirements presentations with no confusion. It has a clear meaning in the
> requirements context (desirable but not essential) and I
> see no reason to avoid its use at this phase.  There will certainly be
> other things that are "nice to have", and it is appropriate
> Though if DTMF quality is known to be inadequate, it
> CH: Isn't time to include the MUST requirement for DTMF testing and SHOULD
> requirement for transmission support into the
> requirements document.

---

From: James Rafferty <[email-masked]>
Date: Fri, 2 Apr 2010 14:48:18 -0400

+1
James Rafferty

From: [email-masked] [mailto:[email-masked]] On Behalf Of stephen botzko
Sent: Friday, April 02, 2010 2:18 PM
To: Christian Hoene
Cc: [email-masked]
Subject: Re: [codec] #5: Mention DTMF in requirements

Stephen Botzko
On Fri, Apr 2, 2010 at 1:34 PM, Christian Hoene <[email-masked]<mailto:[email-masked]>> wrote:
Hi Stephan,

Comments inline:

---------------------------------------------------------------
Dr.-Ing. Christian Hoene
Interactive Communication Systems (ICS), University of Tübingen
Sand 13, 72076 Tübingen, Germany, Phone +49 7071 2970532
http://www.net.uni-tuebingen.de/
From: stephen botzko [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Friday, April 02, 2010 4:50 PM
To: Christian Hoene
Cc: Michael Knappe; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [codec] #5: Mention DTMF in requirements
(a) "MUST NOT" means that it must fail.  Perhaps you meant "need not" or "might not"?
CH: Yes, I meant need not (or the German "muss nicht")

 Christian

---

From: Roman Shpount <[email-masked]>
Date: Fri, 2 Apr 2010 15:02:39 -0400

This is all wonderful, but what seems to be missing here is any type of
standard test for DTMF tone detection. We have a number of tests for things
that should not be detected as DTMF (various talk-off tapes), but there is
no test with samples of valid DTMF signals. In real world you get quite a
variety of tones that different handsets or telephone devices generate when
a digit is pressed. Some of those tones are almost at border line conditions
for a valid DTMF tone.

Second issue, which was already mentioned here, would be performance of DTMF
tone detector in the presence of packet loss or time stretching caused by
jitter buffer. Those things, even if they do not prevent DTMF detection, are
almost guaranteed to split a single long tone in two.

Finally, we seemed to concentrate on DTMF tones only, but in reality all the
signaling/special tones are quite important. Fax Send/Receive tones come to
mind almost immediately as tones that are typically detected inband (very
few gateways encode those tones using RFC 4733/2833)

I think we should formulate this requirement not in terms of DTMF tone but
in terms of accuracy of reproduction of  periodic signals in specified
frequency ranges. To be honest, the most robust solution would be to
incorporate RFC4733/4744 as a part of the codec in a manner similar to
comfort noise frames.
_____________________________
Roman Shpount - www.telurix.com

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Fri, 02 Apr 2010 12:49:13 -0700

On 04/02/2010 12:02 PM, Roman Shpount wrote:
> standard test for DTMF tone detection. We have a number of tests for
> things that should not be detected as DTMF (various talk-off tapes), but
> there is no test with samples of valid DTMF signals. In real world you
> get quite a variety of tones that different handsets or telephone
> devices generate when a digit is pressed. Some of those tones are almost
> at border line conditions for a valid DTMF tone.
> 
> Second issue, which was already mentioned here, would be performance of
> DTMF tone detector in the presence of packet loss or time stretching
> caused by jitter buffer. Those things, even if they do not prevent DTMF
> detection, are almost guaranteed to split a single long tone in two.
> 
> Finally, we seemed to concentrate on DTMF tones only, but in reality all
> the signaling/special tones are quite important. Fax Send/Receive tones
> come to mind almost immediately as tones that are typically detected
> inband (very few gateways encode those tones using RFC 4733/2833)
> 
> I think we should formulate this requirement not in terms of DTMF tone
> but in terms of accuracy of reproduction of  periodic signals in
> specified frequency ranges. To be honest, the most robust solution would
> be to incorporate RFC4733/4744 as a part of the codec in a manner
> similar to comfort noise frames.

Why would you want to do that, when there is perfectly good *Internet* codecs
for telephony signals (RFC 4733, RFC 4734 and RFC 5244) and comfort noise (RFC
3389)?

> Roman Shpount - www.telurix.com <http://www.telurix.com>
> 
> 
>     Cc: Michael Knappe; [email-masked] <mailto:[email-masked]>;
>     [email-masked] <mailto:[email-masked]>
>     (b) SHOULD is commonly used in establishing requirements, and it
>     certainly was used in MARTINI and other working groups at IETF77 in
>     requirements presentations with no confusion. It has a clear meaning
>     in the requirements context (desirable but not essential) and I
>     see no reason to avoid its use at this phase.  There will certainly
>     be other things that are "nice to have", and it is appropriate
>     I agree that language about DTMF might not need to be in the final
>     RFC.  Though if DTMF quality is known to be inadequate, it
>     CH: Isn't time to include the MUST requirement for DTMF testing and
>     SHOULD requirement for transmission support into the
>     requirements document.

-- 
Marc Petit-Huguenin
Personal email: [email-masked]
Professional email: [email-masked]
Blog: http://blog.marc.petit-huguenin.org

---

From: stephen botzko <[email-masked]>
Date: Fri, 2 Apr 2010 16:12:40 -0400

We don't have agreed-upon tests for anything.  I've been assuming (possibly
incorrectly) that after the requirements are understood the group will need
to develop a test plan that indicates how we will test against the various
requirements.

Stephen Botzko

On Fri, Apr 2, 2010 at 3:02 PM, Roman Shpount <[email-masked]> wrote:

> DTMF tone detector in the presence of packet loss or time stretching caused
> by jitter buffer. Those things, even if they do not prevent DTMF detection,
> are almost guaranteed to split a single long tone in two.
>
> the signaling/special tones are quite important. Fax Send/Receive tones come
> to mind almost immediately as tones that are typically detected inband (very
>> requirements presentations with no confusion. It has a clear meaning in
>> the requirements context (desirable but not essential) and I

---

From: Roman Shpount <[email-masked]>
Date: Fri, 2 Apr 2010 16:25:59 -0400

This was somewhat my point: instead of figuring out how to encode telephone
signals in the CODEC encode them using these standards. If we need to make
this robust in situations when these CODECs are not available we can include
a pass though mechanism for their content in the CODEC itself (even though I
really think this is a bad idea). Whatever we are going to do to encode
telephony signals would probably be worse quality, less robust and consume
more bandwidth then the existing work. Not sure why we need to recreate it.

Comfort noise is a different animal altogether. First of all, I think CN is
covered by IPR licensing from G.729. Furthermore, unless I am mistaken, it
is narrow band. I heard suggestions about using AMR-WB comfort noise for
wide band, but this once again might be covered by AMR-WB IPR.
_____________________________
Roman Shpount - www.telurix.com

---

From: Roman Shpount <[email-masked]>
Date: Fri, 2 Apr 2010 16:32:53 -0400

I have a feeling that this requirement as it stands right now is
non-enforceable since there is no industry accepted test on DTMF pass
through.  Creating such test will not be trivial since it will require
collection of the large amount of real life DTMF and other telephony signal
recordings. So, we cannot make this a requirement for a new codec unless we
define this in a form of some minimum frequency fidelity or some other
independently verifiable quality requirement.
_____________
Roman Shpount

---

From: stephen botzko <[email-masked]>
Date: Fri, 2 Apr 2010 16:41:13 -0400

Its my understanding that several other standardized codecs have had
requirements like this one (and have somehow been tested).  I agree that we
will need a more precise and testable statement, and a corresponding test
method.

Though that will be true for *every* requirement we agree upon in this
phase.

Stephen Botzko

On Fri, Apr 2, 2010 at 4:32 PM, Roman Shpount <[email-masked]> wrote:

>> We don't have agreed-upon tests for anything.  I've been assuming
>> (possibly incorrectly) that after the requirements are understood the group
>> will need to develop a test plan that indicates how we will test against the
>> various requirements.
>>
>> Stephen Botzko
>>
>>> but in terms of accuracy of reproduction of  periodic signals in specified

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Fri, 02 Apr 2010 13:43:28 -0700

On 04/02/2010 01:25 PM, Roman Shpount wrote:
> This was somewhat my point: instead of figuring out how to encode
> telephone signals in the CODEC encode them using these standards. If we
> need to make this robust in situations when these CODECs are not
> available we can include a pass though mechanism for their content in
> the CODEC itself (even though I really think this is a bad idea).
> Whatever we are going to do to encode telephony signals would probably
> be worse quality, less robust and consume more bandwidth then the
> existing work. Not sure why we need to recreate it.

Sorry, I misunderstood you then.  I thought that you were proposing to use a
different type of frame inside the codec to carry the DTMF.

> 
> Comfort noise is a different animal altogether. First of all, I think CN
> is covered by IPR licensing from G.729. 

I cannot find any IPR disclosure for RFC 3389 or draft-ietf-avt-rtp-cn.  If you
think that there should be one, then you should fill a third-party IPR disclosure.

> Furthermore, unless I am
> mistaken, it is narrow band. I heard suggestions about using AMR-WB
> comfort noise for wide band, but this once again might be covered by
> AMR-WB IPR.

I do not see why RFC 3389 would not be used fine for the Internet wideband
codec.  There is an example for use with G.722.1:

m=audio 49230 RTP/AVP 101 102
a=rtpmap:101 G7221/16000
a=fmtp:121 bitrate=24000
a=rtpmap:102 CN/16000

> Roman Shpount - www.telurix.com <http://www.telurix.com/>
> 
> 
> On Fri, Apr 2, 2010 at 3:49 PM, Marc Petit-Huguenin <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     <http://www.telurix.com>
>     > On Fri, Apr 2, 2010 at 1:34 PM, Christian Hoene
>     <[email-masked] <mailto:[email-masked]>
>     > <mailto:[email-masked] <mailto:[email-masked]>>>
>     wrote:
>     >
>     >     Hi Stephan,
>     >
>     >     Comments inline:
>     >
>     >
>     >     From: stephen botzko [mailto:[email-masked]
>     <mailto:[email-masked]>
>     >     <mailto:[email-masked]
>     <mailto:[email-masked]>>]
>     >     Cc: Michael Knappe; [email-masked]
>     <mailto:[email-masked]> <mailto:[email-masked]
>     <mailto:[email-masked]>>;
>     >     [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>     <mailto:[email-masked]>>
>     >     CH: Better? "The codec SHOULD support the transmission DTMF at
>     most
>     >     transmission conditions."
>     >
>     >     I also agree to the obvious comment that DTMF detection
>     methods are
>     >     out of scope.
>     >
>     >     CH: Agreed.
>     >

---

From: "Raymond (Juin-Hwey) Chen" <[email-masked]>
Date: Fri, 2 Apr 2010 13:54:58 -0700

Actually, ITU-T Recommendation G.720 gives some guidelines about how to test a codec's performance when passing DTMF signals through in-band.  (It also talks about other non-voice signals.)

In practice, however, we found that some of the test conditions specified in G.720 are fairly extreme corner cases which you normally don't see in real-world recorded DTMF signals.  Real-world DTMF signals are usually easier to pass through a codec without degradation in the DTMF detection error rate than those extreme DTMF conditions specified in G.720.

In our BroadVoice16 paper that I mentioned previously, we described a method for testing DTMF pass-through performance for a codec.  The method is based on G.720 but with some relaxation of the extreme conditions to better reflect the DTMF conditions we observed in a large amount of real-world record DTMF signals.  This is because we are more interested in how a codec would perform when passing typical real-world DTMF signals through.

Raymond

From: [email-masked] [mailto:[email-masked]] On Behalf Of stephen botzko
Sent: Friday, April 02, 2010 1:13 PM
To: Roman Shpount
Cc: [email-masked]
Subject: Re: [codec] #5: Mention DTMF in requirements

We don't have agreed-upon tests for anything.  I've been assuming (possibly incorrectly) that after the requirements are understood the group will need to develop a test plan that indicates how we will test against the various requirements.

Stephen Botzko
On Fri, Apr 2, 2010 at 3:02 PM, Roman Shpount <[email-masked]<mailto:[email-masked]>> wrote:
This is all wonderful, but what seems to be missing here is any type of standard test for DTMF tone detection. We have a number of tests for things that should not be detected as DTMF (various talk-off tapes), but there is no test with samples of valid DTMF signals. In real world you get quite a variety of tones that different handsets or telephone devices generate when a digit is pressed. Some of those tones are almost at border line conditions for a valid DTMF tone.

Second issue, which was already mentioned here, would be performance of DTMF tone detector in the presence of packet loss or time stretching caused by jitter buffer. Those things, even if they do not prevent DTMF detection, are almost guaranteed to split a single long tone in two.

Finally, we seemed to concentrate on DTMF tones only, but in reality all the signaling/special tones are quite important. Fax Send/Receive tones come to mind almost immediately as tones that are typically detected inband (very few gateways encode those tones using RFC 4733/2833)

I think we should formulate this requirement not in terms of DTMF tone but in terms of accuracy of reproduction of  periodic signals in specified frequency ranges. To be honest, the most robust solution would be to incorporate RFC4733/4744 as a part of the codec in a manner similar to comfort noise frames.
_____________________________
Roman Shpount - www.telurix.com<http://www.telurix.com>

Comments inline:

 Christian

---

From: Brian West <[email-masked]>
Date: Fri, 2 Apr 2010 16:14:41 -0500

The biggest issue with relaxed DTMF is screaming kids can trigger stray DTMF detection as can some people's voices.

/b

On Apr 2, 2010, at 3:54 PM, Raymond (Juin-Hwey) Chen wrote:

> Actually, ITU-T Recommendation G.720 gives some guidelines about how to test a codec’s performance when passing DTMF signals through in-band.  (It also talks about other non-voice signals.)
>  
> In practice, however, we found that some of the test conditions specified in G.720 are fairly extreme corner cases which you normally don’t see in real-world recorded DTMF signals.  Real-world DTMF signals are usually easier to pass through a codec without degradation in the DTMF detection error rate than those extreme DTMF conditions specified in G.720.
>

---

From: "Raymond (Juin-Hwey) Chen" <[email-masked]>
Date: Fri, 2 Apr 2010 14:43:32 -0700

Hi Brian,

You probably misunderstood me.  I am not suggesting relaxing the DTMF detector, which may increase the probability of detecting speech as DTMF.  I was just talking about relaxing the corner-case conditions imposed on the DTMF signal itself.  The DTMF detector remains the same.  For example, G.720 specify 50 ms on, 40 ms off tone duration, but all the automatic DTMF dialers we recorded has significantly longer on durations than 50 ms, and even when I deliberately tried to press the button for as short a duration as possible, I was never able to produce a DTMF tone duration shorter than about 60 ms. As I said in my last email, most recorded DTMF signals we collected have on durations of 90 to 160 ms. Therefore, if we want to get a better idea of how a codec would perform in passing real-world DTMF signals, we may want to use a tone duration longer than 50 ms, perhaps in the 75 to 105 ms range like we did in our BroadVoice16 paper.

Of course, if necessary, we can still use exactly the corner conditions listed in G.720 and measure the DTMF detection error rate of a codec for each condition.  That will tell us the worst-case DTMF pass-through performance of a codec (versus the "typical-case" performance for real-world DTMF signals described above).

In reality, DTMF tones were carefully chosen so that there is no harmonic relationship between any pair of the 8 possible frequencies.  This makes the probability of falsely detecting speech as a valid DTMF tone extremely low since high-energy voiced signal have harmonically spaced spectral peaks.  Just to give an idea, we passed the 2.5 hours of speech in the Bellcore DTMF test tapes through a codec then a commercial DTMF detector.  When there was no codec (i.e. 16-bit linear PCM was used), only 40 isolated DTMF digits were detected out of 2.5 hours of speech.  When the codec was 64 kb/s mu-law PCM, 39 digits were detected.  When the codec was BV16, only 31 digits were detected. These numbers are much lower than what's specified in the requirement R15-21 of the Bellcore General Requirements GR-506-CORE, which states that a DTMF receiver shall not register the 16 possible DTMF symbols more than 670 times when decoding the 2.5 hours of signals contained in the three Bellcore cassette tapes.

Raymond

From: Brian West [mailto:[email-masked]]
Sent: Friday, April 02, 2010 2:15 PM
To: Raymond (Juin-Hwey) Chen
Cc: stephen botzko; Roman Shpount; [email-masked]
Subject: Re: [codec] #5: Mention DTMF in requirements

/b

Raymond

---

From: "Raymond (Juin-Hwey) Chen" <[email-masked]>
Date: Fri, 2 Apr 2010 18:05:26 -0700

Comments in-line.

Raymond

From: [email-masked] [mailto:[email-masked]] On Behalf Of Roman Shpount
Sent: Friday, April 02, 2010 12:03 PM
To: [email-masked]
Subject: Re: [codec] #5: Mention DTMF in requirements

This is all wonderful, but what seems to be missing here is any type of standard test for DTMF tone detection. We have a number of tests for things that should not be detected as DTMF (various talk-off tapes), but there is no test with samples of valid DTMF signals. In real world you get quite a variety of tones that different handsets or telephone devices generate when a digit is pressed. Some of those tones are almost at border line conditions for a valid DTMF tone.
[Raymond]: I commented on this earlier. ITU-T G.720 can be used for general guidelines. There were some published papers on codec DTMF pass-through testing based on G.720. Our BroadVoice16 paper is one of them (even though we relaxed some corner conditions of G.720 to get a better idea of real-world performance). If people are interested, I can share more details about the test method.
Second issue, which was already mentioned here, would be performance of DTMF tone detector in the presence of packet loss or time stretching caused by jitter buffer. Those things, even if they do not prevent DTMF detection, are almost guaranteed to split a single long tone in two.
[Raymond]: Packet loss certainly will cause problems, but if the packet loss rate is relatively low, there is still a good chance that DTMF digits can get through.
Finally, we seemed to concentrate on DTMF tones only, but in reality all the signaling/special tones are quite important. Fax Send/Receive tones come to mind almost immediately as tones that are typically detected inband (very few gateways encode those tones using RFC 4733/2833)

I think we should formulate this requirement not in terms of DTMF tone but in terms of accuracy of reproduction of  periodic signals in specified frequency ranges. To be honest, the most robust solution would be to incorporate RFC4733/4744 as a part of the codec in a manner similar to comfort noise frames.
[Raymond]: I don't think we can do it in terms of "accuracy of reproduction of periodic signals in specified frequency ranges".  That may work for some single-frequency signaling tones, but it won't work for DTMF, since DTMF signals are not periodic in any sense as the two frequencies of the sine waves are not harmonically related.
_____________________________
Roman Shpount - www.telurix.com<http://www.telurix.com>

Comments inline:

 Christian

---

From: "Raymond (Juin-Hwey) Chen" <[email-masked]>
Date: Fri, 2 Apr 2010 18:05:59 -0700

+1

Raymond

From: [email-masked] [mailto:[email-masked]] On Behalf Of James Rafferty
Sent: Friday, April 02, 2010 11:48 AM
To: stephen botzko; Christian Hoene
Cc: [email-masked]
Subject: Re: [codec] #5: Mention DTMF in requirements

+1
James Rafferty

Stephen Botzko

Comments inline:

 Christian

---

From: Roman Shpount <[email-masked]>
Date: Fri, 2 Apr 2010 21:25:38 -0400

I understand your points. There are few questions that I have:

1. Do we need a real world DTMF and telephone signals recording database to
implement this? Correct me if I am wrong, you've done your tests for BV16
using your own DTMF generator and unspecified "commercial" DTMF detector.
How would define this test in a way it is possible to replicate it by third
parties and reference it in a standard?

2. Did you ever test your CODEC for other telephony signals, such as /ANSam,
ANS, CED, and CI tones? These tones are needed in order to detect modems and
fax machines and switch to an appropriate payload. What about progress
tones?

3. Is this criteria feasible for large frame CELP CODECs? Are we going to
exclude them based on this criteria?
_____________________________
Roman Shpount - www.telurix.com

On Fri, Apr 2, 2010 at 9:05 PM, Raymond (Juin-Hwey) Chen <[email-masked]

>  Comments in-line.
>
>
>
> Raymond
>
>
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Roman Shpount
> *Sent:* Friday, April 02, 2010 12:03 PM
> *To:* [email-masked]
> [Raymond]: I commented on this earlier. ITU-T G.720 can be used for general
> guidelines. There were some published papers on codec DTMF pass-through
> testing based on G.720. Our BroadVoice16 paper is one of them (even though
> we relaxed some corner conditions of G.720 to get a better idea of
> real-world performance). If people are interested, I can share more details
> about the test method.
>
> [Raymond]: Packet loss certainly will cause problems, but if the packet
> loss rate is relatively low, there is still a good chance that DTMF digits
> can get through.
>
> [Raymond]: I don’t think we can do it in terms of “accuracy of reproduction
> of periodic signals in specified frequency ranges”.  That may work for some
> single-frequency signaling tones, but it won’t work for DTMF, since DTMF
> signals are not periodic in any sense as the two frequencies of the sine
> waves are not harmonically related.
>

---

From: "Raymond (Juin-Hwey) Chen" <[email-masked]>
Date: Mon, 5 Apr 2010 14:43:16 -0700

Roman,

Answers to you questions in-line.

Raymond

From: Roman Shpount [mailto:[email-masked]]
Sent: Friday, April 02, 2010 6:26 PM
To: Raymond (Juin-Hwey) Chen
Cc: [email-masked]
Subject: Re: [codec] #5: Mention DTMF in requirements

1. Do we need a real world DTMF and telephone signals recording database to implement this? Correct me if I am wrong, you've done your tests for BV16 using your own DTMF generator and unspecified "commercial" DTMF detector. How would define this test in a way it is possible to replicate it by third parties and reference it in a standard?
[Raymond]: We can use real-world recorded DTMF signals if we want to, but we don't have to. Yes, in our DTMF testing of BV16 we used a DTMF generator software to generate the waveform of 20,000 random DTMF symbols for each of 24 DTMF test conditions, for a total of nearly half a million DTMF symbols. After passing through the codec, the DTMF signals were passed through a PC executable program which simulated a DTMF decoder that was widely deployed in commercial VoIP applications. The decoded DTMF symbols were then compared with the original DTMF keys used to generate the symbols to calculate the error rates. As you can see, such a test method is purely based on software and can easily be duplicated by third parties.  If the codec WG decides not to do anything with DTMF, then obviously nothing further needs to be done, but if the WG decides that the codec candidate needs to be tested for DTMF pass-through, I will try to dig up our DTMF test software package and get internal approval to donate it to IETF for this purpose.
2. Did you ever test your CODEC for other telephony signals, such as /ANSam, ANS, CED, and CI tones? These tones are needed in order to detect modems and fax machines and switch to an appropriate payload. What about progress tones?
[Raymond]: No, we didn't.  However, if these are single-frequency tones, I would expect that it should be easier for a typical codec to pass these tones than to pass the dual-frequency DTMF signals without causing significant degradation in the subsequent processing of these tones.
3. Is this criteria feasible for large frame CELP CODECs? Are we going to exclude them based on this criteria?
[Raymond]: If a CELP codec with a large frame size is designed such that it can provide high encoding fidelity for DTMF signals, then it is still feasible for such a CELP codec to pass DTMF signals without causing subsequent DTMF decoding errors, so being a large frame CELP codec does not automatically mean it will be excluded.
_____________________________
Roman Shpount - www.telurix.com<http://www.telurix.com>

On Fri, Apr 2, 2010 at 9:05 PM, Raymond (Juin-Hwey) Chen <[email-masked]<mailto:[email-masked]>> wrote:
Comments in-line.

Raymond

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Roman Shpount
Sent: Friday, April 02, 2010 12:03 PM
To: [email-masked]<mailto:[email-masked]>
Subject: Re: [codec] #5: Mention DTMF in requirements

Comments inline:

 Christian

---

From: "Kevin P. Fleming" <[email-masked]>
Date: Tue, 06 Apr 2010 06:47:51 -0500

Raymond (Juin-Hwey) Chen wrote:

> 2. Did you ever test your CODEC for other telephony signals, such as
> /ANSam, ANS, CED, and CI tones? These tones are needed in order to
> detect modems and fax machines and switch to an appropriate payload.
> What about progress tones?
> [Raymond]: No, we didn’t.  However, if these are single-frequency tones,
> I would expect that it should be easier for a typical codec to pass
> these tones than to pass the dual-frequency DTMF signals without causing
> significant degradation in the subsequent processing of these tones.

This is pretty much a dead-end based on the other comments on this list,
but in general, no, most of these tones are *not* single-frequency
simple tones. ANSam, for example, is a single frequency, but is
amplitude modulated. There are also variants that have phase reversals,
and these must be preserved for them to be discriminated from the
non-phase-reversal versions.

-- 
Kevin P. Fleming
Digium, Inc. | Director of Software Technologies
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
skype: kpfleming | jabber: [email-masked]
Check us out at www.digium.com & www.asterisk.org

---

From: stephen botzko <[email-masked]>
Date: Tue, 6 Apr 2010 10:49:29 -0400

I agree that there is a pretty strong consensus that carriage of "analog"
fax and modem signals is a non-requirement.  If by "payload switching" we
are talking about switching codecs, then that is beyond the charter scope.

I am thinking that we are getting a bit too focused on testing methods.  In
my view it would be more efficient to capture use cases, and derive MUST and
SHOULD requirements from them first.  Then go back and figure out what we
will test and how we will test it.

Stephen Botzko

---

From: Koen Vos <[email-masked]>
Date: Wed, 07 Apr 2010 10:28:12 -0700

The distinction between use case and testing makes sense to me.

DTMF is a real use case. For example some PSTN gateways still don't  
support out-of-band DTMF.

It's unclear what to test about DTMF though. The Codec will support a  
range of sample rates and bitrates. The requirements specify coding of  
music at near-transparent quality. I don't see how a such a codec  
could not provide sufficient quality for in-band DTMF. At the same  
time, the requirements talk about bitrates down to 8 kbps, where it  
may be quite tricky to get DTMF to work well. Therefore, specifying  
that the codec SHOULD or MUST encode DTMF with sufficient quality is  
meaningless. If anything, the bitrate settings that allow in-band DTMF  
could be characterized after the Codec is formed.

Nevertheless it's good to be aware of these use cases during  
development. For instance, with SILK we included DTMF signals in the  
LSF quantizer training database.

Similarly, I see tandem coding as a use case to keep in mind during  
development, but again don't see a need for testing or MUST/SHOULD  
requirements.

koen.

Quoting stephen botzko:

>> > [Raymond]: No, we didn't.  However, if these are single-frequency tones,

---

From: stephen botzko <[email-masked]>
Date: Thu, 8 Apr 2010 07:23:01 -0400

We are getting closer to agreement, but maybe aren't all the way there yet.

For me, requirements need to be solidly anchored in use cases.  So if we
identify reasonably common use cases, then the functionality that is needed
for use cases has to be reflected in the requirements (either as SHOULDs or
MUSTs).  If the requirements are not clearly articulated and documented,
then some will get lost as we move through the development / seleciton /
testing process.

We certainly agree that the developers will need to keep all the
requirements in mind.  After we have the full list of requirements, we will
then need to figure out the test plan.

I think we also agree that we won't succeed if we have a very long list of
requirements.  One way to try to prevent that is to narrow our application
focus.  At the moment that seems a bit vague, and perhaps to broad.

As an aside, I would prioritize tandeming well above DTMF.  We are not
seeing any trend towards client side mixing (either with SIP phones or video
systems).  Also conference bridges are typically deployed for a very long
time, so the existing server-side mixing will be with us for the foreseeable
future..

Stephen Botzko

On Wed, Apr 7, 2010 at 1:28 PM, Koen Vos <[email-masked]> wrote:

> DTMF is a real use case. For example some PSTN gateways still don't support
> out-of-band DTMF.
>
> It's unclear what to test about DTMF though. The Codec will support a range
> of sample rates and bitrates. The requirements specify coding of music at
> near-transparent quality. I don't see how a such a codec could not provide
> sufficient quality for in-band DTMF. At the same time, the requirements talk
> about bitrates down to 8 kbps, where it may be quite tricky to get DTMF to
> work well. Therefore, specifying that the codec SHOULD or MUST encode DTMF
> with sufficient quality is meaningless. If anything, the bitrate settings
> that allow in-band DTMF could be characterized after the Codec is formed.
>
> Nevertheless it's good to be aware of these use cases during development.
> For instance, with SILK we included DTMF signals in the LSF quantizer
> training database.
>
>> On Tue, Apr 6, 2010 at 7:47 AM, Kevin P. Fleming <[email-masked]
>> >wrote:
>>

---

From: "Wyss, Felix" <[email-masked]>
Date: Thu, 8 Apr 2010 09:27:08 -0400

I still fail to see a reason to include support for DTMF transparency as a requirement.  It simply does not seem worth the cost to even test (DTMF) tones other than to ensure the codec doesn't produce overly unpleasant audio artifacts.

Your statement that some PSTN gateways currently don't support RFC#2833 seems immaterial to me as they also do not support this yet to be designed codec.

--Felix

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Koen Vos
> Sent: Wednesday, April 07, 2010 13:28
> To: [email-masked]

---

From: stephen botzko <[email-masked]>
Date: Thu, 8 Apr 2010 13:51:56 -0400

On this thread I see 6 posters who see at least a SHOULD requirement, 4
posters who see a NON requirement and 3 posters for which I can't tell.

There seems to be general agreement that the use case is real, the
disagreement is whether RFC 2833 eliminates the need for DTMF transparency
(or not).

I suggest a hum in July might be the best way to see where the community as
a whole is.

Stephen Botzko

On Thu, Apr 8, 2010 at 9:27 AM, Wyss, Felix <[email-masked]> wrote:

> I still fail to see a reason to include support for DTMF transparency as a
> requirement.  It simply does not seem worth the cost to even test (DTMF)
> tones other than to ensure the codec doesn't produce overly unpleasant audio
> artifacts.
>
> Your statement that some PSTN gateways currently don't support RFC#2833
> seems immaterial to me as they also do not support this yet to be designed
> codec.
>
> --Felix
>
> > > my view it would be more efficient to capture use cases, and derive
> MUST
> > and