Subject: Re: [rtcweb] Working group last call for draft-ietf-rtcweb-audio

From: Tim Panton <[email-masked]>
Date: Tue, 26 Jan 2016 12:15:06 +0000

> On 19 Dec 2015, at 00:10, Cullen Jennings (fluffy) <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
>> On Dec 18, 2015, at 3:24 PM, Roman Shpount <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> This applies to all DTMF tones, not A-D specifically. What I am saying is that WebRTC endpoints must be able to generate all DTMF tones including A-D and send them to legacy PSTN (gateway).
> 
> The thing people keep asking is why? what is the use case ?
> 
> I don’t feel like I have seen an answer to this. 

More specifically - Under what circumstances will we need them where 
the same result couldn’t be achieved by injecting the correct notes into the stream using webAudio?
It has been suggested that webaudio tone generation might not work with certain sets of codecs.
But surely the (few) legacy systems (coal mines for example) that use A-D will be using G711
for which tone generation should work well.

(I might add that tone _detection_ can also be done in webaudio so developers have a symmetrical solution).

Tim.

> 
> 
> _______________________________________________
> rtcweb mailing list
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/rtcweb

Tim Panton - Web/VoIP consultant and implementor
www.westhawk.co.uk <http://www.westhawk.co.uk/>

---

From: Roman Shpount <[email-masked]>
Date: Tue, 26 Jan 2016 14:59:57 -0500

> developers have a symmetrical solution).

Any tone generated using inband audio will be less reliable then
telephone-events tone. For telephone-events 3 or more packets in a row need
to be lost for the tone to be lost or interrupted. For inband loosing a
single packet can break tone in two distinct tones. I am not even talking
about all the extra work and interop required to properly implement tone
generation. To generate all DTMF tone telephone-events you need no extra
code in the browser you need no additional code.

Finally, if you want WebRTC to be backwards compatible with RFC 2833 (which
is the most common implementation of telephone-events in PSTN universe),
then you will need to deal with this
https://tools.ietf.org/html/rfc2833#section-3.3:

   A compliant implementation MUST support the events listed in Table 1
   with the exception of "flash". If it uses some other, out-of-band
   mechanism for signaling line conditions, it does not have to
   implement the other events.

Here is table 1 (https://tools.ietf.org/html/rfc2833#section-3.10):

   Table 1 summarizes the DTMF-related named events within the
   telephone-event payload format.

                     Event  encoding (decimal)
                     _________________________
                     0--9                0--9
                     *                     10
                     #                     11
                     A--D              12--15
                     Flash                 16

                     Table 1: DTMF named events

So, all DTMF digits 0-9, *, #, A-D are required by RFC 2833.

So, if you want to break backwards compatibility please explain your
motivation. These tones (as well as all the other telephone-events) are
only needed for backwards compatibility. Implemented all or some of them
requires the same amount of effort. Not implementing all of them will make
life harder for the few people who use them.

Finally, I have wrote several other things regarding DTMF tone generation
which are considerably more important. Does everyone agree with the
following or no one cares:

Generated events MUST have duration of no more than 6000 ms and no less
than 40 ms with the recommended default duration of 100 ms for each tone.
The gap between events MUST be no less then 30 ms with the recommended
default duration of 70 ms. Event SHOULD start on a regular audio packet
border and event and gap duration SHOULD be rounded up to the next regular
audio packet border.

During the time events are generated no audio SHOULD be sent for the same
audio stream. When gaps between events are generated, silence
and not the background audio SHOULD be sent using regular audio encoding.

If multiple audio sampling rates are supported, audio/telephone-event
payload
SHOULD be present for each supported sampling rate. Endpoints SHOULD use
audio/telephone-event format parameters during the offer/answer to
indicate which events are supported.

Receivers MUST be able to consume any audio/telephone-event events
in such a way that it will not generate audio artifacts, jitter buffer
adjustments, payload mismatches, or invalid RTCP statistics calculation.
Receivers MAY generate audio corresponding to the received events
but are also allowed to discard them in a manner that does not affect
regular audio processing.

Regards,
_____________
Roman Shpount

---

From: Jean-Marc Valin <[email-masked]>
Date: Tue, 26 Jan 2016 15:26:43 -0500

Hi Roman,

On 01/26/2016 02:59 PM, Roman Shpount wrote:
> Finally, I have wrote several other things regarding DTMF tone 
> generation which are considerably more important. Does everyone
> agree with the following or no one cares:

The text below is what you propose for the rtcweb-audio draft? I
personally have no opinion on these events (see comment inline for
clarification though) so I'm happy to include that text if everyone
else is OK with it.

> Generated events MUST have duration of no more than 6000 ms and no
> less than 40 ms with the recommended default duration of 100 ms for
> each tone. The gap between events MUST be no less then 30 ms with
> the recommended default duration of 70 ms. Event SHOULD start on a
> regular audio packet border and event and gap duration SHOULD be
> rounded up to the next regular audio packet border.
> 
> During the time events are generated no audio SHOULD be sent for
> the same audio stream. When gaps between events are generated,
> silence and not the background audio SHOULD be sent using regular
> audio encoding.

I think this would require a bit of clarification. What does "no
audio" mean here? Silence or no codec packets at all (in that case
what about PLC)? Also, keep in mind that any silence may not be
perfectly aligned with theoretical frame boundaries, due to look-ahead.

> If multiple audio sampling rates are supported,
> audio/telephone-event payload SHOULD be present for each supported
> sampling rate. Endpoints SHOULD use audio/telephone-event format
> parameters during the offer/answer to indicate which events are
> supported.
> 
> in such a way that it will not generate audio artifacts, jitter
> buffer adjustments, payload mismatches, or invalid RTCP statistics
> calculation. Receivers MAY generate audio corresponding to the
> received events but are also allowed to discard them in a manner
> that does not affect regular audio processing.

Cheers,

	Jean-Marc

---

From: Roman Shpount <[email-masked]>
Date: Tue, 26 Jan 2016 16:21:41 -0500

> clarification though) so I'm happy to include that text if everyone

Yes, this is the text I am proposing for rtcweb-audio draft.

> perfectly aligned with theoretical frame boundaries, due to look-ahead.
First of all I was trying to copy all the normative tone duration
information from W3C document to rtcweb-audio draft.

Second, as far as "no audio" is concerned, what I mean here is that RFC4733
telephone-events are alternative audio encoding similar to CN (see
https://tools.ietf.org/html/rfc4733#section-2.5.2.2). For RTP timestamp
interval covered by telephone-event no non-event audio packets, such as
Opus or G.711, should be present since they are redundant and cause interop
problems. PLC does not apply here since remote gateways is supposed to
generate audio based on the information in telephone-events packets. This
is essentially the same requirement that end point should not send packets
with two different audio encodings (i.e. send Opus and G.711 RTP packets)
at the same time. This can work but can also cause interop problems.

Packet boundaries of audio encoded using codecs such as G.711 or Opus
should create a continuous stream as far as RTP timestamps are concerned
with RFC 4733 telephone event packets unless discontinuous RTP is used. It
should be possible to align decoded audio and telephone-events based on RTP
timestamps.

I also wanted to provide clarification for
https://tools.ietf.org/html/rfc4733#section-2.5.1.2. This section says:

   A source has wide latitude as to how often it sends event updates.  A
   natural interval is the spacing between non-event audio packets.
   (Recall that a single RTP packet can contain multiple audio frames
   for frame-based codecs and that the packet interval can vary during a
   session.)  Alternatively, a source MAY decide to use a different
   spacing for event updates, with a value of 50 ms RECOMMENDED.

I am proposing that WebRTC should recommend that sending telephone-event
packet at natural interval between non-event audio packets is
strongly preferred.

Also, blocking input audio for a short period of time after
telephone-events are transmitted prevents a duplicate DTMF digit audio from
being present in the encoded audio stream which some times happen due to
microphone picking up audio feedback tones played by some DTMF keypad
implementations. This also improves DTMF tone recognition by a receiving
IVR system after audio stream passes through PSTN gateway since having
background audio as a separator between DTMF digits can negatively affect
some tone detection algorithms.

Please let me know if you need more information.

Regards,
_____________
Roman Shpount

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 27 Jan 2016 12:17:58 -0500

On 01/26/2016 04:21 PM, Roman Shpount wrote:
> Second, as far as "no audio" is concerned, what I mean here is
> that RFC4733 telephone-events are alternative audio encoding
> similar to CN (see
> https://tools.ietf.org/html/rfc4733#section-2.5.2.2). For RTP 
> timestamp interval covered by telephone-event no non-event audio 
> packets, such as Opus or G.711, should be present since they are 
> redundant and cause interop problems. PLC does not apply here
> since remote gateways is supposed to generate audio based on the
> information in telephone-events packets. This is essentially the
> same requirement that end point should not send packets with two
> different audio encodings (i.e. send Opus and G.711 RTP packets) at
> the same time. This can work but can also cause interop problems.

The problem I see is that unlike G.711, modern codecs (not just Opus)
aren't stateless. It's not a simple thing to just stop and restart
transmission during telephony events -- especially if you want to
avoid glitches and want all implementations to behave the same. You
need to consider look-ahead (6 ms for Opus, with 5-10 ms being
typical), but also the "memory" from the previous frame.

> Packet boundaries of audio encoded using codecs such as G.711 or
> Opus should create a continuous stream as far as RTP timestamps are
> concerned with RFC 4733 telephone event packets unless
> discontinuous RTP is used. It should be possible to align decoded
> audio and telephone-events based on RTP timestamps.

What do you mean "create a continuous stream"? I thought you wanted
the codec to stop transmitting.

	Jean-Marc

---

From: Roman Shpount <[email-masked]>
Date: Wed, 27 Jan 2016 12:43:25 -0500

> typical), but also the "memory" from the previous frame.

In this regard Opus is no different then G.729 which was used with
telephone-events for ages. Sending a telephone-event packet is an audio
codec switch. Its behavior should be no different then switch from Opus to
PCMU or from Opus to CN and back. I assume statefull codec such as Opus
should reset its state before starting to send audio after such switch
occurs. Is this correct? From what I know this was never defined anywhere
and it might be a good idea from the codec designers to specify what is the
best behavior for codec switches from telephone-event or CN. Since most of
codecs converge very quickly I see this implemented equally often with or
without codec state reset. Also, since telephone-events are normally
followed by a period of silence (intertone gap), codec state plays much
smaller role in practice.

> the codec to stop transmitting.

Yes, I do want the code to stop transmitting. I want the clean codec switch
from non-event audio codec to telephone-event and back. What I mean here is
if you have PCMU packet with timestamp 1000 and duration 160 the following
telephone-event start timestamp should be 1160. The transition in other
direction should also happen without the timestamp gap so that if
telephone-event start timestamp is 1160 and the total tone duration is 800
then next non event audio packet should start with timestamp 1960.
Furthermore, I like to recommend to switch from non-event to
telephone-event and back on the regular packet boundary. This means no
partial audio packets (17 ms audio packet before telephone event) and no
gaps or overlaps after telephone-event (no telephone events with duration
117 ms which causes subsequent audio packet either start before the end of
the telephone-event or with a 3 ms gap after the event or exactly after the
telephone event but with 3 ms offset in comparison with audio packets
before telephone events).

Sorry for spending so much time on telephone-event generation but equipment
generating broken telephone-event streams and then causing havoc in the
phone network is something that is something that wasted couple of weeks of
my time this month alone. I simply want to provide the list of
recommendations for rtcweb which will cause the least amount of interop
trouble in the future.

Regards,
_____________
Roman Shpount