Subject: [codec] Technical Considerations additions

From: Alexander Chemeris <[email-masked]>
Date: Fri, 31 Jul 2009 00:46:17 +0400

Hello,

Just to make a note of proposed additions to technical considerations,
which were seen on the jabber conference:

1) Requirement/non requirement of stereo and/or multichannel
2) Good performance in the presence of packet loss
3) Compatibility with negotiation schemes widely used over
the Internet.

Points (1) and (2) are obvious and seem to leak from the consideration
just by occasion, but I want to slightly elaborate on point (3).

A non-technical point, which was not clearly defined, but I think is
very important:

* Considered codecs must be standardized. In ITU, IETF, wherever,
but there should be a clean standard allowing creation of compatible
3rd party implementation.

= Introduction for non-VoIP developers =

SIP is one of the most widely used and well known VoIP signaling and it
do use venerable SDP (RFC2327) for codec negotiation. The recommended
way to negotiate codecs with SDP is a so-called Offer/Answer Model
(RFC3264). Simply speaking, one side offers a codec set and the other
side can either accept a specific codec or reject its use.

= The problem =

SDP was clearly designed for the use with old-fashion codecs which
supported only one or few operating modes, usually just a set of few
different bitrates. But modern codecs tend to be a beasts of different
kind they offer a plenty of configuration options users have to choose
from. I know two examples of such codecs - Speex and AMR(-WB),
probably IPMR goes a similar way, but I'm not familiar with it.
For RTP payload and SDP negotiation of Speex look into RFC 5574,
for AMR and AMR-WB look into RFC5574.

So, the tale is about AMR(-WB) which has few horrible statements
in the RFC about SDP negotiation, worst of which is:

      -  Each combination of the RTP payload transport format
         configuration parameters (octet-align, crc, robust-sorting,
         interleaving, and channels) is unique in its bit-pattern and
         not compatible with any other combination. [...]

This means that if you want to support several combinations of
parameters, SDP offer will blow up into a huge list of different
parameter combinations. That's the problem, because SDP plus
SIP header should fit into UDP packet size (you can use TCP,
but it's much better to fit into UDP, isn't it?) and wasting payload
type numbers.
But this problem won't be that sharp if every VoIP developer read
RFC carefully with the full understanding of them and implement
everything as written. Instead, many developers assume that
defaults are fine and don't have time or willing to read or implement
such a complex operation. No need to say I've seen implementations
which ignored even "octet-align" fmtp parameter, trying to guess
it from the actual data.
When I implemented AMR(-WB) payload for our SIP stack I some
more inconveniences and even one collision in a RFC, but alas
I can't recall details now. I should have documented the problem
that time and not rely on my memory..

Speex does a better job here, as it does not have incompatible
sets of features. I.e. every stream, produced by an encoder can
be at least correctly passed if not decoded by a decoder.

The moral of that story is that ideal VoIP codec should not
assume a fixed number of parameter sets in endpoints, as
AMR does, and rely on SDP for features offer. Rather it should
use in-band signaling as much as possible to offload SDP
and make codec API "programmer-friendly".

-- 
Regards,
Alexander Chemeris.

SIPez LLC.
SIP VoIP, IM and Presence Consulting
http://www.SIPez.com
tel: +1 (617) 273-4000

---

From: "Slava Borilin" <[email-masked]>
Date: Fri, 31 Jul 2009 00:51:47 +0400

Thanks Alexander, 

that's exactly the type of input which is important define the
"internet" requirements to the codec, which will lead us to the either
selection, or design of the one which satisfy the technical goals, while
being easily accessible and freely distributable.

And that's (one more) proof why BOF agreed today that we @IETF have the
expertise to define the scope of work (and probably we have even more
expertise in this specific area then other SDOs).

best regards,
Slava Borilin

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Alexander Chemeris
Sent: Thursday, July 30, 2009 10:46 PM
To: [email-masked]
Subject: [codec] Technical Considerations additions

Hello,

= The problem =

-- 
Regards,
Alexander Chemeris.

SIPez LLC.
SIP VoIP, IM and Presence Consulting
http://www.SIPez.com
tel: +1 (617) 273-4000
_______________________________________________
codec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/codec

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 30 Jul 2009 14:14:50 -0700

Hi Slava,
I don't think the BOF "agreed" to anything, nor it has a mandate to agree to
anything.  

Hi Alexander,
According to my dim recollection of the discussions at the time, the reasons
for the complexity of the negotiation process in the AMR-WB payload (RFC
4867) lies in the (IMHO very desirable) backward compatibility of AMR-WB to
AMR---the by far widest deployed speech codec on the planet.  Whenever you
have an organic and backward compatible development of a technology, you add
options and make the capability exchange more complex.  Assuming that this
will not happen for a from-scratch designed codec is shortsighted---simply
because, over time, one will want to support more modes (and or more
codecs).  The main issue is that SIP has been extremely weak in its
capability exchange mechanisms.  This is about to get fixed.

With respect to the in-band signaling: I think there may be a small
misunderstanding here.  AMR-WB, AFAIK, contains a complete set of inband
signals---the bitstream is self-contained.  The designers of the AMR-WB
payload format simply gave the users the option to select their preferred
modes, instead of requiring the support of all of those modes.  The tradeoff
that matters here is between implementation complexity in the signal
processing part, and implementation complexity in the capability-exchange
(SDP) part.  I personally believe that a lot of payload formats have gotten
this tradeoff wrong, and the AMR-WB payload format may be one of those.  But
that's an issue that's most easily fixed in future payload designs (which
clearly is in the IETF's realm), and has very little to do with the codec
design itself.

Regards,
Stephan

---

From: Alexander Chemeris <[email-masked]>
Date: Fri, 31 Jul 2009 02:02:34 +0400

Hi, Stephan

On Fri, Jul 31, 2009 at 01:14, Stephan Wenger<[email-masked]> wrote:
> codecs).

Yeah, mode selection is a part of the problem, but it's natural and
can't be solved without a SDP. I rather blame the set of incompatible
codec packaging options. If you want octet-aligned stream with crc it
is not even parsable by decoder which expects octet-aligned stream
without crc. Why not to specify that for VoIP we want octet-aligned
(or not octet-aligned) version of the codec? I think that diversity of
settings were developed by ITU-T to make every single use-case
possible, to be possible to optimize for the slightly lower bandwidth
in one case and for slightly less CPU power in other case, etc. But
IMHO gained diversity does not cost the increased implementation
burden. Look at MPEG video codecs - you may not support some features,
but you are always able to at least parse the stream. What VoIP
"end-developers" want is a simple codec, which produce "frames", which
can be simply put into RTP without a lot of headache. Not this hell of
AMR packaging.

> The main issue is that SIP has been extremely weak in its

Right. But I'm not aware that it will be fixed before this Christmas. Do you? ;)

> modes, instead of requiring the support of all of those modes.

No, read my rants incompatible encodings above.

> The tradeoff

Not exactly. This CAN be fixed in IETF by defining a cool RTP payload
which will add additional in-band signalling, but isn't it nicer to
have a self-contained codec, which just produces "frames" and does not
need any fancy packaging and additional signaling? Look at Speex as an
example of pretty good implementation of this paradigm.

-- 
Regards,
Alexander Chemeris.

SIPez LLC.

---

From: "Kevin P. Fleming" <[email-masked]>
Date: Thu, 30 Jul 2009 17:24:31 -0500

> example of pretty good implementation of this paradigm.

This is also an important issue for SIP B2BUAs, when media streams from
two endpoints must be connected together even though they may not have
originally been connected (think call transfers, for example). With SDP
complexity of the sort present with AMR-WB (or video codecs), it's not
practically possible for the B2BUA to determine whether the current
media formats being used by the endpoints are compatible, even though
they may be same codec and sample rate. With Speex, at least, it's
always safe to send any Speex stream to a decoder, even if it wasn't
produced using the parameters the decoder specified during negotiation,
but I'm not aware of any other existing codecs for which that would work.

This results in the B2BUA having to completely understand all the
interactions of the codec parameters to determine whether two streams
are 'compatible', or whether renegotiation (or worse, transcoding) is
needed.

Moving the codec parameter negotiation into the codec bitstream resolves
this issue completely, and means that only the actual endpoints that
contain implementations of the codec have to understand the interactions
between its parameters.

-- 
Kevin P. Fleming
Digium, Inc. | Director of Software Technologies
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
skype: kpfleming | jabber: [email-masked]
Check us out at www.digium.com & www.asterisk.org

---

From: Steve Underwood <[email-masked]>
Date: Fri, 31 Jul 2009 09:32:07 +0800

> contain implementations of the codec have to understand the interactions
Codec mode information in the RTP stream is always fragile, due to the 
possibility of packet loss. However, codec mode information in the RTP 
stream during route switching is a truly terrible idea. It is very 
common for the first X packets of an RTP stream to go missing, for 
various reasons. Anything while relies on a perfectly clean packet flow 
at startup is a non-starter.

Steve

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Fri, 31 Jul 2009 07:46:05 +0200

Hi

I believe that in the AMR case it is very important to be able to distinguish between the apples and the oranges. AMR was originally designed for circuit switched access and therefore some extra parameters such as allowed mode sets, mode change period and mode change neighbor is in the negotiation, these parameters are there simply make interworking between a PS and a CS domain possible (and avoid transcoding). AMR is heavily used in mobile telephones (CS) and just because we start to think VoIP we cannot just pretend that CS does not exist overnight.
There is even a Q bit to tell that part of the payload is bad (something that in practice only happens in a CS environment, may happen with UDP-Lite)
Now for a PS to PS call, all that stuff is not needed anymore UDP-Lite may be used though but I have to see evidence of it). If you envision only PS to PS you can skip all the above things in your negotiation.
The octet aligned and bandwidth efficient modes are there to satisfy the requirements.
If you look into 3GPP spec TS26.114 you will find more info about this.   
Or why not have a look in this book 
http://www.wiley-vch.de/publish/dt/books/bySubjectEE00/bySubSubjectEE25/0-470-05855-2/?sID=nqk9adkt7hm033ice0kp7jh3v4

/Ingemar

> From: Alexander Chemeris [mailto:[email-masked]] 
> Sent: den 31 juli 2009 00:03
> To: Stephan Wenger
> Cc: Slava Borilin; [email-masked]
> Subject: Re: [codec] Technical Considerations additions
> 
> Hi, Stephan
> 
> > According to my dim recollection of the discussions at the 
> time, the 
> > reasons for the complexity of the negotiation process in the AMR-WB 
> > payload (RFC
> > 4867) lies in the (IMHO very desirable) backward compatibility of 
> > AMR-WB to AMR---the by far widest deployed speech codec on 
> the planet.  
> > Whenever you have an organic and backward compatible 
> development of a 
> > technology, you add options and make the capability exchange more 
> > complex.  Assuming that this will not happen for a from-scratch 
> > designed codec is shortsighted---simply because, over time, 
> one will 
> > want to support more modes (and or more codecs).
> 
> Yeah, mode selection is a part of the problem, but it's 
> natural and can't be solved without a SDP. I rather blame the 
> set of incompatible codec packaging options. If you want 
> octet-aligned stream with crc it is not even parsable by 
> decoder which expects octet-aligned stream without crc. Why 
> not to specify that for VoIP we want octet-aligned (or not 
> octet-aligned) version of the codec? I think that diversity 
> of settings were developed by ITU-T to make every single 
> use-case possible, to be possible to optimize for the 
> slightly lower bandwidth in one case and for slightly less 
> CPU power in other case, etc. But IMHO gained diversity does 
> not cost the increased implementation burden. Look at MPEG 
> video codecs - you may not support some features, but you are 
> always able to at least parse the stream. What VoIP 
> "end-developers" want is a simple codec, which produce 
> "frames", which can be simply put into RTP without a lot of 
> headache. Not this hell of AMR packaging.
> 
> capability 
> >exchange mechanisms.  This is about to get fixed.
> 
> > misunderstanding here.  AMR-WB, AFAIK, contains a complete set of 
> > inband signals---the bitstream is self-contained.  The designers of 
> > the AMR-WB payload format simply gave the users the option 
> to select 
> > their preferred modes, instead of requiring the support of 
> all of those modes.
> 
> > (SDP) part.  I personally believe that a lot of payload 
> formats have 
> > gotten this tradeoff wrong, and the AMR-WB payload format 
> may be one 
> > of those.  But that's an issue that's most easily fixed in future 
> > payload designs (which clearly is in the IETF's realm), and 
> has very 
> > little to do with the codec design itself.
> 
> Not exactly. This CAN be fixed in IETF by defining a cool RTP 
> payload which will add additional in-band signalling, but 
> isn't it nicer to have a self-contained codec, which just 
> produces "frames" and does not need any fancy packaging and 
> additional signaling? Look at Speex as an example of pretty 
> good implementation of this paradigm.
> 
> --
> Regards,
> Alexander Chemeris.
> 
> SIPez LLC.

---

From: Henry Sinnreich <[email-masked]>
Date: Thu, 30 Jul 2009 23:51:31 -0700

I am not so sure about

> 2) Good performance in the presence of packet loss

For two reasons:

 1.  Broadband networks are not plagued by significant packet loss,
 2.  If packet loss occurs, it is in the form of long bursts of packet loss, cutting out words or whole sentences and no codec in the world can correct this.

Henry

On 7/30/09 10:46 PM, "Alexander Chemeris" <[email-masked]> wrote:

Hello,

= The problem =

--
Regards,
Alexander Chemeris.

SIPez LLC.

---

From: Ron <[email-masked]>
Date: Fri, 31 Jul 2009 16:44:13 +0930

>  2.  If packet loss occurs, it is in the form of long bursts of packet loss, cutting out words or whole sentences and no codec in the world can correct this.

At the very low latencies that codec developers are trying achieve now
(which are very important for QOS of VoIP systems), even a small increase
in network delay equates to packet loss.  That can be adapted to, but in
the meantime, you still have packet loss to deal with, even if your
throughput and reliability are generally quite high.

 Ron

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 31 Jul 2009 09:15:30 +0200 (CEST)

On Thu, 30 Jul 2009, Henry Sinnreich wrote:

> 2.  If packet loss occurs, it is in the form of long bursts of packet 
> loss, cutting out words or whole sentences and no codec in the world can 
> correct this.

In the case of wireless networks you have pretty reliable transport but 
you're plagued by sometimes huge intermittent jitter.

So, it's also an interesting distinction whether a packet received after 
the receiver jitter buffer ran out should be classified as a packet loss 
or not. From the codec point of view, it is, as the codec didn't get this 
information.

So, in my mind the codec needs to handle jitter buffer growth and 
reduction by shrinking/dragging out sound to in a graceful manner to adapt 
to the jitter buffer size dynamically changing over time, thus it needs to 
be tightly integrated with transport.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: "Roni Even" <[email-masked]>
Date: Fri, 31 Jul 2009 11:28:51 +0300

Hi,
The issue is not negotiation the codec parameters in band. What you describe
is that the decoder can accept any combination that the encoder can produce.

This is easy when there is no backward compatibility or no specific profiles
based on parameters.
The problem with this solution is that the receiver cannot control the codec
options he wants to receive. The bit rate may be controlled by b= parameter.
Using RTP or RTCP to negotiate send and receive capabilities rae not the
right way
Roni Even

> Of Kevin P. Fleming
> Sent: Friday, July 31, 2009 1:25 AM
> Cc: [email-masked]

---

From: Alexander Chemeris <[email-masked]>
Date: Fri, 31 Jul 2009 13:18:56 +0400

Hi, Ingemar

On Fri, Jul 31, 2009 at 09:46, Ingemar Johansson
S<[email-masked]> wrote:
> Now for a PS to PS call, all that stuff is not needed anymore UDP-Lite may be used though but I have to see evidence of it).

Right, that's what I was talking about. May I was not very clear.

> If you envision only PS to PS you can skip all the above things in your negotiation.

True, but only in the half.
Problem is that you still should offer at least two (!) modes, because
they are not bitstream compatible (!). As per RFC:

         [...], the offerer is RECOMMENDED to also offer a payload
         type containing only the octet-aligned or bandwidth-efficient
         configuration with a single channel

I understand the desire to make seamless integration of mobile and
VoIP networks, but all this adds to implementation complexity and
decrease codec popularity. Internet is all about a lot of different
implementations and the Internet standards should be "as simple as
possible (but no simpler)". Here is my view on two ways how AMR
payload could be done better:

1) Fix octet-aligned mode as the only allowed in RTP. Convert all
bandwidth efficient packets to octet-aligned ones on the border. Keep
in mind, this is not transcoding, just bits moving. Other params as in
(2).
2) Prepend every payload with 4 bits, by a bit for each of
octet-align, crc, robust-sorting and interleaving settings. Then
decoder will be able to know about bitstream format without any
out-of-band means. Given that this is required only once per packet
(not per frame) that's not a big addition to the bandwidth, but will
greatly simplify operation.

This AMR discussion is going to be off-topic. Actually I just wanted
to contrast AMR to Speex as two examples of codecs less and more
suitable for the "ideal VoIP codec" place by the "SDP and
end-programmer friendliness" parameter.

> http://www.wiley-vch.de/publish/dt/books/bySubjectEE00/bySubSubjectEE25/0-470-05855-2/?sID=nqk9adkt7hm033ice0kp7jh3v4

Sorry, I don't have the spec. Could you copy-paste here a wording which
leads to the need of octet-aligned and bandwidth-efficient modes? I was
always wondering about this.

-- 
Regards,
Alexander Chemeris.

SIPez LLC.