Subject: [codec] Architectural Considerations

From: stephen botzko <[email-masked]>
Date: Fri, 31 Jul 2009 10:43:50 -0400

Now that Mikael and I have found we were in violent agreement, I would like
to move on (retitling the thread).

The basic architecture for SIP and RTP is well established, and I thought
was well accepted.  However, there are several comments on this list that
make me wonder.

Today, adaptation to network conditions is a shared responsibility, with
some requirements on the codec layer, others on the transport layer, and
still others on the application.  Several comments (mostly criticism) of the
ITU-T codecs indicate that many of the posters here have a different view of
how this architecture is supposed to work.  Many comments are about
inadequate response to jitter for instance are not relevant to the codec.
They *are* relevant to the RTP transport and to the application's
implementation choices, but not to the codec itself.

I'd like go back to the G.719 example (of course I like that codec, but the
main reason is that it is representative of current work, and allows me to
make my architectural point).

(1) The codec itself (defined in the ITU-T) provides a range of output
bitrates that is useful over a pretty wide range of link capacities.  The
state carried over between audio frames is limited, and self-synchronizing.
So the decoder automatically re-syncs to the encoder state after packet
loss. Frame erasure concealment (e.g. packet loss concealment) is built into
the algorithm.

(2) The packetization RFC (defined in the IETF), provides the facilities to
handle jitter and congestion control.  It allows multiple audio frames to be
placed in a packet.  These frames can be interleaved or redundant.  The
codec bitrate can be changed on a frame by frame basis without signaling.
SDP signaling assures that the relevant facilities for the application can
be negotiated.  For instance, interleaved transmission is generally not
appropriate for conversational applications, so the signaling allows its use
to be negotiated.  Also, the variable bitrate features of the algorithm
might not be possible through some gateways, so it is possible to negotiate
a CBR mode of operation.  In addition, general RTP facilities (such as
forward error correction, etc) are  available in codec-independent fashion.
RTCP provides feedback to the sender on the quality of the transmission.

(3) Although the IETF provides the facilities to handle jitter and
congestion control, the application is responsible to use them. For
instance, it is the application's job to provide a suitable depth for the
dejittering/packet reordering queue, and to manage any delay that the queue
creates.  Additional packet loss concealment (beyond what is built in) can
also be implemented as a post-process.

On the whole, I find this architecture to be well-reasoned.  The main issue
with it from a quality point of view is that there is perhaps not enough
guidance for the application designer to handle (3).

Let me switch to G.711/G.722, and look again at the three layers.  The story
is quite consistent, though comfort noise is probably done on the wrong
level.

(1) Although these codecs predate the internet, they are capable of running
at three different bitrates (48, 56, 64 kbits).  This is not as broad a
range as one would like, but still it is something.  They are both
sample-based codecs, so they have quite low algorithmic delay.
Sample-to-sample state is non-existent with G.711, and is quickly
self-synchronized with G.722.  Specific Packet loss procedures are
recommended by the ITU, though they are not mandated (since they do not
apply for all applications, and there are billions of old implementations
that predate the internet).

(2) The packetization RFC for both these codecs allows for basically any
number of samples to be placed in an RTP packet.  The reduced bitrates of 48
and 56 kbs can be used, but the *RFC* requires them to be padded to 64 kbs,
so there is no benefit.  General RTP facilities, including forward error
concealment *and* audio redundancy encoding (RFC 2198) can be used.  Average
bit rate of the codecs can be reduced using the comfort noise payload
specified in RFC 3389.  RTCP provides feedback to the sender on the quality
of the transmission.

(3) Application designers have the same responsibility here as they do with
G.719.

Looking at what is missing here from a system design perspective, I see a
couple of things.

(a) There is no clear guidance on how to measure congestion, and no
reference model on how to respond.  The underlying tools are present, but
there is no guidance on how to use them effectively or interoperably.  In my
view, this is mostly a transport/application issue, and not properly viewed
as a codec issue.

(b) Although the benefits of adaptively sized dejitter buffering, etc are
well understood, they are often not implemented in real applications.
Likewise RTCP support is commonly omitted in many applications, so there is
no feedback to the source on packet loss, and often the application makes no
attempt to adapt to it. The packet loss concealment method that is already
specified is not always used.

Standardization (preferably in the IETF) is a reasonable way to deal with
(a).  Most of that work is clearly outside the scope of a codec group.  I
don't see how standardization can help with (b).

My main argument here is that the codec group should be focused on the first
layer of the existing architecture, and not expand the work into the
transport and application.  Some of the posts/discussion comments seem to
imply otherwise -  either by explicitly indicating that transport work
should be included, or by implicitly by raising deficiencies of existing
implementations that are not due to the codec at all, but are properly fixed
in the transport or in the applicaton implementation.

Perhaps I am wrong, but I do think some clarity on this subject might be
helpful in working out the charter.

Regards,
Stephen Botzko
Polycom

On Fri, Jul 31, 2009 at 7:22 AM, Mikael Abrahamsson <[email-masked]>wrote:

> On Fri, 31 Jul 2009, stephen botzko wrote:
>
>  Ok.  You ended up agreeing with what I said in my first post. I pointed
>> out
>> the redundant frames/interleaved frame transmission in the G.719 rfc, but
>> said you should also switch to a lower bandwidth mode if the network were
>> congested.
>>
>
> Absolutely, I never meant to advocate *increasing* bw by means of
> maintaining codec bw but transmitting more redundant information, thus
> increasing the overall IP bits/s used.
>
> --
> Mikael Abrahamsson    email: [email-masked]
> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec
>

---

From: "Slava Borilin" <[email-masked]>
Date: Fri, 31 Jul 2009 18:55:14 +0400

Stephen, I tend to agree with you - WG should focus on codec (as this is
interop requirement), and leave the rest to transport and not even
necessarily standardize the rest, but leave as "open air" for people to
implement differently.

best regards,

Slava Borilin

________________________________

From: [email-masked] [mailto:[email-masked]] On Behalf
Of stephen botzko
Sent: Friday, July 31, 2009 6:44 PM
To: [email-masked]
Subject: [codec] Architectural Considerations

Now that Mikael and I have found we were in violent agreement, I would
like to move on (retitling the thread).

The basic architecture for SIP and RTP is well established, and I
thought was well accepted.  However, there are several comments on this
list that make me wonder.  

Today, adaptation to network conditions is a shared responsibility, with
some requirements on the codec layer, others on the transport layer, and
still others on the application.  Several comments (mostly criticism) of
the ITU-T codecs indicate that many of the posters here have a different
view of how this architecture is supposed to work.  Many comments are
about inadequate response to jitter for instance are not relevant to the
codec.  They are relevant to the RTP transport and to the application's
implementation choices, but not to the codec itself.

I'd like go back to the G.719 example (of course I like that codec, but
the main reason is that it is representative of current work, and allows
me to make my architectural point).

(1) The codec itself (defined in the ITU-T) provides a range of output
bitrates that is useful over a pretty wide range of link capacities.
The state carried over between audio frames is limited, and
self-synchronizing.  So the decoder automatically re-syncs to the
encoder state after packet loss. Frame erasure concealment (e.g. packet
loss concealment) is built into the algorithm. 

(2) The packetization RFC (defined in the IETF), provides the facilities
to handle jitter and congestion control.  It allows multiple audio
frames to be placed in a packet.  These frames can be interleaved or
redundant.  The codec bitrate can be changed on a frame by frame basis
without signaling.  SDP signaling assures that the relevant facilities
for the application can be negotiated.  For instance, interleaved
transmission is generally not appropriate for conversational
applications, so the signaling allows its use to be negotiated.  Also,
the variable bitrate features of the algorithm might not be possible
through some gateways, so it is possible to negotiate a CBR mode of
operation.  In addition, general RTP facilities (such as forward error
correction, etc) are  available in codec-independent fashion.  RTCP
provides feedback to the sender on the quality of the transmission.

(3) Although the IETF provides the facilities to handle jitter and
congestion control, the application is responsible to use them. For
instance, it is the application's job to provide a suitable depth for
the dejittering/packet reordering queue, and to manage any delay that
the queue creates.  Additional packet loss concealment (beyond what is
built in) can also be implemented as a post-process.

On the whole, I find this architecture to be well-reasoned.  The main
issue with it from a quality point of view is that there is perhaps not
enough guidance for the application designer to handle (3).

Let me switch to G.711/G.722, and look again at the three layers.  The
story is quite consistent, though comfort noise is probably done on the
wrong level.

(1) Although these codecs predate the internet, they are capable of
running at three different bitrates (48, 56, 64 kbits).  This is not as
broad a range as one would like, but still it is something.  They are
both sample-based codecs, so they have quite low algorithmic delay.
Sample-to-sample state is non-existent with G.711, and is quickly
self-synchronized with G.722.  Specific Packet loss procedures are
recommended by the ITU, though they are not mandated (since they do not
apply for all applications, and there are billions of old
implementations that predate the internet).

(2) The packetization RFC for both these codecs allows for basically any
number of samples to be placed in an RTP packet.  The reduced bitrates
of 48 and 56 kbs can be used, but the RFC requires them to be padded to
64 kbs, so there is no benefit.  General RTP facilities, including
forward error concealment and audio redundancy encoding (RFC 2198) can
be used.  Average bit rate of the codecs can be reduced using the
comfort noise payload specified in RFC 3389.  RTCP provides feedback to
the sender on the quality of the transmission.

(3) Application designers have the same responsibility here as they do
with G.719.

Looking at what is missing here from a system design perspective, I see
a couple of things.

(a) There is no clear guidance on how to measure congestion, and no
reference model on how to respond.  The underlying tools are present,
but there is no guidance on how to use them effectively or
interoperably.  In my view, this is mostly a transport/application
issue, and not properly viewed as a codec issue.

(b) Although the benefits of adaptively sized dejitter buffering, etc
are well understood, they are often not implemented in real
applications.  Likewise RTCP support is commonly omitted in many
applications, so there is no feedback to the source on packet loss, and
often the application makes no attempt to adapt to it. The packet loss
concealment method that is already specified is not always used.

Standardization (preferably in the IETF) is a reasonable way to deal
with (a).  Most of that work is clearly outside the scope of a codec
group.  I don't see how standardization can help with (b).  

My main argument here is that the codec group should be focused on the
first layer of the existing architecture, and not expand the work into
the transport and application.  Some of the posts/discussion comments
seem to imply otherwise -  either by explicitly indicating that
transport work should be included, or by implicitly by raising
deficiencies of existing implementations that are not due to the codec
at all, but are properly fixed in the transport or in the applicaton
implementation.

Regards,
Stephen Botzko
Polycom 

On Fri, Jul 31, 2009 at 7:22 AM, Mikael Abrahamsson <[email-masked]>
wrote:

	Ok.  You ended up agreeing with what I said in my first post. I
pointed out
	the redundant frames/interleaved frame transmission in the G.719
rfc, but
	said you should also switch to a lower bandwidth mode if the
network were
	congested.