Subject: [codec] Codec proposal: draft-valin-celt-codec-00.txt

From: Jean-Marc Valin <[email-masked]>
Date: Sun, 05 Jul 2009 00:13:27 -0400

Hi everyone,

Here's a first codec proposal:
http://www.ietf.org/internet-drafts/draft-valin-celt-codec-00.txt

Cheers,

	Jean-Marc

---

From: Stephan Wenger <[email-masked]>
Date: Sat, 04 Jul 2009 21:54:28 -0700

Hi,
Three things that will hopefully be seen as constructive :-):
First, posting reference code in text format, while quite common in speech
standards, is not that overly helpful IMHO, especially considering the
number of lines of code.  At least during the I-D phase, a reference to an
URL where one can retrieve a stable version of the code, plus a promise to
copy-paste that code into a final RFC if that were the consensus, seems to
suffice.
Second, if you add code into your I-D, it would IMHO make sense if you would
start using the required license---a BSD derivate.  While I personally
appreciate your very broad license, it is not directly compatible with the
BSD license we require for code sections of RFCs.
Third, it would be helpful if you would explicitly state what part of your
I-D actually is the normative specification: source code, or textual
description.  I note that in most ITU and 3GPP speech codec standards, the
code is actually setting the normative standard, and the textual description
is informative.  If I recall correctly, in the IETF's LBC effort the same
holds (I didn't check this---too lazy).  To me, using source code as the
normative standard makes sense---for speech coding only.  If that's what is
intended here, please make sure that it is prominently advertised.
Regards,
Stephan

On 7/4/09 9:13 PM, "Jean-Marc Valin" <[email-masked]> wrote:

> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec

---

From: Jean-Marc Valin <[email-masked]>
Date: Sun, 05 Jul 2009 08:17:18 -0400

> copy-paste that code into a final RFC if that were the consensus, seems to

I agree that the 72-column text format isn't great for source code (took
me a while just to make it fit!). Right now, the source code in the RFC
actually differs from the downloadable code in that it has been
simplified to only support floating point (the main reference code can
switch from float to fixed-point through macros). In any case if the
consensus is to leave it out during drafting, that's what I'll do.

> BSD license we require for code sections of RFCs.

Not sure what you mean here. The license on the files I appended *is*
the BSD license.

> intended here, please make sure that it is prominently advertised.

Actually, I couldn't find such a statement in iLBC (though I may have
missed it). Unless there's a good reason not to do that that I'm unaware
of, I agree that it's safer to make the C code normative.

Cheers,

	Jean-Marc

---

From: Stephan Wenger <[email-masked]>
Date: Sun, 05 Jul 2009 12:11:06 -0700

Hi Jean-Marc,
Re point 2, the license to be used (once the I-D were to become an RFC) can
be found here: http://trustee.ietf.org/docs/IETF-Trust-License-Policy.pdf.
It's a BSD-style license, like yours, but the attribution is different.
That's what I meant, and I'm sorry for having been unclear.
Regards,
Stephan
P.s.: AFAIK, there is no formal requirement to have the correct license and
attribution in an I-D, so there is nothing wrong with it for now.

---

From: Gregory Maxwell <[email-masked]>
Date: Mon, 6 Jul 2009 09:05:11 -0700

Stephan Wenger [[email-masked]] wrote:
[snip]
> copy-paste that code into a final RFC if that were the consensus, seems to
[snip]
> description.  
[snip]

I'm strongly in favour of leaning heavily on the source code as the normative specification: Some parts, specifically any parts involving precise integer arithmetic, are very difficult to describe in english with the necessary precision without creating a difficult to understand convoluted mess.

But at the same time, if the source is the normative specification I think it would be inappropriate to leave it out of the draft even temporarily. 

There is also the issue that not all of the source is expected to have the same degree of mandatory compliance.  A convention for distinguishing these parts will probably need to be adopted. Currently the text denotes operations which must be bit-exact, so perhaps that is sufficient.

---

From: Jean-Marc Valin <[email-masked]>
Date: Mon, 6 Jul 2009 11:28:23 -0400

Hi Henry,

Quoting Henry Sinnreich <[email-masked]>:
> >It is primarily designed for transmission over packet networks
>
> ?? Were we not supposed to work on an Internet voice codec?
> Maybe this intent should be reflected in the text as well?
>
> If other "packet networks" may be suitable, which would they be and why?
> (Does this can of worms need to be opened here?)

Sorry, I was being a bit too vague. Will change to "over the Internet" for the
next version.

Cheers,

   Jean-Marc

> Thanks, Henry
>
>
> On 7/4/09 11:13 PM, "Jean-Marc Valin" <[email-masked]> wrote:
>
> Hi everyone,
>

---

From: Henry Sinnreich <[email-masked]>
Date: Mon, 6 Jul 2009 08:10:29 -0700

Congratulations Jean-Marc for the so very promising start!

Thanks, Henry

Hi everyone,

Cheers,

        Jean-Marc

---

From: Koen Vos <[email-masked]>
Date: Tue, 07 Jul 2009 22:13:04 -0700

Hi all,

We have recently submitted two Internet Drafts  
(https://developer.skype.com/silk?action=AttachFile&do=get&target=draft-vos-silk-00.txt and https://developer.skype.com/silk?action=AttachFile&do=get&target=draft-spittka-silk-payload-format-00.txt). These were submitted manually since we had problems with the web submission and may take a few days to show up on the mailing list. We are proposing SILK as a candidate codec for the Internet Wideband Audio Codec BOF in Stockholm. We've submitted an initial description of the SILK codec as well as a payload format draft. Comments are  
welcome.

best regards,
Koen

---

From: Christian Hoene <[email-masked]>
Date: Wed, 08 Jul 2009 09:03:44 +0200

Hello Koen,

> (https://developer.skype.com/silk?action=AttachFile&do=get&target=draft-vos-silk-00.txt and https://developer.skype.com/silk?action=AttachFile&do=get&target=draft-spittka-silk-payload-format-00.txt…

It is really a nice Internet speech codec design. I especially liked the
controlling of encoding parameters on a per frame basis:

3.1.1.  Control Parameters

   The encoder with control parameters specifying the operating point is
   depicted in Figure 1.  All control parameters can be changed during
   regular operation of the codec, when inputting a frame of audio data,
   without interrupting the audio stream from encoder to decoder.  The
   codec control parameters are described in Section 3.1.1.1 to
   Section 3.1.1.5.

                 Sampling rate ---------------+
                       Bitrate -------------+ |
                   Packet rate -----------+ | |
              Packet loss rate ---------+ | | |
                    Complexity -------+ | | | |
                       Use DTX -----+ | | | | |
                                    | | | | | |
                                    \/\/\/\/\/\/
                                  +-------------+
                  Input signal -->|   Encoder   |--> Bitstream
                                  +-------------+

These is a quite useful feature for VoIP applications. 
I have a question regarding the variable packet rate:

3.1.1.3.  Packet Rate

   SILK encodes frames of 20 milliseconds at a time and can combine 1,
   2, 3, 4 or 5 of these frames in one payload, thus creating one packet
   every 20, 40, 60, 80 or 100 milliseconds.

Why did you decide to implement a variable packet rate by combining multiple frames into one packet? Isn't it is better to define individual frame formats for different packet rates. Then you can quantized the frame parameters more fine grained, don't you?

With best regards,

 Christian

---

From: "Slava Borilin" <[email-masked]>
Date: Wed, 8 Jul 2009 11:18:50 +0400

Putting several frames into one packet is useful for packet grouping,
which in turn is very useful for bandwidth adaptation and network usage
efficiency, I would say?

best regards,
Slava Borilin

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Christian Hoene
Sent: Wednesday, July 08, 2009 11:04 AM
To: Koen Vos
Cc: [email-masked]
Subject: Re: [codec] Codec proposal: draft-vos-silk-00.txt

Hello Koen,

> We have recently submitted two Internet Drafts  
(https://developer.skype.com/silk?action=AttachFile&do=get&target=draft-
vos-silk-00.txt and
https://developer.skype.com/silk?action=AttachFile&do=get&target=draft-s
pittka-silk-payload-format-00.txt). These were submitted manually since
we had problems with the web submission and may take a few days to show
up on the mailing list. We are proposing SILK as a candidate codec for
the Internet Wideband Audio Codec BOF in Stockholm. We've submitted an
initial description of the SILK codec as well as a payload format draft.
Comments are  

3.1.1.3.  Packet Rate

Why did you decide to implement a variable packet rate by combining
multiple frames into one packet? Isn't it is better to define individual
frame formats for different packet rates. Then you can quantized the
frame parameters more fine grained, don't you?

With best regards,

 Christian

---

From: "Christian Hoene" <[email-masked]>
Date: Wed, 8 Jul 2009 09:31:01 +0200

Hi Slava Borilin,

yes, you are right. The question is one whether to make the packet grouping
in RTP or whether to change the encoding.
 Latter should be more efficient because the encoder can what advantage of
it by e.g. reducing all inter-subframe redundancies, better parameter
compression and so on.

I am just wondering, why where is no codec which does the frame
concatenation by itself? Is the gain too small?

With best regards,

 Christian

> From: Slava Borilin [mailto:[email-masked]]
> Sent: Wednesday, July 08, 2009 9:19 AM
> To: Christian Hoene; Koen Vos
> Cc: [email-masked]
> Subject: RE: [codec] Codec proposal: draft-vos-silk-00.txt
>

---

From: Christopher Montgomery <[email-masked]>
Date: Wed, 8 Jul 2009 04:31:40 -0400

> concatenation by itself? Is the gain too small?

In general, if you're going to send multiple frames grouped together
anyway, the gain from having a single large frame is even larger than
grouping multiple small frames post-facto, especially in codecs using
a monolithic transform space.

Monty

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 08 Jul 2009 07:28:56 -0400

> a monolithic transform space.

While this is obviously true of a transform codec like Vorbis where a
larger frame means better frequency resolution (and already less so on
CELT), the gain is much smaller on a time-domain codec like SILK,
especially for > 20 ms frames. Making bigger frames instead of using
multiple smaller frames is usually not worth the extra code size and
memory required, both in the codec and for the client to deal with
changing frame sizes. Also, there's a loss of flexibility because you
can't re-packetize in the middle. Considering that:

"The extent to which SILK exploits inter-frame dependencies can be
adjusted on the fly to choose a trade-off between bitrate and amount of
error propagation."

I think it may even be possible to just make use of more inter-frame
correlation for frames 2..N in a packet of N frames and get most of the
benefits it would get from a larger frame size.

Cheers,

	Jean-Marc

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 08 Jul 2009 07:33:33 -0400

Hi Koen,

Not sure what problem you got, but for me I had to use the "experimental
version" of xml2rfc ( http://xml.resource.org/experimental.html ), along
with ipr="trust200902" in the <rfc> tag before the system would accept
my submission.

Cheers,

	Jean-Marc

Koen Vos a écrit :
> (https://developer.skype.com/silk?action=AttachFile&do=get&target=draft-vos-silk-00.txt
> and
> https://developer.skype.com/silk?action=AttachFile&do=get&target=draft-spittka-silk-payload-format-00.txt).
> These were submitted manually since we had problems with the web
> submission and may take a few days to show up on the mailing list. We
> are proposing SILK as a candidate codec for the Internet Wideband Audio
> Codec BOF in Stockholm. We've submitted an initial description of the
> SILK codec as well as a payload format draft. Comments are welcome.
> 
> best regards,
> Koen

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 08 Jul 2009 07:42:59 -0400

Hi,

I'd just like to point out that there is also an RTP payload format
available for CELT:
http://www.ietf.org/internet-drafts/draft-valin-celt-rtp-profile-01.txt

Cheers,

	Jean-Marc

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 8 Jul 2009 09:17:34 -0400

> Hi all,

I think you forgot to include the list in your original message, so I'm quoting
your whole message for the benefit of others.

> > especially for > 20 ms frames.
>
> Can this gain be quantified? Are their any scientific studies?

There are no scientific studies, because it's too hard to measure (it depends on
the implementation), but here's my experience from Speex (I have no reason to
think it doesn't apply to SILK as well). There are mainly three types of
parameters that CELP and other LP-based codecs transmit (from the biggest to
the smallest): quantised residual (innovation), LSP coefficients, and pitch
parameters. First, the quantised residual usually takes up at least 50% of the
bits and in the case of Speex, it is transmitted based on intervals that are
shorter than 5 ms anyway, so larger frames would have no impact whatsoever. In
the case of SILK, it seems to be entropy coded one sample at a time, so
presumably there would be no impact there either (maybe a negligible gain in
the entropy coder, but probably not worth it). Then there's the LSP parameters.
Those represent the speech spectrum and are usually considered stationary on 20
ms blocks. That's why a lot of speech algorithms (e.g. recognition) work on 20
ms blocks even when there's no coding and latency involved. Again, using 40 ms
blocks, would mean having to code the LSPs twice, which means you don't gain
much. The last is the pitch parameters. Those tend to be coded on 5-10 ms
blocks, so again I don't see a point in having frames larger than 20 ms.

> > Making bigger frames instead of using
> > changing frame sizes.
> Nowadays every device has gigabits of memory. Isn't the argumentation on
> extra code size and memory a bit outdated?

Not every device has gigabits of memory. Many low-power DSPs operate with very
little internal (SRAM) memory (e.g. 64 kB for code and data) and transfers
to/from external memory are expensive.

> > Also, there's a loss of flexibility because you
> > can't re-packetize in the middle.
>
> Sorry, JM, this argument is sooo "circuit-switched" ;-)

Last I checked, circuit-switched networks didn't use packets :-)

> In the Internet we have the end-to-end principle, which requires that the
> transport layer is send transparently between two end nodes
> (http://en.wikipedia.org/wiki/End-to-end_principle). If a codec is designed
> for the Internet, then there will not be any transcoding or reframing of RTP
> packets.

That's assuming you're not trying to be compatible with legacy infrastructure.
PSTN gateways will continue to exist for a while and saving two bits on an
Internet codec is not a valid reason to make their lives more complicated.

Cheers,

    Jean-Marc

> > Considering that:
> Yes, I agree. Setting the inter-frame correlation high for the frame within
> a packet and low for frames between two packets considering, could give a
> benefit.
>
> Cheers,
>
>  Christian
>
>
>
>

---

From: "Christian Hoene" <[email-masked]>
Date: Wed, 8 Jul 2009 16:09:58 +0200

Hi all

> to/from external memory are expensive.

I just want to question whether such low-power DSPs will be used for high
quality audio or superwide-speech transmissions over the Internet. If power
needs to be saved, why not use narrow-band speech transmissions?

Also, you must look at the entire system. If you can encode audio better
with less bits, you save transmission energy.

Do you agree with me that this question cannot be solved on a mailing list
but must be analyzed scientifically?

> That's assuming you're not trying to be compatible with legacy
infrastructure.

I think we have to be precise on whether repacketization in a network is
needed and where. Do you (or anybody else) know which kind of infrastructure
requires repacketization? In TURN servers, MGCP media gateways? I am not
aware of a VoIP gateway that does this and that is not coupled to a PSTN
infrastructure.

Also, how to stay compatible if a new codec is introduced? You have to run
an update anyhow that will include the gateway, too.

> PSTN gateways will continue to exist for a while 

Neither SBC, CELT nor SILK will be ever used in a PSTN network. Thus, a PSTN
gateway has to transcode the speech stream anyhow. Repacketization does not
matter then.

My argumentation is the same as with open-source codecs: We are in a
disruptive technology change. Some of the rules that used to be valid need
to be questioned. To remain compatible with legacy infrastructure
(especially PSTN circuit-switched based) should not be an important
requirement for a codec proposed in an Internet standardization body. That's
is the task of ITU or 3GPP.

Prost,

 Christian

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 08 Jul 2009 10:30:23 -0400

> needs to be saved, why not use narrow-band speech transmissions?

Internet devices are no longer just PCs. There's more and more low-power 
portable devices equipped with DSPs. It would be a mistake to restrict 
these devices to narrowband just so you can save a few bits by increasing 
the code size. In the end, it's always a matter of trade-off and seeing 
what you can gain from an increase in code size or complexity. From what 
I've read about SILK so far (we'll see when there's more information and 
source code), I see no reason to question the trade-off that were made.

> but must be analyzed scientifically?

I don't agree. These kinds of questions are related to the requirements and 
should eventually (now or later) be discussed on the list. These are not 
that much scientific issues, but issues about what we are trying to 
achieve, i.e. what is useful.

Cheers,

	Jean-Marc