Subject: Re: [codec] Comments on draft-vos-silk-00.txt

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 16 Jul 2009 07:41:43 -0400

Hello,

Koen and I have been having a technical discussion in the past few days
before we realised that it would be best to do that publicly on the
list. So we're moving the thread to the list and I'm attaching the
discussion we had in private (with only very minor edits).

Cheers,

	Jean-Marc
Date: Tue, 07 Jul 2009 22:33:15 -0400
From: Jean-Marc Valin <[email-masked]>
To: Koen Vos <[email-masked]>, Jason Fischl <[email-masked]>
Subject: Comments on draft-vos-silk-00.txt

Hi Koen, Jason,

I've started reading draft-vos-silk-00.txt and had a few comments. I'm
not posting on the list, since I'm guessing you didn't want to
"officially" announce the draft yet. In no particular order:

1) I think section 2 is a bit strange as it seems to list the
requirements that lead to the proposed codec, rather than requirements
for compliance with the draft standard. Maybe it should be labeled as
"design goals" or something like that, with the MUSTs and SHOULDs removed.

2) Some of the requirements may be useful in another section that is
really about compliance. For example, "The codec MUST be capable of
running with an algorithmic delay of no more than 30 milliseconds" could
be there to say that any implementation of SILK cannot add more than 10
ms lookahead (on top of the 20 ms frame size).

3) It would be nice to have more details on the "Noise Shaping
Quantizer", though I'm guessing that's mostly a matter of lack of time
so far. I see a mention of "pyramid range encoder", but it's not clear
where the pyramid comes from (any relationship with the pyramid vector
quantizer we're using in CELT)?

4) Not sure about that one, but a "comparison with CELP" may be useful
to better understand the algorithm.

5) Are you standardising only the decoder (MPEG-style, where the
reference encoder is merely a suggestion) or are you also standardising
the encoder (ITU-style, minus bit-exactness)?

I haven't read through all the details yet, so I may have more comments
later. If you have any comments on the CELT draft, I'd be glad to hear
them so I can address then before the final cut-off.

Cheers,

	Jean-Marc

------------------------------------------------------------

Date: Wed, 08 Jul 2009 14:01:13 -0700
From: Koen Vos <[email-masked]>
To: Jean-Marc Valin <[email-masked]>
Cc: Jason Fischl <[email-masked]>
Subject: Re: Comments on draft-vos-silk-00.txt

Hi Jean-Marc,

Thanks for the very useful feedback.

The requirements should indeed be replaced by design goals later on.  
For now these are also meant as a discussion point in Stockholm about  
what the codec(s) should provide.

We'll fill in the details about noise shaping quantizer. FYI: the  
noise shaping quantizer generates quantization indices, but without  
the constraints for "tracks" and total number of pulses found in a  
CELP codec. This makes it possible to process the signal  
sample-by-sample, which is simpler than CELP's search.  The  
quantization indices are entropy coded with a pyramid quantization  
scheme that is indeed similar to the one in CELT.

We haven't made up our mind about what we want to standardize. Besides  
the two options you mention, even the full ITU approach including  
bit-exactness is a candidate as it guarantees quality (at the cost of  
innovation opportunity).

I'll get back to you with feedback on the CELT draft shortly.

best,
koen.

----------------------------------------------------------------

Date: Wed, 08 Jul 2009 19:29:43 -0400
From: Jean-Marc Valin <[email-masked]>
To: Koen Vos <[email-masked]>
CC: Jason Fischl <[email-masked]>
Subject: Re: Comments on draft-vos-silk-00.txt

Hi Koen,

Koen Vos a écrit :
> We'll fill in the details about noise shaping quantizer. FYI: the noise
> shaping quantizer generates quantization indices, but without the
> constraints for "tracks" and total number of pulses found in a CELP
> codec. This makes it possible to process the signal sample-by-sample,
> which is simpler than CELP's search. 

Do you have a higher-complexity option where the scalar quantization
takes into account the "N-best solutions"? This is something I've done
with the split-VQ innovation codebook in Speex (only at complexity 3 and
above).

> The quantization indices are
> entropy coded with a pyramid quantization scheme that is indeed similar
> to the one in CELT.

Do you mean that you entropy-code the total number of pulses and then
you convert the PVQ value to a single integer?

Considering that both CELT and SILK use a range coder and assuming the
pyramid quantization in SILK is similar enough to what CELT does, would
it make sense to specifically attempt to make SILK and CELT share common
code? That way a DSP implementation could support both codecs without
having to completely duplicate the code size (and the porting effort).
Not sure it's practical, but I think it's at least worth checking
whether there's enough practical similarities.

> We haven't made up our mind about what we want to standardize. Besides
> the two options you mention, even the full ITU approach including
> bit-exactness is a candidate as it guarantees quality (at the cost of
> innovation opportunity).

I think it may be useful to have a "if you're bit-exact with this you
know you're OK" type of reference, but it would probably would not be a
good idea to force everyone to be bit-exact with that. That would not
only favour some DSP manufacturers over others (depending on how you
implement the basic operators), but it would also prevent float
implementations, which can be faster on some machines. For example, the
float version of Speex can be almost double the speed of the fixed-point
version on a PC. That can be important for applications like PSTN gateways.

> I'll get back to you with feedback on the CELT draft shortly.

I look forward to having your feedback.

Cheers,

	Jean-Marc

-------------------------------------------------------------------

Date: Mon, 13 Jul 2009 04:03:06 -0700
From: Koen Vos <[email-masked]>
To: Jean-Marc Valin <[email-masked]>
Cc: Jason Fischl <[email-masked]>
Subject: Re: Comments on draft-vos-silk-00.txt

Hi Jean-Marc,

Sorry for the slow response - here are some answers inline, followed  
by feedback on the CELT draft.

> Do you have a higher-complexity option where the scalar quantization
> takes into account the "N-best solutions"? This is something I've done
> with the split-VQ innovation codebook in Speex (only at complexity 3 and
> above).

Yes, we use trellis coding with 2 and 4 states/best solutions in the  
medium and high complexity settings. It helps quite a bit, saving  
several kbps.

> Considering that both CELT and SILK use a range coder and assuming the
> pyramid quantization in SILK is similar enough to what CELT does, would
> it make sense to specifically attempt to make SILK and CELT share common
> code? That way a DSP implementation could support both codecs without
> having to completely duplicate the code size (and the porting effort).
> Not sure it's practical, but I think it's at least worth checking
> whether there's enough practical similarities.

I like the idea of sharing part of the code base.  As discussed, the  
PVQ in SILK is somewhat different from the one in CELT, and uses  
trained probabilities to describe how a given number of pulses in a  
16-sample block is distributed. We could try CELT's PVQ but it might  
slightly increase the bitrate.  The range encoder could be shared  
(although it would break the bitstream format for one codec). In any  
case, there should be many low-level functions that can be shared.

Feedback on the CELT draft:

We already discussed transients over the phone a bit.
In the description of the time-domain windowing, reference is made to  
a "transient time"; it was not immediately clear to me how this time  
is determined: is it fixed on the transition between first and second  
MDCT window?
Have you considered using shorter transitions for the intra-frame  
overlaps in "short blocks" mode?  I wonder if that could accomplish  
the same as the time-domain windowing (so that you wouldn't need that  
mode anymore).

The coarse energies are predicted based on energies in the previous  
frame and lower bands. Are the coarse or fine energies from the  
previous frame used as a basis for prediction?  In theory it could be  
more efficient to predict and encode fine energies directly, but that  
would result in a variable number of bits spend for the energy  
encoding (which might be acceptable, don't know). Alternatively, to  
reduce such bit rate variation, the energies could be encoding one  
band at a time, while varying quantization accuracy depending on how  
many bits were spent so far.
It also feels like the energy quantization coarseness for one band  
could be made dependent on its energy relative to the preceding band:  
if the band below had a lot more energy, than the current band's  
energy can be quantized more coarsely. Companding could achieve that,  
for example.

How well does the pitch prediction work?  Does it remove most of the  
energy in the pitch bands for voiced speech?  Is the pitch lag  
estimation accurate when the pitch lag is not a multiple number of  
frames?  I wonder how the frequency domain correlation can be accurate  
in that case, because with an MDCT, a shift in time does not  
correspond to a phase rotation like with a Fourier transform.   
Probably I should just take a look at the code to understand better  
what goes on..

Anyway, it's nice to see a codec with lots of original ideas that make  
perfect sense.

best,
koen.

-----------------------------------------------------------------------

Date: Mon, 13 Jul 2009 07:43:05 -0400
From: Jean-Marc Valin <[email-masked]>
To: Koen Vos <[email-masked]>
CC: Jason Fischl <[email-masked]>
Subject: Re: Comments on draft-vos-silk-00.txt

Hi Koen,

Thanks very much for the review. Comments below.

Koen Vos a écrit :
> We already discussed transients over the phone a bit.
> In the description of the time-domain windowing, reference is made to a
> "transient time"; it was not immediately clear to me how this time is
> determined: is it fixed on the transition between first and second MDCT
> window?

I guess I should make this clearer. The transient time is determined by
the encoder and transmitted in the bit-stream. It's the exact time (in
samples) where the transient begins.

> Have you considered using shorter transitions for the intra-frame
> overlaps in "short blocks" mode?  I wonder if that could accomplish the
> same as the time-domain windowing (so that you wouldn't need that mode
> anymore).

The time-domain windowing (I guess I could call it a time-domain
pre-emphasis because it's compensated at the decoder) is more precise in
time than just short blocks, but it's true that it may be possible to
reduce the overlap inside the frame. I'll have a look at this.

> The coarse energies are predicted based on energies in the previous
> frame and lower bands. Are the coarse or fine energies from the previous
> frame used as a basis for prediction?

The prediction is based on the fine quantisation of the previous frame.
Will try and make that clearer.

>  In theory it could be more
> efficient to predict and encode fine energies directly, but that would
> result in a variable number of bits spend for the energy encoding (which
> might be acceptable, don't know). 

The 6 dB coarse energy quantisation was selected because that's roughly
where prediction stops being useful. At this point, the fine energy bits
are mostly random. Also, because the quantisation step is fixed, the
probability model is easier to deal with (one set of probabilities per
band). Another technical issue is that for bands where fine energy can
have a resolution of 0.05 dB, there would be too many symbols to encode
in one step anyway.

> Alternatively, to reduce such bit rate
> variation, the energies could be encoding one band at a time, while
> varying quantization accuracy depending on how many bits were spent so far.
> It also feels like the energy quantization coarseness for one band could
> be made dependent on its energy relative to the preceding band: if the
> band below had a lot more energy, than the current band's energy can be
> quantized more coarsely. 

I'm still considering adjusting the quantisation level of fine and PVQ
with the input spectrum (i.e. dynamic allocation), but haven't yet
figured out a way to do that that sounds better than the current code.

> Companding could achieve that, for example.

Not sure I understand what you mean here.

> How well does the pitch prediction work?  Does it remove most of the
> energy in the pitch bands for voiced speech? 

So far, pitch only seems to be useful for lower bit-rate (<= 48 kbit/s)
speech. It not that it really hurts otherwise, but the improvement it
makes is usually just enough to cover the bits it adds.

> Is the pitch lag
> estimation accurate when the pitch lag is not a multiple number of
> frames?  I wonder how the frequency domain correlation can be accurate
> in that case, because with an MDCT, a shift in time does not correspond
> to a phase rotation like with a Fourier transform.  Probably I should
> just take a look at the code to understand better what goes on..

Oh, the pitch period is estimated using a time-domain weighted
cross-correlation (implemented using an FFT), so the lag can be precise.

> Anyway, it's nice to see a codec with lots of original ideas that make
> perfect sense.

Thanks!

	Jean-Marc

----------------------------------------------------------------------

Date: Mon, 13 Jul 2009 21:07:48 -0700
From: Koen Vos <[email-masked]>
To: Jean-Marc Valin <[email-masked]>
Cc: Jason Fischl <[email-masked]>
Subject: Re: Comments on draft-vos-silk-00.txt

Hi Jean-Marc,

> I'm still considering adjusting the quantisation level of fine and PVQ
> with the input spectrum (i.e. dynamic allocation), but haven't yet
> figured out a way to do that that sounds better than the current code.

I guess a simple way is to operate on the differences between the band  
log energies (subtract, from each band, the log energy of the band  
below).  Then apply non-uniform quantization (or companding + uniform  
quantization).  The non-uniform quantizer would have less accuracy at  
lower levels, where there is more masking from the band below.  The  
resulting indices can be predictively entropy coded, where the  
probability is equal to the integral of the Laplacian over the  
quantization interval. The mean of the Laplacian is set equal to the  
predicted value, and the spread/variance comes from a predefined table.

> Another technical issue is that for bands where fine energy can
> have a resolution of 0.05 dB, there would be too many symbols to encode
> in one step anyway.

Whether the entropy coding is done in one step or in multiple  
refinements could be local to the entropy coder. Although 0.05 dB  
quantization steps means that the quantization noise sits more than 40  
dB below the signal. Is quantizing more coarsely audibly worse?

> So far, pitch only seems to be useful for lower bit-rate (<= 48 kbit/s)
> speech. It not that it really hurts otherwise, but the improvement it
> makes is usually just enough to cover the bits it adds.

It could be interesting to establish an upper bound of what could be  
achieved with the "perfect" pitch lag estimator: do a brute-force  
search over all possibly pitch lags by computing the windowed MDCT for  
each time lag and computing the energy reduction with that lag.   
Although unacceptably complex, it would be relatively easy to  
implement and gives a feel of how much potential there is from  
improving pitch lag estimates.

best,
koen.

----------------------------------------------------------------------

Date: Tue, 14 Jul 2009 00:37:02 -0400
From: Jean-Marc Valin <[email-masked]>
To: Koen Vos <[email-masked]>
CC: Jason Fischl <[email-masked]>
Subject: Re: Comments on draft-vos-silk-00.txt

Hi Koen,

Koen Vos a écrit :
>> I'm still considering adjusting the quantisation level of fine and PVQ
>> with the input spectrum (i.e. dynamic allocation), but haven't yet
>> figured out a way to do that that sounds better than the current code.
> 
> I guess a simple way is to operate on the differences between the band
> log energies (subtract, from each band, the log energy of the band
> below). 

That's similar to what the current quantizer already does if you disable
inter-frame prediction.

> Then apply non-uniform quantization (or companding + uniform
> quantization).  The non-uniform quantizer would have less accuracy at
> lower levels, where there is more masking from the band below. 

I see. I tried something a bit similar (adding a linear offset before
the log and removing at the decoder) in the past without much
improvement. Could be worth revisiting that though.

> The
> resulting indices can be predictively entropy coded, where the
> probability is equal to the integral of the Laplacian over the
> quantization interval. The mean of the Laplacian is set equal to the
> predicted value, and the spread/variance comes from a predefined table.

You mean adjusting the Laplacian probabilities based on the companding?
With the current code, that would require too much computations to be
worth it. I actually tried all kinds of fancy things for coarse energy
prediction/entropy coding and it's really hard to beat the (simple)
current code by more than about one bit.

>> Another technical issue is that for bands where fine energy can
>> have a resolution of 0.05 dB, there would be too many symbols to encode
>> in one step anyway.
> 
> Whether the entropy coding is done in one step or in multiple
> refinements could be local to the entropy coder. Although 0.05 dB
> quantization steps means that the quantization noise sits more than 40
> dB below the signal. Is quantizing more coarsely audibly worse?

The number of fine energy bits is actually computed in the bit
allocation process. If the allocator finds that it has B bits for a
certain band of N bins, then fine energy will get B/N bits and PVQ will
get (N-1)*B/N (in practice, there also an offset, but it's not important
here). So at *very* high bit-rates, CELT can give as much as 7 bits per
sample for some bands, which means the fine energy for those bands can
be 7 bits, so 0.05 dB resolution. In practice, it tends to be coarser
than that though (at very low rates, some bands will not have fine
energy bits at all).

>> So far, pitch only seems to be useful for lower bit-rate (<= 48 kbit/s)
>> speech. It not that it really hurts otherwise, but the improvement it
>> makes is usually just enough to cover the bits it adds.
> 
> It could be interesting to establish an upper bound of what could be
> achieved with the "perfect" pitch lag estimator: do a brute-force search
> over all possibly pitch lags by computing the windowed MDCT for each
> time lag and computing the energy reduction with that lag.  Although
> unacceptably complex, it would be relatively easy to implement and gives
> a feel of how much potential there is from improving pitch lag estimates.

I think the main problem with the pitch does not come from the lag
estimation, but the way the pitch gain is quantised and used. That's
because if the pitch gain is very large, then the PVQ codebook gain
(computed to ensure that the final vector has unit norm) will vary a lot
with the actual codeword selected. Consider the geometric interpretation
if you have a pitch gain of 0.9. The codebook that results from
pitch+PVQ obviously has a higher resolution in the direction of the
pitch vector. However, because the "residual" (after removing pitch)
tends to be mostly orthogonal to the pitch vector, the net result is
that the resolution for the value we're actually quantising isn't
improved by much.

In the past, I've tried to use some kind of companding to the PVQ
codebook to increase the resolution in directions that are orthogonal to
the pitch vector, but it was quite expensive, reduced loss robustness
and gave little improvements overall.

Cheers,

	Jean-Marc

-----------------------------------------------------------------------

Date: Mon, 13 Jul 2009 22:11:09 -0700
From: Koen Vos <[email-masked]>
To: Jean-Marc Valin <[email-masked]>
Cc: Jason Fischl <[email-masked]>
Subject: Re: Comments on draft-vos-silk-00.txt

Hi Jean-Marc,

> You mean adjusting the Laplacian probabilities based on the companding?

Yes, that would be one implementation of it.

> I actually tried all kinds of fancy things for coarse energy
> prediction/entropy coding and it's really hard to beat the (simple)
> current code by more than about one bit.

Better keep it simple then :)

> I think the main problem with the pitch does not come from the lag
> estimation, but the way the pitch gain is quantised and used. That's
> because if the pitch gain is very large, then the PVQ codebook gain
> (computed to ensure that the final vector has unit norm) will vary a lot
> with the actual codeword selected. Consider the geometric interpretation
> if you have a pitch gain of 0.9. The codebook that results from
> pitch+PVQ obviously has a higher resolution in the direction of the
> pitch vector. However, because the "residual" (after removing pitch)
> tends to be mostly orthogonal to the pitch vector, the net result is
> that the resolution for the value we're actually quantising isn't
> improved by much.

The variations of the PVQ codebook gain, could they be avoided by  
orhogonalizing the PVQ contribution relative to the pitch vector?   
Given the pitch gain, you would then automatically know the PVQ gain:
   gain_PVQ = sqrt( 1 - gain_pitch^2 );
(assuming the PVQ part in normalized in energy after the orthogonalization).

best,
koen.

------------------------------------------------------------------------

Date: Tue, 14 Jul 2009 07:17:04 -0400
From: Jean-Marc Valin <[email-masked]>
User-Agent: Thunderbird 2.0.0.22 (X11/20090608)
MIME-Version: 1.0
To: Koen Vos <[email-masked]>
CC: Jason Fischl <[email-masked]>
Subject: Re: Comments on draft-vos-silk-00.txt

Koen Vos a écrit :
>> I think the main problem with the pitch does not come from the lag
>> estimation, but the way the pitch gain is quantised and used. That's
>> because if the pitch gain is very large, then the PVQ codebook gain
>> (computed to ensure that the final vector has unit norm) will vary a lot
>> with the actual codeword selected. Consider the geometric interpretation
>> if you have a pitch gain of 0.9. The codebook that results from
>> pitch+PVQ obviously has a higher resolution in the direction of the
>> pitch vector. However, because the "residual" (after removing pitch)
>> tends to be mostly orthogonal to the pitch vector, the net result is
>> that the resolution for the value we're actually quantising isn't
>> improved by much.
> 
> The variations of the PVQ codebook gain, could they be avoided by
> orhogonalizing the PVQ contribution relative to the pitch vector?  Given
> the pitch gain, you would then automatically know the PVQ gain:
>   gain_PVQ = sqrt( 1 - gain_pitch^2 );
> (assuming the PVQ part in normalized in energy after the
> orthogonalization).

Yes, that was the original thing we thought about (before the
companding), but we ended up with two major problems:
1) We couldn't find a way to compute the orthogonalization efficiently
(O(N^3) is not an option);
2) If you lose a packet, there is basically no hope of resynchronising.

	Jean-Marc

Date: Tue, 14 Jul 2009 22:50:04 -0700
From: Koen Vos <[email-masked]>
To: Jean-Marc Valin <[email-masked]>
Cc: Jason Fischl <[email-masked]>
Subject: Re: Comments on draft-vos-silk-00.txt

>> The variations of the PVQ codebook gain, could they be avoided by
>> orhogonalizing the PVQ contribution relative to the pitch vector?  Given
>> the pitch gain, you would then automatically know the PVQ gain:
>>   gain_PVQ = sqrt( 1 - gain_pitch^2 );
>> (assuming the PVQ part in normalized in energy after the
>> orthogonalization).
>
> Yes, that was the original thing we thought about (before the
> companding), but we ended up with two major problems:
> 1) We couldn't find a way to compute the orthogonalization efficiently
> (O(N^3) is not an option);

Should be O(N) I think:

PVQ_orth = PVQ - Pitch * (Pitch' * PVQ);           (Eq. 1)

Where Pitch and PVQ are like the adaptive and fixed codebook vectors.  
Under normal operation, Pitch is normalized in energy (but it may have  
less energy after packet loss - see below).  The decoder creates the  
quantized signal as:

S_quant = Pitch * pitch_gain +
           PVQ_orth * sqrt( (1 - pitch_gain ^ 2) / (PVQ_orth' * PVQ_orth) );

> 2) If you lose a packet, there is basically no hope of resynchronising.

Maybe that can be avoided.  After a packet loss, and assuming no  
normalization, the Pitch vector has zero energy so S_quant is equal to  
the scaled PVQ.  With each decoded packet Pitch gradually increases  
until it reaches unit energy.  As a result the orthogonalization in  
the decoder (as described in Equation 1) is only a partial  
orthogonalization in that the result is not completely orthogonal to  
Pitch as long as Pitch has energy below one.  In other words PVQ_orth  
will be more similar to PVQ while the Pitch vector is building up.
(You could of course still choose to normalize the S_quant signal for  
creating the output signal, as long the signal going into the pitch  
buffer is based on the unnormalized S_quant.  This increases  
complexity though.)

There's also a way to balance pitch coding gain and speed of error  
recovery:  When creating the pitch residual signal as input to the  
PVQ, using a pitch gain that is lower than the ideal pitch gain (=  
Pitch' * S) will typically lead to a PVQ vector that is less  
orthogonal to Pitch.  So it contains part innovation but also part a  
description of Pitch itself.  The pitch gain sent to the decoder  
should still be the (quantized) ideal pitch gain.  Without packet  
loss, the orthogonalization removes the part of PVQ that's aligned  
with Pitch, and the S_quant signal is created as usual.  Following a  
packet loss, however, there is no orthogonalization and the full PVQ  
is used which already looks more like the original signal.  And  
subsequent packets, with partial orthogonalization, will converge  
faster to the signal that would have existed without packet loss.

Don't know if this makes any sense...

best,
koen.

-------------------------------------------------------------------------

Date: Wed, 15 Jul 2009 07:31:10 -0400
From: Jean-Marc Valin <[email-masked]>
To: Koen Vos <[email-masked]>
CC: Jason Fischl <[email-masked]>
Subject: Re: Comments on draft-vos-silk-00.txt

Hi Koen,

Koen Vos a écrit :
>> Yes, that was the original thing we thought about (before the
>> companding), but we ended up with two major problems:
>> 1) We couldn't find a way to compute the orthogonalization efficiently
>> (O(N^3) is not an option);
> 
> Should be O(N) I think:
> 
> PVQ_orth = PVQ - Pitch * (Pitch' * PVQ);           (Eq. 1)

This is is exactly (well, we a coefficient before the second term) what
I meant by the companding and that's what CELT was doing until around
0.3.0. The complexity wasn't too bad (it doubled the search complexity
or so), but it was causing resynchronisation problems. For the case
where PVQ is forced to be completely orthogonal, there were even
numerical stability problems. Another issue now is that going back to
the companding idea would prevent one of the search optimisations
(projection on the pyramid to find the first pulses) we added later, so
the cost on the search would be more like 10x or so.

> Where Pitch and PVQ are like the adaptive and fixed codebook vectors.

Did you know that CELT originally stood for "Code-Excited Lapped
Transform" because of the analogy with CELP? In the end I got too many
complaints that it was misleading that I changed it to
Constrained-Energy Lapped Transform. So far I'm impressed that we're
thinking about the exact same solutions for all these issues.

> Under normal operation, Pitch is normalized in energy (but it may have
> less energy after packet loss - see below).  The decoder creates the
> quantized signal as:
> 
> S_quant = Pitch * pitch_gain +
>           PVQ_orth * sqrt( (1 - pitch_gain ^ 2) / (PVQ_orth' * PVQ_orth) );
> 
> 
>> 2) If you lose a packet, there is basically no hope of resynchronising.
> 
> Maybe that can be avoided.  After a packet loss, and assuming no
> normalization, the Pitch vector has zero energy so S_quant is equal to
> the scaled PVQ.  With each decoded packet Pitch gradually increases
> until it reaches unit energy. 

You mean you would force it to be smaller than unity? Because with the
normalisation it usually *has* to have unit norm.

> As a result the orthogonalization in the
> decoder (as described in Equation 1) is only a partial orthogonalization
> in that the result is not completely orthogonal to Pitch as long as
> Pitch has energy below one.  In other words PVQ_orth will be more
> similar to PVQ while the Pitch vector is building up.
> (You could of course still choose to normalize the S_quant signal for
> creating the output signal, as long the signal going into the pitch
> buffer is based on the unnormalized S_quant.  This increases complexity
> though.)

I'm not following here.

> There's also a way to balance pitch coding gain and speed of error
> recovery:  When creating the pitch residual signal as input to the PVQ,
> using a pitch gain that is lower than the ideal pitch gain (= Pitch' *
> S) will typically lead to a PVQ vector that is less orthogonal to
> Pitch.  So it contains part innovation but also part a description of
> Pitch itself.  The pitch gain sent to the decoder should still be the
> (quantized) ideal pitch gain.  Without packet loss, the
> orthogonalization removes the part of PVQ that's aligned with Pitch, and
> the S_quant signal is created as usual.  Following a packet loss,
> however, there is no orthogonalization and the full PVQ is used which
> already looks more like the original signal.  And subsequent packets,
> with partial orthogonalization, will converge faster to the signal that
> would have existed without packet loss.
> 
> Don't know if this makes any sense...

Well it does and as far as I can tell it's pretty close to what we used
to do (I can dig out the source code if you'd like to look at it). In
the end, we judged that it wasn't really worth the effort -- especially
considering that we had to quantise pitch better. That's what it was
removed. Right now, the pitch gain is either 0 or 0.9 (one bit for each
of the 8 "pitch bands") and to me that sounds about as good as all the
fancy things we did. It turns out that even though we're minimizing the
complexity added by pitch, it still nearly doubles the encoder
complexity, and as far as I can tell it only improves lower bit-rate
speech performance (<= 48 kbit/s).

Cheers,

	Jean-Marc

---

From: Gregory Maxwell <[email-masked]>
Date: Thu, 16 Jul 2009 09:45:33 -0700

Jean-Marc Valin [[email-masked]] wrote:
>[SILK_CELT_discussion.txt]

Koen, I'm very curious how well your 'pyramid range coder' works in terms of storage efficiency compared to the indexed PVQ scheme used in CELT.

Given uniformly distributed pyramidal vectors and providing that the sum of absolute values is known (In CELT the vectors are all unit norm, and the PVQ precision is inferred from the current state and allocation table, so this is known without signalling cost) the indexed approach CELT uses is optimal for codebooks of less than 32bits (± the range coder precision).  However, your approach may gain some advantage from entropy coding— in CELT optimal entropy coding of the PVQ vectors appears to be a negligible improvement, but the underlying distribution of your data may be different. 

(CELT also works with vectors of many dimensions, so the memory required for probability tables was one reason the indexing scheme is used).

For codebooks greater than 32bits CELT recursively bisects the vector in dimensions until each leaf fits in 32 bits. This is significantly less efficient, especially since the number pulses on one side is currently coded without entropy coding. But it's hardly applicable to CELT because CELT doesn't encounter codebooks this big at normal rates as most of the bits are used on low frequencies which have small bands.  (The approach CELT is using does work beyond 32bits, but it requires 64-bit arithmetic) 

The approximate storage required for the CELT approach is easy to calculate (for codewords <32bits). Providing the magnitude is know the storage is just the log2 of the number of possible PVQ codewords. If the magnitude must be signalled it would need to be entropy coded (I presume that if need that you're already entropy coding it).

If N is the number of dimensions, and K is the magnitude (sum of absolute values), The cardinality of the codebook, V(N,K) can be determined by the recursive formula: V(N,0)=1  ; V(0,K)=0;  V(N,K)=V(N-1,K) + V(N,K-1) + V(N-1,K-1) 
(What CELT does internally is, obviously, much faster)

Here is a simple python script that calculates this:

---cwrs.py---
#!/usr/bin/python
import math,sys
sys.setrecursionlimit(32768)

memory={}

def V (n,k):
  if (k==0):
    return 1
  if (n==0):
    return 0
  if (n,k) not in memory:
    memory[(n,k)] = V(n-1,k) + V(n,k-1) + V(n-1,k-1)
    return memory[(n,k)]
  return memory[(n,k)]

N=16
for K in range(13):
  print "(%d,%d)"%(N,K),math.log(V(N,K),2)

---end: cwrs.py ---

It outputs:
(16,0) 0.0
(16,1) 5.0
(16,2) 9.0
(16,3) 12.4178525149
(16,4) 15.4262647547
(16,5) 18.1210477887
(16,6) 20.5636729598
(16,7) 22.7971992119
(16,8) 24.853601964
(16,9) 26.7576215488
(16,10) 28.5289824639
(16,11) 30.1837675338
(16,12) 31.7353184363

If would be interesting to know how that compares to your approach, and if codebooks of greater than 32bits are relevant to your usage. 

Cheers,

---

From: "Christian Hoene" <[email-masked]>
Date: Thu, 16 Jul 2009 19:59:02 +0200

Guys, 

please stop coding and start working! Did you call read “RFC 5434:
Considerations for Having a Successful Birds-of-a-Feather (BOF) Session”.
May I cite a few paragraphs?

   This document provides suggestions on how to host a successful BOF at
   an IETF meeting.  It is hoped that by documenting the methodologies
   that have proven successful, as well as listing some pitfalls, BOF
   organizers will improve their chances of hosting a BOF with a
   positive outcome.

   6) During the 2-4 weeks before an IETF (assuming a BOF has been
      approved and scheduled), the focus (on the mailing list) should be
      on identifying areas of agreement and areas of disagreement.
      Since disagreement, or "lack of consensus", tends to be the main
      reason for not forming a WG, focusing on those specific areas
      where "lack of consensus" exists is critically important.  In
      general, only after those disagreements have been resolved will a
      WG be formed; thus, the main goal should be to find consensus and
      work through the areas of disagreement.  Alternatively, a specific
      case should be made about why the charter, as it is written, is
      the best one, in spite of the stated opposition.

   7) Prior to the BOF, it is critical to produce a proposed charter and
      iterate on it on the mailing list to attempt to get a consensus
      charter.  Ultimately, the most important question to ask during a
      BOF is: "should a WG with the following charter be formed?".  It
      goes without saying that a charter with shortcomings (no matter
      how seemingly trivial to fix) will not achieve consensus if folk
      still have issues with the specific wording.

   8) Decide what questions will be asked during the BOF itself.  Since
      the exact wording of the questions is critical (and hard to do on
      the fly), it is strongly recommended that those questions be
      floated on the mailing list and to the ADs prior to the BOF.  This
      will enable people to understand what they will be asked to
      approve and will allow the questions to be modified (prior to the
      BOF) to remove ambiguities, etc.  Likewise, discussing these
      questions in advance may lead to refinement of the charter so that
      the questions can be answered affirmatively.

I do not see any progress on this mailing to address these issues. I have my
doubts whether you are interesting in forming a working group at all: No
progress on the charter, no progress on the agenda, no progress on
understanding the real problem. I only see new CELT versions.

Please, please, do start to act. Stockholm is in 10 days...

Christian

--------------------------------------------------------
Dr.-Ing. Christian Hoene
Interactive Communication Systems (ICS), University of Tübingen
Sand 13, 72076 Tübingen, Germany, Phone +49 7071 2970532
http://www.net.uni-tuebingen.de/

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Gregory Maxwell
> Sent: Thursday, July 16, 2009 6:46 PM
> To: [email-masked]; [email-masked]
> Subject: Re: [codec] Comments on draft-vos-silk-00.txt
> 
> Koen, I'm very curious how well your 'pyramid range coder' works in terms
of
> storage efficiency compared to the indexed PVQ scheme used in CELT.
> 
> Given uniformly distributed pyramidal vectors and providing that the sum
of
> absolute values is known (In CELT the vectors are all unit norm, and the
PVQ
> precision is inferred from the current state and allocation table, so this
is
> known without signalling cost) the indexed approach CELT uses is optimal
for
> codebooks of less than 32bits (± the range coder precision).  However,
your
> approach may gain some advantage from entropy coding— in CELT optimal
entropy
> coding of the PVQ vectors appears to be a negligible improvement, but the
> underlying distribution of your data may be different.
> 
> (CELT also works with vectors of many dimensions, so the memory required
for
> probability tables was one reason the indexing scheme is used).
> 
> For codebooks greater than 32bits CELT recursively bisects the vector in
> dimensions until each leaf fits in 32 bits. This is significantly less
> efficient, especially since the number pulses on one side is currently
coded
> without entropy coding. But it's hardly applicable to CELT because CELT
> doesn't encounter codebooks this big at normal rates as most of the bits
are
> used on low frequencies which have small bands.  (The approach CELT is
using
> does work beyond 32bits, but it requires 64-bit arithmetic)
> 
> The approximate storage required for the CELT approach is easy to
calculate
> (for codewords <32bits). Providing the magnitude is know the storage is
just
> the log2 of the number of possible PVQ codewords. If the magnitude must be
> signalled it would need to be entropy coded (I presume that if need that
> you're already entropy coding it).
> 
> If N is the number of dimensions, and K is the magnitude (sum of absolute
> values), The cardinality of the codebook, V(N,K) can be determined by the
> recursive formula: V(N,0)=1  ; V(0,K)=0;  V(N,K)=V(N-1,K) + V(N,K-1) +
V(N-
> 1,K-1)
> If would be interesting to know how that compares to your approach, and if
> codebooks of greater than 32bits are relevant to your usage.
> 
> Cheers,
> 
> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 16 Jul 2009 17:31:46 -0400

Gregory Maxwell wrote:
> Koen, I'm very curious how well your 'pyramid range coder' works in
> terms of storage efficiency compared to the indexed PVQ scheme used in
> CELT.

Actually, what I'd be curious to know is how SILK computes/stores the 
probabilities for the number of pulses on each side of the "pyramid split". 
Do you store a set of probabilities for all codebook sizes and for all 
number of pulses? Or is there a neat "trick" we could try to reduce the 
splitting overhead on large CELT codebooks?

 > May I cite a few paragraphs?

Since when are coding and working mutually exclusive? And what happened to 
rough consensus and running code? ;-)

Cheers,

	Jean-Marc

---

From: Koen Vos <[email-masked]>
Date: Thu, 16 Jul 2009 15:47:58 -0700

Hi Gregory,

Here's how SILK's version of PVQ works:

The input is a frame of integer-valued quantization indices. Each  
frames is split into blocks of 16 indices. For each block, we  
separately encode: (1) the sum of absolute values (which is known in  
CELT so is not encoded there); (2) the distribution of pulses over the  
the block; and (3) the sign of each non-zero quantization index.

More about step (2):
Each block is recursively split in two equal-sized halves, and the  
distribution of summed pulses over the two halves is losslessly  
encoded. Given that a block contains K pulses, there are K+1 ways in  
which to distribute these K pulses over the first and second halves of  
the block. Each of these K+1 ways has an assigned probability that is  
stored in a table, and was computed heuristically by running on a  
large training set. Starting with a block of 16 indices, it takes 15  
splits to determine the locations of all pulses. For computational  
efficiency, splitting stops whenever a subblock contains 0 pulses.  
Each block of 16 indices may contain up to 18 summed pulses. Subblocks  
of size 8, 4 and 2 may contain up to 12, 8 and 6 pulses respectively.  
If the block or any of its subblocks contains more pulses than  
allowed, an escape mechanism is used which codes the LSBs of the  
pulses separately.

Storage (ROM) size:
Separate tables are used for splitting the blocks of size 16, 8, 4 and 2:
Size 16: sum(3:20) = 207
Size 8:  sum(3:14) = 102
Size 4:  sum(3:10) = 52
Size 2:  sum(3:8) = 33
Total: 394 probability values.

Compared with CELT, this method makes no assumptions on the  
distribution of indices. It should provide an optimal encoding as long  
as all indices within a block are i.i.d. (independent and identically  
distributed). The ROM storage comes at a (small) cost in memory size,  
but gives us the probability values without any computations. We've  
tried increasing the block size to 32, but that gave no real  
performance gain.

As to Jean-Marc's question if there's a trick that could be used to  
reduce the splitting overhead on large CELT codebooks: that might  
indeed be possible.

best,
koen.

Quoting Gregory Maxwell <[email-masked]>:

> terms of storage efficiency compared to the indexed PVQ scheme used  
> in CELT.
>
> Given uniformly distributed pyramidal vectors and providing that the  
> sum of absolute values is known (In CELT the vectors are all unit  
> norm, and the PVQ precision is inferred from the current state and  
> allocation table, so this is known without signalling cost) the  
> indexed approach CELT uses is optimal for codebooks of less than  
> 32bits (± the range coder precision).  However, your approach may  
> gain some advantage from entropy coding— in CELT optimal entropy  
> coding of the PVQ vectors appears to be a negligible improvement,  
> but the underlying distribution of your data may be different.
>
> (CELT also works with vectors of many dimensions, so the memory  
> required for probability tables was one reason the indexing scheme  
> is used).
>
> For codebooks greater than 32bits CELT recursively bisects the  
> vector in dimensions until each leaf fits in 32 bits. This is  
> significantly less efficient, especially since the number pulses on  
> one side is currently coded without entropy coding. But it's hardly  
> applicable to CELT because CELT doesn't encounter codebooks this big  
> at normal rates as most of the bits are used on low frequencies  
> which have small bands.  (The approach CELT is using does work  
> beyond 32bits, but it requires 64-bit arithmetic)
>
> calculate (for codewords <32bits). Providing the magnitude is know  
> the storage is just the log2 of the number of possible PVQ  
> codewords. If the magnitude must be signalled it would need to be  
> entropy coded (I presume that if need that you're already entropy  
> coding it).
>
> If N is the number of dimensions, and K is the magnitude (sum of  
> absolute values), The cardinality of the codebook, V(N,K) can be  
> determined by the recursive formula: V(N,0)=1  ; V(0,K)=0;   
> V(N,K)=V(N-1,K) + V(N,K-1) + V(N-1,K-1)
> If would be interesting to know how that compares to your approach,  
> and if codebooks of greater than 32bits are relevant to your usage.
>
> Cheers,
>
>

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 16 Jul 2009 20:49:49 -0400

Hi Koen,

Koen Vos a écrit :
> distribution of summed pulses over the two halves is losslessly encoded.
> Given that a block contains K pulses, there are K+1 ways in which to
> distribute these K pulses over the first and second halves of the block.
> Each of these K+1 ways has an assigned probability that is stored in a
> table, and was computed heuristically by running on a large training
> set. 

Just curious, have you considered taking into account the pitch period
for adjusting the probabilities? i.e. assuming that pulses are likely to
be bigger at pitch intervals, even after LTP.

> Starting with a block of 16 indices, it takes 15 splits to
> determine the locations of all pulses. For computational efficiency,
> splitting stops whenever a subblock contains 0 pulses. Each block of 16
> indices may contain up to 18 summed pulses. Subblocks of size 8, 4 and 2
> may contain up to 12, 8 and 6 pulses respectively. If the block or any
> of its subblocks contains more pulses than allowed, an escape mechanism
> is used which codes the LSBs of the pulses separately.

Any special reason for the limit? Is it often exceeded?

> Compared with CELT, this method makes no assumptions on the distribution
> of indices. It should provide an optimal encoding as long as all indices
> within a block are i.i.d. (independent and identically distributed). The
> ROM storage comes at a (small) cost in memory size, but gives us the
> probability values without any computations. We've tried increasing the
> block size to 32, but that gave no real performance gain.

Technically, CELT cannot use this method because we need to know in
advance the precise number of bits required. However, it could
potentially be useful for cases where we need to split anyway to make
the codebook fit in 32 bits. It would be challenging though because
unlike SILK, CELT has to deal with arbitrary vector sizes (all
different) and not just 2,4,8,16.

One thing that would be interesting to measure is the performance of the
SILK codebook compared to what the i.i.d. assumption of CELT does. I
remember measuring a while ago and the *upper bound* (measuring the
entropy directly on the "training set") for some CELT codebook was about
1% improvement, which wasn't worth the complexity. I'd be curious to see
if the gain is larger for the data SILK encodes.

Cheers,

	Jean-Marc

---

From: Koen Vos <[email-masked]>
Date: Thu, 16 Jul 2009 19:26:37 -0700

Hi Jean-Marc,

> be bigger at pitch intervals, even after LTP.

If tried to look for such patterns in the sum-of-pulses (per block)  
parameter. This is encoded once per millisecond, in wideband mode, but  
I couldn't find a clear gain.

> Any special reason for the limit? Is it often exceeded?

At low rates it's never exceeded for normal speech signals. At high  
rates it's exceeded maybe a few times per second.  The limit was  
chosen such that increasing it further didn't reduces the bitrate by  
more than a few bits/second.

> if the gain is larger for the data SILK encodes.

Initially we constructed the probability tables using combinatorial  
logic. The table could be filled under Laplacian assumptions or  
Gaussian assumptions or anything in between. The optimum was clearly  
around Gaussian, being perhaps a few percent better than Laplacian. I  
don't remember how much the trained tables improved over the Gaussian  
constructed tables. Not so much for sure.

Frequency domain coefficients are known to have more of a Laplacian  
distribution, whereas a time domain LPC+LTP residual is approximately  
Gaussian (probably because of the least-squares optimization).  That  
could explain why the norm-1 pyramid vector quantizer is so close to  
optimal for CELT.

best,
koen

---

From: Koen Vos <[email-masked]>
Date: Thu, 16 Jul 2009 21:22:44 -0700

Hi Jean-Marc,

Continuing the CELT discussion:

> or so), but it was causing resynchronisation problems.

It's probably enough to only orthogonalize after the search, when  
generating the reconstructed signal in the encoder (for the pitch  
buffer) and in the decoder.

>> After a packet loss, and assuming no normalization, the Pitch  
>> vector has zero energy so S_quant is equal to the scaled PVQ.  With  
>> each decoded packet Pitch gradually increases until it reaches unit  
>> energy.
>

Right, maybe the Pitch vector should not be normalized after packet  
loss before being used in equation 1. Because after losing one or more  
packets, whatever Pitch is, it surely is very different from the  
encoder. Best might be to reset Pitch, as a result of which no  
orthogonalization will occur. One packet later, Pitch still contains  
only a small amount of useful information, and I would argue it should  
still not be normalized to unity. Doing so would build up noise over  
subsequent pitch-predicted decodes with orthogonalization.

Also, forcing S_quant to be unit energy in the decoder when it only  
contains a small/noisy innovation signal after packet loss might  
reduce the quality of the decoded signal?

> Right now, the pitch gain is either 0 or 0.9 (one bit for each
> fancy things we did.

How do you control error propagation when the pitch gain is 0.9?  Turn  
off pitch prediction every so many packets?

> It turns out that even though we're minimizing the

With SILK, turning off the LTP model reduces the PESQ MOS score from  
4.5 to 3.8, at 30 kbps in wideband mode. With LTP on, the 3.8 score is  
reached already at 9 kbps. Is there an explanation of why pitch  
prediction couldn't work well for CELT?

best,
koen

---

From: Jean-Marc Valin <[email-masked]>
Date: Fri, 17 Jul 2009 07:47:04 -0400

Hi Koen,

> buffer) and in the decoder.

This is something we've tried and it's a catastrophe.

> contains a small/noisy innovation signal after packet loss might reduce
> the quality of the decoded signal?

Well, what would you normalise S_quant to? We can artificially lower its
energy, but normalisation is necessary because of how CELT works.

> off pitch prediction every so many packets?

With a pitch gain of 0.9 and no attempt at companding or anything, the
decoder naturally resyncs about as quickly as a CELT coder does (see
figure 9 in http://people.xiph.org/~jm/papers/celt_tasl.pdf ). It even
has the advantage that you still have the right gain and energy envelope
even during the resync. For example, CELP codecs sometimes need pitch
gains greater than unity for onsets whereas CELT's normalisation means
that a gain smaller than zero can still predict it correctly and be more
stable. You just want to make sure not to predict from a block that has
*much* weaker energy to avoid numerical stability problems (it would be
equivalent to having a pitch gain of 10 or something).

> With SILK, turning off the LTP model reduces the PESQ MOS score from 4.5
> to 3.8, at 30 kbps in wideband mode. With LTP on, the 3.8 score is

Well one obvious reason we don't see as much improvement in CELT is that
the LTP-off performance of CELT is much better due to the use of the
MDCT rather than time-domain LPC. There is also the fact that CELT deals
with a larger audio bandwidth than SILK, so the relative size of the
region where LTP is useful is smaller. But that likely does not explain
everything. I think the rest is due to the normalisation and how the
fixed codebook gain is automatically calculated (not encoded). Whether
this is worth trying to improve (given whatever cost it may have) is an
open question. So far I did not find anything that could provide a
significant improvement, but that doesn't mean that no such thing exists.

Cheers,

	Jean-Marc

---

From: Koen Vos <[email-masked]>
Date: Fri, 17 Jul 2009 12:43:00 -0700

> energy, but normalisation is necessary because of how CELT works.

Without packet loss, and asuming the PVQ signal is always normalized,  
the normalization factor applied to S_quant is close to 1.0 (in fact,  
no normalization would be needed with orthogonalization). So my  
thinking was you can just not do any normalization after packet loss.

I guess the question is: what's more important, to get the energy  
right over time-frequency or to have a good SNR in each energy band.  
The answer might be somewhere in the middle.

> MDCT rather than time-domain LPC.

Why is that? From a rate-distortion point of view, the only advantage  
I can think of for MDCT over LPC is that the quantized signal has a  
Laplacian distribution rather than a Gaussian one. Laplacian signals  
can be coded at a lower bitrate, for a given SNR. The gain is < 1 kbps  
though...

> There is also the fact that CELT deals
> region where LTP is useful is smaller.

True, although most bits are spent at frequencies where LTP is used.

koen.

---

From: Jean-Marc Valin <[email-masked]>
Date: Fri, 17 Jul 2009 15:59:08 -0400

Hi Koen,

Koen Vos wrote:
> the normalization factor applied to S_quant is close to 1.0 (in fact, no 
> normalization would be needed with orthogonalization). So my thinking 
> was you can just not do any normalization after packet loss.

I think we're talking about different normalizations. When I'm talking 
about normalisation, I mean taking the PVQ codebook and transforming it 
into something of unit norm (possibly with pitch included). If you don't 
normalise, you have integers with a norm far greater than unity.

> I guess the question is: what's more important, to get the energy right 
> over time-frequency or to have a good SNR in each energy band. The 
> answer might be somewhere in the middle.

 From my experience with CELT, the most important is to get the energy 
right, especially considering that damping everything will not make the 
noise any less audible (the masking threshold goes down by the same amount).

> Why is that? From a rate-distortion point of view, the only advantage I 
> can think of for MDCT over LPC is that the quantized signal has a 
> Laplacian distribution rather than a Gaussian one. Laplacian signals can 
> be coded at a lower bitrate, for a given SNR. The gain is < 1 kbps 
> though...

I only meant that the MDCT has better redundancy reduction than LPC alone 
(no LTP), especially on tonal signals (and even if the freq resolution is 
not that good).

> True, although most bits are spent at frequencies where LTP is used.

Fair enough.

BWT, if you're interested in making some experiments with the pitch in 
CELT, I can give you a hand. Most of the interesting code is in 
quant_bands() (bands.c) and alg_quant() (vq.c), and it's easy to completely 
bypass the decoder and get the synthesis signal straight from the encoder.

Cheers,

	Jean-Marc

---

From: Koen Vos <[email-masked]>
Date: Sat, 18 Jul 2009 07:45:05 -0700

> From my experience with CELT, the most important is to get the  
> energy right, especially considering that damping everything will  
> not make the noise any less audible (the masking threshold goes down  
> by the same amount).

Makes sense.

> I only meant that the MDCT has better redundancy reduction than LPC  
> alone (no LTP), especially on tonal signals (and even if the freq  
> resolution is not that good).

For the redundancy reduction to lead to a lower bitrate requires a  
model to describe how energy is distributed over MDCT bins/bands.   
CELT uses a model with 20 energy bands, which gives a similar  
flexibility in describing the energy distribution as an LPC model of  
order 16.  Beyond that, within a single energy band, there is (almost)  
no coding gain from having all energy sitting in one bin instead of it  
being spread over all bins.  (And the small gain that exists comes  
from the Laplacian rather than Gaussian encoding.)

> BWT, if you're interested in making some experiments with the pitch  
> in CELT, I can give you a hand. Most of the interesting code is in  
> quant_bands() (bands.c) and alg_quant() (vq.c), and it's easy to  
> completely bypass the decoder and get the synthesis signal straight  
> from the encoder.

Sounds interesting for sure. I'll take a look at the code and get back to you.

koen.

---

From: Jean-Marc Valin <[email-masked]>
Date: Sat, 18 Jul 2009 14:11:13 -0400

Koen Vos a écrit :
> For the redundancy reduction to lead to a lower bitrate requires a model
> to describe how energy is distributed over MDCT bins/bands.  CELT uses a
> model with 20 energy bands, which gives a similar flexibility in
> describing the energy distribution as an LPC model of order 16. 

Not strictly related to coding efficiency, but the MDCT band model used
by CELT is closer to a *warped* LPC model. However, the main advantage
is that you get more control over the quantisation error, because you
can control it independently for each band, whereas LSP quantization
error is hard to control efficiently and tends to spread across the
whole spectrum. That being said, I agree that an all-pole model is
probably best when speech is the only thing you want to encode.

> Beyond
> that, within a single energy band, there is (almost) no coding gain from
> having all energy sitting in one bin instead of it being spread over all
> bins.  (And the small gain that exists comes from the Laplacian rather
> than Gaussian encoding.)

The gain is actually pretty big. It's probably not that much in terms of
SNR, but mainly perceptually. I once tried quantising using a "random
rotation matrix" to spread/unspread the PVQ pulses (so that "pulses" end
up looking like noise) and the sound quality went down dramatically.

Cheers,

	Jean-Marc

---

From: Koen Vos <[email-masked]>
Date: Sun, 19 Jul 2009 00:25:29 -0700

Quoting Jean-Marc Valin:

> by CELT is closer to a *warped* LPC model.

True.

> However, the main advantage is that you get more control over the  
> quantisation error, because you can control it independently for  
> each band, whereas LSP quantization error is hard to control  
> efficiently and tends to spread across the whole spectrum.

SILK uses LPC to minimize the residual energy, and, thus, the bitrate.  
LSP/LSF quantization works well for that.

The quantization error in SILK is controlled through noise shaping,  
and is largely independent of the LPC model and LSF quantization.   
Because this noise shaping is done only on the encoder side, it can be  
based on any model for distributing quantization noise over the  
spectrum (including frequency domain masking models), without breaking  
the bitstream or affecting the LPC model.

> up looking like noise) and the sound quality went down dramatically.

Interesting. The same thing applies to LPC in the time domain: LPC is  
good at encoding short impulses in time without time-domain smearing  
artifacts like pre-echoes. So I guess there's a perceptual advantage  
when the basis functions mimic the signal's time-frequency energy  
distribution, because it's easier to handle the large dynamic range.

best,
koen.