Subject: Re: [codec] A pitch filtering method to improve the mode 1 CELT codec

From: Koen Vos <[email-masked]>
Date: Tue, 2 Nov 2010 22:54:29 +0100 (CET)

Hi Raymond, Jean-Marc,

As Jean-Marc and I have been discussing, SILK has a pitch contour coder. It's efficient in terms of bitrate (adding about 4 bits on top of the single pitch lag, to code 4 subframe lags), and has low complexity. It assumes that the pitch lags for all subframes in a frame are nearby and evolve roughly linearly (this is not strictly necessary though).  This seems like a close fit to CELT's pitch estimator also.

best,
koen.

----- Original Message -----
From: "Jean-Marc Valin" <[email-masked]>
To: "Raymond (Juin-Hwey) Chen" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: Monday, November 1, 2010 8:30:47 PM
Subject: Re: [codec] A pitch filtering method to improve the mode 1 CELT codec

Hi Raymond,

On 10-11-01 09:54 PM, Raymond (Juin-Hwey) Chen wrote:
> Regarding the pitch contour for frame sizes above 5 ms, as I mentioned
> before, I recommend that the pitch period still gets updated every 5
> ms even when the frame size is 10 or 20 ms. The simplest way is to run
> the pitch estimator at every 5 ms subframe in the same way as the
> pitch estimator for a frame size of 5 ms, and code the resulting pitch
> period independent of the pitch periods in other 5 ms subframes of the
> current frame. This approach is obviously not as efficient as it can
> be, but it
> is simple, requires essentially no additional code, and has probably
> the lowest risk of unintentional IP infringement.

I'm afraid that independently coding each subframe might actually
require too many bits (and that those bits would be more useful in
quantization). Because the pitch here is used only as a post-filter, the
accuracy on the period isn't quite as critical as it is when using pitch
for prediction. That being said, I think it's more useful to find a more
efficient way to code the contour, especially considering that we want
that contour to be very smooth, not arbitrary.

> We can potentially also use the future subframes in the current frame
> as the "look-ahead" when performing pitch estimation in the earlier
> subframes in order to increase the reliability of the estimated pitch
> period, but
> this will require new code and thus additional code space.

I don't think this is worth adding look-ahead for, though if it's
encoder-side, implementers would be free to do so if they like.

> We can also independently code the pitch period of the last 5 ms
> subframe, and then for the earlier subframes, we interpolate between
> this pitch
> period and the pitch period of the last subframe of the last frame,
> and then code only the difference between the estimated pitch period
> and the
> interpolated pitch period. This approach should significantly reduce
> the encoding bit-rate for the pitch period. However, this approach may
> have a higher risk of patent infringement because something like this
> might have already been patented (unless the idea is so old that any
> related patent has expired).

This is actually the approach I had in mind. It's probably worth
investigating what has been done there.

> As to the range of the pitch period, I think lowering the minimum
> pitch period from the current 30 samples to something lower, such as
> 20 samples,
> or even to 11 samples (to cover the pitch of the highest piano note at
> 4186.01 Hz), will cost almost nothing when compared with the current
> pitch period range,

Right now, I first code the octave number, then the pitch within the
octave. Going to 16 would require another octave (7 instead of 6), which
means 0.22 bit.

> but it will increase the covered pitch frequency range
> substantially. Of course, people may question (1) whether we will get
> accurate enough pitch frequency when the pitch period is constrained
> to be
> integer number of samples at such a low pitch period, and (2) whether
> the pitch prefilter/postfilter approach still get the desired and
> expected perceptual benefit at such a high pitch frequency.

I don't actually think the postfilter would be necessary at that point
as its "resolution" would actually be worse than the resolution of the
CELT bands. The peaks of the response would be too wide to have an
effect within one CELT band and the energy quantization would do a
better job across bands.

> I think one thing worth trying is that rather than searching through
> and coding every integer between the minimum pitch period and the
> maximum pitch period (which takes 10 bits to code independently),
> perhaps we should
> progressively increase the distance between adjacent allowed pitch
> period candidates from 1 sample to 2, 3, or even higher number of
> samples as the
> pitch period increases from the minimum pitch period to the maximum
> pitch period of 960. By doing this, we will not only reduce the search
> complexity of the pitch estimator, but also reduce the number of bits
> required to encode the pitch period from 10 bits to 9 bits, or perhaps
> even 8 bits or lower.

As I said, I'm currently coding in 6 octaves, so for the highest pitches
(small period), only 7.5 bits are required while for lowest pitches we
need 11.5 bits. I think I prefer this to lowering the resolution of the
period for low pitches

Cheers,

Jean-Marc
_______________________________________________ codec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/codec

---

From: "Raymond (Juin-Hwey) Chen" <[email-masked]>
Date: Tue, 2 Nov 2010 21:07:25 -0700

Hi Koen,

I don't really care what kind of pitch quantization scheme is used, so if 
the SILK pitch quantizer has low complexity, high coding efficiency, and 
gets the job done well for the pitch prefilter/postfilter, then I am all 
for it.

On the other hand, if the SILK pitch quantizer was mainly optimized for 
coding speech, I have a feeling that it may need to be modified or re-
tuned to handle music well.  Unlike speech signals where the pitch 
contour is pretty much a continuous function within a voiced segment, the 
pitch contour of music instruments can and do jump abruptly when it goes 
from one note to another note with a different pitch. It's a quantum jump 
rather than a ramp as in speech, so linear interpolation may not work 
quite as well as in speech during such note junctions.  Ideally the pitch 
quantizer needs to be able to handle both types of pitch contours well.

Raymond

-----Original Message-----
From: Koen Vos [mailto:[email-masked]] 
Sent: Tuesday, November 02, 2010 2:54 PM
To: Jean-Marc Valin
Cc: [email-masked]; Raymond (Juin-Hwey) Chen
Subject: Re: [codec] A pitch filtering method to improve the mode 1 CELT codec

Hi Raymond, Jean-Marc,

best,
koen.

----- Original Message -----
From: "Jean-Marc Valin" <[email-masked]>
To: "Raymond (Juin-Hwey) Chen" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: Monday, November 1, 2010 8:30:47 PM
Subject: Re: [codec] A pitch filtering method to improve the mode 1 CELT codec

Hi Raymond,

Cheers,

Jean-Marc