Subject: Re: [codec] A pitch filtering method to improve the mode 1 CELT codec

From: Koen Vos <[email-masked]>
Date: Thu, 28 Oct 2010 10:53:10 +0200 (CEST)

I guess pitch filtering hasn't been used for music before because very little music is actually anywhere near harmonic.  Most tonal music contains chords: multiple tones with different fundamental frequencies.  For these, pitch filtering easily causes more harm than anything else.  But it is true that harmonic sounds like the harpsichord are among the trickiest to code.  

An interesting aspect of string-generated tones is that their harmonic frequencies are not exactly integer multiples, because of the stiffness of the string: http://en.wikipedia.org/wiki/Inharmonicity
I remember from previous experiments with pitch filters that for signals like the harpsichord this effect is quite strong.  Fortunately it's possible to create a pitch filter with similarly "inharmonic" harmonics: just add an allpass filter to the delay line.  The allpass filter creates a shorter delay for higher frequencies, which moves the higher harmonics up a bit.  The allpass coefficient can be fairly easily estimated with for instance a gradient descent algorithm, or even by just trying out a few values and chosing the one with the highest prediction gain.  Might be worth an experiment.  The downside is of course that the inharmonicity coefficient will need to be coded and transmitted.

Another idea is to use a slighlty lower gain in the pitch prefilter than in the postfilter.  This will make the decoder emphasize the harmonics a bit, which creates an effect similar to a traditional harmonic postfilter - for free.  This may work especially well for speech coded at below-transparent quality.

koen.

----- Original Message -----
From: "Raymond (Juin-Hwey) Chen" <[email-masked]>
To: "Jean-Marc Valin" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, "Jean-Marc Valin" <[email-masked]>
Sent: Wednesday, October 27, 2010 11:26:12 PM
Subject: Re: [codec] A pitch filtering method to improve the mode 1 CELT codec

Jean-Marc,

I forgot to comment:
As long as the filter tap codebook adequately covers the full possible
range from purely periodic (no envelope shaping) to effectively turning
off the pitch filter, then even if we can't quite get a perfect pitch
estimator or pitch filter tap determination method that we are
completely happy with before IETF 79 meeting, we can improve them later
even after the possible pitch periods and possible pitch filter tap sets
are determined. The improved pitch estimator and pitch filter tap
determination method developed later will improve the audio quality
without changing the bit-stream format. They may increase the complexity
of the encoder side, but the decoder side remains low complexity and
unchanged. And who knows, maybe with sufficient research, it is possible
that the pitch period and pitch filter tap determination can both be
improved while reducing the encoder side complexity at the same time.

Raymond

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Raymond (Juin-Hwey) Chen
Sent: Wednesday, October 27, 2010 11:10 PM
To: Jean-Marc Valin
Cc: [email-masked]'; Jean-Marc Valin
Subject: Re: [codec] A pitch filtering method to improve the mode 1 CELT
codec
Importance: Low

Hi Jean-Marc,

Thanks for your comments. If the problem is with the pitch estimation,
the good thing is that it does not affect the bit-stream format if we
just use the integer pitch period within a certain range. Even after the
bit-stream format is frozen, the pitch estimation accuracy can still be
improved later as it only affects the encoder side. Therefore, even if
we can't get a very reliable pitch estimator for music before the IETF
79 meeting, it doesn't necessarily need to hold up the bit-stream
definition.

Hence, I think the pitch estimation accuracy is not the most urgent
issue. I think the most urgent issue is to design the pitch filter
tap(s) codebook entries so they all correspond to stable pitch filters
and there are enough variety of filter tap sets in the codebook so they
allow us to shape the envelope of the frequency response of the pitch
filters in a wide range from fully periodic (no envelope shaping) to
something that effectively turns off the pitch filter. To finalize the
bit-stream, we do need to finalize such a filter tap codebook.

I remember I have seen some papers on pitch estimation techniques for
music signals published at IEEE ICASSP conferences, and from the pitch
contour figures in the papers, they seem to work well. I think we can
first try to improve the CELT PLC pitch estimator, BroadVoice pitch
estimator, or SILK pitch estimator to see if we can get satisfactory
pitch estimation performance for both speech and music with low enough
complexity. If not, we can check those music pitch estimation papers out
to see what they have there. We can do such pitch estimator improvement
after IETF 79 if we can't finish it before then.

Raymond

P.S. It just so happens that this week I am completely booked solid
during the weekdays -- a 3.5-day off-site conference followed by a
1.5-day meeting, so I don't have much time to work on this method this
week. This is very rare for me. Sorry about that. Next week I expect to
have more time to work on this.

-----Original Message-----
From: Jean-Marc Valin [mailto:[email-masked]]
Sent: Wednesday, October 27, 2010 6:13 AM
To: Raymond (Juin-Hwey) Chen
Cc: Jean-Marc Valin; [email-masked]'
Subject: Re: [codec] A pitch filtering method to improve the mode 1 CELT
codec

Hi Raymond,

On 10-10-27 02:17 AM, Raymond (Juin-Hwey) Chen wrote:
> Are you saying the problem is in the pitch estimation for that test
> file rather than the pitch filtering?

The fundamental problem is definitely with the estimation. Reliable and
consistent pitch estimation is hard. The question is thus whether it's
possible to either have a reliable enough estimator *or* have a robust
enough filtering scheme that it can tolerate occasional errors in the
estimation. I'm still working on both.

> I will take a look at that file. Is that a raw file? I am not familiar
> with the ".sw" file extension.

That's just a 48 kHz raw 16-bit PCM file (little endian). sw is just the
sox program's extension for "signed word".

> I could hear audible improvement when I tried the method on the other
> solo instrument files that I am working on, although the degree of
> improvement depends on which file it is, as expected.

Yes, I'm also observing that. What I mainly want to improve is how it
works on more complicated signals (like the one I linked to), or
non-harmonic signals. As long as we're in a situation where it improves
on a subset of
the samples while not degrading the others, I'm happy.

> I am starting to wonder how many people in the codec WG email list
> still want to continue to see our low-level technical discussions
> about the
> algorithm development in the reflector. I wonder if some people in the
> codec WG email list would prefer not to be bombarded with such emails.
> Should we consider moving such detailed codec algorithm development
> discussions off the WG mailing list and just between those of us who
> are interested or are directly involved?
>
> Or is this how IETF supposed to work (discussing all technical
> development in emails)?

I think we should keep having this discussion on the list. The IETF
lists are normally meant for technical discussions so this just helps
improve the
SNR :-)

Cheers,

Jean-Marc

_______________________________________________ codec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/codec

_______________________________________________ codec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/codec