Subject: [video-codec] Daala deringing filter vs Thor constrained low pass filter

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 21 Oct 2015 17:07:30 -0400

Hi Steinar, Thomas,

I saw your new constrained low pass filter draft for Thor and decided
to start experimenting with it. Putting the filter in Daala, I saw a
PSNR bd-rate improvement around 2.1% in the 0.05 bpp to 0.1 bpp range
over our ntt-short1 test set (I disabled our deringing filter for this
comparison to avoid the interactions). However, I was unable to see
improvements of 4-7% like you are reporting in the draft.

Of course, it's quite likely that I messed up the implementation or
the adaptation to Daala. It's also possible that somehow Daala cannot
benefit from the CLP as much as Thor does. This is the code I wrote
for filtering a small block of pixels:
https://pastebin.mozilla.org/8850049
I've had to do some adaptation since the Daala pixels are shifted up
by 4 at the point where we're applying the filter. Does this look
correct to you?

The other experiment that would be nice to try is using the Daala
deringing filter in Thor. In Daala, our deringing filter is improving
PSNR bd-rate by about a 6.5% in the 0.05 bpp to 0.1 bpp range. The
deringing filter is documented in this IETF draft:
https://tools.ietf.org/html/draft-valin-netvc-deringing-00
or (for those who don't like ASCII equations) in this pdf:
http://jmvalin.ca/notes/dir_dering.pdf
I uploaded some still image examples at:
http://jmvalin.ca/video/deringing_examples/
Any interest in trying the Daala deringing filter in Thor?

Cheers,

	Jean-Marc

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 22 Oct 2015 11:28:17 -0400

Hi Steinar,

On 10/22/2015 04:48 AM, Steinar Midtskogen wrote:
> It looks like you're adjusting pixels by one if just one or more 
> neighbour is more than 1 higher or lower.  I wouldn't expect that
> to work.  How does this work:
> 
> y[i*ystride + j] = x + ((((x0 > x) + (x1 > x) + (x2 > x) + (x3 > x)
> > 2) - ((x0 < x) + (x1 < x) + (x2 < x) + (x3 < x) > 2)) << 4);

Indeed, I clearly saw the "> 2" in the draft... and then omitted it
from my code. Using your code above, I am now getting a 3.1%
improvement. Although it's still lower than what you report in the
draft, it's not so far considering that our test set does not include
the two clips that are giving you the highest gains. For example, I'm
seeing that CLP at high rate improves BQTerrace by about 8% and Johnny
by about 9%. In fact, at very high rate (> 0.2 bpp), the deringing
filter and the CLP give very similar improvement. Deringing is mostly
providing better gains compared to CLP when the bitrate drops around
0.05 to 0.1 bpp (depending on the clip).

I'm attaching a PDF with the PSNR and PSNR-HVS curves for CLP and
deringing compared to no filter.

> Thomas implemented a simplified SAO for Thor for reference, just
> edge offset and +/-1, but this gives most of the gains in H.265
> anyway.  The objective gains in Thor were useful, but not very
> convincing.  I think Thor needs less deringing than H.265.  Perhaps
> it's because the interpolation filter is different (fewer taps, the
> special 2,2 filter and possibly an improved frequency response).
> 
> I will look into your filter, but I might not be able to implement 
> anything before the meeting.

Well, most of the Daala code we have for the deringing filter should
be directly applicable in Thor. I can help in adapting it too. The
main thing that would be required from the existing Thor codebase is
doing the buffering so that the deringing filter only gets applied on
non-filtered pixels rather than "refiltering" some pixels. This is
important because the filter is longer than for CLP, so we can't just
avoid filtering on the superblock boundaries. So let me know if/when
you're interesting in trying this.

Cheers,

	Jean-Marc

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 22 Oct 2015 12:22:34 -0400

Hi Steinar,

On 10/22/2015 12:05 PM, Steinar Midtskogen wrote:
> 3.1% doesn't sound very unreasonable if this means you're applying
> the filter to all coded blocks without any signalling.
> 
> Thor has a bit for every 64x64 block, except blocks that are
> entirely without coefficients (can still have vectors, though) or
> entirely bi-predicted.  Such blocks are never filtered.  If a 64x64
> block is filtered, subblocks can still be left unchanged if they
> lack coefficients or are bipred.
> 
> The test whether to filter a 64x64 block or not is a simple
> squared difference with the original block.  If the sum for the
> filtered block is less than the sum for the unfiltered block, the
> filtered block is kept.

Actually, the 3.1% is with signalling, in exactly the same way as we
did the signalling for deringing or for the first CLP filter. The test
is based on a PSNR-HVS-like metric compared to the original block. The
flag is entropy coded (with context from the neighbouring blocks),
with an RDO decision. Our superblock size is 32x32 so the cost is a
little bit higher than for Thor, but not significantly so since we
have entropy coding. We do not signal a flag on superblocks that are
entirely skipped. For superblocks that are filtered, we still disable
the filter in areas that are skipped. I believe this is pretty much
what Thor does too.

> It's interesting to see that the deringing also works reasonably
> well for low bitrates where CLP is falling apart (though I believe
> it still is useful subjectively).
> 
> I did some experiments also signalling offsets larger than 1 (same 
> offset for every 64x64 block) which probably helps more at low
> rates than at high rates.  But even if I assumed zero signalling
> costs, the gains weren't big.

Indeed, for lower bitrates you need to be able to have larger offsets.
The key is being able to avoid blurring out edges, which is what our
deringing filter attempts to do.

> An observation in Thor is the CLP filter works better if applied
> after the deblocking filter and not before.  I think SAO in H.265
> has a similar property.

Yes, both CLP and deringing in Daala are applied after our lapping
post-filter. We never tried applying it before, but I would not expect
anything good to come out of it.

Cheers,

	Jean-Marc

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 22 Oct 2015 14:37:06 -0400

On 10/22/2015 12:50 PM, Steinar Midtskogen wrote:
> If it's an easy experiment, what happens if you apply both the
> deringing and the CLP filter (either order), or try to replace the
> secondary deringing filter with CLP?

Just ran the following experiments:
1) running CLP after the second deringing filter
2) running CLP before the first deringing filter
3) running CLP instead of the second deringing filter

Of these, the best one was 2), but all three are slightly worse than
running the deringing filter alone. This isn't so surprising
considering that CLP is attempting to do something quite similar to
deringing.

	Jean-Marc

---

From: Mohammad AlSaleh <[email-masked]>
Date: Thu, 29 Oct 2015 19:41:40 +0300

On Wed, Oct 21, 2015 at 05:07:30PM -0400, Jean-Marc Valin wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> Hi Steinar, Thomas,
> 

Looking at chapel subjectively, I think an option to use a weaker
filter would be helpful.

Generally, the filter gives good subjective results. But some of
the details might get lost (e.g. the last steps leading to the chapel).

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 29 Oct 2015 13:39:48 -0400

> the details might get lost (e.g. the last steps leading to the chapel).

It isn't really a matter of stronger vs weaker filter. The encoder can
turn the deringing filter on/off for every superblock, so any issue
would come down to how the encoder made the decision to keep the
filtered superblock. For example, the distortion function could be
missing something, or the encoder could be deciding to "sacrifice" some
region of the superblock to make another region better.

Cheers,

	Jean-Marc