Subject: [codec] Jan Skoglund's Opus test results (NB/WB/FB)

From: Gregory Maxwell <[email-masked]>
Date: Thu, 31 Mar 2011 09:08:03 -0700

Jan Skoglund  presented listening test results in the WG meeting today:

http://tools.ietf.org/agenda/80/slides/codec-4.pdf   starting at slide 15.

I did statistical tests on the results, which are available at:

http://people.xiph.org/~tterribe/notes/celt/nb-overall-ptests.png
http://people.xiph.org/~tterribe/notes/celt/wb-overall-ptests.png
http://people.xiph.org/~tterribe/notes/celt/music-overall-ptests.png

These are the results of  randomized blocked permutation test with N=100000 and correction for multiple comparisons.
Results with a p-value 0.05 of lower are considered significant— and are colored red or green based worse/better when read across the rows.

The complete data from the testing is available at:

https://people.xiph.org/~greg/Opus2011-Jan/OpusTest2011.xls
(I'm posting it because I had the data from the statistical tests and easy web-hosting available)

It's pretty self-explanatory— listeners on columns, samples on rows, totals driving graphs.