Subject: Re: [codec] Testing Codec Quality

From: Michael Knappe <[email-masked]>
Date: Wed, 12 Jan 2011 01:32:35 -0800

Christian - WOW, what a fantastic document!

I encourage everyone to read this through and provide comments back to the
mailing list. In particular, check out section 10.2 which lists detailed
considerations for the proposed semi-formal testing with the 6+ volunteer
'test houses'. One of the issues we've been dealing with in our discussions
is how to divide up testing of even a simplified list of codec configuration
and operational permutations (Christian easily came up with 48 different
test cases) between the test houses. The plan is to have a very short list
of basic codec test cases that every test house repeats, along with a longer
list of permutations that is divided among the test houses.

We do need to add a section outlining the proposed widescale informal
listening tests via a web based mushra app that will be done alongside the
semi-formal testing. Look for that app soon at www.open-codec.com, we're
working on getting that app up and running as we speak.

Thanks again to Christian for all the effort put into this doc.

Cheers, 

Mike

On 12/3/10 2:12 AM, "Christian Hoene" <[email-masked]> wrote:

> Hi,
> 
> one important question back to the days where iLBC was submitted to the IETF
> was: how do we know whether iLBC is a technical sound contribution to the
> IETF?
> 
> The draft http://www.ietf.org/id/draft-hoene-codec-quality-00.pdf addresses
> this question in the context of this WG. The first chapters describe well
> established procedures on how to test a codec (and goes a bit beyond that
> where needed). Finally, Chapter 10 suggests a procedure on how to test IIAC
> (or Opus).
> 
> Timely feedback is welcomed as the testing shall start soon.
> 
> With best regards,
> 
>  Christian
> 
> 
> ---------------------------------------------------------------
> Dr.-Ing. Christian Hoene
> Interactive Communication Systems (ICS), University of Tübingen
> Sand 13, 72076 Tübingen, Germany, Phone +49 7071 2970532
> http://www.net.uni-tuebingen.de/
> 
> 
> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec