Subject: [codec] Examples of codec test and processing plans

From: Paul Coverdale <[email-masked]>
Date: Sun, 3 Apr 2011 15:07:00 -0400

Colleagues,

Following on from the discussion in Prague of the scope of subjective
testing for Opus, and the need for a test plan, I have attached examples of
plans executed during the development of G.729.1 (formerly G.729EV) for
NB/WB operation, and for extension of G.722 and G.711.1 for SWB operation.
Yes, there is a lot of detail here which may not be necessary for Opus, but
I believe that the material can be used as a starting point.

Paul Coverdale

---

From: Jean-Marc Valin <[email-masked]>
Date: Tue, 5 Apr 2011 13:35:16 -0400

Hi Paul,

Thanks for the examples. That could potentially be useful once we first 
determine what information is actually missing to answer the questions Koen 
and I listed in a previous email (Subject: A concrete proposal for 
requirements and testing). We have not heard comments on these questions so 
far.

	Jean-Marc

On 11-04-03 03:07 PM, Paul Coverdale wrote:
> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec

---

From: Paul Coverdale <[email-masked]>
Date: Tue, 5 Apr 2011 16:08:18 -0400

Hi Jean-Marc,

My concern with the 7 questions listed by you and Koen is that we seem to be
trying to meet the quality requirements of section 4.2 in a "piece-meal"
fashion, by pulling together results from several different experiments with
a small number of subjects and unknown test configurations. This may be OK
for initial qualification testing on Opus (and yes, I have to admit that the
results look encouraging), but it does not seem to me like a formal
evaluation against the requirements.

In any case,  I haven't seen any results in different languages or with
multiple talkers, as specified in section 4.2.

Regards,

...Paul

-----Original Message-----
From: Jean-Marc Valin [mailto:[email-masked]] 
Sent: Tuesday, April 05, 2011 1:35 PM
To: Paul Coverdale
Cc: [email-masked]
Subject: Re: [codec] Examples of codec test and processing plans

Hi Paul,

	Jean-Marc