Subject: Re: [codec] WGLC #2 for draft-ietf-codec-opus-10

From: Jean-Marc Valin <[email-masked]>
Date: Tue, 01 Nov 2011 14:15:41 -0400

This new draft addresses issues that were raised during the first WGLC
by the WG participants and chairs, as well as 3GPP and ITU-T SG16
liaisons. It also fixes several issues that were discovered during
extensive testing of the code. While most of the code modifications were
either encoder changes or reformatting changes, there were a few decoder
changes. Among the changes:
- Fixes for several glitches caused by mode switching and corner cases
(changes bit-stream for SILK-only stereo and hybrid stereo modes, other
modes are unaffected)
- Implementation of CBR for SILK-only, which means that CBR is now
available in all modes
- The code has been checked on multiple architectures, including an
automated build system on x86 and SPARC (https://mf4.xiph.org/jenkins/)
- A single set of opus_* types throughout the code
- Test vectors are now provided (on the website for size reasons)
- Improved documentation of both the encoder and decoder

Cheers,

	Jean-Marc

On 31/10/11 08:58 PM, Jonathan Rosenberg wrote:
> The chairs would like to initiate a ~3-week WGLC on
> draft-ietf-codec-opus-10
> (http://datatracker.ietf.org/doc/draft-ietf-codec-opus/) which will end
> November 19, coincident with the conclusion of the Taipei IETF.
> 
> Please note that this document has several IPR claims against it:
> 
> Xiph.org:    https://datatracker.ietf.org/ipr/1524/
> Broadcom:    https://datatracker.ietf.org/ipr/1526/
> Skype:       https://datatracker.ietf.org/ipr/1602/
> Qualcomm:    https://datatracker.ietf.org/ipr/1520/
> 
> The IETF cannot take a position on validity of these claims. It is up to
> IETF participants to make their own decisions. Participants are
> encouraged to form an opinion about whether you would like to proceed
> with publication of this document under these declarations, or whether
> you would like to suggest changes, which you should do during the last
> call period. As always, we will proceed based on consensus of the
> working group.
> 
> Thanks,
> Jonathan R.
> 
>

---

From: Jean-Marc Valin <[email-masked]>
Date: Tue, 01 Nov 2011 18:35:12 -0400

I forgot to mention, you can also get the source code directly from:
http://downloads.xiph.org/releases/opus/opus-0.9.8.tar.gz
This package includes the same code as in the draft, but it also
contains MSVC project files, an autoconf build system, and a test suite.
Also, the test vectors are located at
http://opus-codec.org/testvectors/opus_testvectors-draft10.tar.gz
Last thing, for those who want to start using Opus, we now have some API
documentation available at: http://opus-codec.org/docs/

Cheers,

	Jean-Marc

On 01/11/11 02:15 PM, Jean-Marc Valin wrote:
> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec

---

From: Erik Norvell <[email-masked]>
Date: Wed, 9 Nov 2011 13:30:17 +0100

Hi all, 

I am not in any way against the use of patented technology in IETF in general, but I think many people agree that this is not the expected outcome of this WG. The main arguments for starting this WG is that it should produce an unencumbered codec. Still, there is an opportunity in trying to achieve an unencumbered codec by ensuring the largest possible freedom in the implementation. For this reason, I suggest the standard compliance defined in terms of closeness to the test vectors should be removed. In consequence, this will render the specified OPUS encoder and decoder c-code to informative only. 

I would like to note in this context that other standards bodies often apply the criterion of compliance to test vectors not with the only objective of ensuring a defined quality level but also to have means of enforcing the use of the IPRs of a reference implementation. Given the original goal of the codec WG of providing an unencumbered codec, this latter aspect should be irrelevant in the context of our group.

The current situation is that there are no constraints on the encoder other than that it should produce a readable bitstream. It may change the quality in any direction. In the interest of maximizing the implementation freedom, the same should apply for the decoder. This means the only constraint for both the encoder and the decoder would be that they can write and read the bit stream format, respectively.

With regards to the desire to achieve the best possible quality when using the IETF codec, I think that - given that there was in any case no plan to have strong encoder compliance criteria - there is no loss in relaxing the compliance criteria even for the decoder. Rather, it can be foreseen that the market will choose the best performing implementation and we can be sure that especially for the decoding end that it will be in the biggest interest of providers of a product to use an implementation that guarantees the highest quality. It might be that implementers have less incentive of providing the best possible encoders since this will actually not impact the directly perceived quality of the product (it would only hurt someone else on the far end). However, we would in any case have had this potential problem and even here I believe that the market mechanisms will work leading to the use of high-quality implementations. On a general perspective, specifying only the bit stream in a normative fashion will allow to continuously improve both encoders and decoders over the (informative-only) reference implementations of the standard. 

Best regards, 
Erik  

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] 
> On Behalf Of Jonathan Rosenberg
> Sent: den 1 november 2011 01:58
> To: [email-masked]
> Subject: [codec] WGLC #2 for draft-ietf-codec-opus-10
> 
> (http://datatracker.ietf.org/doc/draft-ietf-codec-opus/) 
> which will end November 19, coincident with the conclusion of 
> the Taipei IETF.
> 
> The IETF cannot take a position on validity of these claims. 
> It is up to IETF participants to make their own decisions. 
> Participants are encouraged to form an opinion about whether 
> you would like to proceed with publication of this document 
> under these declarations, or whether you would like to 
> suggest changes, which you should do during the last call 
> period. As always, we will proceed based on consensus of the 
> working group.
> 
> Thanks,
> Jonathan R.
> 
> 
> -- 
> Jonathan D. Rosenberg, Ph.D.                   SkypeID: jdrosen
> Skype Chief Technology Strategist
> [email-masked]                              http://www.skype.com
> [email-masked]                            http://www.jdrosen.net
>

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 09 Nov 2011 15:10:35 -0500

Hi Erik,

With respect to IPR, I'd like to refer you to the statement made by
Monty on the rtcweb mailing list:
http://www.ietf.org/mail-archive/web/rtcweb/current/msg02646.html

Now, I do agree in principle with the idea of "ensuring the largest
possible freedom in the implementation" and that's certainly what we've
done for the encoder. Now, when it comes to the decoder, it's a
different matter because it's what defines the bit-stream. If you remove
any requirements based on test vectors, then the spec becomes
meaningless. Even a G.729 decoder would produce some audio output when
sent an Opus bit-stream. What would prevent someone from saying it
implements the Opus specification?

In general, considering the fact that the decoder is fairly
straightforward with little room for alternate implementations (unlike
the encoder), the space of changes which change the output but which
remain usefully compatible is fairly small and probably not useful for
patent avoidance. But if the issue was ever to arise, the IETF could
easily publish an updated spec with new test vectors that remains
compatible with the original spec.

Cheers,

	Jean-Marc

On 09/11/11 07:30 AM, Erik Norvell wrote:
> With regards to the desire to achieve the best possible quality when using the IETF codec, I think that - given that there was in any case no plan to have strong encoder compliance criteria - there is no loss in relaxing the compliance criteria even for the decoder. Rather, it can be foreseen that the market will choose the best performing implementation and we can be sure that especially for the decoding end that it will be in the biggest interest of providers of a product to use an implementation that guarantees the highest quality. It might be that implementers have less incentive of providing the best possible encoders since this will actually not impact the directly perceived quality of the product (it would only hurt someone else on the far end). However, we would in any case have had this potential problem and even here I believe that the market mechanisms will work leading to the use of high-quality implementations. On a general perspective, specifying only the bit 
st
>  ream in a normative fashion will allow to continuously improve both encoders and decoders over the (informative-only) reference implementations of the standard. 
> 
> Best regards, 
> Erik  
>

---

From: Måns Nilsson <[email-masked]>
Date: Tue, 8 Nov 2011 11:27:35 +0000

1 nov 2011 kl. 01.58 skrev Jonathan Rosenberg:

> The chairs would like to initiate a ~3-week WGLC on draft-ietf-codec-opus-10 

I have read the document and TTBOMK the IPR statements and support it being advanced up the standardisation ladder. 

-- 
Måns Nilsson    Sveriges Radio
+46 8 784 2326      +46 705989668

Två saker verkar vara oundvikliga i miljöer med dåligt chefsskap: 
Det första är horribla beslut,
det andra att det inte går att föra en diskussion om förhållandena.
                             Mats Svegfors 2010

---

From: Erik Norvell <[email-masked]>
Date: Tue, 15 Nov 2011 09:05:16 +0100

Hi Jean-Marc,

Defining the bit stream does not require a normative decoder. It is defined by the encoder and decoder specification, regardless of whether they are normative or informative. Yes, G.729 could decode whatever raw data you input to some nonsense audio, but I cannot see why anyone would do that and call it Opus. It is not more likely than anyone using the G.729 encoder and calling the output an Opus bitstream, which the current setup permits in the same line of argumentation. What will count in the end on the marketplace is the quality of specific encoder and decoder implementations a provider can offer. If this quality is not limited since there is still the implementation freedom it will just be a plus.

One problem with the proposed way of conformance testing with normative decoder becomes already apparent with the separate test vectors for the FIXED and FLOAT implementations. At least for the mono test vectors 3 and 4, the FIXED implementation fails when comparing to the FLOAT test vector and vice versa. How could we define conformance in such a way that two different but definitely valid implementations fail meeting the conformance criterion of the respective other implementation? 

If already the difference between FIXED and FLOAT is enough to make the conformance test fail, it definitely shows that the proposed way of conformance verification needs to be revised.

Best regards,
Erik 

> From: Jean-Marc Valin [mailto:[email-masked]] 
> Sent: den 9 november 2011 21:11
> To: Erik Norvell
> Cc: [email-masked]
> Subject: Re: [codec] WGLC #2 for draft-ietf-codec-opus-10
> 
> Hi Erik,
> 
> With respect to IPR, I'd like to refer you to the statement 
> made by Monty on the rtcweb mailing list:
> Now, I do agree in principle with the idea of "ensuring the 
> largest possible freedom in the implementation" and that's 
> certainly what we've done for the encoder. Now, when it comes 
> to the decoder, it's a different matter because it's what 
> defines the bit-stream. If you remove any requirements based 
> on test vectors, then the spec becomes meaningless. Even a 
> G.729 decoder would produce some audio output when sent an 
> Opus bit-stream. What would prevent someone from saying it 
> straightforward with little room for alternate 
> implementations (unlike the encoder), the space of changes 
> which change the output but which remain usefully compatible 
> is fairly small and probably not useful for patent avoidance. 
> But if the issue was ever to arise, the IETF could easily 
> publish an updated spec with new test vectors that remains 
> > I am not in any way against the use of patented technology 
> in IETF in general, but I think many people agree that this 
> is not the expected outcome of this WG. The main arguments 
> for starting this WG is that it should produce an 
> unencumbered codec. Still, there is an opportunity in trying 
> to achieve an unencumbered codec by ensuring the largest 
> possible freedom in the implementation. For this reason, I 
> suggest the standard compliance defined in terms of closeness 
> to the test vectors should be removed. In consequence, this 
> will render the specified OPUS encoder and decoder c-code to 
> informative only. 
> > 
> > I would like to note in this context that other standards 
> bodies often apply the criterion of compliance to test 
> vectors not with the only objective of ensuring a defined 
> quality level but also to have means of enforcing the use of 
> the IPRs of a reference implementation. Given the original 
> goal of the codec WG of providing an unencumbered codec, this 
> latter aspect should be irrelevant in the context of our group.
> > 
> > The current situation is that there are no constraints on 
> the encoder other than that it should produce a readable 
> bitstream. It may change the quality in any direction. In the 
> interest of maximizing the implementation freedom, the same 
> should apply for the decoder. This means the only constraint 
> for both the encoder and the decoder would be that they can 
> write and read the bit stream format, respectively.
> > 
> > With regards to the desire to achieve the best possible 
> quality when 
> > using the IETF codec, I think that - given that there was 
> in any case 
> > no plan to have strong encoder compliance criteria - there 
> is no loss 
> > in relaxing the compliance criteria even for the decoder. 
> Rather, it 
> > can be foreseen that the market will choose the best performing 
> > implementation and we can be sure that especially for the 
> decoding end 
> > that it will be in the biggest interest of providers of a 
> product to 
> > use an implementation that guarantees the highest quality. 
> It might be 
> > that implementers have less incentive of providing the best 
> possible 
> > encoders since this will actually not impact the directly perceived 
> > quality of the product (it would only hurt someone else on the far 
> > end). However, we would in any case have had this potential problem 
> > and even here I believe that the market mechanisms will 
> work leading 
> > to the use of high-quality implementations. On a general 
> perspective, 
> > specifying only the bit
> st
> >  ream in a normative fashion will allow to continuously 
> improve both encoders and decoders over the 
> (informative-only) reference implementations of the standard. 
> > 
> > Best regards,
> > Erik
> > 
> >> From: [email-masked] [mailto:[email-masked]] On 
> >> Behalf Of Jonathan Rosenberg
> >> which will end November 19, coincident with the conclusion of the 
> >> Taipei IETF.
> >>
> >> Participants are encouraged to form an opinion about whether you 
> >> would like to proceed with publication of this document 
> under these 
> >> declarations, or whether you would like to suggest 
> changes, which you 
> >> should do during the last call period. As always, we will proceed 
> >> based on consensus of the working group.
> >>
> >> Thanks,
> >> Jonathan R.
> >>
> >>
> >> --

---

From: "Benjamin M. Schwartz" <[email-masked]>
Date: Tue, 15 Nov 2011 10:39:02 -0500

On 11/09/2011 07:30 AM, Erik Norvell wrote:
> The current situation is that there are no constraints on the encoder other than that it should produce a readable bitstream. It may change the quality in any direction. In the interest of maximizing the implementation freedom, the same should apply for the decoder.

The above suggestion is impossible as a matter of logic, because our
purpose is interoperability.

If the decoder is normatively specified, then an encoder may be designed
so that the output of a compliant decoder sounds like the original input
audio.  This is our current situation.

If the encoder is normatively specified, then a decoder may be designed so
that its output sounds like the original input audio encoded by a
compliant encoder.

If neither the encoder nor the decoder is normatively specified, then
neither can be designed with any expectation of what sort of output will
be produced.

Without any normative specification, it would be trivial to build a
compliant but non-interoperable encoder-decoder pair.  Such an encoder
would write valid Opus bitstreams, but these bitstreams would only sound
like the original input audio when played through the corresponding
decoder.  Similarly, the decoder would not produce correct audio output
when connected to other encoder implementations.

Of course, this is all old news.  We have already reached consensus on
this issue, which is why it is all explicitly built into the document that
guides our process here:

http://datatracker.ietf.org/doc/draft-ietf-codec-guidelines/?include_text=1

See Sections 4.1 and 4.3.

--Ben

---

From: "Christian Hoene" <[email-masked]>
Date: Tue, 15 Nov 2011 22:01:05 +0100

> One problem with the proposed way of conformance testing with normative
> decoder becomes already apparent with the separate test vectors for the
> FIXED and FLOAT implementations. At least for the mono test vectors 3 and
> 4, the FIXED implementation fails when comparing to the FLOAT test vector
> and vice versa.

[Christian Hoene] Is this a bug that needs to be fixed?

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 16 Nov 2011 06:52:55 +0800

> [Christian Hoene] Is this a bug that needs to be fixed?

No (I checked). It's merely due to the comparison tool being quite
strict, which is also why we decided to provide both float and fixed
testing tools. I've prepared a slide to discuss this topic in more
details during the meeting, so don't worry.

Cheers,

	Jean-Marc

---

From: Erik Norvell <[email-masked]>
Date: Wed, 16 Nov 2011 09:38:01 +0100

Dear Ben,

I am afraid but I strongly disagree with your logic.

Interoperability is given through the definition of the meaning of each bit of the bitstream.  

One way of achieving such a definition is to require
- for an(y) encoder to interoperate with the specified (reference) decoder and vice-versa
- for an(y) decoder to interoperate with the specified (reference) encoder.

While such a definition obviously relies on a reference encoder and a reference decoder, it is by no means necessary to make any of them normative. 

Best regards,
Erik

> From: Benjamin M. Schwartz [mailto:[email-masked]] 
> Sent: den 15 november 2011 16:39
> To: Erik Norvell
> Cc: [email-masked]
> should apply for the decoder.
> 
> The above suggestion is impossible as a matter of logic, 
> because our purpose is interoperability.
> 
> If the decoder is normatively specified, then an encoder may 
> be designed so that the output of a compliant decoder sounds 
> like the original input audio.  This is our current situation.
> 
> If the encoder is normatively specified, then a decoder may 
> be designed so that its output sounds like the original input 
> audio encoded by a compliant encoder.
> 
> If neither the encoder nor the decoder is normatively 
> specified, then neither can be designed with any expectation 
> of what sort of output will be produced.
> 
> Without any normative specification, it would be trivial to 
> build a compliant but non-interoperable encoder-decoder pair. 
>  Such an encoder would write valid Opus bitstreams, but these 
> bitstreams would only sound like the original input audio 
> when played through the corresponding decoder.  Similarly, 
> the decoder would not produce correct audio output when 
> connected to other encoder implementations.
> 
> Of course, this is all old news.  We have already reached 
> consensus on this issue, which is why it is all explicitly 
> built into the document that guides our process here:
>

---

From: "Christian Hoene" <[email-masked]>
Date: Wed, 16 Nov 2011 11:37:12 +0100

Hi,

I would suggest to split two things: 

First, we need a notion for standard conformance because of the legal IPR
requirements (can be more relaxing). 
Second, we need some tests in respect to interoperability (should be more
straight).

 Both tests need to fulfill different requirements.

Also, I like Erik's idea to allow for algorithms to replace patented stuff
in the codec - even if the quality is a bit worse. 

With best regards,

 Christian Hoene

> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Erik Norvell
> Sent: Wednesday, November 16, 2011 9:38 AM
> To: [email-masked]
> Cc: [email-masked]
> Interoperability is given through the definition of the meaning of each
bit of
> the bitstream.
> While such a definition obviously relies on a reference encoder and a
> reference decoder, it is by no means necessary to make any of them
> normative.
> 
> Best regards,
> Erik
>

---

From: Ron <[email-masked]>
Date: Thu, 17 Nov 2011 05:48:14 +1030

>  Both tests need to fulfill different requirements.

I don't see these as different requirements at all.

The IPR in this case is not being used to protect a revenue stream garnered
from unwitting users of the technology - it is being used to protect the
interoperability guarantees that can be offered to end users.

Separating these two things, and watering down the protection offered by
the IPR does not offer any benefit to users, only to implementers who might
be tempted to compromise on interoperability.  Possibly for 'practical'
reasons, and possibly for 'commercial' reasons of their own - but their
reasons don't matter, only the end effect, a degraded user experience, is
the important consideration (to avoid) here.

There is one requirement.  Good interoperability.  IPR grant conformance
is the only binding method we have of ensuring this.  As heretical as it
may seem to the FRAND profit seekers - the real profit to be made here
from patents is an enhanced and assured user experience.

Of course, since I don't actually own any of the IPR, I can't speak for
the people who do, or for their motives - but this is what I see as their
most valuable gift to us, above and beyond the actual technology itself.

> in the codec - even if the quality is a bit worse.

I think this is a fallacy.  It was argued that this extra freedom might
allow people to produce a 'better' decoder.  That seems like nonsense to
me, the decoder is never going to be able to recover information that
simply isn't there - the best it can do is recover all of it with tight
fidelity.  Which is what the current conformance test tries to measure.

It was also argued that this freedom might allow people to produce a
worse decoder.  That seems like something we should strongly avoid.
There seem to be only three reasons to do this:

 - Save cycles on underpowered systems.
   But the requirements already include assessing complexity guarantees
   are within reasonable bounds.

 - Sidestep the IPR to produce systems that are not fully interoperable
   but can still claim (some semblance of) conformance.

 - Provide distortive 'enhancements' that some users may assess as being
   pleasing on a first brief listen, but which reduce the real fidelity
   of the output.   (eg. simulate mp3 distortion for the people who
   listening tests actually prove like that more than lossless original)

Both those latter cases carry a danger of sounding like crap if future
(real) enhancements are made to the encoder which actually does increase
the real fidelity and information content of the encoded signal.

The likelihood of such improvements being made to the encoder is high,
so I think this is a real jeopardy which we should not inflict upon
end users or limit future encoder work by.

The decoder should decode the bitstream accurately.  If it cannot, then
it should not claim to decode Opus.  People are always free to sidestep
the IPR and make something that sounds worse.  They just shouldn't be
encouraged or allowed to then also misrepresent that as Opus and mislead
the users that this group set out to serve.  As some of the liaisons
have reminded us, we're here because we wanted to *raise* the bar, not
to just continue Business As Usual in patented codec proliferation.
So we should do just that.  And make everyone happy :)

IMO

 Ron

---

From: Koen Vos <[email-masked]>
Date: Thu, 17 Nov 2011 03:48:30 +0100 (CET)

(Changed the thread subject)

Ron wrote:
> There is one requirement.  Good interoperability.

I agree completely.  All compliant future encoders and decoders should work well together.  This also prevents embrace-and-extend tactics.

However, within the space of guaranteed interop, there are benefits to being lenient:

1. The decoder may be improved for specific use cases, by finding a different tradeoff between quality, delay, complexity, etc.  For instance in a non-real time application it may help to have a resampler with higher quality, at the cost of higher delay and complexity.

2. A limited decoder may be better.  For instance, an Opus-to-PSTN gateway only needs a decoder to generate 8 kHz output signals.  Allowing such a decoder enables optimizations in terms of memory and CPU.  (Of course the decoder would still have to decode any Opus bitstream, including one with more than NB content.)  

3. The current test disqualifies a huge space of decoders that sound (virtually) identical.  Simply getting the sign wrong of the output signal shouldn't make a company liable for a patent lawsuit.

A non-goal for the WG, in my opinion, is to ensure that all users get the quality they are deemed to deserve.  While nobody wants decoders that sound like crap, few users will stick with an application that plays out a jingle whenever an Opus bitstream is received.  And we already allow crappy _encoders_.  I prefer the market place to weed out crap over threats of litigation.

While there's no perfect solution, I think Jean-Marc and I will come up with a compare tool that provides a small -but useful- amount of freedom to change the decoder without breaking the interop guarantee.

koen.

----- Original Message -----
From: "Ron" <[email-masked]>
To: [email-masked]
Sent: Thursday, November 17, 2011 3:18:14 AM
Subject: Re: [codec] WGLC #2 for draft-ietf-codec-opus-10

IMO

 Ron

---

From: "Christian Hoene" <[email-masked]>
Date: Thu, 17 Nov 2011 10:32:58 +0100

Hi Ron,

> interoperability guarantees that can be offered to end users.

[Christian Hoene] Sorry, this would be the first standard (that I am aware
of) that you can be suited for a standard implementation that is not
interoperability - no matter whether it is intentionally or not just a bug.
Sorry, but if having bugs in software implementations is not allowed we will
have any software at all. Each complex software product has bugs. 

If you can be charged for every bug in your Opus implementation that limits
interoperability, then using Opus will become more expensive as there might
be a risk that you have to pay royalties. Insuring this risk is costly.

Also, too strict rules would limit any technology progress. This should be
avoided in any case.

With best regards,

 Christian

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 17 Nov 2011 17:57:19 +0800

> be a risk that you have to pay royalties. Insuring this risk is costly.

Please keep in mind that the conformance test is restricted to test
vectors, so there is no risk. Either you pass or you fail and you know
that very early on. If it turns out that your decoder fails to correctly
decode something other than the test vectors, then you're still
"compliant" and the IPR licenses still apply. So regardless of how wide
or narrow the compliance test is, there will not be any uncertainty
because it will always only apply to the test vectors.

That being said, Koen and I are working on the actual test and we should
soon have something that is worth showing.

	Jean-Marc

---

From: Erik Norvell <[email-masked]>
Date: Thu, 17 Nov 2011 14:20:56 +0100

Ron,

You are talking about interoperability guarantees and that the IPRs in the codec would serve the purpose to offer these guarantees rather than to protect a revenue stream garnered from unwitting users of the technology. 
 
I don't believe that the threat of potential litigation is the only way to maintain interoperability between implementations. Further, I would regard any discussion about the purpose of IPRs in OPUS as useless. In general terms it is however an undisputable fact that OPUS is not as unencumbered as it was the original desire of the group. Hence, please respect that every implementer may want to make his own balance of the quality of his product using OPUS on the one hand and the licensing cost/risk stemming from the encumbrance of the codec on the other hand. With this background I simply request to respect that implementers may want to have their implementation freedom.

Concerning interoperability itself you seem to confuse this term with a guarantee to offer the best possible quality. I don't think that we can go that far. It is definitely not logical given that the encoder would in any case only be informative. So even if you are using a bit exact decoder implementation you have no quality guarantees because the encoder might be crappy. (In fact, you don't even have interoperability guaranteed.) Hence, it makes no sense to be extremely relaxed on the encoding end and at the same time to be very restrictive on the decoding end. A reasonable definition of the term interoperability would be required first. In my view interoperability of a given encoder/decoder pair can only mean that the decoder can produce a 'useful output' from any bitstreams produced by the encoder operated in any possible setting. The term useful output should relate to intelligibility, but quality expectations should not be associated with interoperability.

Finally, you seem certain that decoder enhancements are not possible while encoder enhancements would be. I think there are numerous examples where the decoding end of existing (speech) codecs were enhanced. Such enhancements would be disqualified by a too narrow conformance test even though the decoder should be considered interoperable.
 
Best regards
Erik 

> On Behalf Of Ron
> Sent: den 16 november 2011 20:18
> To: [email-masked]
> > First, we need a notion for standard conformance because of 
> the legal 
> > IPR requirements (can be more relaxing).
> > Second, we need some tests in respect to interoperability 
> (should be 
> > more straight).
> > 
> The IPR in this case is not being used to protect a revenue 
> stream garnered from unwitting users of the technology - it 
> is being used to protect the interoperability guarantees that 
> can be offered to end users.
> 
> Separating these two things, and watering down the protection 
> offered by the IPR does not offer any benefit to users, only 
> to implementers who might be tempted to compromise on 
> interoperability.  Possibly for 'practical'
> reasons, and possibly for 'commercial' reasons of their own - 
> but their reasons don't matter, only the end effect, a 
> degraded user experience, is the important consideration (to 
> avoid) here.
> 
> There is one requirement.  Good interoperability.  IPR grant 
> conformance is the only binding method we have of ensuring 
> this.  As heretical as it may seem to the FRAND profit 
> seekers - the real profit to be made here from patents is an 
> enhanced and assured user experience.
> 
> Of course, since I don't actually own any of the IPR, I can't 
> speak for the people who do, or for their motives - but this 
> is what I see as their most valuable gift to us, above and 
> beyond the actual technology itself.
> 
> 
> > Also, I like Erik's idea to allow for algorithms to replace 
> patented 
> > stuff in the codec - even if the quality is a bit worse.
> 
> I think this is a fallacy.  It was argued that this extra 
> freedom might allow people to produce a 'better' decoder.  
> That seems like nonsense to me, the decoder is never going to 
> be able to recover information that simply isn't there - the 
> best it can do is recover all of it with tight fidelity.  
> Which is what the current conformance test tries to measure.
> 
> It was also argued that this freedom might allow people to 
> produce a worse decoder.  That seems like something we should 
> strongly avoid.
> (real) enhancements are made to the encoder which actually 
> does increase the real fidelity and information content of 
> the encoded signal.
> 
> The likelihood of such improvements being made to the encoder 
> is high, so I think this is a real jeopardy which we should 
> not inflict upon end users or limit future encoder work by.
> 
> The decoder should decode the bitstream accurately.  If it 
> cannot, then it should not claim to decode Opus.  People are 
> always free to sidestep the IPR and make something that 
> sounds worse.  They just shouldn't be encouraged or allowed 
> to then also misrepresent that as Opus and mislead the users 
> that this group set out to serve.  As some of the liaisons 
> have reminded us, we're here because we wanted to *raise* the 
> bar, not to just continue Business As Usual in patented codec 
> proliferation.

---

From: Erik Norvell <[email-masked]>
Date: Thu, 17 Nov 2011 14:42:22 +0100

Hi Koen,

I agree with your arguments. When it comes to the solution I believe we can allow even a large amount of freedom and still show interoperability.

Best regards,
Erik 

> On Behalf Of Koen Vos
> Sent: den 17 november 2011 03:49
> To: Ron
> Cc: [email-masked]
> Subject: [codec] Compare tool sensitivity
> 
> I agree completely.  All compliant future encoders and 
> decoders should work well together.  This also prevents 
> embrace-and-extend tactics.
> 
> 1. The decoder may be improved for specific use cases, by 
> finding a different tradeoff between quality, delay, 
> complexity, etc.  For instance in a non-real time application 
> it may help to have a resampler with higher quality, at the 
> cost of higher delay and complexity.
> 
> 2. A limited decoder may be better.  For instance, an 
> Opus-to-PSTN gateway only needs a decoder to generate 8 kHz 
> output signals.  Allowing such a decoder enables 
> optimizations in terms of memory and CPU.  (Of course the 
> decoder would still have to decode any Opus bitstream, 
> including one with more than NB content.)  
> 
> 3. The current test disqualifies a huge space of decoders 
> that sound (virtually) identical.  Simply getting the sign 
> wrong of the output signal shouldn't make a company liable 
> for a patent lawsuit.
> 
> A non-goal for the WG, in my opinion, is to ensure that all 
> users get the quality they are deemed to deserve.  While 
> nobody wants decoders that sound like crap, few users will 
> stick with an application that plays out a jingle whenever an 
> Opus bitstream is received.  And we already allow crappy 
> _encoders_.  I prefer the market place to weed out crap over 
> threats of litigation.
> 
> While there's no perfect solution, I think Jean-Marc and I 
> will come up with a compare tool that provides a small -but 
> useful- amount of freedom to change the decoder without 
> breaking the interop guarantee.
> 
> koen.
> 
>

---

From: Ron <[email-masked]>
Date: Fri, 18 Nov 2011 01:13:36 +1030

Hi Erik,

> Finally, you seem certain that decoder enhancements are not possible while encoder enhancements would be. I think there are numerous examples where the decoding end of existing (speech) codecs were…

I think you give an excellent meta-example here of the dangers inherent
to "enhanced decoding".  You seem to have extrapolated quite broadly
from the bitstream I presented you, and I declare your output to be
clearly non-conformant to the reference :)

I didn't say anything about guaranteeing absolute quality.  That's a
function of the encoder.  In order to be able to optimise that function
though we need a decoder that is deterministic.

The reference encoder is Free, both in licence and royalty terms, and
is basically transparent over a wide range of bitrates.  I don't think
we need to worry about the people who might write crappy encoders.
Nobody will use them, or nobody will listen to their content if they
do.  That's quite a different issue from people selling hardware with
crap decoders in their firmware, or people publishing listening tests
with the crap but conformant decoder they wrote over the weekend.

If you really can't live without distorting 'decoder enhancements' like
bass boost or whatever, you can always add that as a post processing
stage.  Defining compliance in terms of "my mom thinks it sounds ok"
or something similarly handwavy is not good engineering.

I agree with Koen that we should not preclude things which reasonable
people, with an interest in the codec actually succeeding, would
consider an acceptable implementation.  I strongly disagree with the
attempts to define "I could understand 5 out of the 7 words" as being
within that bound of acceptable.  I'm pretty sure you'll still be able
to do that without treading on anyone's IPR.

So let's just let Koen and Jean-Marc find a middle ground, shall we.
I'm pretty sure they'll do as good a job on that as they have with
the codec to date.

Faithfully

 Ron

---

From: "Michael Ramalho (mramalho)" <[email-masked]>
Date: Thu, 17 Nov 2011 09:55:46 -0600

All,

Re: " Further, I would regard any discussion about the purpose of IPRs
in OPUS as useless. In general terms it is however an undisputable fact
that OPUS is not as unencumbered as it was the original desire of the
group."

Sorry for dropping in late on this thread, and my comment is not
specific to Ron's comment (Ron, your email is simply convenient - don't
read my reply to your email to be referencing you specifically).

I think Ron's comment is particularly interesting given Skype's IPR
statement (Skype:       https://datatracker.ietf.org/ipr/1602/ ).

Assuming that MSFT does close on the Skype acquisition, the royalty-free
language appears to me to come with the condition that the receiving
entity agrees to not sue MSFT for ANY MSFT patent (even those patents
NOT RELATED to OPUS).

I am not a lawyer, but here is the language:

<quote_from_URL_above>
Skype will retain the right to terminate the license and assert its
patents (including the right to claim past royalties) against any
licensee that asserts or whose affiliate asserts >>>ANY<<< patent
(either directly or indirectly) against Skype or any of Skype's
affiliates or >>>SUCCESSORS IN TITLE<<<.
</quote_from_URL_above>

That, to some companies, may be a really big legal concession to make
for use of an audio codec.

I am not a lawyer, but I did stay at a Holiday Inn Express last night
(joke only to be understood by US attendees) ...

Michael Ramalho, Ph.D.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Erik Norvell
Sent: Thursday, November 17, 2011 8:21 AM
To: Ron; [email-masked]
Subject: Re: [codec] WGLC #2 for draft-ietf-codec-opus-10

Ron,

You are talking about interoperability guarantees and that the IPRs in
the codec would serve the purpose to offer these guarantees rather than
to protect a revenue stream garnered from unwitting users of the
technology. 
 
I don't believe that the threat of potential litigation is the only way
to maintain interoperability between implementations. Further, I would
regard any discussion about the purpose of IPRs in OPUS as useless. In
general terms it is however an undisputable fact that OPUS is not as
unencumbered as it was the original desire of the group. Hence, please
respect that every implementer may want to make his own balance of the
quality of his product using OPUS on the one hand and the licensing
cost/risk stemming from the encumbrance of the codec on the other hand.
With this background I simply request to respect that implementers may
want to have their implementation freedom.

Concerning interoperability itself you seem to confuse this term with a
guarantee to offer the best possible quality. I don't think that we can
go that far. It is definitely not logical given that the encoder would
in any case only be informative. So even if you are using a bit exact
decoder implementation you have no quality guarantees because the
encoder might be crappy. (In fact, you don't even have interoperability
guaranteed.) Hence, it makes no sense to be extremely relaxed on the
encoding end and at the same time to be very restrictive on the decoding
end. A reasonable definition of the term interoperability would be
required first. In my view interoperability of a given encoder/decoder
pair can only mean that the decoder can produce a 'useful output' from
any bitstreams produced by the encoder operated in any possible setting.
The term useful output should relate to intelligibility, but quality
expectations should not be associated with interoperability.

Finally, you seem certain that decoder enhancements are not possible
while encoder enhancements would be. I think there are numerous examples
where the decoding end of existing (speech) codecs were enhanced. Such
enhancements would be disqualified by a too narrow conformance test even
though the decoder should be considered interoperable.
 
Best regards
Erik

---

From: Ralph Giles <[email-masked]>
Date: Fri, 18 Nov 2011 13:54:06 -0800

Please find attached a patch fixes some editorial issues with opus-10 draft.

 -r

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Fri, 18 Nov 2011 14:12:39 -0800

> Please find attached a patch fixes some editorial issues with opus-10 draft.

Thanks. I'd already corrected a couple of those locally, but I'll make 
sure the rest get applied.

---

From: Ralph Giles <[email-masked]>
Date: Fri, 18 Nov 2011 16:17:31 -0800

Here are two more small patches, fixing issues with the API
documentation in the implementation headers.

 -r