Subject: [codec] Authors and disclosures, regarding Last Call: <draft-ietf-codec-opus-12.txt>

From: Joe Baker <[email-masked]>
Date: Tue, 1 May 2012 19:33:38 +0000

The 'possible royalty/fee' in some the of IPR disclosures is unacceptable
for my applications.  After a rather time consuming review my team has
advised that I do not need to be concerned about the currently disclosed
non-RF patents but we were unsure about undisclosed applications.

Fortunately the majority of the codec components have been disclosed
to the public for many years in earlier drafts and open source releases
by the draft authors.  I assume that the closed source usage of these
components goes back even further.

Can the authors of the draft please indicate when the major components
of the codec were first offered to the public, in any form, so we can
better gauge the risk of substantial components being covered by
undisclosed applications?

If there are any royalty bearing components in the codec they must be
removed if the the codec is to be usable for me.  Unfortunately, the
Huawei disclosures are so vague that they can hardly be considered
disclosures because they claim unpublished applications but do not
specify the subject matter.  These were only reported long after the
last call on the draft which might complicate removing anything in
response to them.  With such late and non-specific disclosures how
are we to express our preference to avoid non-RF encumbered
technologies?

I have audited the entire WG discussion archives, meeting minutes and
source code repositories and was not able to find any technical
contributions by anyone from Huawei.  I assume that the IETF policy
does not permit working group participants to follow the working group
and secretly patent the working group's inventions as their own?

It would be helpful if anyone here is aware that their specific technical
contributions will be covered by currently undisclosed applications which
may not have royalty free licensing if they would point them out to the
group so that we can decide if we should remove those parts.

I did notice one potentially important discrepancy.  There are many
more authors of the draft which are not are listed on it as authors. Are
these people treated equally to the listed authors in terms of the IETF
IPR disclosure requirements?  I believe some of them have never posted to
this list although I may have failed to match some of the names.

+- Descriptive text ---------------------+
|         Author           Lines  Pct Msg|
+----------------------------------------+
|Timothy B. Terriberry      4803  60%  Y |
|Jean-Marc Valin            2428  30%  Y |
|Koen Vos                    473   6%  Y |
|Gregory Maxwell             141   2%  Y |
|Kat Walsh                   112   1%  Y |
|Benjamin M. Schwartz         48  <1%  Y |
|Ralph Giles                  10  <1%  Y |
|Total--------------------- 8015 100%  - |
+----------------------------------------+

+- Formal specification -----------------+
|         Author           Lines  Pct Msg|
+----------------------------------------+
|Jean-Marc Valin           21152  42%  Y |
|Gregory Maxwell           20497  41%  Y |
|Koen Vos                   4111   8%  Y |
|Timothy B. Terriberry      3057   6%  Y |
|Ralph Giles                1329   3%  Y |
|John Ridges                  24  <1%  Y |
|Benjamin M. Schwartz         20  <1%  Y |
|Karsten Vandborg Sorensen    12  <1%  Y |
|Wessel Lubberhuizen           7  <1%  N |
|Thorvald Natvig               7  <1%  Y |
|Alfred E. Heggestad           2  <1%  Y |
|Christian Hoene               2  <1%  Y |
|Benjamin Jemlich              2  <1%  N |
|David Schleef                 1  <1%  N |
|Kat Walsh                     1  <1%  Y |
|Total-------------------- 50224 100%  - |
+----------------------------------------+

The file AUTHORS in the repository also lists
Soren Skak Jensen and the file COPYING lists
Mark Borgerding and Erik de Castro Lopo but
I am not able to find any revisions or list
posts by them.

Thank you.

---

From: Jean-Marc Valin <[email-masked]>
Date: Tue, 01 May 2012 20:16:36 -0400

> undisclosed applications?

Back when we had the first codec BoF (Jul 2009, Stockholm), both SILK
and CELT were already being deployed, and though they were further
modified after that point, the "core technology" hasn't changed that
much. As far as I can tell, SILK was "first offered to the public",
early 2009
(http://www.skypejournal.com/2009/02/silk-skype-new-audio-codec-sets-new.html),
while CELT was first released late 2007. Since January 2011, the CELT
layer has been completely frozen and the SILK layer only got minor fixes.

> group so that we can decide if we should remove those parts.

As far as I know, the contributions of Huawei have been limited to
providing feedback, comments, review, and requirements rather than code,
algorithms, or design.

> this list although I may have failed to match some of the names.

As far as I know, the IETF only allows about 5 authors to be listed at
the top of the draft, even though many more people may have contributed
to the document. You are correct that some people listed in the git logs
have never posted. There are two reasons for this. Some people have
merely sent us small bug fixes, draft typos, comment fixes and the like.
In the case of Mark Borgerding and Erik de Castro, they are authors of
pre-existing BSD code that we decided to include in Opus. This code
implements an FFT and fast float-integer conversion functions, so
nothing I'd be too worried about when it comes to IPR. As for Soren Skak
Jensen, he was a Skype employee, so his contributions are already covered.

On top of the names you found, there's actually one more significant
contribution. This was made by Raymond Chen on the mailing list. Because
I completely re-implemented that contribution in C, his name does not
show up in the git logs. However, Broadcom has properly disclosed IPR
and licensed it under the same license as the one Xiph.Org used.

Hope that answers your questions.

	Jean-Marc

---

From: Gregory Maxwell <[email-masked]>
Date: Tue, 1 May 2012 17:22:53 -0700

For the sake of historic accuracy, 

> |Timothy B. Terriberry      3057   6%  Y |

Even with a SCM it can be tricky to sort out who did what.

I didn't write half the codec, in particular git misattributes about 15,000
lines of the silk code to me because I corrected formatting issues. Much of
that should be assigned to the silk team. My contribution to the silk layer
mostly was finding bugs, removing code, dependencies on undefined behavior,
etc.

Similarly, a few hundred lines of Tim's code is misattributed to me because
he and the compiler have a disagreement about the importance of parentheses
around weakly binding operators like <<. At some point I reviewed and fixed
all the warnings. The numbers also look like they might include the testing
code which I wrote most of but isn't a part of the draft.

These sorts of line counting metrics don't rightfully attribute many of the
contributions which didn't directly change the implementation or the text--
like the fact that Tim wrote a nearly complete second implementation of the
codec, significant software testing by myself, psycho-acoustic testing plus
tuning by Monty, Igor C., Kat, and others, or Ben Schwartz's consistent and
thoughtful shed-painting of the entire codec design (he convinced Jean-Marc
that the mode switching could be made glitchless without boiling the oceans
by figuring out how to do it with minimal modifications and writing running
code for it, though that wasn't the specific code that made it in).

---

From: "Kevin P. Fleming" <[email-masked]>
Date: Wed, 02 May 2012 07:44:37 -0500

> layer has been completely frozen and the SILK layer only got minor fixes.

While it's true that SILK was available at that time, it was only 
available in binary form embedded inside products delivered by Skype, 
and those products heavily obscure and encrypt their behavior. It seems 
unlikely that anyone who had IPR in any of its methods would have been 
able to determine that SILK was potentially infringing before its source 
code was made available. I would consider the 'first public disclosure' 
date to be the first time the source code was published.

-- 
Kevin P. Fleming
Digium, Inc. | Director of Software Technologies
Jabber: [email-masked] | SIP: [email-masked] | Skype: kpfleming
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at www.digium.com & www.asterisk.org

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 02 May 2012 09:39:58 -0400

> date to be the first time the source code was published.

As far as I can tell, the actual source code was released as open source
in March 2010, though it may have been available before that to
developers willing to sign some agreement. Also, fortunately prior art
gets established based on the time Silk was available even in obfuscated
form.

	Jean-Marc

---

From: "Kevin P. Fleming" <[email-masked]>
Date: Wed, 02 May 2012 09:39:37 -0500

> gets established based on the time Silk was available even in obfuscated

In the legal sense that is absolutely true, sure. The OP's comment 
though was that he wasn't sure whether potential IPR holders had had 
enough time to determine whether SILK/OPUS may be infringing on their 
IPR. In that case, the extra year of SILK availability in binary-only 
obfuscated form likely won't make much difference.

Also, we were one of the licensees of SILK prior to the source code 
publication (although we never released any products using it) and the 
codec was only offered to us in binary form.

---

From: Koen Vos <[email-masked]>
Date: Thu, 3 May 2012 22:57:19 +0000

Jean-Marc wrote:
> Also, fortunately prior art gets established based on the time Silk was available even in obfuscated
form.

The first incarnation of SILK was included in the Skype application in January 2009.  While there are some algorithmic differences with SILK in Opus, most of the methods are the same.  You can download the old SILK from the Skype developer web site, if you wanted to compare the two.

best,
koen.

________________________________________
From: [email-masked] [[email-masked]] on behalf of Jean-Marc Valin [[email-masked]]
Sent: Wednesday, May 02, 2012 6:39 AM
To: Kevin P. Fleming
Cc: [email-masked]
Subject: Re: [codec] Authors and disclosures, regarding Last Call: <draft-ietf-codec-opus-12.txt>

        Jean-Marc
_______________________________________________
codec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/codec