Subject: [codec] Royalty Free codec standards -- don't settle for less

From: Rob Glidden <[email-masked]>
Date: Tue, 10 Nov 2009 17:22:20 -0800

Here is my view, perhaps you share it, perhaps you don't.

What the world needs now is royalty-free, standardized codecs.  This is 
critical to the future of the Web, and the progress the Internet has 
brought to the world, and will bring to the world.

Video, audio, transport, the whole thing.  Evaluated, vetted for 
patents. Under an appropriate, responsible and complete royalty free 
process.  No less.

IETF, ITU, and ISO/MPEG should all get going on this important activity 
-- after all why shouldn't all of these organizations include this as 
core to their mission.

I have, and no doubt you have too, seen countless explanations why this 
should not, could not, will not, rather not, might not, or can not 
happen.  Some well meaning and sincere, some from vested interests.  
There are too many "powerful" interests against it.  "Important" 
commercial interests are ambivalent.  It is too hard "legally" or 
"politically" or "technically".  It is just too confusing to think 
through.  There is no longer a critical mass that cares enough about 
keeping the future of the Open Internet open and royalty free.  The well 
meaning are ignorant, or naive.  Etc.

Don't settle.  Take the issue of royalty free, standardized codecs all 
the way to the top of these organizations. Do what it takes.  If it 
requires new organizations, start them. It it requires revised 
processes, revise them. This is the spirit that built the Web and the 
Internet, this is the spirit that is its lifeblood, and this is the 
spirit that needs to be at the heart of its future.

Don't settle.  Don't let those who have tried hard already, or have only 
half-heartedly tried, justify the status quo or their half-heartedness.  
Encourage them to focus on how to take the next steps.  Don't let 
convenient "interpretations" of standards processes be an excuse for 
never starting, never finishing, or never setting up processes that will 
work.  Need more legal background?  Find it. More technical information? 
Get it.

Don't settle.  The world has plenty of patent-encumbered media 
standards, plenty of proprietary solutions, and plenty of standards in 
other domains that have figured out how to deliver royalty free.  But 
the world does not have enough royalty-free codec standards, so this is 
the task that needs to be addressed.

Rob

---

From: Rob Glidden <[email-masked]>
Date: Tue, 10 Nov 2009 19:07:24 -0800

Eric:

Perhaps you would be willing to elaborate?

Rob

Eric Burger wrote:
> If I'm not mistaken, do you not have legal training? What you ask for 
> is just not possible with our legal system in the U.S. or the WIPO, 
> either.
>
> I take that back, and this is why I rewrote the draft charter to place 
> a preference on technologies that are at least 20 years old. That 
> guarantees that any extant IPR has expired, even if we are not aware 
> of it.
>
> Rather than asking for perfection, I think the industry will be OK 
> with good enough. Unless your goal is to kill this work. In that case, 
> rallying people for royalty free only will ensure the engineer's 
> companies will prevent them from doing any work.
>
>
>
> On Nov 11, 2009, at 10:22 AM, Rob Glidden wrote:
>
>> What the world needs now is royalty-free, standardized codecs.  This 
>> is critical to the future of the Web, and the progress the Internet 
>> has brought to the world, and will bring to the world.
>>
>> IETF, ITU, and ISO/MPEG should all get going on this important 
>> activity -- after all why shouldn't all of these organizations 
>> include this as core to their mission.
>>
>> I have, and no doubt you have too, seen countless explanations why 
>> this should not, could not, will not, rather not, might not, or can 
>> not happen.  Some well meaning and sincere, some from vested 
>> interests.  There are too many "powerful" interests against it.  
>> "Important" commercial interests are ambivalent.  It is too hard 
>> "legally" or "politically" or "technically".  It is just too 
>> confusing to think through.  There is no longer a critical mass that 
>> cares enough about keeping the future of the Open Internet open and 
>> royalty free.  The well meaning are ignorant, or naive.  Etc.
>>
>> Don't settle.  Take the issue of royalty free, standardized codecs 
>> all the way to the top of these organizations. Do what it takes.  If 
>> it requires new organizations, start them. It it requires revised 
>> Don't settle.  Don't let those who have tried hard already, or have 
>> only half-heartedly tried, justify the status quo or their 
>> half-heartedness.  Encourage them to focus on how to take the next 
>> steps.  Don't let convenient "interpretations" of standards processes 
>> be an excuse for never starting, never finishing, or never setting up 
>> processes that will work.  Need more legal background?  Find it. More 
>> technical information? Get it.
>>
>> standards, plenty of proprietary solutions, and plenty of standards 
>> in other domains that have figured out how to deliver royalty free.  
>> But the world does not have enough royalty-free codec standards, so 
>> this is the task that needs to be addressed.
>>
>> Rob
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> codec mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/codec
>
>

---

From: Koen Vos <[email-masked]>
Date: Tue, 10 Nov 2009 19:26:50 -0800

1. Technological progress saturates.
2. Patents expire.
Therefore, the performance advantage of royalty-bearing standards  
diminishes with time, and high-quality, royalty-free standards are  
unavoidable. I'm convinced that today we have reached this point of  
commoditization for audio and speech coding technology.

koen.

Quoting Rob Glidden:
> processes, revise them. This is the spirit that built the Web and  
> the Internet, this is the spirit that is its lifeblood, and this is  
> the spirit that needs to be at the heart of its future.
>
> steps.  Don't let convenient "interpretations" of standards  
> processes be an excuse for never starting, never finishing, or never  
> setting up processes that will work.  Need more legal background?   
> Find it. More technical information? Get it.
>

---

From: Stephan Wenger <[email-masked]>
Date: Wed, 11 Nov 2009 13:15:30 +0900

The existence of at least a dozend of projects in the speech coding field
today suggests to me that we have not yet reached the point of technology
progress saturation in this field.  Other that this minor point, I agree.

Stephan

---

From: Eric Burger <[email-masked]>
Date: Wed, 11 Nov 2009 14:17:22 +0900

If the goal is to ensure we do not produce anything for ten years,  
sure, let us form task groups with tens of members to research all of  
the patent literature, technical literature, and press materials;  
catalog the information; and come to written, published, and peer- 
reviewed expert opinion, we MIGHT have a near guarantee of a Royalty  
Free codec. That will cost the industry at least USD 75M (average 50  
people, average cost of US$ 150,000/year, 10 years). Since most of the  
participants have day jobs or are consultants paid by corporations,  
well run companies would say $10M in royalties is a bargain. With  
these numbers, the intellectual property system works, and the world  
is better off paying inventors royalties.

If the goal is to have a reasonable chance of producing a royalty free  
codec in the next three years, then I would offer the proposed charter  
has a high chance of success. Use codecs that are known to have no  
asserted IPR, use codecs that are older than 20 years old, and use  
codecs that have been out for a few years that no one has asserted  
rights against, and we have a reasonable expectation the result will  
be royalty free.

On Nov 11, 2009, at 12:07 PM, Rob Glidden wrote:

>> If I'm not mistaken, do you not have legal training? What you ask  
>> for is just not possible with our legal system in the U.S. or the  
>> WIPO, either.
>>
>> I take that back, and this is why I rewrote the draft charter to  
>> place a preference on technologies that are at least 20 years old.  
>> That guarantees that any extant IPR has expired, even if we are not  
>> aware of it.
>>
>> with good enough. Unless your goal is to kill this work. In that  
>> case, rallying people for royalty free only will ensure the  
>> engineer's companies will prevent them from doing any work.
>>
>>
>>
>>> What the world needs now is royalty-free, standardized codecs.   
>>> This is critical to the future of the Web, and the progress the  
>>> Internet has brought to the world, and will bring to the world.
>>>
>>> patents. Under an appropriate, responsible and complete royalty  
>>> free process.  No less.
>>>
>>> this should not, could not, will not, rather not, might not, or  
>>> can not happen.  Some well meaning and sincere, some from vested  
>>> confusing to think through.  There is no longer a critical mass  
>>> that cares enough about keeping the future of the Open Internet  
>>> open and royalty free.  The well meaning are ignorant, or naive.   
>>> Etc.
>>>
>>> all the way to the top of these organizations. Do what it takes.   
>>> If it requires new organizations, start them. It it requires  
>>> revised processes, revise them. This is the spirit that built the  
>>> Web and the Internet, this is the spirit that is its lifeblood,  
>>> and this is the spirit that needs to be at the heart of its future.
>>>
>>> Don't settle.  Don't let those who have tried hard already, or  
>>> have only half-heartedly tried, justify the status quo or their  
>>> processes be an excuse for never starting, never finishing, or  
>>> never setting up processes that will work.  Need more legal  
>>> background?  Find it. More technical information? Get it.
>>>
>>> standards, plenty of proprietary solutions, and plenty of  
>>> standards in other domains that have figured out how to deliver  
>>> royalty free.  But the world does not have enough royalty-free  
>>> codec standards, so this is the task that needs to be addressed.
>>>
>>> Rob
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>

---

From: Eric Burger <[email-masked]>
Date: Wed, 11 Nov 2009 15:29:48 +0900

In practical terms, I agree that there will be better and better  
codecs in the future. However, I would offer the older codecs are good  
enough for our purposes and will be safe.

---

From: Rob Glidden <[email-masked]>
Date: Wed, 11 Nov 2009 09:10:56 -0800

No argument on the recipe -- build a royalty free house on a royalty 
free foundation with royalty free bricks and royalty free inspection, etc.

But it wasn't the brick house that saved the three little pigs in the end.

It is about going the distance in the face of the inevitable shilling, 
calls to drive into a ditch, meeting-stacking et al.

And it is not about convincing the convinced -- it about proving 
marketplace confidence. It is done all the time, codecs aren't the 
unique special case some need them to be.

Of course vet contributions for blocking patents and other loopholes.

Some noteworthy stuff below.

Rob

2001

http://www.itscj.ipsj.or.jp/sc29/29w12911jvt.pdf
N4400, December 2001

Terms of Reference for a Joint Project between ITU-T Q.6/SG16 and 
ISO/IEC JTC 1/SC 29/WG11
for the Development of new Video Coding Recommendation and International 
Standard
...
"10.0 Patent and Copyright Issues
The project and joint group will progress the project work in compliance 
with the Intellectual Property
Rights (“IPR”) policies and IPR reporting requirements and procedures of 
both organisations
(http://www.itu.int/ITU-Databases/TSBPatent/ and 
http://www.itscj.ipsj.or.jp/sc29/29w7ipr.htm).
*
JVT will define a “baseline” profile. That profile should be 
royalty-free for all implementations.* The
performance of this profile will particularly be the subject of 
performance verification tests.
JVT’s rules for the implementation of the IPR policy are contained in 
Annex 3."

2002

http://lists.mpegif.org/pipermail/discuss/2002-May/000347.html

[M4IF Discuss] RE: [M4IF News] To those concerned about MPEG-4 Licensing ...
Fernando Pereira fp lx.it.pt
Mon May 13 18:40:38 EDT 2002

* Previous message: [M4IF Discuss] RE: [M4IF News] To those concerned 
about MPEG-4 Licensing ...
* Next message: [M4IF Discuss] RE: [M4IF News] To those concerned about 
MPEG- 4 Licensing ...
* Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi ! Yuval Fisher wrote:
 >
 > > As for me, I'm looking at MPEG-4 Part 10 and hoping that it is
 > > patent-free ;-)
 > > I wouldn't hold your breath. Many companies are nervous about submarine
 > patents. There's a strong anti-free-license movement in MPEG.

Let me disagree with this statement ! As the current chairman of MPEG 
Requirements, I would claim that MPEG is doing everything to support the 
royalty free approach for the baseline profile of MPEG-4 part 10 (AVC) 
... and there is a very large support within MPEG members. Of course 
there is also people with a (legitimate) different opinion but I would 
personally claim this is a minority.

Let me also explain that the approach is not simply everything royalty 
free but a combination of a royalty free baseline profile with other 
more complex profiles not necessarily royalty free (but RAND). This 
combination may provide a good compromise between the two possible 
extreme alternatives. Finally let me inform that 2 profiles were defined 
for AVC/H.264 last week in Fairfax: BASELINE (to be royalty free) and 
MAIN (not necessarily royalty free).

Regards Fernando Pereira -- Fernando Manuel Bernardo Pereira, Ph.D., 
Professor
Instituto Superior Técnico - Instituto de Telecomunicações
Av. Rovisco Pais, 1049-001 Lisboa, PORTUGAL
Phone: + 351 21 8418460 Fax: + 351 21 8418472
E-mail: Fernando.Pereira lx.it.pt WWW: http://www.img.lx.it.pt/~fp/

[M4IF Discuss] RE: [M4IF News] To those concerned about MPEG- 4 
Licensing ...
Rob Koenen rkoenen intertrust.com
Fri May 3 20:59:52 EDT 2002

* Previous message: [M4IF Discuss] To those concerned about MPEG-4 
Licensing ...
* Next message: [M4IF Discuss] RE: [M4IF News] To those concerned about 
MPEG- 4 Licensing ...
* Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

 > > As for me, I'm looking at MPEG-4 Part 10 and hoping
 > that it is
 > > patent-free ;-)
 >
 >
 > I wouldn't hold your breath. Many companies are nervous about
 > submarine
 > patents. There's a strong anti-free-license movement in MPEG.

Sweeping statements like these are very unhelpful.

It may indeed be unlikley that full JVT codec is going to be RF
(Royalty-Free).
There is, however, a strong desire among many parties to try and
establish a RF baseline. There is an ongoing effort to see how such
an RF baseline could be established. It is area in which people like
to maneuver carefully, for obvious reasons.

Rob

2003

http://lists.mpegif.org/pipermail/discuss/2003-November/000541.html
Rob Koenen to Iain Richardson
/Thu Nov 20 07:47:47 EST 2003
/

(1) Does this mean that the goal of a royalty-free Baseline Profile is not
going to happen ?

...

I'll answer for what I know and understand today.
 
1) Via Licensing's website states that their proposed terms cover "use of
Baseline, Main, and Extended Profiles". MPEG LA's announcement states "These
terms cover the entire AVC Standard regardless of which Profile(s) are
used". I think that gives you the answer.

2004

H.264/MPEG-4 Part 10 is also subject to a number of essential. patents.
However, in order to make the new standard as accessible as possible,
the JVT has attempted to make the Baseline Profile (see Chapter 6)
'royalty free'. During the standardisation process, holders of key
patents were encouraged to notify JVT of their patent claims and to
state whether they would permit a royalty free license to the
patent(s). These patent statements have been taken into account during
the development of the Profiles with the aim of keeping the Baseline
free of royalty payments. As this process is voluntary and relies on the
correct identification of all relevant patents prior to standardisation,
it is not yet clear whether the goal of a royalty-free Profile will be
realised but initial indications are positive [1].

1 In March 2003, 31 companies involved in the H.264 development process
and/or holding essential patents confirmed their support for a
royalty-free Baseline Profile.

Iain E.G. Richardson, H.264 and MPEG-4 Video Compression: Video Coding
for Next-Generation Multimedia at 274 (John Wiley & Sons, 2004).

2007:

(look also at what happened to the attorneys)

"Thus, while the language of the JVT IPR policies may not expressly 
require disclosure by all participants in all circumstances (e.g., if 
relevant IPR is not disclosed despite the use of best efforts), it at 
least incorporates a best efforts standard (even apart from the 
submission of technical proposals).
...
In sum, we conclude that Qualcomm, as a participant in the JVT prior to 
the release of the H.264 standard, did have IPR disclosure obligations, 
as discussed above, under the written policies of both the JVT and its 
parent organizations"

http://www.cafc.uscourts.gov/opinions/07-1545.pdf

Trial court decision:

http://www.klgates.com/files/upload/eDAT_Qualcomm_8_6_07_Order_on_Remedy.pdf 

2007 SC29 revised patent policy

http://www.itscj.ipsj.or.jp/sc29/29w7ipr.htm

"Although royalty-bearing patented technologies may be included in SC 29 
standards, SC 29 suggests to its WGs to promote, whenever possible, the 
inclusion of technologies that either do not require a patent license, 
or that only require a RAND license without a royalty or license fee."

2007 ISO/ITU common patent policy

2009:

MPEG HVC

---

From: Eric Burger <[email-masked]>
Date: Thu, 12 Nov 2009 05:44:03 +0900

Am I missing something here?  I cannot tell if what you are saying is  
the JVC EG was able to produce a RF baseline (the last paragraph) or  
they were not able to produce a RF baseline (the second to last  
paragraph)?

One thing that is clearly different is IPR declaration in the IETF is  
not optional: it is mandatory and the policy is clear.  Moreover,  
there is established case law that if a participant neglects to  
declare their IPR, and the IPR gets incorporated into a standard, the  
participant loses their IP rights.

On Nov 12, 2009, at 2:10 AM, Rob Glidden wrote:

> It is about going the distance in the face of the inevitable  
> shilling, calls to drive into a ditch, meeting-stacking et al.
>
> for the Development of new Video Coding Recommendation and  
> International Standard
> ...
> The project and joint group will progress the project work in  
> compliance with the Intellectual Property
> Rights (“IPR”) policies and IPR reporting requirements and  
> procedures of both organisations
> (http://www.itu.int/ITU-Databases/TSBPatent/ and http://www.itscj.ipsj.or.jp/sc29/29w7ipr.htm) 
> .
> *
> JVT will define a “baseline” profile. That profile should be royalty- 
> free for all implementations.* The
> JVT’s rules for the implementation of the IPR policy are contained  
> in Annex 3."
>
>
> 2002
>
> * Previous message: [M4IF Discuss] RE: [M4IF News] To those  
> concerned about MPEG-4 Licensing ...
> * Next message: [M4IF Discuss] RE: [M4IF News] To those concerned  
> about MPEG- 4 Licensing ...
> Let me disagree with this statement ! As the current chairman of  
> MPEG Requirements, I would claim that MPEG is doing everything to  
> support the royalty free approach for the baseline profile of MPEG-4  
> part 10 (AVC) ... and there is a very large support within MPEG  
> members. Of course there is also people with a (legitimate)  
> different opinion but I would personally claim this is a minority.
>
> Let me also explain that the approach is not simply everything  
> royalty free but a combination of a royalty free baseline profile  
> with other more complex profiles not necessarily royalty free (but  
> RAND). This combination may provide a good compromise between the  
> two possible extreme alternatives. Finally let me inform that 2  
> profiles were defined for AVC/H.264 last week in Fairfax: BASELINE  
> (to be royalty free) and MAIN (not necessarily royalty free).
>
> * Next message: [M4IF Discuss] RE: [M4IF News] To those concerned  
> about MPEG- 4 Licensing ...
> require disclosure by all participants in all circumstances (e.g.,  
> if relevant IPR is not disclosed despite the use of best efforts),  
> it at least incorporates a best efforts standard (even apart from  
> the submission of technical proposals).
> ...
> In sum, we conclude that Qualcomm, as a participant in the JVT prior  
> to the release of the H.264 standard, did have IPR disclosure  
> obligations, as discussed above, under the written policies of both  
> the JVT and its parent organizations"
>
> "Although royalty-bearing patented technologies may be included in  
> SC 29 standards, SC 29 suggests to its WGs to promote, whenever  
> possible, the inclusion of technologies that either do not require a  
> patent license, or that only require a RAND license without a  
> royalty or license fee."
>
>> codecs in the future. However, I would offer the older codecs are  
>> good enough for our purposes and will be safe.
>>
>>>>> This
>>>>> free
>>>>> can
>>>>> that
>>>>> If
>>>>> never
>>>>> standards

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 12 Nov 2009 07:40:18 +0900

Hi Eric,

> they were not able to produce a RF baseline (the second to last

FWIW, I was one of the "architects" of JVT's royalty-free baseline approach,
and I believe I have some limited insight in the realities today.  My
position in our discussion is in part a reaction to these realities.

The short answer to this question is (as usual): it depends whom you ask.
However, clearly, most lawyers would probably advise a business to practice
H.264 baseline only after having obtained a license from the pool and
possibly other known rightholders.

At present, one can pick a (royalty-bearing, though cheap, comparatively
speaking) license from the MPEG-LA patent pool which contains patent claims
allegedly relevant to practice baseline and/or many other profiles.  MPEG-LA
does not distinguish between profiles on the out-licensing side; which is a
wise business decision on their side, I believe. Further, there are
rightholders outside the pool (according to ITU/ISO/IEC declarations on file
in the respective organizations), who may or may not charge for a license of
their patents related to baseline (or other profiles).  No one really knows
for sure.  

OTOH, some say that the assurances every contribution had to be accompanied
with (RF licensing in case the contribution were accepted, under a
reciprocity condition) continues to provide an essentially RF licensing
environment for baseline.  Note that the requirement for these assurances
were implemented in JVT only; they are not a rack ITU or ISO/IEC policy.
I'm personally aware of only a single ruling related to alleged infringement
of H.264 baseline (although H.264 patents have been litigated more than
once), and this ruling was quite positive for the alleged infringer;
however, the legal grounds for the ruling had AFAIK little or nothing to do
with the assurances mentioned above, but only with other process violations
and (more importantly) legal grounds to subtle to discuss here.  It should
further be noticed, that many folks in the community consider the ruling
"bad law".

Finally, the JVT Royalty-free baseline project did explicitly NOT target
anything but royalty-free *licensing*.  As it has been pointed out already,
by the strictest interpretation of the GPL and other similarly restrictive
"free" or open source licenses, the mere fact that one has to obtain a
license already renders a technology "incompatible" with the license.
That's why I have pointed out time and time again that you guys need to
shoot for non-assert covenants from known rightholders rather than RF
licensing provisions.  That said, there are tons of open source projects
(under GPL and other licenses) that implement H.264 and similarly
known-as-encumbered standards.

> not optional: it is mandatory and the policy is clear.

In the ITU as well.  Arguably: clearer than in the IETF.

> Moreover,  

I would think your statement may be a tad too strong.  But I agree, it has
been getting more and more difficult to enforce patent rights when clear
process violations are shown, independent from the organization involved.
One significant aspect of the case mentioned above is that even violations
of the common conduct in the committee in question (something I have called
"culture" in a previous posts) may endanger the enforceability of patents,
even if one cannot, by the strictest interpretation of policy documents,
find support of a process violation under the written policy documents.

Not simple, this stuff.

Stephan

>> Rights (³IPR²) policies and IPR reporting requirements and
>> JVT will define a ³baseline² profile. That profile should be royalty-
>> JVT¹s rules for the implementation of the IPR policy are contained
>> in Annex 3."
>> 
>> 
>> 2002
>>