Subject: Response to WG last call, Problem Statement: Thoughts on the IET F problem statement

From: Robert Snively <[email-masked]>
Date: Wed, 19 Nov 2003 17:23:49 -0800

In consideration of:

	draft-ietf-problem-issue-statement-05.txt

I'm sorry not to have identified this document earlier, but it was brought
to my attention only today by a news publication.

INTRODUCTION:

First let me briefly introduce myself.  I have created and led the
creation of national and international standards in the INCITS, IEEE,
and IETF environments.  I have additionally created defacto standards
in industry interest organizations, both commercial and voluntary.
As a result, I have a fairly diverse background in the creation of
standards and can speak directly to the challenges as seen from
the working group and area levels.  From my experiences, I believe that
I can provide some helpful consideration to the overall IETF process
up to the IESG level.

BASIC ISSUE IS CULTURE:

I believe your IETF draft has correctly identified many problems
and issues.  However, I believe that there is a more fundamental
problem that must be discussed.

Perhaps one of the most important issues is raised in clause 2.1. 

   "The IETF creates standards and is therefore necessarily a Standards
   Development Organization (SDO) but many participants would like to
   differentiate the IETF and its way of working from the 'conventional'
   SDOs which emphasize corporate involvement and mandated delegates."

This attempt to differentiate and create an "academic" culture is perhaps 
the biggest flaw in an organization architecting a world-wide commercial 
infrastructure with very high standards of reliability and security and 
with immense complexity.  From that attempt flow most of the other problems.
The obvious solution, following some of the principles of the other large
successful SDOs, has consistently slipped from IETF's grasp.

PROBLEM STATEMENT: NO MEMBERSHIP REQUIREMENTS

Perhaps the most important word qualifying the internet today is
"commercial".  Companies spend money in their engineering development
organizations to develop and implement equipment in compliance with
IETF documents.  Time to market is a key aspect of such development
efforts.  Complete, accurate, and timely standards are a requirement.
Involving an arbitrary number of standards developers having no commercial
stake in the process of creating a standard is perhaps the largest
single problem in the IETF.

This is touched upon in clause 2.8, but rather than choosing to
guide the culture toward a more effective process, it was treated as
a failure to properly train the participants.  The real issue is that
membership should be earned.

Membership should be earned in each working group or area by attendance, 
by activity, by participating in reviews, and by payment of a fee.  

Membership should be probably be at an organizational level, thus partially
identifying the otherwise hidden interests and agendas of representatives.
The underlying commercial motivation of most participants must be
recognized and brought to light.

PROBLEM STATEMENT: OBSOLETE WORKING RULES

Tradition is a powerful and helpful guide, but must be supplemented with
wisely selected improvements. This is touched on in clause 2.6.5, but its
effects are far-reaching.  It also appears in various guises in clause 2.2.

  OBSOLETE WORKING RULES:  DOCUMENT FORMAT

	It took the ips group over a year to gain
	permission to post drafts in portable document format (pdf).  Even
then
	the documents had to be posted in text as well.  All other standards
	organizations have used sophisticated word processing capabilities
since
	the early 1980s, enabling their documents to contain diagrams
	and relieving the editor of painful document formatting duties.

  OBSOLETE WORKING RULES:  DEFINITIONS OF CONSENSUS

	As you pointed out in clause 2.7, the definition of consensus is
	rather vague.  In every other standards organization there are
formal
	mechanisms that solicit and measure consensus, usually with a
recorded vote
	by organization or member.  That provides a very effective mechanism
	for managing "consensus by exhaustion" issues. 

  OBSOLETE WORKING RULES:  ALL WORK OVER THE INTERNET

	Face-to-face meetings of sufficient duration to do actual work
	provide high bandwidth interchange and understanding among those
	motivated to attend and participate.  As one example, the iSCSI
	activity held two interim face-to-face meetings co-located with the
	INCITS TC T11 SCSI Committee, bringing together the foremost minds
	in SCSI and the Internet.  In each of those all day meetings, more
was
	accomplished in correcting major flaws in the iSCSI document than
	had been accomplished in three months of e-mail interchanges.
	By documenting those meetings in careful minutes, most of the
	decisions made during that period were not challenged later.
	The personal understandings achieved by those meetings were also
	very helpful.

	The IETF meeting weeks, while sometimes helpful, generally do not
have enough
	time allocated to each project within a working group to achieve
helpful technical
	interchange.  At the same time, the number of interested and
knowledgeable 
	participants for each project is a small percentage of those
	attending the working group, making the IETF meetings even less
useful
	for technical interchange.

	You address this briefly in a number of places, but the real issue
	is that the working groups should have autonomy within certain
	limits to hold meetings and teleconferences to accelerate their
work.
	This autonomy issue is partially addressed in several sub-clauses in
2.6.

PROBLEM STATEMENT:  DICTATORIAL STRUCTURE AND TECHNICAL REVIEW

Clause 2.6 and its sub clauses address the following issue, but less
brutally than I would suggest.  Again, this is a flaw in the IETF culture.

In every standards organization I have participated in, except IETF,
the administrative hierarchy is concerned with proper completion
of the technical process, not with the technical content.  The technical
content is worked out by organizations roughly at the level of the
working group in IETF.  As part of the procedure enforced by the
administrative hierarchy, liaison with other working groups, other
Accredited Standards Developers, and with other industry organizations
is established so that architecturally consistent standards are 
developed.  The administrative hierarchy guarantees a fair and open
process and the proper scoping of a standards document.  It also provides
a public review and a public protest mechanism.  It does not 
perform a technical review.

The process of sending documents up through the IESG for technical
consideration shows a lack of trust in the capabilities of the
working group technical participants, a recognition that the process
is unsuccessful in achieving consistent technical development, and
a certain academic arrogance, akin to grading papers.

Without such delegation of the responsibilities for liaison and
technical excellence, IETF will not prove scalable.

This process is also a major impediment to the timely publication
of documents.  As an example, we have about $3 billion dollars worth
of commercial market value backed up against three IETF documents 
that are being held up by a security document that most 
configurations will never use.

PROBLEM STATEMENT:  LACK OF FORMAL RECOGNITION

Clause 2.5.1 suggests that recognition is an important issue.
Formal recognition should be given in every document.  In systems
as complex as the internet, it is absurd to think that there is
one person that authors an RFC.  All meaningful RFCs are the product
of dozens or hundreds of people.  However, the academic background of
IETF, requires (probably because of the "publish or perish" syndrome)
that only one or a few people be listed as authors.  If 
IETF were being honest, it would identify an editor, a chairperson,
perhaps a vice-chair and/or secretary, and cite the full list of 
participating members as authors.  I have heard direct and explicit 
criticisms of this aspect of IETF culture a number of times on several 
IETF documents.

PROBLEM STATEMENT:  NIH

The previous arrogance of IETF administration has prevented it from
looking at very good models for organizational structure and 
standards development, implemented
by such organizations as ANSI and ISO.  It is refreshing that IETF has
finally realized how severe its problems are and is beginning to
cast about for proper solutions.  This was pointed to somewhat more
politely in several sub clauses of 2.6.  This is one part of the culture
that needs to be corrected.

MY RECOMMENDED PROBLEM RESOLUTION, NOT PART OF THE PROBLEM STATEMENT:

My own suggestion would be to
review documents like the INCITS RD-2 (available at www.incits.org),
the ANSI policy documents (available at www.ansi.org), and the ISO
policy documents (www.iso.org) for the many constructive ideas they
have.  Better yet, IETF should recognize that it now representing
a mature industry and become an accredited standards
developer under ISO, IEC, or ANSI, adopting the open, efficient, and
scalable policies of its new organizational parent.

This is not an internet draft, and will not be because of the "obsolete
working rules, document format" issue, mentioned above.

---

From: Pekka Savola <[email-masked]>
Date: Thu, 20 Nov 2003 11:46:09 +0200 (EET)

> I can provide some helpful consideration to the overall IETF process
[...]

This clearly shows that different people have different missions and
roles in mind for the IETF; some would prefer to have it as an
official SDO, in every aspect (with the problems that causes); some
others want to avoid it at all costs (with the problems we're aware
of).

I certainly would not wish to be part of a "full SDO" IETF.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Keith Moore <[email-masked]>
Date: Thu, 20 Nov 2003 21:53:55 -0500

I disagree with most of these problem statements, especially those 
regarding IETF's "culture" and organizational goals.  In particular, It 
would be a very bad move for IETF to regard the Internet as exclusively 
or even mostly "commercial" in nature, especially given the extensive 
history (which continues to this day) of noncommercial efforts 
producing a significant number of useful innovation, widely-used 
products, and widely-used protocols.   The Internet and IETF both 
continue to evolve, and even if most of the current participants are 
"commercial" we should not insist that they shall be so in the future - 
indeed, I'd argue that to be successful in the future we must find a 
way to attract more participants who take a long view.

Many of these statements are not "problems" but rather assumptions 
about how IETF should be that are far from universally shared.

I do believe that some form of membership should be considered as part 
of a "solution" to some other problems (at least, it shouldn't be 
rejected outright), and that our procedures for the operation of 
working groups in particular are obsolete and need refinement.  But 
these are really topics for another group.

One interesting point regarding document format.  IETF has a very long 
history of exchanging and providing standards documents over the 
network - far more than other SDOs, and this certainly influenced 
IETF's choice of document format.  Nearly 40 years after its initial 
adoption, ASCII text remains a very versatile and portable format, 
sufficient for most (not all) of IETF's purposes.  Few networking SDOs 
can claim that several of their 20-year old technical specifications 
are even accessible and readable, much less usable in other than a 
historical sense.  Specifications that are written to be usable for a 
long time should be produced in a format that will also continue to be 
usable into the future, and this argues for simplicity rather than 
aesthetic appeal.  Despite its name, PDF is already becoming somewhat 
nonportable, and the other popular formats are even worse.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Fri, 21 Nov 2003 10:45:31 -0800

Robert,

let me just say for the record that I disagree with you.

I believe the fact that the IETF has technically competent leaders who are 
expected to use that competence when leading the organization is a central 
reason why the IETF has been successful in a number of areas where ISO, 
ANSI, ITU and many others have failed.

I also disagree with your comments about the value of corporate membership 
and the value of doing work (almost) only at face-to-face meetings (while I 
agree that sometimes more face-to-face meetings are useful).

Differentiation is not a goal in itself.
But neither is being like others.

                         Harald

---

From: "todd glassey" <[email-masked]>
Date: Fri, 21 Nov 2003 10:44:13 -0800

So Harald - what are the goals then? To provide this open and fair
environment for the creation of Internet Standards?

Todd

----- Original Message -----
From: "Harald Tveit Alvestrand" <[email-masked]>
To: "Robert Snively" <[email-masked]>;
<[email-masked]>
Sent: Friday, November 21, 2003 10:45 AM
Subject: Re: Response to WG last call, Problem Statement: Thoughts on the
IET F problem statement

---

From: Marshall Rose <[email-masked]>
Date: Fri, 21 Nov 2003 11:41:54 -0800

> But neither is being like others.
    
i'm going to take this opportunity to agree with, and amplify, what
harald is saying.
    
each different style of organization has its own set of advantages, and
frankly, none of the styles have succeeded to the point of achieving
"one size fits all status" -- it is still the case where the "best tool
should be used for the job".
    
i believe now, as i did during the ietf's "post-kobe revolution", that
the fundamentals of the ietf remain strong. hence, i'm interested in
seeing what can be done in the way of an ietf "point release" (moving
from 0.91 to 0.92).
    
/mtr