Subject: Two document stages: Proposed and Full

From: "Charles E. Perkins" <[email-masked]>
Date: Tue, 10 Jun 2003 11:43:59 -0700

Hello folks,

I think this has been suggested before.  I'd like to
suggest it again.  Maybe we can get some mileage out of
having two stages of standardization:

Proposed Standard: requires working group approval.  IESG comments
	taken into account, but full IESG approval not needed before
	publication.  Interoperability testing completed, no known
	flaws that present danger to the Internet.  Component protocols
	are O.K., as long as applicability (usefulness) is documented.

Full Standard: Full IESG approval required, according to current
	procedures.  Component protocols published as part of
	document suites, or separately, according to IESG discretion.

Since the IESG still controls charters and chair selection, one may
presume that generally sensible people will be involved.  In this
model, for instance, a working group might publish a component
protocol, and another working group with more relevant expertise
might publish another associated component protocol that could fit
together into a larger system.

Maybe this is too much "solution space" -- if so, I'll try to
bring it up whenever I can find that forum.  It's gotta be
around here somewhere.

Regards,
Charlie P.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 10 Jun 2003 14:14:58 -0700

Charles,

First, many thanks for your earlier thread and the exchange with John K.
I think that you've developed an issue that is serious and poses a real
dilemma.

One thought on component-vs-system is that we might want to make that
distinction more explicitly and even formally, making it relatively easy
to do component specifications, but still difficult to do system
specifications.

With respect to your suggestion on standards classifications:

CEP> Proposed Standard: requires working group approval.  IESG comments
CEP> taken into account, but full IESG approval not needed before
CEP> publication.  Interoperability testing completed, no known
CEP> flaws that present danger to the Internet.  Component protocols
CEP> are O.K., as long as applicability (usefulness) is documented.

This is rather interesting.  Most Proposed specs, today, are not
required to demonstrate interoperability, whereas Draft is.

The question is whether the working group's reliably exert enough
quality control on their output. For all that one might express
legitimate concern about IESG push-back, there is also a legitimate
concern about some working groups that produce poor work.

CEP> Full Standard: Full IESG approval required, according to current
CEP> procedures.  Component protocols published as part of
CEP> document suites, or separately, according to IESG discretion.

What is the market motivation for anyone to pursue this classification?

Note that the world lives on Proposed Standard now.  Though we might
want to change the meaning of the term, so that it carries less import,
but we have no assurance that the market will notice or respond to that
change.

What about using existing language for Proposed, but adding the
interoperability requirement, and changing the enforcement back to "no
known bugs" rather than "no known bugs and contains full system
features".

And for Full, simply require only: 1) substantial deployment and use,
and 2) the specification accurately documents what is deployed.

d/
--
 Dave Crocker <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 10 Jun 2003 23:41:33 +0200

I actually think it might be useful to have a couple of people draft their 
thoughts about specific classification schemes/thoughts (as Ted has done) 
and kick the specifics around on the solutions mailing list.... the problem 
that there's a mismatch between our nomenclature and the needs of the 
community has been well discussed, and just needs to be wordsmithed - the 
specifics on how we solve it is a separable issue....

            Harald

---

From: "Charles E. Perkins" <[email-masked]>
Date: Tue, 10 Jun 2003 14:50:29 -0700

Hello Dave,

> to do component specifications, but still difficult to do system

I'm not exactly sure why it has to be difficult to do system
specifications, even though it's bound to be more complicated
than simple component protocols.   For the cases I had in mind,
it would be more the case that vendors could expect to get a
pretty full set of instructions on how to build the systems
they wanted to build, at least from the networking standpoint.
APIs and so on would still have to be done separately.  The
difference would be that a vendor using only a component protocol
would be left somewhat to their own devices when it comes to
filling in the missing pieces.

> required to demonstrate interoperability, whereas Draft is.

I think that's a mistake.  I'd rather have it that Proposed Standard
requires interoperability.  Do you happen to know when the interoperability
requirement was relaxed?

> concern about some working groups that produce poor work.

I don't imagine that the ADs will suddenly stop paying attention
to the working groups in their area.  Their contribution to the
discussion would be valued.  If something is going horribly wrong
and the working group is on the road to perdition, then maybe a
process of dechartering could be initiated before publication of
the offending document.  I don't think this would happen, though.
Furthermore, it is conceivable even that working groups would take
more pride in their work if their documents had to stand on their
own merits before ever getting to full IESG consideration.  I'm
sure that is debatable, but maybe we ought to at least try to
change things for the better.

I think if working groups are indeed producing poor work, then it's
better to recharter or disband instead of haggling over ill-fated
documents.  But, again, work with limited scope, or even incomplete
work, is not necessarily poor quality work.

> What is the market motivation for anyone to pursue this classification?

I'm projecting here, but the motivation would be that building to the full
protocol suite is more guaranteed to not have any missing features (from
the networking perspective).

> but we have no assurance that the market will notice or respond to that

Assurances about the market seem scarce as hens' teeth.

> known bugs" rather than "no known bugs and contains full system

This would be a good subject for discussion when we get to
solution space.  I honestly don't know.  I thought that working
group approval would galvanize the interest, and free up the
IESG for focussing on the more global aspects of promulgating
full standards, reducing their workload.  They might again become
more functional working group members, and in this their experience
would be very, very useful.

Regards,
Charlie P.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 10 Jun 2003 15:49:03 -0700

Charles,

(Harald's request the this solution discussion be moved to the solutions
list is duly noted.  So the reply on this list is non-solution related.)

CEP> I think that's a mistake.  I'd rather have it that Proposed Standard
CEP> requires interoperability.  Do you happen to know when the interoperability
CEP> requirement was relaxed?

It was never 'relaxed'.  When Proposed/Draft were first defined, Draft
called for interoperability and Proposed mostly did not.

The "mostly" is the interesting part.  The language I recall for
requiring interoperability at Proposed was something like:

The specification pertains to essential (ie, fragile) Internet
infrastructure, or the specification is sufficient complex (ie, its
physic are poorly understand) so that an interoperability demonstration
is needed to provide a degree of comfort about basic viability and/or
safety of the specification.

For most specification, we rely on a static review of the document, to
convince us that we understand how it behave and that it will not damage
the infrastucture. When we can't develop that code just from reading, we
require implementation.

A separate point that Marshall Rose has been good at pointing out is,
essentially, a marketing issue for a specification: If it's been
implemented, it is much more difficult for people to make abstract
claims about failings of the spec. Hence it is more difficult to reject
a request for Proposed status.

d/

---

From: Ralph Droms <[email-masked]>
Date: Tue, 10 Jun 2003 19:04:25 -0400

At 03:49 PM 6/10/2003 -0700, Dave Crocker wrote:
>[...]

I agree, and I wrote earlier:

  My recent experience with specifications in the dhc WG
  has been that "running (prototype) code" trumps
  (potentially ad infinitum) speculative discussion about
  whether the protocol is correct and completely
  specified.  

Running code can't, of course, address *every* claim about
the failing of a spec.  But it does rule out a lot of
claims...

So, it seems we need some way to allow and enforce implementation
experience.

- Ralph

---

From: Randy Bush <[email-masked]>
Date: Wed, 11 Jun 2003 08:44:19 +0900

> claims about failings of the spec.

[ aside from that it is always easy to raise abstract claims, but
  can be hard to substantiate them ]

if and only if it has multiple, separately developed from only the
spec, interoperable implementations.  there are a fair number of
critical protocols where one could not really do an implementation
from the spec, and it required 'inside knowledge' and this was
used, let's be kind and say unintentionally, to raise a bar against
new market entries.

randy

---

From: Keith Moore <[email-masked]>
Date: Tue, 10 Jun 2003 20:34:00 -0400

> Maybe we can get some mileage out of

can we please avoid discussion of solutions? 
it's not in scope for this group.

it's one thing to offer a strawman solution to help people understand
the problem description you are proposing, but we shouldn't dig in.
because I really want to tear into this half-baked proposal, but this
is not the place to do so, and silence should not be taken as consent.
fair is fair.  please withdraw the proposal until we can fight over
if fairly.

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 11 Jun 2003 11:07:55 +0200

Problem related comment:

We have ample evidence that WGs don't consistently produce
documents that meet even today's criteria for PS. I am strongly
against giving WGs the ability to approve PSs without enforced peer 
review that is specifically independent of the WG and specifically
consider IETF-wide and Internet-wide issues. This proposal would
only make the problem worse than today, by allowing WGs to ignore
peer review. 

    Brian

---

From: Marcus Brunner <[email-masked]>
Date: Wed, 11 Jun 2003 14:40:33 +0200

Brian,

If we accept that the core value is ".. and running code" the document and 
how it is written does not really matter. Peer review is naturally very 
useful for the final spec, but do we need it before we have implementation 
experience? The problem is the crap sitting in PS and being happy with 
this. (Which from a marketing perspective is enough, getting and market 
products with RFC numbers)

Marcus

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 11 Jun 2003 17:40:22 +0200

Having documents sitting at PS that either need revision or
need to become Historic is certainly a problem. My view is that
by having a high barrier to going "up" from PS causes most
authors to simply not bother about this problem. (That is 
certainly true of myself, so I am guilty of some sleeping
PS's.)

   Brian