Subject: RE: Document Series

From: "Bound, Jim" <[email-masked]>
Date: Sat, 7 Jun 2003 22:19:35 -0400

This is exactly what is happening in the IETF. 

/jim

> -----Original Message-----
> From: Bob Hinden [mailto:[email-masked]] 
> Sent: Tuesday, June 03, 2003 8:03 PM
> To: Dave Crocker
> Cc: [email-masked]
> Subject: Re: Document Series
> 
> 
> Dave,
> 
> >Here is my own thought on useful stages:
> >
> >1.  Technical discussion and convergence -- form community 
> consensus 2.  
> >Technical refinement -- lock down the details 3.  Proof of 
> concept and 
> >debugging 4.  Stability -- community operational acceptance
> >
> >The first is pure mayhem, and that's good. People explore, suggest, 
> >debate. Eventually, they should start to form group preferences and 
> >focus on getting the details worked into a specification 
> that provides 
> >detail which readers can understand and see as coherent and 
> sufficient 
> >for achieving a specific set of useful functions. Then they test the 
> >thing, to see whether interoperable implementations can be 
> developed. 
> >Then they actually use the thing, eventually achieving some sort of 
> >critical mass of deployment.
> >
> >Now, the clever reader will observe that we already have 
> exactly these 
> >stages. (Personal I-D, WG ID -> PS, Draft, Full.) Yet the 
> latter 2 are 
> >virtually unused.
> 
> As you correctly point out the IETF (and as implemented by 
> the IESG) is not 
> using the standards process we have defined.  It has been 
> changed where the 
> initial barrier has been raised very high at Proposed 
> Standard and hardy 
> anything gets to Draft standard, let alone (Full) Standard.
> 
> I think that many of the problems folks are complaining about 
> stem from 
> this.  It causes the IESG to worry about letting a document 
> get to PS that 
> is not perfect.  This results in many detailed reviews and 
> documents being 
> blocked for what might seem minor reasons.  This in turn 
> creates a work 
> load on the IESG that is close to impossible.  The resulting 
> delays cause 
> frustration between authors, chairs, working groups, AD, etc. 
>  I think that 
> a lot of this stems from trying to make PS the biggest hurdle.
> 
> I think that if we implemented the standard process as 
> written (and as 
> intended) that many of these problems would go away.  
> Perfection would only 
> have to be achieved at Standard (and close to perfection at Draft 
> standard).  This would mean that problems with specifications 
> could be 
> found by people implementation the protocols and running into 
> problems, and 
> then bringing the issues back into the working groups.  
> Interoperability 
> problems would be found.  This would spread the load of improving the 
> quality of the specifications to a much larger community of 
> people than 
> just the IESG members.   Running code and interoperability 
> would be the 
> real test for quality.
> 
> Enforcing the time outs would create incentives to move the 
> specifications 
> through the standards process.  I also suspect that some 
> (perhaps many) 
> specifications never really get implemented, so as long as we 
> enforced the 
> time outs as specified, that these would just go away.
> 
> There is a lot of parallels with the multi-step IETF 
> standards process as 
> with building products.  First versions of products always have 
> problem.  Most successful product companies learn that trying 
> to make the 
> first product perfect is counter productive because it 
> creates long delays 
> and the real test is getting the product to the costumers.  Then the 
> company really learns how well the product works and meets 
> the market need 
> (i.e., requirements).  It through creating new versions of 
> the products 
> that problems are fixed and quality is improved.  Companies 
> that delay the 
> first release until the product is perfect usually go out of 
> business.  I 
> wonder if that is what is happening with the IETF.
> 
> Bob
> 
>

---

From: "Bound, Jim" <[email-masked]>
Date: Sat, 7 Jun 2003 22:21:32 -0400

I completely disagree with you Mr. Halpern.  The standard is not to be
as robust or mean't to be at PS.  Please re-read Bob Hinden's mail more
carefully.

/jim

> From: Joel M. Halpern [mailto:[email-masked]] 
> Sent: Tuesday, June 03, 2003 8:14 PM
> To: [email-masked]
> I think we are each making assumptions about the history that 
> led to this, 
> and it would not surprise me if we are making different 
> assumptions. Note that I do not have data to back my 
> understanding / memory, so I may 
> well be confused.
> 
> My understanding is that the higher bar to PS arose as a 
> consequence of 
> things being widely deployed at PS and things not advancing 
> to Draft rather 
> than the deployment and non-advancement being a consequence 
> of the high bar.
> 
> This is important in the sense that if the lack of 
> advancement came first, 
> then simply lowering the bar will not help us get better 
> standards, and in 
> fact could result in our ending up with lower quality 
> documents permanently.
> 
> Yours,
> Joel M. Halpern
> 
> At 05:03 PM 6/3/2003 -0700, Bob Hinden wrote (in response to 
> Dave Crocker):
> the IESG) is
> >not using the standards process we have defined.  It has 
> been changed 
> >where the initial barrier has been raised very high at 
> Proposed Standard 
> >and hardy anything gets to Draft standard, let alone (Full) Standard.
> >
> >I think that many of the problems folks are complaining 
> about stem from
> >frustration between authors, chairs, working groups, AD, 
> etc.  I think 
> >that a lot of this stems from trying to make PS the biggest hurdle.
> 
> 
>

---

From: "Bound, Jim" <[email-masked]>
Date: Sat, 7 Jun 2003 22:28:22 -0400

My view is:

PS is for base prototype of a spec and then implementation.
DS is for pre-product implementation and if lots do it product
implementation.
S(full) is lots of entities are using it and it does not have to be the
entire Internet.

Anything more than this will result in late time to market and the
result of that is being seen today, and soon we will see protocol
proposals to other than the IETF to other SDOs if we don't fix this and
the initial death of this SDO except for maintenance of the current IP
(which is a large job and we call it in business the cash cow).  But the
new and interesting work required for the market will go elsewhere if we
don't fix this.  I think that should be stated as a problem too.  Most
of us don't want to have to deal with multiple SDOs to work on the IP
standards but we are to a minor degree already.

/jim

> From: Ralph Droms [mailto:[email-masked]] 
> Sent: Wednesday, June 04, 2003 9:23 AM
> My experience with DS and full Standard is that no one is 
> interested because of the lack of perceived difference 
> between the requirements between the standards levels and the 
> lack of perceived added value in progressing a spec to DS or 
> full standard.  That is, I disagree with Brian's statement 
> about the requirements for DS and full standard being too 
> high - I think the problem is a lack of differentiation among 
> the three standards levels.
> 
> Here's how I see the standards track:
> The required correctness for PS is high enough that a PS spec 
> is "good enough" for vendor implementation and deployment.  
> The reward for moving to DS and full standard is not worth 
> the investment of effort.
> 
> The tension I see is the need for a specification that is 
> sufficiently baked and stable to be used for prototyping, 
> interoperability testing and proof-of-correctness for the 
> spec doc, but that does *not* result in deployment.  It's the 
> premature deployment that (a) casts goofs in the spec in 
> stone and (b) eliminates any motivation to move the 
> specification through the standards track.
> 
> - Ralph
> 
> At 02:58 PM 6/4/2003 +0200, Brian E Carpenter wrote:
> >Keith Moore wrote:
> >> 
> >> O> My understanding is that the higher bar to PS arose as a 
> >> O> consequence of
> >> > things being widely deployed at PS and things not advancing to 
> >> > Draft rather than the deployment and non-advancement being a 
> >> > consequence of the high bar.
> >> 
> >> that's my understanding also.
> >> 
> advancement came 
> >> > first, then simply lowering the bar will not help us get better 
> >> > standards, and in fact could result in our ending up with lower 
> >> > quality documents permanently.
> >> 
> >> I'm in strong agreement with this.  What we need to do is 
> find a way 
> >> for standards to at least the level that is currently 
> required for PS 
> >> more quickly, not to lower the bar for PS.
> >
> >Fully agree.
> >
> >> We might even need to raise the bar slightly.
> >
> >Do you mean that we are frequently releasing PS documents 
> that contain 
> >significant defects? We certainly can't expect PS documents to be 
> >perfect.
> >
> >Another problem I see is that the barrier to Standard is so 
> high that 
> >nobody is interested, and the barrier to DS is high enough that very 
> >few people are interested. That being so, apart from some academic 
> >ideals, I question the value of having 3 grades at all. 
> Since we hardly 
> >use the top two grades, why have them?
> >
> >   Brian
> 
>