Subject: Re: Time required to write down "wisdom" (Re: "Adult supervision")

From: Aaron Falk <[email-masked]>
Date: Mon, 12 May 2003 10:49:52 -0700

[email-masked] wrote:
> Hi Harald,
> 
> > for a worked example of something that everything we've said here says 
> > "should have been published sooner", consider draft-iab-sec-cons, on how to 
> > write security considerations.
> > 
> > the history of that document, which started life as an emailed note in 
> > 1997, is probably instructive when considering the bars to timely 
> > publication.
> 
> This sounds strangely like the experience of getting an RFC published 
> by a WG.  It seems that the IAB faces many of the same troubles that
> WGs face in moving a document to RFC.  
> 
> It seems that the 'process' has some how gotten twisted to disable the
> quick publication of important documents.  Maybe WGs have been spending
> too much time trying to find creative ways to excede their charters
> & destroy the Internet; and the IESG spends too much time flying
> around in black helicopters.
> 
> I think that we, as an organization, do need to do better.
> 
> John

John-

It may just be that it takes time to write good documents, esp. ones
that are expected to have long shelf life.

--aaron

---

From: Thomas Narten <[email-masked]>
Date: Mon, 12 May 2003 16:55:33 -0400

> quick publication of important documents.

This may true if the assumption is that the *process* is what caused
the problems.

>From my perspective, the real issue often tends to be:

1) Good documents don't pop out in the -00 version.

2) Iteration is essential. Iteration means a small number of people
   (e.g., 1-5) read the document, provide good feedback, and then a
   new revision is produced.

3) process is repeated at least a few more times, with a different set
   of reviewers providing the review and feedback each time.

4) Process terminates, because subsequent reviews don't uncover
   significant issues and the reviewers think the document is good
   enough to ship.

You can't rush a document (if you want it to be good). Indeed, when I
write documents, I personally find that if I reread something I wrote
a month earlier, I often find obvious things that need fixing. I often
don't see these if I review the document a few days after last working
on it. The point here is that good documents just don't happen on the
first version and time is needed to properly review and iterate.

Where the "process" sometimes goes wrong is that the sequence of
reviews and iterations haven't happened properly/optimally. Either not
enough iterations, or too long between iterations.

IMO, there is a problem here that bears further examination. Getting
good reviews and then subsequent revisions in a timely fashion is
something I see too much of.

> I think that we, as an organization, do need to do better.

Yep. IMO, we should look hard at ways of ensuring that the needed
iteration on revisions happens in a timely fashion. But not too
timely, as that leads to documents being pushed forward before they
are truly ready.

Thomas

---

From: "Jeanette Hofmann" <[email-masked]>
Date: Tue, 13 May 2003 00:37:15 +0200

> timely, as that leads to documents being pushed forward before they

One of the few good things that came out of the ICANN reform process is the 
introduction of a policy development process, PDP. It is the attempt to define 
and standardize the steps and overall timeframe of decision making 
processes, as it were a finetuning of the milestones procedure. There are no 
practical experiences with the PDP yet. Moreover, ICANN is not exactly a 
good recommendation for best practice models, I know. Yet, perhaps it is 
worth a thought whether a similar model might improve the milestones 
function. 

Jeanette

---

From: Keith Moore <[email-masked]>
Date: Mon, 12 May 2003 23:15:00 -0400

Good summary.  

The other pitfall I see with this process is that sometimes we keep iterating
on bad documents in the vain hope that they'll get better, and sometimes those
documents get approved due to exhaustion on the part of the reviewers. 
(or worse, in an attempt to avoid embarassment at having invested so much
effort in a bad idea)

But if we could figure out a way to get enough review cycles, at about the
right intervals, and at the same time learn how to prune bad ideas or
incomplete ideas that weren't ready to avoid investing too much in them, we
could drastically improve the quality and relevance of our output.

Keith