Subject: RE: Time required to write down "wisdom" (Re: "Adult supervision")

From: <[email-masked]>
Date: Tue, 13 May 2003 06:09:54 +0300

Thomas,

I think my original mail sounded a bit more cranky than I wanted it,
but I was more responding to Harald's original mail - by the
way is there an archive of this list?

Harald mentioned that the document he was talking about sat around,
the token passed between a number of editors, etc.  It sounded to
me like the process for editing the document, soliciting comments,
producing revisions and so forth, fell apart, resulting in a
much longer than needed process.

I am not advocating pushing documents forward without adequate
review, but much more for having pro-active editors, WG chairs
and sheparding ADs to make sure the documents are getting done
in a timely fashion, especially ones identified as important.
This is more about breakdown in management, in my opinion.

Anyway, this mail is coming close to solutions - which is not
really my intention of posting the original mail.  I think that
one of the problems in getting work done in a timely fashion
is the breakdown of the document management process.

br,
John

> -----Original Message-----
> From: ext Thomas Narten [mailto:[email-masked]]
> Sent: 12 May, 2003 23:56
> To: Loughney John (NRC/Helsinki)
> Cc: [email-masked]; [email-masked]
> Subject: Re: Time required to write down "wisdom" (Re: "Adult
> supervision") 
> 
> 
> > It seems that the 'process' has some how gotten twisted to 
> disable the
> > quick publication of important documents.
> 
> This may true if the assumption is that the *process* is what caused
> the problems.
> 
> From my perspective, the real issue often tends to be:
> 
> 1) Good documents don't pop out in the -00 version.
> 
> 2) Iteration is essential. Iteration means a small number of people
>    (e.g., 1-5) read the document, provide good feedback, and then a
>    new revision is produced.
> 
> 3) process is repeated at least a few more times, with a different set
>    of reviewers providing the review and feedback each time.
> 
> 4) Process terminates, because subsequent reviews don't uncover
>    significant issues and the reviewers think the document is good
>    enough to ship.
> 
> You can't rush a document (if you want it to be good). Indeed, when I
> write documents, I personally find that if I reread something I wrote
> a month earlier, I often find obvious things that need fixing. I often
> don't see these if I review the document a few days after last working
> on it. The point here is that good documents just don't happen on the
> first version and time is needed to properly review and iterate.
> 
> Where the "process" sometimes goes wrong is that the sequence of
> reviews and iterations haven't happened properly/optimally. Either not
> enough iterations, or too long between iterations.
> 
> IMO, there is a problem here that bears further examination. Getting
> good reviews and then subsequent revisions in a timely fashion is
> something I see too much of.
> 
> > I think that we, as an organization, do need to do better.
> 
> Yep. IMO, we should look hard at ways of ensuring that the needed
> iteration on revisions happens in a timely fashion. But not too
> timely, as that leads to documents being pushed forward before they
> are truly ready.
> 
> Thomas
>

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 13 May 2003 05:20:07 +0200

John, others:

since this is a repeated question:

> by the

1) the list is an IETF WG list. All IETF WGs have the archive listed in the 
WG's charter.

2) the list is a Mailman list. In the headers of every single message on 
the list, the following appears:

List-Unsubscribe: 
<http://eikenes.alvestrand.no/mailman/listinfo/problem-statement>,
	<mailto:[email-masked]?subject=unsubscribe>
List-Archive: <http://eikenes.alvestrand.no/pipermail/problem-statement>
List-Post: <mailto:[email-masked]>
List-Help: <mailto:[email-masked]?subject=help>
List-Subscribe: 
<http://eikenes.alvestrand.no/mailman/listinfo/problem-statement>,
	<mailto:[email-masked]?subject=subscribe>

Just pointing out two common ways of finding the answer to the question 
without asking...

                  Harald

---

From: Keith Moore <[email-masked]>
Date: Tue, 13 May 2003 01:28:33 -0400

> This is more about breakdown in management, in my opinion.

It's one thing to require this of WG documents.  It's somewhat harder to
imagine how to apply it to individual submissions.

And in general, having people be more pro-active means giving them more time
to work on things, which might be beyond our control.

---

From: Thomas Narten <[email-masked]>
Date: Tue, 13 May 2003 10:53:08 -0400

Hi John.

> way is there an archive of this list?

Email can be funny that way...

> much longer than needed process.

Yep. 

> This is more about breakdown in management, in my opinion.

Yep. I think it would be good to understand when and why this process
seems to breakdown, as it leads to stalled documents much more
frequently than we perhaps realize. And these breakdowns can lead to
an extra month or two or three of delay. Or worse.

A couple more things I might add to me original post.

Thomas Narten <[email-masked]> writes:

> > It seems that the 'process' has some how gotten twisted to disable the

>    new revision is produced.

One thing that needs to be done here as well is that the original
person who reviewed the document needs to sign off on the
revision. That is one of the essential parts of our process (when it
works). Person reviews, provides feedback, a new revision comes a
long, and original reviewer says "works for me". Everyone is happy,
document is truely more of a consensus document. 

But, this doesn't always happen. What sometimes happens is the
revision is so long in coming, it becomes hard to get the attention of
the original reviewer, who has since forgotten about the details of
the document (in the worst case, this leads to the death spiral where
the delays get increasing longer...). Or, the author revises the
document, and *thinks* they have addressed the comments, but there is
no confirmation. When the original reviewer goes back, they may not be
happy with the result. Now there is bad feeling on top of everything
else, in that a reviewer feels like their input was ignored and not
given proper consideration.

Or, the author just responds to the issues clarifying them in
email. While this might explain what is meant, it doesn't mean the
document has gotten any better. Author thinks input has been dealt
with "I responded to that", while the reviewer feels like they have
been ignored. Document further stalls while it is unclear who has the
token...

I'm sure there are other examples. I think your point is right though
that there is a document "management" function that we often don't pay
sufficient attention to, and this does lead to long delayed documents.

Oh, and note also that one also needs to deal with the situation where
the author/reviewer can't agree, so that we don't get deadlock. This
also can block documents for inappropriately long periods of time.

---

From: Margaret Wasserman <[email-masked]>
Date: Tue, 13 May 2003 11:05:57 -0400

Hi Thomas,

>sufficient attention to, and this does lead to long delayed documents.

This is related to the document editor function that I keep
discussing in various fora...  Each document should have an
editor who is managing the document, dealing with and resolving
issues, etc.

In some SDOs, this is a centralized (even paid) function.  But
in the IETF, we have a tendency to just assume that the original
author(s) of the document will serve in this function.  The defacto
editor may not understand what they are being asked/expected to
do, they have no training, no formal role definition, no tools
to help with the job, nothing...

>also can block documents for inappropriately long periods of time.

IMO, this is a WG chair function.  When the editor can't establish
agreement with a reviewer, the WG chair needs to determine what
WG consensus is on the issue.

Margaret

---

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 13 May 2003 10:22:12 -0500

Just following up to Margaret's post -

> to help with the job, nothing...

(Your mileage may vary, but I'm also seen some documents derailed
because of editor job changes)

One of the points that we're making in the current WG chair training (thank
you,
Margaret) is that the decisions about who the editor is, and whether an
editor
continues to be the editor, really are WG chair decisions, even though they
are often
defaulted to "original author, who edits forever".

One might argue that the author skills for good -00 individual submissions
aren't
even the same skills as the editor skills for good -08 working group
revisions.

Spencer

---

From: "Charles E. Perkins" <[email-masked]>
Date: Tue, 13 May 2003 09:14:39 -0700

Hello Thomas,

Perhaps we can agree that it takes time to produce
quality documents, and that sometimes procedural
difficulties extend that time to seemingly unreasonable
lengths.

My original suggestion was that an Area Director should
be expected to be able to point to documentation when
raising major architectural objections to a working group
document.  This expectation should rise linearly as the
working group document progresses.  It is NOT too much
to expect an Area Director to write a two page explanation
when killing or significantly devaluing the efforts of
dozens of people.

I don't want that point to get lost in tautologies about
how documents are hard to write.

Even an Internet Draft would be better than the sometimes
terse snippets of lore that substitute for documentation
in recent memory.  At least then there is the possibility
that some process can lead to some conclusion according to
professional ethics, instead of what could appear to be
arbitrary quashing of something the AD just doesn't like.

Plus, it's not all black and white.  My point is that it's
too gray today.

Regards,
Charlie P.

---

From: Jari Arkko <[email-masked]>
Date: Tue, 13 May 2003 20:09:07 +0300

> to help with the job, nothing...

In the paid, centralized function model you'd probably see
not just more formal role definitions, but the whole process
would likely become more formal. We'd be sending detailed
Change Requests all over the place ;-( [I'm not sure whether
that would be good or bad.]

Anyway, *if* its true that IETF editors in general don't understand
what the job is, have no training or tools... perhaps we need
a BCP or training material for this purpose. I took a quick look
at what's easily found from www.ietf.org, and it seems that
there isn't that much information, at least if you don't know
what to look for. The ID guidelines and nits are mainly tailored for
someone who is making his first draft. The chairs training
concentrates on, well, chairs issues. Newcomers orientation
doesn't talk about these issues either. The directly available
material doesn't even mention xml2rfc, for instance. But to
do more than submit an initial draft to the IETF, editors need
to do many things. Perhaps it would be useful to have some
material that talks about the following:

o Tool support
   - Editing
   - ID nits checks
   - State machine drawing / verification tools (oh well,
     the IETF uses formalisms very little...)
   - Difference visualization
   - Issue tracking tools
o Specific text guidance
   - How to write IANA section (point to RFC 2434 )
   - How to write security considerations section (...)
   - Nits
   - Checklists for specific issues (beyond ID nits)
o Process
   - Role of editor
   - Tasks upon writing a new document, producing
     a new revision, list discussion, getting it
     approved at IESG, ...
o ...

--Jari