Subject: WG last call - problem description document

From: Melinda Shore <[email-masked]>
Date: Mon, 17 Nov 2003 12:06:25 -0500

This is to announce the beginning of working group last call for
"IETF Problem Statement"  
(http://www.ietf.org/internet-drafts/draft-ietf-problem-issue- 
statement-05.txt).
Last call closes on Monday, 24 November 2003.  Please send comments to
the mailing, and note that we're not kidding about the closing date.

Melinda

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 18 Nov 2003 16:19:00 +0100

Just ship it. We could spend 5 years improving it, but the need for
action is now and it just isn't worth further tuning.

  Brian

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 18 Nov 2003 11:53:37 -0800

Speaking as participant:

I have read the document, and support its publication in its current form.

            Harald

---

From: Alex Conta <[email-masked]>
Date: Tue, 18 Nov 2003 17:00:58 -0500

Here are a few comments regarding
"draft-ietf-problem-issue-statement-05.txt"  document. They are marked 
"minor" and "major".

a. Minor

Section titles are too long, too specific in describing a problem, which 
becomes ennoying when the text in the body of the section does NOT 
follow the title.

For instance, the name "The IETF Management Structure is not Matched to 
the Current Size and Complexity of the IETF" of Section 2.6 does not 
reflect adequately the content of all its subsections. For instance 
"Procedural Blockages" subsection 2.6.3 may have little to do with a 
mismatch between the size and complexity of the IETF Management structure.

A possible solution is shortening the section names, for instance:

current:
  "Participants in the IETF do not have a Common Understanding of the 
its Mission.

change to:
"the Understanding of the IETF Mission"

current:
"The IETF Management Structure is not Matched to the Current Size and 
Complexity of the IETF"

change to:
"the IETF Management Structure"

b.  Major

subsection 2.6.3 is addressing "procedural blockage"  at the IESG (Area 
Director) level. This is not the only "procedural blockage". The 
document should address the same type of problem at WG (WG Chair) level 
as well. This should be done either in the same subsection, or in a new 
subsection.

The cause of such WG procedural blockage problem can be multiple, but 
one that may be encountered in IETF, more often than it should, is when 
a chair, or both chairs have vested interest in directing the work on a 
certain path. This happens often when a chair or the chairs of a WG are 
directly involved in writing drafts for that WG. In some WGs, the chairs 
are authors or co-authors of a large number of drafts or RFCs; in such 
cases, it is likely that they will favor work or drafts that are not 
challenging, or are in alignment with their documents.

In many Standards Development Organizations, for limiting the cause of 
such impartiality, a WG chair's role excludes participation or 
authorship of drafts, and it is rather limited to moderating the WG 
meetings, and driving the WG process along, according to the 
organization's rules. Perhaps this is a rule that IETF should adopt.

c. Major

subsection 2.6.6 "Concentration of Influence in Too Few Hands"
mentions the possible negative effect of too long continuity in IESG and 
IAB memebership. However it does not mention the same negative effect in 
case of too long continuity in WG chairmanship. Possible fix is 
virtual, or by statute, term limits for WG chairs.

This subsection should address also the perception that the WG chair 
selection is not an open process, or not as open as it should be.  WG
members have in general no saying in chair selecting, or in making a 
choice between several candidates.

Regards,
Alex Conta

---

From: "Spencer Dawkins" <[email-masked]>
Date: Wed, 19 Nov 2003 19:57:13 -0600

Echoing the esteemed Brother Carpenter - just ship it. In addition to
everything Brian said, this document was cited by many of the
proposals now underway - the discussion has served its purpose.

Spencer

---

From: Keith Moore <[email-masked]>
Date: Thu, 20 Nov 2003 15:18:34 -0500

1. The document title is misleading; it should be changed to something
like "Perceived Problems in IETF" to make clear that while these
perceptions are widely held, there's not consensus on the list of
problems.

2. Remove the text

   The result will be used to guide
   the next stage of the process in the Problem Statement working group
   which is to recommend the structures and processes that will carry
   out the corrections.

from the abstract - as it is presumptive.

3. In section 2.2, the following text:

   o  The charter may not be sufficiently detailed to document the
      process and timeline to be followed by the WG.  Additional
      documents may be needed such as a roadmap or detailed plans.

could be read as indicating a problem with WG charters.  It's more 
realistic to say that there's a natural gap between the level of detail
specified in a WG's charter and the level of detail specified in a WG's
first draft protocol specifications, and our WG processes do not
currently give much guidance as to how to narrow that gap.  

IMHO IETF's biggest problem is that its working groups rarely develop
specifications by any kind of incremental refinement.  Proposals are
expected to be fairly complete when they first appear as
Internet-Drafts; otherwise they are shot down for lack of completeness. 
On the other  hand, detailed proposals are less likely to get sufficient
review because they are lengthy, and sometimes we shoot down entire
proposals  because they get some of the details wrong - because those
details were specified prematurely.  (And given that few people can be
found to read detailed proposals at early stages, is it any wonder why
we consume precious meeting time with boring, detailed presentations?)

Working groups should be expected to take problem descriptions at the
charter level and iterate them through increasing levels of detail, e.g.

- problem definition/statement
- goals document (NOT requirements, though there may be a few goals 
  that are non-negotiatble)
- functional specification / block diagram
  that broadly specifies how the functional pieces interact 
- rough specification
  which might specify protocol elements and their semantics but not to
  exact detail of presentation
- prototype specification
  which specifies a subset of the protocol in enough detail that it
  can be implemented and tested (without being ready for prime time
  in any sense - for instance, things like authentication might be 
  specified in a trivial fashion at this phase, even though security
  goals and requirements need to have already been specified) 
- draft specification
  which is a fairly complete specification for the protocol as it
  is intended to be deployed but may have bugs or a few unspecified
  areas
- PS candidate specification
  which is believed to meet the requirements for Proposed Standard

This seems like a lot of documents, but the first several of these 
documents should be quite short.

Cross-area review should happen early, probably by the functional
specification phase and certainly by the rough specification phase.

Early documents are not carved in stone, they should be revised
as it becomes apparent that changes are needed - so that there is
always a current problem definition, goals statement, functional 
spec, etc.

Not every WG should have to do each of these - it depends on various
factors such as the overall complexity of the group's work, the 
size of its user community, the amount of interaction with other
protocols and other areas, whether there are compatibility issues,
etc.  The important thing is the pattern of incremental refinement
and review, not the specific set of steps.

suggested action: reword to something like:

   o  Currently there is no expectation that WGs define their
      processes or timelines beyond the level of detail specified 
      in the group's charter; however this level of detail is rarely
      sufficient to direct a WG's activities.   Additional documents
      such as a roadmap or detailed plans are rarely produced.

4.  same section

   o  Charters regularly fail to set sufficiently granular milestones at
      which progress of WGs, individuals and documents can be evaluated.
      Also WGs often do not make more detailed work plans to refine the
      charter plans.

Again, while true, this seems phrased to put the burden on charters and
I don't think it belongs there.

I don't think it's reasonable to specify this level of detail in the
WG's  charter. To be useful, dated milestones need to be expressed in
terms of  meaningful, measurable results, and "submit an Internet-Draft
on topic XXX", while measureable, isn't meaningful.  To be meaningful
the goal needs to be understood well enough in terms of things that have
been done before.

IMHO the charter should specify an expectation of the minimum steps that
a WG will follow in producing a specification (say, in terms of the list
above) and a vague expectation of the timeframe in which the steps will
be undertaken - but should not be expected to serve as a detailed 
schedule.  Instead, WG chairs should amplify on the vague outline 
provided by the charter and produce schedules (separate from the charter
outline) for the next few months outlining specific, measurable steps 
(revise functional spec; decide on issues X, Y, Z; review documents
A, B, C) and timeframes that actually have some basis in reality.

suggested action: reword to something like:

   o  The timeframes specified in charter milestones are generally not
      grounded in reality, nor is it reasonable to expect tight
      adherence to schedules that are specified at the level of detail
      appropriate for a working group charter.

5. section 2.4

   o  The three stage hierarchy is, accordingly, seen to be excessive.

we really haven't demonstrated that three stages are excessive;
what we've seen is that the stages we have are a poor impedance match
between our energies to refine our output and the input accepted by
our "customers"

suggest change to:

   o  The three stage hierarchy thus appears to be a poor match for
      vendor expectations, market expectations, and/or energy to
      complete the process.