Subject: Re: WG Quality Processes WG

From: Margaret Wasserman <[email-masked]>
Date: Mon, 02 Jun 2003 09:12:00 -0400

I was hoping that someone else would say this, so that I
wouldn't have to argue with myself in public again, but
here goes...

At 10:05 PM 5/31/2003 -0400, Margaret Wasserman wrote:
>In particular, the process document currently indicates
>that we should start a WG to improve WG Quality Processes,
>as described below.
>
>So, do people think we need a WG like this?  If so, does
>it make sense to hold a BOF in Vienna?

I do think that we should hold a BOF in Vienna, but I _don't_
agree with my own WG description anymore...

IMO, the description delves too far into "solutions space".
It also makes the assumptions that all of the recommendations
for short term improvement to the document production process
will be process-oriented changes to how WGs function, and could
therefore be deployed on a per-WG basis.

Given some of the excellent suggestions made recent (which may
also be too much in "solutions space" for this WG), those
assumptions are invalid.

Instead, I think that we should hold a BOF, with the intention
of starting a WG to:

         Improve the quality, timeliness and predictability of
         the document production process.

And, I think that we should leave it to that WG to determine
what process makes sense for the introduction of each proposal
that it undertakes.

IMO, we should have a single WG for this function, so that the
group can consider various proposals, prioritize them, and
implement them in the way that is least likely to cause chaos
and mutual interference...  What do folks think of that?  The
alternative would be to consider/implement various improvements
to the document production process separately -- either through
grass-roots efforts or through separate WGs.

What do folks think?

If we do want a WG, I think it makes sense to push for a BOF
in Atlanta.  This would let us put together a leadership
team and vet a few proposals now, rather than waiting until
Minneapolis in November to get this effort off the ground.

Would folks like to use the "solutions" mailing list as a
rendezvous point to discuss the formation of this group?
I'll send a message over there to try to get discussion
started.

Margaret

---

From: Scott W Brim <[email-masked]>
Date: Mon, 2 Jun 2003 12:23:35 -0400

I'm just reluctant to make it a WG without an immutable time-to-live,
less than a year.  I want them to take on just one or two possible
changes at a time.  A change which is so major that it takes longer
should be shepherded by the IESG (at this group's instigation?).  It can
be recreated (not just rechartered) every time.  We have a precedent,
the NomCom.  I don't want the group to get stuck in its ways.

On Sat, May 31, 2003 10:05:24PM -0400, Margaret Wasserman allegedly wrote:
>     A working group should be formed in the General Area of the IETF to
>     oversee improvements to the quality processes used in IETF WGs, and
>     to increase the effectiveness of IETF reviews at all levels.  This
>     group should take an experimental, iterative approach to these
>     improvements:
> 
>          - Identify and prioritize a set of promising proposals for
>            improvement.
>          - Figure out what each proposal is trying to improve (in
>            measurable terms) and define a metric to measure performance
>            in that area.
>          - Determine the current level of performance against the
>            defined metric.
>          - Institute each change in a few representative WGs (on a
>            volunteer basis).
>          - Measure the results to determine if each change was
>            successful.
>          - Make successful changes available IETF-wide, by publishing
>            them in BCP RFCs.
>          - As necessary, train WG chairs and other participants on the
>            how to implement the successful improvements in their WGs.
>          - Repeat as necessary.

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 02 Jun 2003 13:48:20 -0400

Hi Scott,

>the NomCom.  I don't want the group to get stuck in its ways.

I understand and share your concerns about establishing
an ongoing process bureaucracy...

However, I'm not sure what, exactly, you are proposing as
an alternative.  Everyone seems to believe that there is
a serious problem with the quality, timeliness and predictability
of WG output.  But, how do we initiate an effort to improve in
this area?

We have the strange gap between the problem-statement group
and the efforts to make actual improvements in the IETF...

The problem-statement group isn't supposed to talk about
specific solutions.  So, it is virtually impossible for us
to start efforts that are focused on accomplishing
particular solutions...  Basically, all we can do is to
encourage the creation of groups (or efforts of any kind)
that _will_ (finally!) focus on solutions.

Margaret

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 2 Jun 2003 14:24:35 -0500

Hi, Scott,

We had already started a discussion this morning about
how we organize to do "small chunks of work". It seems
to me that we are headed toward small process changes,
as opposed to "feed the IESG to pigs and start over",
so the "small chunks of work" discussion seems applicable
to process work as well.

Margaret told me that she'd gotten approval for a BoF
in 15 minutes, on one occasion. My limited experience 
has been that it takes me longer than that to get approval 
for a BoF, and about two IETF meeting intervals to charter
a working group after a BoF is proposed.

I'm also thinking that if the IESG has to choose between
considering BoF requests for process work and BoF
requests for protocol work, the protocol work will get
the nod.

If you're really good at sliding WG creation requests
through, please stay engaged. If I'm going to be involved,
I'm concerned about the inertia I've experienced when
spinning WGs up from scratch, and expect that process
WGs would take more time, rather than less.

Spencer

p.s. I'm not complaining about the amount of inertia - I work
in TSV, and we tend to NOT "go crazy" and redo TCP every
year, for good reasons. I'm just not sure the amount of
inertia I've experienced will help us improve our processes
before IETF 60 (to pick a date).

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 02 Jun 2003 15:45:06 -0400

At 02:24 PM 6/2/2003 -0500, Spencer Dawkins wrote:
>in 15 minutes, on one occasion.

That would be an exaggeration (not saying I didn't say
it).  I've only proposed two BOFs myself, both of which
were approved quite quickly (~6 hours and ~2 days)
after the final proposal was submitted.

It takes a good deal of time, though, to put together
a reasonable BOF request, and compared to that, the
time to get the BOF approved by the IESG has been
trivial (in my experience).

Margaret

---

From: Keith Moore <[email-masked]>
Date: Mon, 2 Jun 2003 16:11:09 -0400

> trivial (in my experience).

well of course some BOF requests are no-brainers 
(yep, that's obviously a good idea, shouldn't be too controversial,
make it so)

and others aren't
(gee, do we really want to open *that* pandora's box?)
or
(this seems to encompass multiple areas, how do we coordinate?)
or
(is this even in scope for IETF at all?)
or
(haven't we had 2 BOFs on this topic already?  why do we think we'll get
better results this time?)