Subject: RE: pausable explanation for the Document Series

From: <[email-masked]>
Date: Fri, 6 Jun 2003 09:24:50 +0300

Hi Pekka,

> I partially disagree.  It's often too challenging to later change 
> fundamental design decisions made early on: just shipping a spec with a 
> thought like "we'll fix the issues later" seems like a recipe for 
> disaster.
> 
> Naturally, when updating the documents some features are  ripped off, some 
> clarifications added etc., but this *typically* cannot influence the basic 
> way specs work.  And therefore, the core things in specs must 
> be in very good shape.
> 
> In consequence, "security will be described in other documents" or 
> "deployment will be considered in other documents" doesn't sound like a 
> good idea.  These can often hide some very nasty surprises which affect 
> your design decisions on the core spec.

Of course the core of any protocol developed in the IETF should be 
extremely stable even at the PS level.  However, if how can you
get deployment concerns for many protocols that are not yet PS?  There
is a big chicken & egg problem here.  For protocol work, we should
have a tight scope on what is contained in the draft (& later PS).

John

---

From: Pekka Savola <[email-masked]>
Date: Fri, 6 Jun 2003 09:30:05 +0300 (EEST)

> have a tight scope on what is contained in the draft (& later PS).

In real world, people implement the protocols for pilot/test purposes in 
any case.  This can be done when the I-D has reached reasonable level of 
stability.

Of course, we could try to add some "experimental" category below PS, to 
try to document these pilot protocols -- but this is slightly problematic 
also, as we have to make sure people understand that if protocol is 
modified for PS, no backward-compatibility must be required..

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Fri, 06 Jun 2003 09:07:16 +0200

> modified for PS, no backward-compatibility must be required..

two examples of trying exactly that approach....

RFC 2362
     Protocol Independent Multicast-Sparse Mode (PIM-SM): Protocol
     Specification. D. Estrin, D. Farinacci, A. Helmy, D. Thaler, S.
     Deering, M. Handley, V. Jacobson, C. Liu, P. Sharma, L. Wei. June
     1998. (Format: TXT=159833 bytes) (Obsoletes RFC2117) (Status:
     EXPERIMENTAL)

RFC 3208
     PGM Reliable Transport Protocol Specification. T. Speakman, J.
     Crowcroft, J. Gemmell, D. Farinacci, S. Lin, D. Leshchiner, M. Luby,
     T. Montgomery, L. Rizzo, A. Tweedly, N. Bhaskar, R. Edmonstone, R.
     Sumanasekera, L. Vicisano. December 2001. (Format: TXT=244637 bytes)
     (Status: EXPERIMENTAL)

I think the general idea seems attractive, but Real Life has twisted our 
expectations at least in the case of PIM-SM, and I have heard rumblings 
about the same thing wrt PGM - that "the RFC is out there, and works for 
the 95% case I need" has taken a great deal of interest away from working 
on other possible solutions - or even pushing the thing onto the standards 
track at all.

It's possible that "permanently archived I-D copy" would do better. Then 
again, maybe not.

Of course, I'm not an expert in either field, so if I have to hurriedly 
retract my reading of the situation when the real experts chime in, I 
apologize in advance.....

                       Harald

---

From: Margaret Wasserman <[email-masked]>
Date: Fri, 06 Jun 2003 08:45:48 -0400

I hate to do this, but...

Not that I have any official authority to say this, but...

At this point, this mailing list is almost exclusively
discussing solutions to problems, and refinements to
those solutions.

I, too, find it more fun and constructive to discuss
solutions, but this WG is only chartered to produce:

	1 - A problem statement
	2 - Recommendations for a process to solve
		the problems.

Are people happy enough with those documents that we're
ready to publish them and move on?  If so, let's do
that. If not, please send feedback on the documents to
this list, and keep solutions discussions on the solutions
mailing list.

Thanks,
Margaret

---

From: Melinda Shore <[email-masked]>
Date: Fri, 06 Jun 2003 09:24:36 -0400

> this list, and keep solutions discussions on the solutions

I'm also very concerned about keeping the documents moving
forward, but I do think that the notion of what we need to
do about short-term solution efforts that would be hampered
by the overhead associated with working group creation do
need to be addressed and I think some interesting ideas have
been popping out of the discussion.

One of the problems with the discussion, however, is that we
still don't have a way to accommodate ad-hoc, short-lived,
individual voluntary, or what-have-you efforts, and that
does need to be dealt with.  Personally (i.e. not as chair)
I'm not crazy about things like what's going on with the
SIRS proposal (going off and collecting volunteers without
real agreement from the cohort that this is a good thing to
do) because it feels pre-emptive and unco-operative, but on
the other hand given our decision-making structure I'm not
sure what the alternatives are if anything is going to
happen in real time.  If anybody has wisdom about how to do
short-term efforts within the IETF, this would be an
excellent time to discuss it.

Melinda

---

From: Keith Moore <[email-masked]>
Date: Fri, 6 Jun 2003 10:03:28 -0400

> For protocol work, we should

tight scope is good, but there's a limit.  We tried to keep tight scope in
MIME, and in so doing, we failed to recognize some of MIME's limitations until
it was too late to change it.  I think it's useful to limit the scope of a
core protocol spec and to make it extensible, but it's also useful to 
at least map out how you expect likely extensions to work, and even to 
try a few of them - just don't put them into the base spec.

---

From: Dave Crocker <[email-masked]>
Date: Fri, 6 Jun 2003 16:15:35 -0700

Melinda,

MS> I'm not crazy about things like what's going on with the
MS> SIRS proposal (going off and collecting volunteers without
MS> real agreement from the cohort that this is a good thing to
MS> do) because it feels pre-emptive and unco-operative,

Oh boy, do we have different views about such efforts.

Let's be clear about the nature of this sort of "preemptive" and
"uncooperative" effort:

1. At the last IETF meeting, Brian came up with an idea, chatted it up,
got good reactions and suggestions, recruited someone to help with the
writing a few paragraphs on it and then circulated the idea informally.

2. The writeup was chatted up, got good suggestions and reactions, and
turned into an I-D.

3. The I-D got chatted up, got good suggestions and reactions, and
turned into an experiment.

4. The experiment has, so far, gotten volunteers, started to offer
itself for public use, and will prove useful -- or not -- on its merits.
It has no force of law, or anything other than basic credibility.

It also has not had months and months of delay.

So, in fact, this is about as far as you can get from being preemptive or
uncooperative.

On the other hand, yes, it is also very far from relying on bureaucratic
approval.

Rather than being worried that it has occurred, perhaps we should be
worried that more such efforts have not?

d/
--
 Dave Crocker <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>

---

From: John C Klensin <[email-masked]>
Date: Sat, 07 Jun 2003 10:01:34 -0400

Dave,

I need to _slightly_ support Melinda's position on this.  I 
certainly wouldn't have used words like "preemptive" or 
"uncooperative".  But...

In a cultural problem that goes back long before we, as you 
observed elsewhere, started putting more focus on static 
description than on demonstrated interoperability (and, IMO, 
blew it on both), we have had a habit of proposing, documenting, 
and sometimes even trying "experiments" which lack evaluation 
criteria... or any way, other than force of personality, to 
really determine whether they "worked".  Nor do we have real 
procedures for documenting what we have learned.

In that regard, SIRS has been, I think, better than the average. 
But it is a handy example, so let's look at it, with the 
understanding that I'm deliberately broadening the example a bit 
to illustrate a problem with the way we often do business ... 
some of that example may not actually apply to SIRS.

It establishes goals as:

>      The procedure described in this document is intended to
>      solve, or palliate, a number of related problems that
>      have been observed in the IETF process [PROBLEM]:
>
>           *    overload of the IESG (leading to delay)
>
>           *    perception that authority and influence
>                in the IETF are concentrated in too few
>                hands
>
>           *    submission of documents to the IESG that
>                still have significant problems (leading
>                to delay)
>
>           *    failure to detect fundamental problems
>                and Internet- wide issues at an early
>                stage

Certainly worthwhile.  But, quoting from your note, "...started 
to
offer itself for public use, and will prove useful -- or not -- 
on its merits.   How are those merits to be evaluated so as to 
determine whether this is a success?  The document then goes on 
to say:

>      However,
>      it is likely that drafts with at least three positive
>      reviews from SIRs in different areas will experience
>      much shorter IESG review cycles than drafts with fewer
>      positive reviews.
[...]
>      In all likelihood, Drafts without reviews will get
>      worse IESG response time than today, whereas Drafts
>      with reviews will be processed much more rapidly,
>      especially as the IESG's confidence in the SIR
>      procedure increases.

Plausible hypotheses indeed.  But, for a given document coming 
out of a given WG, we have no practical and objective way to 
accurately estimate how long it would have taken to get through 
the IESG without SIRS support, so it is not possible to 
objectively know whether SIRS sped thing up.

I don't think it likely, but a counter-hypothesis is that the 
IESG, upon receiving a document with three or five SIRS signoffs 
but disagreeing anyway, would decide that it needed to give more 
extensive explanations to those senior people, or discuss issues 
with them even before going back to discuss them with the WG. 
That would probably positively leverage quality, but would have 
a negative effect on speed.

And, of course, trying to guess at whether SIRS improved quality 
in comparison to what the IESG would have done, to/with a given 
document without it. is even more difficult.

So, you collect some reviewers, and some WGs choose to use them. 
For some of those WGs --perhaps the ones who were in good shape 
anyway but that, too, is pretty subjective-- the reviews help 
the IESG expedite the process.  For others, they don't appear 
to.  Do we then declare the experiment a success and try to make 
it mandatory?   Does anyone who doesn't think it has proven 
itself a success get to speak up without being overwhelmed in 
rhetoric and forceful assertions?  Do we infer that ADs who 
don't quickly sign off on SIRS-endorsed documents are 
obstructionist nit-pickers and need to be fired?   And so on.

As I hope my recent notes make clear, I'm not in favor of trying 
to invent procedures to alleviate any of the above, or of 
delaying things until such procedures reach some extraordinary 
consensus.  But I'd be much happier about the idea --even as an 
experiment-- had the IESG been willing to stand up and say 
"fascinating idea, lets try it".  And it would have been even 
better, at least from an "experimental" context, had the IESG 
chosen to say "It would be good to really do this as an 
experiment and therefore to be able to make some comparisons. 
So let's look at the following mix of areas, or specific groups 
of WGs, first, leave the others with status quo, and see how 
things work out.

I'm disappointed that type of response from the IESG hasn't 
happened.  Maybe they don't think they have been asked.   And 
the latter is, of course, what my recent notes have been about: 
if we need a formal document and request to publish it as an RFC 
and/or a formal WG call for consensus and response, in order to 
get an IESG response to a "we think this would be worth trying, 
do you have reactions or suggestions/requirements about how to 
try it", then, IMO, we are in deep trouble.    But, going ahead 
with something on the grounds that it is an experiment, without 
objective evaluation criteria or any real possibility of them, 
and without IESG comment, input, or participation...  That seems 
to me to be an opportunity for some future demagogue, with some 
other experiment, to try something, claim it succeeded, and then 
insist --loudly and by whispering campaign-- it was a success 
and that IESG failure to immediately adopt it indicates that all 
of those rascals should be thrown out and the procedural 
structure blown up (and replaced with one of the demagogue's 
liking) to mark their way.

        john

--On Friday, 06 June, 2003 16:15 -0700 Dave Crocker 
<[email-masked]> wrote:

> Let's be clear about the nature of this sort of "preemptive"
> and "uncooperative" effort:
>
> 1. At the last IETF meeting, Brian came up with an idea,
> chatted it up, got good reactions and suggestions, recruited
> someone to help with the writing a few paragraphs on it and
> then circulated the idea informally.
>
> 2. The writeup was chatted up, got good suggestions and
> reactions, and turned into an I-D.
>
> 3. The I-D got chatted up, got good suggestions and reactions,
> and turned into an experiment.
>
> 4. The experiment has, so far, gotten volunteers, started to
> offer itself for public use, and will prove useful -- or not
> -- on its merits. It has no force of law, or anything other
> than basic credibility.
>
> So, in fact, this is about as far as you can get from being
> preemptive or uncooperative.
>
> On the other hand, yes, it is also very far from relying on
> bureaucratic approval.
>
> Rather than being worried that it has occurred, perhaps we
> should be worried that more such efforts have not?
>
> d/
> --

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Sat, 07 Jun 2003 16:19:38 +0200

--On lørdag, juni 07, 2003 10:01:34 -0400 John C Klensin 
<[email-masked]> wrote:

> As I hope my recent notes make clear, I'm not in favor of trying to
> invent procedures to alleviate any of the above, or of delaying things
> until such procedures reach some extraordinary consensus.  But I'd be
> much happier about the idea --even as an experiment-- had the IESG been
> willing to stand up and say "fascinating idea, lets try it".

Full disclosure: I said "fascinating idea, let's try it". Or the semantic 
equivalent thereof.
I thought at the time, and still think, that the IESG doesn't have to give 
blessing to this experiment - so I spoke only as myself.

> And it
> would have been even better, at least from an "experimental" context, had
> the IESG chosen to say "It would be good to really do this as an
> experiment and therefore to be able to make some comparisons. So let's
> look at the following mix of areas, or specific groups of WGs, first,
> leave the others with status quo, and see how things work out.

Given the variability between areas that Eric Rescorla's numbers showed up, 
and the variability between groups, I think this is difficult to do.
I'd be very interested in seeing someone propose metrics to measure whether 
or not the experiment succeeded.
But I don't really see why the IESG should be the one doing that.

> I'm disappointed that type of response from the IESG hasn't happened.
> Maybe they don't think they have been asked.   And the latter is, of
> course, what my recent notes have been about: if we need a formal
> document and request to publish it as an RFC and/or a formal WG call for
> consensus and response, in order to get an IESG response to a "we think
> this would be worth trying, do you have reactions or
> suggestions/requirements about how to try it", then, IMO, we are in deep
> trouble.    But, going ahead with something on the grounds that it is an
> experiment, without objective evaluation criteria or any real possibility
> of them, and without IESG comment, input, or participation...  That seems
> to me to be an opportunity for some future demagogue, with some other
> experiment, to try something, claim it succeeded, and then insist
> --loudly and by whispering campaign-- it was a success and that IESG
> failure to immediately adopt it indicates that all of those rascals
> should be thrown out and the procedural structure blown up (and replaced
> with one of the demagogue's liking) to mark their way.

Point.

*Someone* needs to do quality control and measurement on experiments.
But .... the IESG has other jobs. Our standard-and-often-recommended 
practice when asked to do something is to figure out who to ask to do it.

Who should be asked in this case, and who should do the asking?

                  Harald

---

From: Margaret Wasserman <[email-masked]>
Date: Sat, 07 Jun 2003 10:28:48 -0400

>Who should be asked in this case, and who should do the asking?

The process document currently suggests forming a WG to evaluate
improvements to the WG process, develop metrics, run experiments,
etc...  IMO, the group should focus on making iterative
improvements to the quality, timeliness and predictability of
WG output that are expected to have near-term results.  I think
that there is a lot of low-hanging fruit regarding the process
used by WGs, including milestone management, reviews, etc.  So,
I think that we can make quite an improvement in the operation
of the IETF through process improvements in this area.

There has been some support for the creation of this WG, and for
holding a BOF in Vienna.  However, there have also been folks who
have argued that a WG is "too heavyweight" to do this, and
that we should be performing this function in a more adhoc manner.
In particular, John has suggested that the IESG run the process.

Although I accept that it is within the area of responsibility
of the IESG to run this function, we also have an acknowledged
overload problem for individual IESG members.  So, I think that
this is a perfect example of a responsibility that the IESG can,
and should, delegate to members of the community with expertise
in process improvement, metrics development, etc.

Because I think that this activity should be open to the
community, I favor a WG.

What do others think?

It would be nice to get some consensu on this, so I know what to
say in the document.

Margaret

---

From: John C Klensin <[email-masked]>
Date: Sat, 07 Jun 2003 10:50:45 -0400

--On Saturday, 07 June, 2003 16:19 +0200 Harald Tveit Alvestrand 
<[email-masked]> wrote:

>> As I hope my recent notes make clear, I'm not in favor of
>> trying to invent procedures to alleviate any of the above, or
>> of delaying things until such procedures reach some
>> extraordinary consensus.  But I'd be much happier about the
>> idea --even as an experiment-- had the IESG been willing to
>> stand up and say "fascinating idea, lets try it".
>
> Full disclosure: I said "fascinating idea, let's try it". Or
> the semantic equivalent thereof. I thought at the time, and
> still think, that the IESG doesn't have to give blessing to
> this experiment - so I spoke only as myself.

Good to know.  I agree that the IESG shouldn't have to bless 
these things.  But it _might_ have been helpful had your 
observation/ comment been a bit better advertised.  For many of 
these things, I (and perhaps others) get nervous if there is no 
appearance of _anyone_ on the "inside" saying, at least, "ok, 
doesn't seem harmful, try it".   If at least someone does, and 
no one speaks up and says "terrible, risky, doom, 
destruction,...", then going ahead seems rational.   Put 
differently, I don't see any need for formal IESG consensus. 
But an informal sanity check with the IESG seems prudent.

>> And it
>> would have been even better, at least from an "experimental"
>> context, had the IESG chosen to say "It would be good to
>> really do this as an experiment and therefore to be able to
>> make some comparisons. So let's look at the following mix of
>> areas, or specific groups of WGs, first, leave the others
>> with status quo, and see how things work out.
>
> Given the variability between areas that Eric Rescorla's
> numbers showed up, and the variability between groups, I think
> this is difficult to do. I'd be very interested in seeing
> someone propose metrics to measure whether or not the
> experiment succeeded. But I don't really see why the IESG
> should be the one doing that.

I didn't want to suggest that the IESG should do any 
evaluations.  But, if the IESG has insights about why some 
things take longer than others, and what might or might not 
impact that, which have not be shared with the community (and my 
impression is that few of them have), then it is probably time. 
With comments like the above, please distinguish between things 
that I think it would be good if the IESG would do if they knew 
how and had the time and things that I think the community 
should task the IESG to do, feasible or not.    If I wasn't 
clear enough about the above falling into the first category, I 
apologize.   And "that type" below referred to some response 
along the lines of "go ahead and try it", not the experimental 
design.

>> the latter is, of course, what my recent notes have been
>> about: if we need a formal document and request to publish it
>> as an RFC and/or a formal WG call for consensus and response,
>> in order to get an IESG response to a "we think this would be
>> worth trying, do you have reactions or
>> suggestions/requirements about how to try it", then, IMO, we
>> are in deep trouble.    But, going ahead with something on
>> the grounds that it is an experiment, without objective
>> evaluation criteria or any real possibility of them, and
>> without IESG comment, input, or participation...  That seems
>> to me to be an opportunity for some future demagogue, with
>> some other experiment, to try something, claim it succeeded,
>> and then insist --loudly and by whispering campaign-- it was
>> a success and that IESG failure to immediately adopt it
>> indicates that all of those rascals should be thrown out and
>> the procedural structure blown up (and replaced with one of
>> the demagogue's liking) to mark their way.
>
> Point.
>
> *Someone* needs to do quality control and measurement on
> experiments. But .... the IESG has other jobs. Our
> standard-and-often-recommended practice when asked to do
> something is to figure out who to ask to do it.

I wish that "figure out who to ask to do it" were true; I think 
significant evidence contradicts that assertion.  But that is a 
different problem and thread.

> Who should be asked in this case, and who should do the asking?

Harald, my notion of "propose an experiment" is that the people 
doing the proposing are obligated to propose the mechanisms by 
which it can be evaluated and its success or failure determined. 
Without that, it isn't a proposal for an experiment, it is a 
suggestion about some sort of more-or-less ritualized exercise. 
The SIRS draft is especially interesting in that regard because 
it makes a series of assertions in likelihood terms.  Are those 
assertions plausible?  Yes.  But likelihood assertions need to 
be testable -- at least going out and maybe coming in.  And the 
responsibility for demonstrating how they can be tested lies 
with the authors, not the IESG.   The IESG's sole responsibiity 
in this area, IMO, is that, if one of these things comes along 
that the IESG needs to process (and that doesn't included 
informal questions, or individual submissions to the RFC 
Editor-- although I would hope that they would apply similar 
criteria), and it proposes an "experiment" without a clue about 
evaluation, that ought to be sufficient grounds to bounce it.

All of that is pretty theoretical.   What I'm worried about 
specifically is the risk of bad experimental chains tying our 
ability to make rational protocol or procedural decisions in 
knots.  There, SIRS is a lousy example: it seems like a good 
idea, and the authors have been pretty careful about review. 
If someone can say "we will go off and try this 'experiment'"... 
"ok, it succeeded, therefore I invoke the 'running code' 
principle and the IETF needs to adopt it", we slide toward a 
process model that I find almost as frightening as I do the 
canonizing of any statement the IESG, or any AD, or the 
Secretariat, chooses to make about procedures or the belief that 
we can't do anything without formal community action.  We can 
tie ourselves in terrible knots at either extreme.

     john

---

From: Dave Crocker <[email-masked]>
Date: Sat, 7 Jun 2003 08:26:03 -0700

John,

JCK> blew it on both), we have had a habit of proposing, documenting,
JCK> and sometimes even trying "experiments" which lack evaluation 
JCK> criteria...

First, we are currently experiencing organizational lock-jaw, and the
jaw is mostly locked shut. We are focused, instead, on discussing the
problem rather than doing anything about it. That kind of situation
calls for more a more aggressive approach to taking action. Aggressive,
not brash.

Second, meaningful, objective measures for the effectiveness of such
actions are tough, as you know better than most IETF participants.
Absent really good objective measures, I am a fan of simplistic
measures.  When a solution proposal has a reasonably coherent and
constructive logic, and folks get engaged in debating and refining it,
and others are willing to try it out, then I think we have a pretty good
subjective measure of likely utility.  When we've done the experiment
for awhile, I'm guessing that the relative utility of it will be
obvious, one way or the other.

(I'll note that your question highlights a long-standing difference in
our approach to things.  I entirely agree with the desire to do
evaluations, but believe that taking reasoned action is more important
than waiting for complete analysis, evaluation or the like.  My own bias
is to view this in the category of best being the enemy of the good.
I'd guess your own bias is brashness is the enemy of long-term
effectiveness.)

JCK> It establishes goals as:
...
JCK> How are those merits to be evaluated so as to
JCK> determine whether this is a success?  The document then goes on 
>>      much shorter IESG review cycles than drafts with fewer
...
JCK> Plausible hypotheses indeed.

And your concern (criticism) about this section of the document is
entirely reasonable -- even valid.  Frankly I made a point of not
challenging this part of Brian's writing, and focused only on tuning it.
There were a number of reasons for my choice.

First, this particular stated outcome is good to state -- as a
communication act to the IESG (ie, lobbying.)  To the extent that
working groups feel a lack of standing in a 'negotiation' with an AD,
this should help their case, but only if their case has legitimate
substance.

Second, this realm is sufficiently complex to warrant the cook's
attitude of "if all of the ingredients are good stuff, the odds are high
that the result will be too."

That is, this seems to me such a patently good mechanism to put in, I
frankly do not see any potential downside. (Yes, there are opportunities
for abuse, and all institutions ossify and get abused, and ...) This is
fundamentally a "staff" position and its used is controlled by the
working group.

And, by the way, our community has shown a complete lack of skills at
anything involving social analysis, nevermind social evaluation.  So,
waiting for development and agreement on the measures you seek is an
infinite task.

We don't/can't even do valid, serious measurement of whether our
specifications are actually used.  And *that* as quite a bit easier than
the measurements you are calling for.

JCK> I don't think it likely, but a counter-hypothesis is that the
JCK> IESG, upon receiving a document with three or five SIRS signoffs 
JCK> but disagreeing anyway, would decide that it needed to give more 
JCK> extensive explanations to those senior people, or discuss issues 
JCK> with them even before going back to discuss them with the WG. 
JCK> That would probably positively leverage quality, but would have 
JCK> a negative effect on speed.

a) i strongly doubt it, in the long run; in the short run, I've no doubt
there is a "training" and adjusting process that will need to take
place.

b) we are slow enough, now, so that getting slower would not be noticed;
no i do not mean it's ok to get slower, but rather than this is an
unlikely outcome and that -- given that it is unlikely -- the downside
of it's occurring is tolerable, in an odd sort of way.

JCK> And, of course, trying to guess at whether SIRS improved quality 
JCK> in comparison to what the IESG would have done, to/with a given 
JCK> document without it. is even more difficult.

We have no concept of quality measurement now. So, effectively, you are
stacking a series of essentially impossible dependencies in the way of
taking localized, subjectively reasonable -- and apparently reasonably
popular -- action. Again, this is where our philosophies of project
management diverge utterly.

JJCK> For some of those WGs --perhaps the ones who were in good shape
JCK> anyway but that, too, is pretty subjective-- the reviews help 
JCK> the IESG expedite the process.  For others, they don't appear
JCK> to.  Do we then declare the experiment a success and try to make 
JCK> it mandatory?   Does anyone who doesn't think it has proven 
JCK> itself a success get to speak up without being overwhelmed in 
JCK> rhetoric and forceful assertions?  Do we infer that ADs who 
JCK> don't quickly sign off on SIRS-endorsed documents are 
JCK> obstructionist nit-pickers and need to be fired?   And so on.

The 'and so on' is key. My experimentalist training entirely agrees with
you. My project management experience knows that that your points really
are a trap that ensures taking no action. The and so on highlights the
fact that there are an infinite set of unknowns. We cannot wait until we
resolve all of them. For that matter, the ones you list are enough to
make clear that we would take essentially forever before being able to
start the 'experiment'. And, no, this is not a controlled experiment.

JJCK> consensus.  But I'd be much happier about the idea --even as an
JCK> experiment-- had the IESG been willing to stand up and say 
JCK> "fascinating idea, lets try it".

Well, I made a point of not citing this in the note to Melinda, simply
to keep the focus on our difference in philosophy, but... Brian cleared
this with Harald first. I classed that as a courtesy, rather than a
requirement, which is why it does not satisfy your point.  But it IS
relevant to it.

JCK>   And it would have been even 
JCK> better, at least from an "experimental" context, had the IESG 
JCK> chosen to say "It would be good to really do this as an 
JCK> experiment and therefore to be able to make some comparisons.

Having the IESG take initiative on such things would be fine, but since
that has not happened, we need to take constructive action anyway.

JCK>   That seems
JCK> to me to be an opportunity for some future demagogue, with some 
JCK> other experiment, to try something, claim it succeeded, and then 
JCK> insist --loudly and by whispering campaign-- it was a success 
JCK> and that IESG failure to immediately adopt it indicates that all

1.  In my note to Melinda, I made a point of documenting just how
extensive the community involvement in the SIRS effort had already been.
This was intended to mitigate concerns about individual rashness.

2.  The plan in fact got very strong positive encouragement from folks.
That is another reason that concern over personal demagoguery should be
reduced.

3. The serious threat from demagogues won't be reduced by any of this;
they work on their own dynamics and use whatever is convenient to their
purposes. If we give the worry about demagogues a veto over all change,
we will remain frozen forever.

d/

---

From: Scott W Brim <[email-masked]>
Date: Sat, 7 Jun 2003 15:24:17 -0400

> of the IETF through process improvements in this area.

All sounds good.

> holding a BOF in Vienna.  However, there have also been folks who have
> argued that a WG is "too heavyweight" to do this, and that we should
> be performing this function in a more adhoc manner.  In particular,
> John has suggested that the IESG run the process.
> 
> Although I accept that it is within the area of responsibility of the
> IESG to run this function, we also have an acknowledged overload
> problem for individual IESG members.  So, I think that this is a
> perfect example of a responsibility that the IESG can, and should,
> delegate to members of the community with expertise in process
> improvement, metrics development, etc.
> 
> Because I think that this activity should be open to the community, I
> favor a WG.
> 
> What do others think?

Why do we have design teams?  For that same reason, we want a relatively
small group to do most of the work in fashioning ideas and trying to get
experiments done.  WGs aren't the most fast-moving of structures.
However, refreshing new ideas from refreshing new people are going to be
very valuable, and WG design teams can get static.  So what you want is
a small group of people, with membership which must be churned
periodically (but not too often, say once a year), who take input from a
large group and come up with short-term process experiments and then
push them to completion.  You can put this in the working group
structure if you like, and we can use NomCom's procedures for
determining membership in the small group, but the only reason to do so
is that WGs are all we know how to do.  I would just call it a task
force, an ad hoc group, a kind of directorate, some group type with a
good acronym, but not bother to make it a WG.

.swb

---

From: Keith Moore <[email-masked]>
Date: Wed, 11 Jun 2003 17:19:35 -0400

> worried that more such efforts have not?

I think Melinda's concern is reasonable.  I don't have a big problem 
with SIRS specifically because according to my understanding of this 
organization I don't think SIRS is likely to usurp any real authority.  
(A big part of this is knowing the individuals involved.)  But I'd hate 
to see thousands of flowers bloom in this space.

---

From: Randy Bush <[email-masked]>
Date: Thu, 12 Jun 2003 06:26:08 +0900

the big question about the sirs (and 'greybeards'?  what, no
women?)  effort is whether it will produce solid useful reviews.
we need such badly.  and the proof is in the pudding.

randy

---

From: Dave Crocker <[email-masked]>
Date: Thu, 12 Jun 2003 11:38:35 -0700

Randy,

RB> the big question about the sirs (and 'greybeards'?  what, no
RB> women?)

1. It's odd, Randy, that you appear to be unaware of the history of the
term.  For your edification, a note is on the home page for the site.

2. Given the workload on area directors, one would think that they have
better things to do with their time than to level irrelevant, critical
witticisms against the names of sites that are donating resources to
IETF activities. No doubt such cleverness will encourage others to
donate their resources.

RB> effort is whether it will produce solid useful reviews.
RB> we need such badly.  and the proof is in the pudding.

No kidding.

One would almost think that is the reason for initiating the effort,
wouldn't one?

d/