Subject: Re: Document Blocking (Was: I-D

From: Scott Bradner <[email-masked]>
Date: Fri, 16 May 2003 13:03:24 -0400 (EDT)

some background on the blocking documents thread based on my experience on
the IESG

lets split documents that the IESG looks at into two piles
	1/ documents forwarded to the IESG by the RFC Editor to get the
IESG's recommendation 
	2/ documents that are the product of IETF working groups or
independent documents (mostly for standards track) that are being evaluated
as if they were WG documents 

case 1: 
the document is normally assigned to a specific AD - that AD does an
evaluation and comes up with a suggestion as to what the IESG should say to
the RFC Editor - this can take an arbitrarily long time

when the designated AD has done the review the document is put onto the
agenda for an IESG teleconference

the IESG then discusses the document during the teleconference - most of
the time the IESG winds up sending a note to the RFC Editor saying that it
has no objections to the document being published

sometimes the AD will work with the document authors/editors to deal with
issues that the AD finds during their review, sometimes this happens before
the document getting on the IESG agenda sometime after the IESG discussion

it is not common for the IESG to recommend that the RFC Editor not publish
a document (but it does happen a few times a year) - when it does happen
the RFC Editor is sent a 'do not publish' (DNP) recommendation.  The DNP is
written by one or two ADs - this can take an arbitrarily long time  

The RFC Editor uses the IESG response in deciding whether to publish the
document

case 2:
it is quite common for issues/questions to be raised by one or more ADs
during the IESG evaluation of a document - if the AD(s) feel strongly
enough that there is an issue that needs to be addresses they vote
"discuss" - this will block a document until the AD(s) are satisfied by
revisions in the document or as a result of discussion

when an AD has an issue with a document and has voted "discuss" the
document and the issues are discussed during an IESG teleconference -
sometimes the discussion results in the AD changing their evaluation and
removing their "discuss"

generally when an AD keeps their "discuss" after the IESG teleconference
there is some level of consensus in the IESG that the issues raised are
real and do need to get fixed - in this case it is generally the case that
other ADs to not also vote "discuss" to indicate their agreement, they
delegate one AD as the discuss holder - that AD will evaluate the document
changes and give a OK when they are happy - i.e. the fact that only one
security AD is recorded as having a discuss on a document should not be
read to say that the rest of the IESG does not support that discuss

in my experience, from time to time it was the case that I did not see IESG
consensus support for the concerns of a specific AD but the normal IESG
process does not make it easy to get around a single AD's discuss - there
is a process that was defined to do this but it has never been used, and
that process would not get around a case where two ADs had issues that the
rest of the IESG did not share

I do not recall that the IESG has ever specifically decided to not publish
a document in this group but since it can take an arbitrarily long time for
the working group (or document authors/editors) to respond to the concerns
and an arbitrarily long time for the back and forth between the AD for the
working group, the AD holding the discuss, the working group chairs, and
the working group the effect of the process may not be distinguishable from
a decision to not publish.

Scott

---

From: Keith Moore <[email-masked]>
Date: Fri, 16 May 2003 14:36:35 -0400

> some background on the blocking documents thread based on my
> experience on the IESG
> 
> independent documents (mostly for standards track) that are being
> evaluated as if they were WG documents 
> 
> case 1: 
> evaluation and comes up with a suggestion as to what the IESG should
> say to the RFC Editor - this can take an arbitrarily long time

does the RFC editor no longer impose a timeout on IESG feedback for
such documents?  (where such feedback could say "we need more time"
or "we're discussing with the author")

> case 2:
> it is quite common for issues/questions to be raised by one or more
> ADs during the IESG evaluation of a document - if the AD(s) feel
> strongly enough that there is an issue that needs to be addresses they
> vote "discuss" - this will block a document until the AD(s) are
> satisfied by revisions in the document or as a result of discussion
> 
> sometimes the discussion results in the AD changing their evaluation
> and removing their "discuss"
> 
> generally when an AD keeps their "discuss" after the IESG
> teleconference there is some level of consensus in the IESG that the
> issues raised are real and do need to get fixed - in this case it is
> generally the case that other ADs to not also vote "discuss" to
> indicate their agreement, they delegate one AD as the discuss holder -
> that AD will evaluate the document changes and give a OK when they are
> happy - i.e. the fact that only one security AD is recorded as having
> a discuss on a document should not be read to say that the rest of the
> IESG does not support that discuss

maybe now that the votes are being publicized the IESG might want to
consider changing that.  it was handy to have only one discuss holder
because only one vote had to be changed to move the document forward.
but now perhaps it would be useful for IESG to separate the notion of
"who thinks this document has problems" from the notion of "who has the
token to say when the problems with this document are fixed"

> in my experience, from time to time it was the case that I did not see
> IESG consensus support for the concerns of a specific AD but the
> normal IESG process does not make it easy to get around a single AD's
> discuss - there is a process that was defined to do this but it has
> never been used, and that process would not get around a case where
> two ADs had issues that the rest of the IESG did not share

concur.  very occasionally, it seemed to me that another IESG member's
concerns were either unfounded or too trivial to hold up passage of the
document.  and at least while I was there, we didn't have a good way
to work past those problems.  

(I should clarify that IMHO the vast majority of IESG Discuss concerns
were well-founded and appropriate.)

Keith

---

From: Scott Bradner <[email-masked]>
Date: Fri, 16 May 2003 15:27:38 -0400 (EDT)

> or "we're discussing with the author")

formally, they do, but it is meangingless

Scott

---

From: Pete Resnick <[email-masked]>
Date: Fri, 16 May 2003 14:37:48 -0500

On 5/16/03 at 2:36 PM -0400, Keith Moore wrote:

>[Scott Bradner:]
>>in my experience, from time to time it was the case that I did not 
>>see IESG consensus support for the concerns of a specific AD but 
>>the normal IESG process does not make it easy to get around a 
>>single AD's discuss - there is a process that was defined to do 
>>this but it has never been used, and that process would not get 
>>around a case where two ADs had issues that the rest of the IESG 
>>did not share
>
>concur. very occasionally, it seemed to me that another IESG 
>member's concerns were either unfounded or too trivial to hold up 
>passage of the document. and at least while I was there, we didn't 
>have a good way to work past those problems.

I don't understand how that jibes with:

>>The problem with the current process (as I understand it) is that 
>>it allows documents to be blocked by one or two IESG members 
>>without the consensus of the group.
>
>I don't see this as a problem at all.

You concur with the problem outlined by Scott above: There have been 
(perhaps rare) cases where someone claimed there to be a serious 
enough problem to hold up a document where the other IESG member's 
concerns were unfounded, yet there was no way to work past this hold 
up.

>Many of the issues that hold up document publication are not easily 
>understood without expertise in that particular area.

But if the expert in that area can't explain to the rest of the IESG 
why all should agree with the "discuss", then:

a) Perhaps it's unfounded
b) How on earth is that "expert" going to explain it sufficiently to 
the working group whose document is being held up.

As Dave said, "No matter how stellar the expertise of someone, if 
they cannot convince others that their views are correct, something 
is very, very wrong."

>Also, IESG does not "block" documents, it explains what is wrong 
>with them and sends them back to the working group.

But, as Scott said:

>>...since it can take an arbitrarily long time for the working group 
>>(or document authors/editors) to respond to the concerns and an 
>>arbitrarily long time for the back and forth between the AD for the 
>>working group, the AD holding the discuss, the working group 
>>chairs, and the working group the effect of the process may not be 
>>distinguishable from a decision to not publish.

Quite often, "discuss" (especially in the absence of an 
understandable explanation) ends up being, for all intents and 
purposes, identical to "block".

>It's simply infeasible to expect all of IESG to reach consensus on 
>every issue that requires a change to a document. For a deep 
>technical issue, there *might* be four people on IESG who really 
>have enough appreciation for that issue to express an informed 
>opinion - and two of those might have to stretch to understand it.

We're not talking about cases where some members of the IESG have a 
technical issue and the rest say, "Well, I don't understand that well 
enough to have a technical opinion on this, but you've convinced me 
it's a serious enough issue to warrant discussing the document 
further." That *is* consensus to discuss the document. It's when the 
objecting members *don't* explain the problem sufficiently to 
convince the others that there even *is* a serious issue. The IESG 
certainly *must* come to consensus that there is a real issue before 
stopping a document from moving along.

>A common case is to have one or two Yes votes, one or two Discuss 
>votes, and the rest No Objection. That's not a case of one or two 
>people holding up the document against an overwhelming majority. 
>It's more like a tie.

I would be deeply concerned if there were one or two Discuss, two 
Yes, and the rest No Objection *and* the Discuss voter(s) couldn't 
convince any of the Yes or No Objection people to change their votes 
to Discuss. Unfortunately, the current process allows that to hold up 
a document, and as both you and Scott have said, it has happened at 
least a few times during each of your tenors on the IESG.

>(I should clarify that IMHO the vast majority of IESG Discuss 
>concerns were well-founded and appropriate.)

That might be true. Understand, though, that given the current 
process, it is impossible to tell the difference between the ones the 
IESG considers well-founded and the one's it does not. Hence:

>>the fact that only one security AD is recorded as having a discuss 
>>on a document should not be read to say that the rest of the IESG 
>>does not support that discuss
>
>consider changing that.

I agree completely. But that doesn't change the fact that in those 
cases where the IESG does not support the discuss of one or two 
members, there should be some (simple) way to over-rule the "discuss" 
and move the document forward. That's why there needs to be a process 
change.

pr
-- 
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Aaron Falk <[email-masked]>
Date: Fri, 16 May 2003 13:15:24 -0700

> token to say when the problems with this document are fixed"

This seems like a good suggestion.

--aaron

---

From: Keith Moore <[email-masked]>
Date: Fri, 16 May 2003 16:27:43 -0400

> concerns were unfounded, yet there was no way to work past this hold 

I think my biggest problem was with the suggestion that IESG consensus
should be required to block a document, when that's clearly not
feasible.  

I do think it would be useful to have some sort of mechanism (lighter
weight than an appeal) by which a WG or IESG could push back on a single
IESG member's objections when they seemed trivial or poorly founded. 
I don't have a good suggestion for how to do that, but I do feel certain
that requiring IESG consensus on every Discuss is too onerous.

> a) Perhaps it's unfounded

Perhaps, but on balance it is safer to give the objecting IESG member
the benefit of the doubt.  And these days the co-AD often serves as a
moderating influence in practice, even if he doesn't have formal power
to veto his co-AD's Discuss.

> the working group whose document is being held up.

Often this is difficult, because the WG lacks the very expertise that is
needed to understand the problem.  But we shouldn't blame the IESG for
the WG's lack of expertise.

> they cannot convince others that their views are correct, something 

And sometimes Dave is very, very wrong.  What I might argue instead is
that if success of a protocol hinges on something that is too subtle for
most of its implementors and designers to understand, that success is
doubtful even if the protocol specification properly considers that
subtlety. But even that isn't always true. 

> >>distinguishable from a decision to not publish.

Right, but IESG has timeouts in place for almost all phases of its
deliberative process.   It's the WG that can hold up things for
arbitrary amounts of time (sometimes because the authors and/or chair
are sitting on the document and the WG doesn't know what is happening,
sometimes because the WG refuses to make changes and doesn't know what
to do next).   

Now it might be useful to have a well-documented process by which
IESG provides feedback to a WG on a document, including public
disclosure of the feedback on a web page, and to have an explicit
timeout by which a WG is expected to:

a. revise the document and resubmit to IESG
b. outline a plan for fixing the document and ask for approval of 
   that plan
c. abandon the document
d. appeal

But it would be even more useful to try to get an explicit dialogue
going between the relevant ADs and the relevant WG participants,
because most problems can be solved in this way.  

It's a real trick to outline a process that both streamlines getting
things fixed that need to be fixed (hopefully the normal case), and at
the same time provides mechanisms to safeguard against abuse.  Most
problems will be solved by dialogue, but it's hard to engage in
productive dialogue under a deadline, say if you know you've got to
resolve things in X days or you'll get burned.  

> stopping a document from moving along.

Sometimes only one person on IESG really understands the problem. 
Maybe that AD should need to come up with some more support for his
position somehow, if not on IESG than somewhere else.  Maybe he should
get to write a dissenting opinion.   But I don't think it's reasonable
to dismiss those concerns simply because nobody else on IESG understands
them.  And I think that at least initially the presumption should be
that the AD is right - because he usually is, and he's having to stick
out his neck to raise this objection.

(Even before we had public disclosure of the IESG ballots, new ADs
learned fairly quickly that raising substantial objections could cost
them both in terms of ease of working with other IESG peoople, and in
temrs of the time spent in discussion with the relevant parties to try
to work out the differences.  So there's a significant incentive to
avoid raising non-trivial objections, sometimes even when they're fairly
important.  One result is that ADs raise trivial objections when more
serious objections are really warranted.)

> to Discuss.

That's not the way it works.  Since every Discuss vote has to be
explicitly changed before the document can move forward, issues get
resolved faster when there are fewer Discuss votes, and the polite thing
to do is to not vote Discuss when someone else has already raised the
same objection.  so if you get multiple Discuss votes only when there
is a very strong and widespread objection to a document, or when there
are widely varying objections to a document.

> IESG considers well-founded and the one's it does not.

I don't know whether this can be fixed.  What seems to one person
to be a trivial or unfounded concern might actually be deep wisdom.

Keith

---

From: Pete Resnick <[email-masked]>
Date: Fri, 16 May 2003 19:28:51 -0500

On 5/16/03 at 4:27 PM -0400, Keith Moore wrote:

>I do think it would be useful to have some sort of mechanism 
>(lighter weight than an appeal) by which a WG or IESG could push 
>back on a single IESG member's objections when they seemed trivial 
>or poorly founded.

Agree completely. I think that's what I've been looking for.

>I don't have a good suggestion for how to do that, but I do feel 
>certain that requiring IESG consensus on every Discuss is too 
>onerous.

Actually, I think with such a lightweight mechanism in place, this is 
exactly what you get: If someone says "discuss" and the rest of the 
IESG thinks it is reasonable (even if they don't exactly understand 
the details of the problem), then the IESG as a whole has rough 
consensus to discuss. If a large proportion of the IESG thinks it is 
trivial or poorly founded, then the IESG does not have consensus and 
the mechanism can be used to push back on the "discuss".

Don't get hung up on "consensus" when thinking about it in the 
context of the current process. In the current process, I consider 
one "discuss" and the rest "no objection and the discuss seems legit" 
to be consensus to discuss. It's just that in the current process, 
that is indistinguishable from one "discuss" and the rest "no 
objection and the discuss sounds like complete nonsense to me and no 
one has explained to me what the problem is." The latter isn't 
consensus, and I think we agree that we don't like the latter, 
however rare it might be.

>And sometimes Dave is very, very wrong.

Thanks, that really helped progress this discussion. :-(

(Let's try to keep this nonsense out of the discussion from now on.)

>What I might argue instead is that if success of a protocol hinges 
>on something that is too subtle for most of its implementors and 
>designers to understand, that success is doubtful even if the 
>protocol specification properly considers that subtlety. But even 
>that isn't always true.

Perhaps, but again, if these occurrences are rare anyway, I don't see 
why we cannot expect an expert to explain this to the satisfaction of 
at least the rest of the IESG, if not the WG.

>arbitrary amounts of time (sometimes because the authors and/or 
>chair are sitting on the document and the WG doesn't know what is 
>happening, sometimes because the WG refuses to make changes and 
>doesn't know what to do next).

Let's not get into finger pointing here. We both know of situations 
where a document is stuck in the WG (for both unreasonable and 
reasonable cause), we both know of situations where the IESG has 
ignored timeouts (for both unreasonable and reasonable cause), and we 
both know of cases where the token has gotten lost (which can occur 
even now with the tracker, when there is a discuss item that the WG 
doesn't understand and can't get enough feedback to fix). We've been 
talking about unsticking at one point, within the IESG. It is a 
separate problem to deal with unsticking within the WG, which is 
probably even trickier.

>b. outline a plan for fixing the document and ask for approval of  that plan
>c. abandon the document
>d. appeal

Agree wholeheartedly.

>because most problems can be solved in this way.

Most of the time, I think that actually exists. I think what we are 
talking about here is solving the problem for the times that it 
doesn't, because those failures are what sticks out like a sore thumb 
(and engenders distrust between the IESG and the rest of the IETF).

>learned fairly quickly that raising substantial objections could 
>cost them both in terms of ease of working with other IESG peoople, 
>and in temrs of the time spent in discussion with the relevant 
>parties to try to work out the differences. So there's a significant 
>incentive to avoid raising non-trivial objections, sometimes even 
>when they're fairly important. One result is that ADs raise trivial 
>objections when more serious objections are really warranted.)

I've heard this from several former ADs, and perhaps it indicates a 
further problem: The idea that because one AD raises substantial 
objections, other ADs would make life difficult for him or her, and 
that trivial objections do not cause that, is unbelievably bad. It is 
a further indication of distrust among the ADs in the IESG, the very 
same thing that would make anyone think that "a way to stop bad 
things from happening" type of veto is necessary. I find the idea 
that any AD would behave this way instant grounds for recall. The 
IESG is supposed to be acting as a group to do global-issue review. 
If new ADs are forced to hold their tongues, we need to start 
replacing the old ADs.

As far as time spent discussing the problems, it seems that a 
knowledgeable directorate would solve that problem.

>>convince any of the Yes or No Objection people to change their 
>>votes to Discuss.
>
>resolved faster when there are fewer Discuss votes, and the polite 
>thing to do is to not vote Discuss when someone else has already 
>raised the same objection. so if you get multiple Discuss votes only 
>when there is a very strong and widespread objection to a document, 
>or when there are widely varying objections to a document.

I know that's not the way it works now. But now that the process is 
becoming more public (due to the tracker), I really think it needs to 
be that way. At the very least, you should never see a single Discuss 
and two Yes votes; that would be an indication (to an outside 
observer) of serious disagreement in the IESG and something that 
needs explanation.

pr

---

From: Keith Moore <[email-masked]>
Date: Fri, 16 May 2003 21:26:41 -0400

> ignored timeouts (for both unreasonable and reasonable cause),

actually, no.  I don't know of a case where the latter has occurred.

> and we both know of cases where the token has gotten lost (which can 
> occur even now with the tracker, when there is a discuss item that 
> the WG doesn't understand and can't get enough feedback to fix). 
> We've been talking about unsticking at one point, within the IESG. 
> It is a separate problem to deal with unsticking within the WG, 
> which is probably even trickier.

I'm all for unsticking things.  But I suspect we're to the point that IESG is
largely unstuck, and that most of the documents that get stuck are stuck
either within a WG or because of poor communications.  So I'm interested in
focusing on these sources of stickiness.  Also, I think that unsticking IESG
is better left to IESG - i.e. to people who understand the subtleties of that
environment, and who have more than adequate interest in working more
effectively.

> that any AD would behave this way instant grounds for recall. 

I agree with you that ADs don't (or at least didn't when I was on IESG) always
have enough trust of one another.  I hope things are better now, but I suspect
some of the distrust is inherent in the combination of diversity of
backgrounds and workload.  Regarding the veto and recall, I don't follow what
you are saying.

I don't know that anyone deliberately tried to make life difficult for me
because I raised technical objections when I was on IESG.  I do think there
was an atmosphere of "hey, we have enough work to do as it is, please don't
make things more difficult unless you have a good reason" along with
(sometimes) a failure to recognize good reasons.

> observer) of serious disagreement in the IESG and something that 

Well, again, I think changes to how IESG works internally are better left for
IESG to make.  They have plenty of incentive and they are more likely to
understand the effects of those changes than non-members or even former members.

Keith

---

From: John C Klensin <[email-masked]>
Date: Sat, 17 May 2003 08:01:13 -0400

A few observations on "document blocking".  Others are covering 
the "discuss" theme, but there is another one that might be 
slipping toward the cracks...

In his note of Friday, 16 May, 2003 13:03 -0400 Scott Bradner 
<[email-masked]> wrote:

> 	1/ documents forwarded to the IESG by the RFC Editor to get
> the IESG's recommendation
> independent documents (mostly for standards track) that are
> being evaluated as if they were WG documents
>
> case 1:
> the document is normally assigned to a specific AD - that AD
> does an evaluation and comes up with a suggestion as to what
> the IESG should say to the RFC Editor - this can take an
> arbitrarily long time
>...
> sometimes the AD will work with the document authors/editors
> to deal with issues that the AD finds during their review,
> sometimes this happens before the document getting on the IESG
> agenda sometime after the IESG discussion

And sometimes the AD has other things on his or her mind, and 
this can take an arbitrarily long time no matter how responsive 
the author is.

> it is not common for the IESG to recommend that the RFC Editor
> not publish a document (but it does happen a few times a year)
> - when it does happen the RFC Editor is sent a 'do not
> publish' (DNP) recommendation.  The DNP is written by one or
> two ADs - this can take an arbitrarily long time
>...

> case 2:
> it is quite common for issues/questions to be raised by one or
> more ADs during the IESG evaluation of a document - if the
> AD(s) feel strongly enough that there is an issue that needs
>...
> to be addresses they vote "discuss" - this will block a
> document until the AD(s) are satisfied by revisions in the
> document or as a result of discussion
>
> when an AD has an issue with a document and has voted
> "discuss" the document and the issues are discussed during an
> IESG teleconference - sometimes the discussion results in the
> AD changing their evaluation and removing their "discuss"

And sometimes, it results in the AD saying, more or less, "I 
will explain my reasons in a note or draft writeup".  At that 
stage, unless there is a clear consensus that the AD is off 
base, the IESG will give the AD as much time as needed to do 
that writeup (after all, each AD might be in the same position 
next time).  That can take an arbitrarily long time.

> I do not recall that the IESG has ever specifically decided to
> not publish a document in this group but since it can take an
> arbitrarily long time for the working group (or document
> authors/editors) to respond to the concerns and an arbitrarily
> long time for the back and forth between the AD for the
>...

Also, Keith raised a question and Scott responded on Friday, 16 
May, 2003 15:27 -0400:

>> does the RFC editor no longer impose a timeout on IESG
>> feedback for such documents?  (where such feedback could say
>> "we need more time" or "we're discussing with the author")
>

And Keith later wrote, on Friday, 16 May, 2003 16:27 -0400:

> Right, but IESG has timeouts in place for almost all phases of
> its deliberative process.   It's the WG that can hold up
> things for arbitrary amounts of time (sometimes because the
> authors and/or chair are sitting on the document and the WG
> doesn't know what is happening, sometimes because the WG
> refuses to make changes and doesn't know what to do next).

Now, my first observation is that phrases equivalent to 
"arbitrarily long time" appear much too often above, especially 
when they are linked to experience in which things have taken a 
very long time.   There is certainly a difference procedurally 
and formally between blocking by "discuss"  and "just tie it up 
for an "arbitrarily long time" while something is ostensibly 
being done.  But, in practical terms, there really is no 
difference at the end of the day: the document isn't getting out 
and isn't getting cleanly rejected with clear reasons either.

Part of the problem is that, as with the "RFC Editor timeout" on 
non-WG/ non-standards track submissions (Scott's "case 1"), the 
timeouts to which Keith refers are, in my observation, routinely 
ignored.

Certainly sometimes, probably often, the delays can be pinned on 
the WGs as Keith suggests.  But there is a loop in that 
particular bit of finger-pointing too.  Most of us who have 
observed or worked with long-lived WGs have noticed that it is 
possible for them to reach a point of exhaustion in which 
everything happens slowly, if at all, getting a reasonable 
number of people to focus on an issue or decision or to 
carefully review a document.  So, suppose I were an AD holding 
the token on a WG document and wanted to _cause_ that to happen 
(for the record, I'm not aware of any cases in which this has 
been done deliberately.  But the intent is less important than 
the effect).  I would send the WG a long-ish list of things that 
I insisted be fixed, some important, some trivial, and at least 
a few guaranteed to irritate them (possibly by raising old 
issues that the WG thought it had already become exhausted 
discussing).  I would wait for a response, possibly a whole new 
draft.   I might insist that the new draft be Last Called within 
the WG, just to be sure that consensus still existed.  If the WG 
asked me questions about my position, I'd take my time 
answering.   I'd count all of the time the WG spent on that 
process as "WG delay" and the current structure of the tracking 
system would help me document that.   Then I'd do it again -- 
I'd review the revised document, find more stuff to object too 
(even if it was completely trivial) and send the document back 
again.  If the WG Chair and Editor(s) shared the issues with the 
WG, the odds are good that frustration and exhaustion would set 
in quickly.   If they didn't, but tried to negotiate the trivia 
(remembering that opinions about what is, and is not, trivial 
can vary widely depending on one's perspective) privately, but 
that took a while (it always does), the WG members would get 
frustrated about the apparent "black hole" situation.

My conclusion is that we need to have timeouts, and they need to 
be meaningful.   Meaningless timeouts may be worse than no 
timeouts at all.  The problem is how to make, and keep, them 
meaningful, especially when the rules we already have are --it 
seems regularly-- ignored.  And we should figure out ways to do 
that which don't compromise quality of review.  Examples (I'm 
not really proposing solutions here, just possible ways to think 
about the problem):

* After an AD gets a document from a WG with a request for Last 
Call, or after Last Call completes, he or she has N weeks to get 
the Last Call issued or to get the document in front of the 
IESG.  The document may, instead, be returned to the WG during 
that window with an explanation.   But, if it is neither 
returned nor processed within N weeks, the AD must provide a 
public explanation of why the added delay is needed.  And, at 2 
or 3 times N, there needs to be IESG consensus --and another 
public statement-- as to the importance and reasonableness of 
more delay.

* For "case 1" documents, remember that "sometimes the AD will 
work with the document authors/editors to deal with issues that 
the AD finds during their review" (from Scott's explanation 
quoted above)is a process violation.  For those documents, the 
IESG is permitted only to tell the RFC Editor that there is an 
end-run going on, or that the suggestion, if implemented, would 
be harmful to the Internet.   "Make an AD happy" or "resolve any 
'issues' the AD has" are not on that very short list.  Certainly 
we should not discourage an AD who has the time and interest 
from trying to sort "issues" out with an author.  But, until and 
unless we change the procedures, that should occur by the AD 
--more or less as an individual member of the community-- saying 
to the author "I've got some problems with this and would like 
to work with you on them".  The author can then pull the 
document out of the queue and get to work.   Or he or she can 
decline to do that, at which point the AD --again, acting 
strictly as an individual-- could reasonably tell the RFC Editor 
that the discussion/interaction occurred as input into the RFC 
Editor decision about publication.  But an AD's sitting on an 
individual submission to "work on issues" under the umbrella of 
doing "an evaluation and com[ing] up with a suggestion as to 
what the IESG should say..." is abuse of authority.  There will 
certainly be times when the initial "timeout" period is 
insufficient to conduct even the minimal review that 2026 
requires and permits.   But, then, let's get a public notation 
that a delay has been requested and an explanation for it: it 
might help move things along, or, at worst, provide the 
community with a paper track to help evaluate whether the 
particular AD should continue to hold that position.

     john

---

From: Keith Moore <[email-masked]>
Date: Sat, 17 May 2003 10:25:34 -0400

> next time).  That can take an arbitrarily long time.

My recollection is that during the time that I was on IESG it became much
less acceptable for an AD to explain his reasons for a Discuss at a later
time.  By the end of my 2nd term we were expected to exlain our reasons at the
time we voted Discuss.  The explanations could be brief, or the AD could vote
Defer for one telechat (usually two weeks) and then change it to Discuss once
the explanation was supplied, but "I'll write it up later" wasn't acceptable
anymore.  And a Defer automatically changed to No Objection if not explicitly
changed by the next telechat.

---

From: Margaret Wasserman <[email-masked]>
Date: Sat, 17 May 2003 20:53:55 -0400

At 10:25 AM 5/17/2003 -0400, Keith Moore wrote:
>time we voted Discuss.

Explain them to whom?  The rest of the IESG?  What good does that
do for the WG, if the reasons aren't documented and mailed to the
WG mailing list?

Margaret

---

From: Keith Moore <[email-masked]>
Date: Sun, 18 May 2003 00:17:04 -0400

> do for the WG, if the reasons aren't documented and mailed to the

well...there's a fair amount of presumption in those questions.

providing an explanation to IESG forced the objecting AD to come up 
with some explanation within two weeks, rather than waiting until he 
found the time to figure out how to explain what was wrong.  
essentially, it made providing such explanations a high-priority task 
where it wasn't so beforehand, and this substantially reduced the 
amount of time it took to get feedback to WGs.  it also made it at 
least somewhat more difficult for an IESG member to block a document on 
trivial grounds, since the other IESG members could and did push back 
on ADs whose objections seemed trivial.

of course the responsible AD _was_ expected to communicate those 
objections to the appropriate parties, which generally meant the 
document authors for minor fixes and the WG mailing list for major 
changes.  and these days the IESG member's discuss comments also end up 
on the web via the document tracker.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 19 May 2003 12:02:11 +0200

--On 17. mai 2003 08:01 -0400 John C Klensin <[email-masked]> wrote:

> And sometimes, it results in the AD saying, more or less, "I will explain
> my reasons in a note or draft writeup".  At that stage, unless there is a
> clear consensus that the AD is off base, the IESG will give the AD as
> much time as needed to do that writeup (after all, each AD might be in
> the same position next time).  That can take an arbitrarily long time.

replying to a sub-point of a sub-point in this thread (more later...)

actually we imposed a deadline on ourselves last year.
Discuss writeups are due before the next telechat, or the Discuss will be 
considered lifted. Most Discusses that are communicated vocally on the call 
have their writeups on the list before the end of the call; Discusses that 
come in on email have their writeup with them, of course.

Note that not all writeups need to be long - for instance, Randy's DISCUSS 
comment on EPP was about one sentence long, but it re-triggered a huge 
debate within the WG about appropriate considerations for privacy.

(tangential..... I am sometimes frustrated that WGs seem to take a Discuss 
as "declaration from on high that a technical decision needs to be 
changed", rather than as a challenge asking them to explain their positions 
better; afrter all, what has happened is that the IESG has failed to 
understand that the WG position is reasonable; either the WG is wrong, or 
the documents have insufficient convincing power - increasing the 
convincing power SHOULD be the right answer in some cases. But that seems 
rare....)

                       Harald

---

From: John C Klensin <[email-masked]>
Date: Mon, 19 May 2003 09:26:28 -0400

--On Monday, 19 May, 2003 12:02 +0200 Harald Tveit Alvestrand 
<[email-masked]> wrote:

> Discuss writeups are due before the next telechat, or the
> Discuss will be considered lifted. Most Discusses that are
> communicated vocally on the call have their writeups on the
> list before the end of the call; Discusses that come in on
> email have their writeup with them, of course.

This is a significant improvement.  Have you seen the difference 
in throughput?

Just for clarification, this applies to _all_ documents -- 
standards-track and not, and both WG-originated and individual 
submissions.  Right?

> (tangential..... I am sometimes frustrated that WGs seem to
> take a Discuss as "declaration from on high that a technical
> decision needs to be changed", rather than as a challenge
> asking them to explain their positions better; afrter all,
> what has happened is that the IESG has failed to understand
> that the WG position is reasonable; either the WG is wrong, or
> the documents have insufficient convincing power - increasing
> the convincing power SHOULD be the right answer in some cases.
> But that seems rare....)

It seems to me that this is a call for more WG training and/or 
for an explicit, boilerplate, cover note from IESG members to 
WGs (and individual authors) indicating precisely that 
additional explanation would be as welcome as a change.   The 
latter, at least, would seem to be easy to implement.

    john

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 19 May 2003 08:51:18 -0500

I'm glad to see Harald and John making this point.

I see two common reactions, based on whether the WG sees IESG as
gatekeepers ("you can't do this") or as questioners ("we don't understand
why
you want to do this").

Using a feedback template that emphasizes the IETF' role as "questioners"
seems helpful.

Spencer

----- Original Message -----
From: "John C Klensin" <[email-masked]>
To: "Harald Tveit Alvestrand" <[email-masked]>;
<[email-masked]>
Sent: Monday, May 19, 2003 8:26 AM
Subject: Re: Document Blocking (Was: I-D

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 19 May 2003 10:22:17 -0400

At 12:02 PM 5/19/2003 +0200, Harald Tveit Alvestrand wrote:
>changed", rather than as a challenge asking them to explain their 
>positions better;

I agree with other comments that this could be made clearer in
how IESG DISCUSS comments are communicated to the WG.  For
example, why doesnt' the IESG member with the DISCUSS send a
message directly to the WG raising the issue?

>after all, what has happened is that the IESG has failed to understand 
>that the WG position is reasonable; either the WG is wrong, or the 
>documents have insufficient convincing power - increasing the convincing 
>power SHOULD be the right answer in some cases. But that seems rare....)

Why are there only two choices that "the WG is wrong, or the
documents have insufficient convincing power"?  While I don't
think it happens very often, couldn't the IESG be wrong?

Also, some AD review comments and DISCUSS comments are really
about matters of taste -- the belief that a section should be
removed from a document because it is redundant (not wrong,
just redundant), the opinion that some historical note should be
added explaining why something was done a particular way, etc.
In those cases, there may be no right or wrong.  In general,
the IESG wins because they can block publication of the document.
Do you think that's reasonable?

Margaret

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 19 May 2003 17:00:56 +0200

--On 19. mai 2003 09:26 -0400 John C Klensin <[email-masked]> wrote:

> This is a significant improvement.  Have you seen the difference in
> throughput?

We haven't really measured throughput... Eric Rescorla's numbers might show 
a change....?
> standards-track and not, and both WG-originated and individual

Just for standards-track, since we don't formally ballot informationals.

we've actually found that tracking the comments on informationals was 
problematic - ie we have had cases where we had comments on WG documents, a 
new version arrived 3 months later that purported to fix the comments, and 
we couldn't figure out from the logs what the original comment was.

We've got a new system for ballot handling being built now (as part of the 
I-D tracker), and we decided that we'd simply add ballots for 
non-standards-track too - but with different approval rules (no need to 
wait for "enough" people to say no-objection; the document is OK if it's 
been on a telechat agenda and nobody objects).

Trying to fix things where we can.......

                    Harald

---

From: Keith Moore <[email-masked]>
Date: Mon, 19 May 2003 10:56:41 -0400

> >(tangential..... I am sometimes frustrated that WGs seem to take a
> >Discuss as "declaration from on high that a technical decision needs
> >to be changed", rather than as a challenge asking them to explain
> >their positions better;
> 

Sometimes they do, or they engage the document authors.  But this
doesn't always work well.  It can produce long protracted discussions
that don't terminate.  If multiple IESG members have objections then
it can be hard to keep track of when all the objections are satisfied.

At the same time, my experience was that there were often disconnects in
communicating Discuss concerns to WGs and document authors - not because
of a failure to try, but because the ADs and WG members were working
from very different sets of assumptions (often not well-articulated)
and/or speaking different languages.  The situation was made worse when
one AD had to be an intermediary between another AD and the WG.

So I think some form of direct communication from the objecting ADs to
the WG is appropriate, but having the responsible AD send the Discuss
comments to the WG mailing list would serve that purpose.  I also think
it's useful if the WG chairs and document authors could get the relevant
ADs on a conference call - it's often easier to resolve differences over
the phone than over email, and this could save weeks of delay.

> >after all, what has happened is that the IESG has failed to
> >understand that the WG position is reasonable; either the WG is
> >wrong, or the documents have insufficient convincing power -
> >increasing the convincing power SHOULD be the right answer in some
> >cases. But that seems rare....)
> 

Of course.  Though if the document is misunderstood by IESG the chances
that it will be misunderstood by ordinary readers seems high.

> Do you think that's reasonable?

Yes -- not because the IESG is always right, but because *somebody* has
to decide, and the IESG is in a better position to make a determination
as to whether a document is suitable for the Internet as a whole than
the WG is.  You could move the burden of that decision to some other
body, but you'd still have the same problem.  Either somebody can trump
the other's desicion or you end up with a process that doesn't
terminate.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 21 May 2003 14:13:28 +0200

--On 19. mai 2003 10:22 -0400 Margaret Wasserman <[email-masked]> wrote:

>> Discuss  as "declaration from on high that a technical decision needs to
>> be  changed", rather than as a challenge asking them to explain their
>> positions better;
>

Sometimes he will, sometimes there are reasons not to....

one reason is that he wants the sponsoring AD's feedback first... after 
all, he might be wrong, or it might not be important enough. Or someone 
else might be better placed to take it - a common call is "Steve, this 
looks like a security issue to me - I'll say no-ob if you say it's OK; 
otherwise, either you or I could take it".
at times, too, the sponsoring AD has suggested that an issue would be 
better handled by the document editor or even with a note to the RFC 
Editor, rather than inviting the WG to reexamine an issue where the WG had 
a rough time coming to consensus, and the IESG only wants the result to be 
more clearly documented, not changed.
The AD for the WG is the IESG member best placed to give advice about how 
the message should go back to the WG.

and also - sending mail to a WG list is not always simple, unless you have 
interacted with that list before - just about all lists bounce mail from 
nonsubscribers, and not all of them have active moderators who will forward 
such mail, no matter what guidelines say.

> think it happens very often, couldn't the IESG be wrong?

of course the IESG could be wrong!
But they failed to be convinced by the documents.
So more discussion is needed....

> Do you think that's reasonable?

No. Those issues should be labelled COMMENT, not DISCUSS.