Subject: [Fwd: Re: rough consensus (was Re: "trouble maker")]

From: James Seng <[email-masked]>
Date: Wed, 16 Jul 2003 16:52:43 +0800

following from the brief discussion at the mike on "rough consensus", I 
would like to raise the ambiguity of "rough consensus" as a problem.

The current rough consnsus _process_ as defined in RFC 2418 basically 
say "It is up to the Chair to determine if rough consensus has been reach."

Because of this ambiguity, there is a general perspection that the 
"Chair has all the power => top-down". And what "rough consensus" the 
Chairs determined may also be disagree by a some people, a few may be 
become dissent, likewise due to this ambiguity.

<solutionism>
While a formal definition may be hard to nail down, it would be useful 
to cite several examples of past experiences. It also helpful to 
document some of the consideration and the thinking process the wg 
chairs takes to determine rough consensus.
</solutionism>

-James Seng

-------- Original Message --------
Subject: Re: rough consensus (was Re: "trouble maker")
Date: Fri, 27 Jun 2003 03:43:09 +0800
From: James Seng <[email-masked]>
To: Avri Doria <[email-masked]>
CC: [email-masked],	Harald Alvestrand 
<[email-masked]>

I believe the general problem is our current process of letting wg
chairs determine rough consensus without a general definition. (Yes, I
know it is a feature too :P)

Since people have different opinion of the "rough consensus", those
(especially the doc authors or someone with more then casual
interest[1])  who disagree with the chair become resentful of the IETF
process.

Personally, I already lost count how many times it happens. It is not
healty.

[1] "more then casual interest" != vested interest. (See Keith v.s NAT,
no offense intended :-)

-James Seng

Avri Doria wrote:
> Hi,
> 
> This has been a very interesting discussion, and I beleive that
> it is important that the details of this issue be debated and resolved.
> I do not believe that should be done on this list.  It was good to get
> the problem discussed so that people could do an abstraction of it
> for the problem statement, but I would like to avoid dwelling on  the
> specific arguments relating to it any longer.
> 
> At this point  I ask that people concentrate on the general language
> for the problem statement so that we can make sure that the problem
> is reflected adequately in  the problem statement.
> 
> Thanks
> 
> Avri
> co-chair
> 
> 
> 
> 
> On torsdag, jun 26, 2003, at 13:45 Asia/Seoul, Harald Tveit Alvestrand 
> wrote:
> 
> 
>

---

From: Charlie Perkins <[email-masked]>
Date: Wed, 16 Jul 2003 02:06:43 -0700

Hello folks,

I'd like to suggest the possible problem for inclusion in the
problem statement draft.  I'm obviously not suggesting a
complete solution.  If this is already present in the draft,
then please excuse my inadvertent duplication.

Problem: There often seem to be adversarial relationships
               within the IETF that are not necessary, and which
               substantially interfere with accomplishing timely
               work.  These adversarial relationships exist, at times,
               between:
               - IESG members and document editors, and
               - IESG members and working group chairs
               - possibly other cases I am not aware of.

Partial solution(s):
               Making all process steps and expectations evident
               as early as possible.  Arrange things so that each
               party expects the best, instead of the worst, from
               the other party.  Clarify or modify IETF processes
               to eliminate sources of differing expectations.  Avoid
               all situations where one party knowingly allows the
               other party to harbor false expectations.

Regards,
Charlie P.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 16 Jul 2003 12:29:11 +0200

the RFC 2418 definition is as follows (section 3.3):

   Working groups make decisions through a "rough consensus" process.
   IETF consensus does not require that all participants agree although
   this is, of course, preferred.  In general, the dominant view of the
   working group shall prevail.  (However, it must be noted that
   "dominance" is not to be determined on the basis of volume or
   persistence, but rather a more general sense of agreement.) Consensus
   can be determined by a show of hands, humming, or any other means on
   which the WG agrees (by rough consensus, of course).  Note that 51%
   of the working group does not qualify as "rough consensus" and 99% is
   better than rough.  It is up to the Chair to determine if rough
   consensus has been reached.

<solutionism>
how could this definition be modified to be more useful?
</solutionism>

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 16 Jul 2003 12:54:00 +0200

> how could this definition be modified to be more useful?

I doubt if it can, unless we radically change our open door
policy. 
 
   Brian

---

From: James Seng <[email-masked]>
Date: Wed, 16 Jul 2003 19:19:05 +0800

There are two parts here : Rough consensus process and rough consensus 
itself.

RFC 2418 defines the process but leave the definition of rough consensus 
vague, leaving it to the chair.

I am not saying the process have problem. I am saying we need to clarify 
the latter so everyone at least have some baseline understanding of what 
"rough consensus" is.

-James Seng

---

From: [email-masked]
Date: Wed, 16 Jul 2003 13:29:42 +0200

When I was writing a recent draft on alternative consensus mechanisms, I felt
like the IETF process needed a little more context than 2418.  The text is
below, and improvements on it would be welcome 
(from draft-hardie-alt-consensus-00.txt)

. Introduction.

     Dave Clark's much-quoted credo for the IETF cites "rough consensus
     and running code" as the key criteria for decision making in the
     IETF.  Aside from a pleasing alliteration, these two touchstones
     provide a concise summary of the ideals which guide the IETF's
     decision making.  The first implies an open process in which any
     technical opinion will be heard and any participant's concerns
     addressed; the second implies a recognition that any decision must
     be grounded in solid engineering and the known characteristics of
     the network and its uses.  The aim of the IETF is to make the best
     possible engineering choices and protocol standards for the
     Internet as a whole, and these two statements guide it in making
     its choices and standards.

<snip>

2.  Rough Consensus as a baseline approach.

     The Conflict Research Consortium at the University of Colorado
     outlines the pros and cons of consensus as:

	The advantage of consensus processes is that the resulting
	decision is one that meets the interests of all the parties
	and that everyone can support.Ê The disadvantage is that
	developing such a decision can be a very slow process,
	involving many people over a long period of time.Ê There is
	also a relatively high probability of failure. If a quick
	decision is needed, the consensus approach may not work.Ê
	Consensus rule processes also tend to favor those that oppose
	change and want to preserve the status quo. All these people
	have to do is refuse to support any consensus compromises and
	they will win (at least as long as they can delay
	change). (CONFLICT)

     Using "rough consensus" as a guideline limits some of
     disadvantages of consensus processes by ensuring that individuals
     or small factions cannot easily block a decision which has
     otherwise general support.  The second touchstone of "running
     code" can also limit the disadvantages of consensus processes by
     requiring that statements opposing particular proposals be
     technically grounded.

     These limitations do not change the core mechanisms of
     consensus-building, however, and the IETF process continues to
     require individual participants both to use their best engineering
     judgment to select among proposals and to balance their own
     interests with those of the Internet as a whole.  Active
     participation and a willingness to compromise, possibly on key
     points, are needed.  Historically, this has worked because a large
     majority of participants have recognized that the Internet's
     growth and enhancement are more important overall than any
     specific short-term advantage.

     In other words, the use of "rough consensus" is sufficient in most
     cases in the IETF to ensure that individuals or small groups are
     heard when they raise technical objections, but that they cannot
     block progress when a general agreement has been reached.  This
     document does not suggest changing the usual mechanisms for
     achieving forward progress; it proposes mechanisms for use when a
     working group has consensus that it must make a decision, but it
     cannot make that decision by the usual rules.

At 7:19 PM +0800 7/16/03, James Seng wrote:
>There are two parts here : Rough consensus process and rough consensus itself.
>
>RFC 2418 defines the process but leave the 
>definition of rough consensus vague, leaving it 
>to the chair.
>
>I am not saying the process have problem. I am 
>saying we need to clarify the latter so everyone 
>at least have some baseline understanding of 
>what "rough consensus" is.
>
>-James Seng
>
>Brian E Carpenter wrote:
>
>>policy.     Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 16 Jul 2003 13:29:46 +0200

"In general, the dominant view of the working group shall prevail. "

Doesn't that define rough consensus?

   Brian

-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Brian E Carpenter 
Distinguished Engineer, Internet Standards & Technology, IBM 
On assignment at the IBM Zurich Laboratory, Switzerland

---

From: Scott W Brim <[email-masked]>
Date: Wed, 16 Jul 2003 13:33:10 +0200

> the latter so everyone at least have some baseline understanding of what 

I don't know if that's preferable.  The situations encountered are so
different, even within a single working group.  I think that trying to
define rough consensus better than we have already will disregard the
innate, and rather excellent, understanding of groups and group
consensus which we have been developing for many millions of years.

---

From: James Seng <[email-masked]>
Date: Wed, 16 Jul 2003 19:56:50 +0800

In some cases, it is easy to differential. But in most cases, it is a 
fuzzy line.

For example, a proposal which have most people supportive but a couple 
of objections, do we move forward? In general, the answer is "yes".

But what if the couple of objections involves all the major vendors for 
that protocol. Do we still move forward a proposal we know which will 
not be deploy at all?

Another example, a proposal have only a handful of people supporting it 
passionly but the rest of the group is in the class 'don't care but no 
object', 'dont like it but wont bother to object', 'finds the whole 
excerise waste of time'.

So when you call for a hum, you get some very loud hum but still weak. 
You call for objections, silent. Do you move forward?

<solutionism>
So basically, there are a lot of factors influencing whether the Chairs 
will declare rough consensus or not. While I feel the Chairs should 
still have pregorative to decide when to move forward, the consideration 
should be captured and make clearer to the participants so that people 
wont scream so much when the Chairs decided to move (or decided not to 
move).
</solutionism>

-James Seng

---

From: Bill Sommerfeld <[email-masked]>
Date: Wed, 16 Jul 2003 08:06:15 -0400

> You call for objections, silent. Do you move forward?

well, if the silent majority is truly silent, the trick is to
distinguish the case you cited from "small number of enthusiastic
prospective users and implementors, many apathetic don't minds"
meaning that there's a small community of interest in a feature which
does no appreciable harm to the larger community as whole.

In most working groups I frequent, there are plenty of people who are
not shy about announcing that they dislike a proposal..

						- Bill

---

From: James Seng <[email-masked]>
Date: Wed, 16 Jul 2003 20:21:50 +0800

Ignore the solutionism portion. I am trying to point out that informal 
definition of rough consensus has been stretched too far.

A Chair handwaving and say 'rough consenus, you know, is..well, you 
know, consensus which is rough so not everyone may agree' may works when 
we are much smaller.

But it dont build confidences in the Chairs. Chairs ultimately suffer 
the bulk of the attack from the people who disagree.

Personally, I seen lots of people angry & upset over the rough 
consensus. I have people telling me 'I spend 3 years of my time, 
travelling to every IETF and I am disillusioned now and will not attend 
anymore'. And they dont.

Now, you may say that is their problem...Part of it yes. But could part 
of it be our problem?

-James Seng

---

From: Scott W Brim <[email-masked]>
Date: Wed, 16 Jul 2003 14:27:58 +0200

I like the idea that Chairs should document why they declared consensus
(or the lack of it).

---

From: Scott W Brim <[email-masked]>
Date: Wed, 16 Jul 2003 14:39:49 +0200

So, in problem-issue-statement-02, somewhere around this text:

   Participants are frequently allowed to re-open previously closed
   issues just to replay parts of the previous discussion without
   introducing new material.  This may be either because the decision
   has not been clearly documented or it may be a maneuver to try to get
   a decision changed because the participant did not concur with the
   consensus originally.

.. add that participants might abandon the IETF altogether.  Stress
that lack of clarity and justification for the Chair declaring consensus
(or the lack of it) aggravates this problem.

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 16 Jul 2003 15:01:09 +0200

> I like the idea that Chairs should document why they declared consensus

Agreed. But another thing that may be going on is Chairs making consensus 
calls too late.

If you ask a WG to accept a draft that's been in development for a year
and the WG says no, you asked the question too late. It's not a
problem with the consensus process, but with the decision tree.

      Brian

---

From: Erik Guttman <[email-masked]>
Date: Wed, 16 Jul 2003 15:21:46 +0200

> Agreed. But another thing that may be going on is Chairs making consensus 

The hardest part of calling consensus is its subtlety.  After 100s of
emails have been exchanged and a rough consensus has emerged it doesn't
really exist (have any reality) until the consensus has been put into
words by the chair.  Without a coherent statement posted to the list
describing a consensus it is hard to know
  - what actions it implies
  - whether all dissenting views have been considered
  - what compromises were made among those forming the consensus
    (what were their positions originally and what did they agree to?)

Defending or questioning a consensus call requires mailing list
archeology, which is a tedious and inexact discipline.  We should
do better and we can.

============================
Solution discussion:

I have found that by documenting the

  - final decision and action to take
  - summary of the dominant consensus position taken, including
    where they aren't in accord
  - well worked out dissenting views
  - notes from the WG chair (if needed), to clarify how a difficult
    decisions was made and why it is reasonable.

This sounds like a lot of writing.  It usually comes down to a page
or two, even for very complex decisions.  Advantages of putting this
in the record are

a) You can point at it when someone wants to bring the topic up
    again.

b) If the chair overlooks something, it is easy to reopen the
    consensus call *for a very specific reason* without reopening
    the whole debate.  (Note:  Sometimes WG chairs do not fully
    understand the issues they are making a consensus call on.
    By writing this consensus document, it is easy to determine
    whether the argument has been correctly heard and evaluated.)

c) It is a extremely useful to have definitive instructions *what
    to do* with a consensus decision.

d) Once a consensus statement is accepted by the working group
    we have something much more concrete than a record in WG
    minutes that such and such was decided upon, etc.

Erik

---

From: "todd glassey" <[email-masked]>
Date: Wed, 16 Jul 2003 06:22:10 -0700

----- Original Message ----- 
From: "Harald Tveit Alvestrand" <[email-masked]>
To: "James Seng" <[email-masked]>; <[email-masked]>
Sent: Wednesday, July 16, 2003 3:29 AM
Subject: Re: [Fwd: Re: rough consensus (was Re: "trouble maker")]

>    this is, of course, preferred.

So OK then Harald - who decides what constitutes a concensus then? The WG
Chair? and then what is the minimum number of members that need to vote, or
how is a quorum created? becuase without some form of voting, I see that
there is no real way to measure anything... let alone a concensus.

>    In general, the dominant view of the
>    persistence, but rather a more general sense of agreement.)

OK So all the parties on whoever's payroll all vote the same.

>    Consensus
>    which the WG agrees (by rough consensus, of course).

So then the WG, can select its own type of concensus, making NONE of the
efforts of achieving concensus comparable to each other. This is a glaring
flaw in the process since it essentially says that each WG can do whatever
then hell it wants with no regard for consistancy.

>   Note that 51%

So here again - we have some basic number stating that more than 52% is
considered a concensus. But that above 52%, and its the WG Chair's
responsibility to declare the concensus reached. So let me paint a picture
hypothetically...

So I come up with a new backbone protocol so revolutionary that it threatens
to rock the world, and I expose it to the IETF under terms not totally
inline with the full releases in RFC2026's SS10. Cisco or other large IETF
participant notices that I have something that potentiallty could kick their
ass in the marketspace, and so they flood the WG that I got the IETF to
allow me to start on this with bodies. These bodies do two things, they
create chaos in the process and effectively STOP me from advancing the
protocol...

The question is how the IETF's processes can prevent the scenario that I
just painted, and my apologies to Big John and the Tasman Avenue crowd, no
offense was intended here, but the fact is that any large player in the IETF
has a clear, everpresent, and very unfair advantage here, and the IETF's
processes are setup to allow this to happen and that's the issue.

> how could this definition be modified to be more useful?

    1)    Define the formal vetting process and the steps involved - this
will take a revision to RFC2026 and the standards process.

    2)    Define the minimum number of participants needed to constitute a
vetting process, and  allow for any number of standards initiatives in the
same WG. It is not the WG's choir to pick the dominent player in the
marketplace. Its their responsibilty to "create interoperable standards" not
lock others out of the marketplace.

    3)    Put in place a real oversight process instead of the loosely
defined methods now woven through RFC2026 and the other Operating Documents
herein. This includes Management Oversignt as well and having some formal
mechanism for the members of a WG to oust the WG Chair and how many members
it takes to do that.

---

From: "todd glassey" <[email-masked]>
Date: Wed, 16 Jul 2003 06:23:10 -0700

You mean that they should be accountable for their decisions?

Todd

---

From: "todd glassey" <[email-masked]>
Date: Wed, 16 Jul 2003 06:30:18 -0700

----- Original Message ----- 
From: "todd glassey" <[email-masked]>
To: "Harald Tveit Alvestrand" <[email-masked]>; "James Seng"
<[email-masked]>; <[email-masked]>
Sent: Wednesday, July 16, 2003 6:22 AM
Subject: Re: [Fwd: Re: rough consensus (was Re: "trouble maker")]

> same WG. It is not the WG's choir

Chore...  (sorry what  meant to say is that its not the WG's chore to...)

>  to pick the dominent player in the

---

From: Erik Guttman <[email-masked]>
Date: Wed, 16 Jul 2003 16:43:04 +0200

Ted Lemon wrote:
> On Wednesday 16 July 2003 15:21, Erik Guttman wrote:
> 
> I think it's important to note here that silence does not indicate consent.   
> There are cases where a discussion in a working group becomes so voluminous 
> that people stop reading the mailing list, or even if they don't stop, they 
> may miss the consensus call, or may not have time to respond to it because 
> they are so behind in their day job because of keeping up with the mailing 
> list.   Or they may just be so discouraged from what has happened that they 
> don't say anything because it seems futile to do so.   Or they may be 
> reluctant to say anything because they feel that opening up the debate again 
> will just drive more working group members away.   When that happens, you 
> can't really say the failure of those parties to protest means that the 
> working group agrees with your sense of what the consensus is.

Ted,

I agree.

However, I think one can improve the current situation.

It is much easier to read and consider a 'consensus statement' than
the full-on fire hose of a working group mailing list discussing a
divisive topic.  The consensus statement I am describing summarizes
all arguments for and against and makes a definitive call as well as
its implications.

Voicing effective criticism often requires full engagement with WG 
discussion to participate *at all*.  We often lack time to do that.
I believe that this is largely due to a failure on the part of WG
chairs to document decisions and what led up to them.  Given such a 
document, we (as lurkers/outsiders) can take issue or agree *with the
document* without needing to master the huge volume of correspondence
which preceded it.

Erik

---

From: Ted Lemon <[email-masked]>
Date: Wed, 16 Jul 2003 17:22:46 +0200

> document* without needing to master the huge volume of correspondence

I am concerned that those who participate in this way may not get a helpful 
summary from the wg chair, since the arguments are necessarily filtered 
through the chair's understanding of the situation, and will tend to be 
colored by the chair's leanings on the matter.

This seems like it's proposed as a solution to the problem of a too-loud 
objector on the wg mailing list, but I don't think that it solves the 
problem.   My personal experience from having seen this solution implemented 
is that I do not think it successfully captures a consensus.   When I saw 
this implemented in zeroconf, my impression was that it degenerated into a 
simple majority vote, which is not the same thing as rough consensus.

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 16 Jul 2003 20:23:20 -0700 (PDT)

On Wed, 16 Jul 2003, Scott W Brim wrote:

> I like the idea that Chairs should document why they declared
> consensus (or the lack of it). 

I agree.

//cmh

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 17 Jul 2003 10:56:09 +0200

> Agreed. But another thing that may be going on is Chairs making consensus

I think the problem WG meeting's discussion of the problem document on 
Tuesday actually was a fairly telling example of some of the subtleties of 
the rough consensus process.

We had a number of "issues thought to be open".
On every issue, we had a large number of speakers, trying to improve the 
WG's understanding that the issue was indeed real, and exploring subtle 
ramifications of the issue. People just observing the people at the mike, 
and not having read the drafts, would have a hard time detecting what the 
consensus of the room was.

But when the WG chair called for consensus on *whether the document text 
was a good enough description of the issue*, the consensus of the room was 
pretty clear - on some issues, near-unanimous consent that it was good 
enough; on other issues, a clearly divided opinion that means that the 
issue has to be rehashed on the list.

Calling consensus after just listening to the discussion is very hard.
Calling consensus after asking a specific question and seeing the response 
is a lot easier - and a LOT easier to record.

My 2 cents....

                     Harald

---

From: "todd glassey" <[email-masked]>
Date: Thu, 17 Jul 2003 07:12:47 -0700

Harald's right. Consensus needs to be obtained on specific items and not
generalizations. The reasoning for this is that Consensus is something that
MUST be demonstrable in retrospect if its to be of any value at all in
demonstrating fair-play... Otherwise, there is an issue...

Todd
----- Original Message ----- 
From: "Harald Tveit Alvestrand" <[email-masked]>
To: "Brian E Carpenter" <[email-masked]>; "Scott W Brim"
<[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, July 17, 2003 1:56 AM
Subject: Re: [Fwd: Re: rough consensus (was Re: "trouble maker")]