Subject: RE: OPEN ISSUE: Standards Track

From: <[email-masked]>
Date: Wed, 21 May 2003 14:32:52 -0700

Hi everybody,

> -----Original Message-----
> From: ext Keith Moore [mailto:[email-masked]]
> Sent: Friday, May 16, 2003 9:02 AM
> > *what sort of bugs, if detected, would make a protocol 
> unsuitable for 
> > Proposed Standard?*
> 
> the way Proposed is treated by industry now, anything that would cause
> problems in wide deployment should make a protocol unsuitable for
> Proposed.
> 
> certainly failure to interoperate would be in this list,
> but so would failure to scale, lack of robustness, failure to provide
> adequate security, and a lot of other problems.

I do not really think it is just the industry that is considering Proposed ready for wide development, but how we are going around in the IETF about the proposed standard actually makes it feel like it should be ready for wide scale implementation. When I first started coming to the IETF back in '99, I first read all the training material on the IETF web page. To my it was quite clear that proposed standard is no way stable, but either draft or full standard would be then really stable. Sadly, I had to be rehabilitated in the IETF to understand that proposed standard is mostly the last stage a protocol ever takes. 

And now for something a bit different: I think we should have a new stability stage (proposed standard?) to the documentation process, where the specification is relatively stable. That would mean that the bugs can be fixed, and the the work has by no means stopped. However, the technical concept would already be quite stable. At the moment, the specifications can change quite a bit in the last stages before going to proposed. The relatively-stable-stage could help the vendors to standart to implement the protocol and find out the bugs, and not to have to change the implementation completely when the next version of the protocol is out. For instance, it could be considered that the specs are "functionally frozen", but the last bugs are to be taken out.

BTW, yes, I think too this is a real issue.

Cheers,

Jonne.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 22 May 2003 11:07:26 +0200

> BTW, yes, I think too this is a real issue.

The answer to this is to recycle at Proposed. The last thing we need to do 
is to add a step to the process. It would be entirely recursive in effect -
after a few years, someone would be be asking "shall I implement
draft-jonnes-protocol-17.txt, or should I wait for the relatively-stable 
draft?"

We need to remove steps, not add them.

   Brian

---

From: Pekka Savola <[email-masked]>
Date: Thu, 22 May 2003 12:50:17 +0300 (EEST)

> We need to remove steps, not add them.

It cuts both ways.  

We may either have many smaller steps or only few big ones.

If we add a step which should be easy to get to ("Experimental") and
actually try to make it so, we might still leave the folks enough
incentive to finish it and hone it to get it to the next level ("Proposed
Standard") -- or kill it completely as a bad idea.

I'm not sure whether that's really a good idea, but neither is the 
alternative..

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 22 May 2003 06:22:33 -0500

We're into specific solutions here, but to return to the drafts...

The process document current says:

>There is also a more fundamental issue with the IETF's engineering
>practices.  Although our current standards track contains three
>levels of maturity (Proposed Standard, Draft Standard and Full
>Standard), we do not have sufficient differentiation regarding the
>quality and completeness of documents required at each stage.  The
>bar is set very high for publication at Proposed Standard, and very
>few documents advance beyond this stage. [OPEN ISSUE: Do we have
>IETF consensus that this is a problem?]

We're hearing proposed solutions to this problem, so it looks like there
are folks who agree that it's a problem.

Are there folks who DON'T agree that this is a problem?

Spencer

---

From: Keith Moore <[email-masked]>
Date: Thu, 22 May 2003 08:38:07 -0400

> We need to remove steps, not add them.

actually I also think that we need one or two baby steps, prior to what
we now think of as proposed, to ensure that certain considerations
get dealt with early in the design phase, and also to ensure that 
proposals get wide review long before the Last Call for proposed.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 22 May 2003 15:42:50 +0200

> proposals get wide review long before the Last Call for proposed.

Agreed. But these problems are clearly identified in the problem
statement draft (and clearly targetted by at least one extant
proposed solution).

   Brian

---

From: Randy Bush <[email-masked]>
Date: Thu, 22 May 2003 08:17:42 -0700

> Are there folks who DON'T agree that this is a problem?

how does this 'problem' do damage to

   the ietf's goal is to produce high quality, relevant, and timely
   standards for internet technology.

randy

---

From: Keith Moore <[email-masked]>
Date: Thu, 22 May 2003 11:40:11 -0400

> > We're hearing proposed solutions to this problem, so it looks like
> > there are folks who agree that it's a problem.
> > 

well, what we currently have is for most purposes a single stage of
review.  perhaps we'd produce higher quality and more relevant documents
if we imposed some earlier review, and perhaps we'd get those documents
done in a more timely fashion if the early reviews identified problems
that are expensive (or time consuming) to fix if not discovered until
later.

---

From: Randy Bush <[email-masked]>
Date: Thu, 22 May 2003 08:58:49 -0700

> that are expensive (or time consuming) to fix if not discovered until

i do not disagree with you.  

but that is not at all what the problem statement above says to me
it seems to say that
  o the bar is too high
  o there is insufficient differentiation between ps, ds, and fs
with which i disagree

randy

---

From: "Charles E. Perkins" <[email-masked]>
Date: Thu, 22 May 2003 09:01:49 -0700

Hello Randy,

I think it is a serious problem.  Lengthy gestation periods
for Proposed Standards means that we don't get experience with
using the technology except in limited deployments and labs.
With faster publication of a Proposed Standard that would
be subject to further refinement, we would get better
operational experience sooner, and overall faster progress.

As it is, even with the too-long process, we see that hasty
decisions are taken at the end to satisfy all possible
stakeholders, even those who have not done any work on the
protocol.  Since it takes so long to do the process, at the
end the hardest working people are burned out and just want
to get it over with, so compromise is shunned because that
takes too much discussion, so the specification just gets
axed in order to avoid debate.  This leads to some unwise
elimination of tested, interoperable, known-good features
only in the name of getting the damned process finished.

I'd prefer more interoperability assurance than longer
processing time.  If dozens of teams can build it and get
good results, I think that says something more relevant
about the readiness of the specification, than whether anyone
in any corner of the IESG can think of something that might
possibly go wrong under some hypothetical set of circumstances.

Of course I do not mean that this is all black and white,
and that interoperability trumps all architectural
considerations.

I just think it's tilted much too far in the "what if"
direction than the "what is" direction.

As an example, I think it was a terrible mistake to delay
Mobile IPv6 for so long pending the completion of the world's
most sophisticated and super robost security solution (slight
exaggeration, but ...).  I guess I don't have to describe the
effect this can have in a world moving at what used to be
called "Internet speed".

Regards,
Charlie P.

---

From: Keith Moore <[email-masked]>
Date: Thu, 22 May 2003 12:05:57 -0400

> > review.  perhaps we'd produce higher quality and more relevant
> > documents if we imposed some earlier review, and perhaps we'd get
> > those documents done in a more timely fashion if the early reviews
> > identified problems that are expensive (or time consuming) to fix if
> > not discovered until later.
> 

yes, I think that bit of the problem statement needs some refinement.

Keith

---

From: Pete Resnick <[email-masked]>
Date: Thu, 22 May 2003 09:08:43 -0700

On 5/22/03 at 8:58 AM -0700, Randy Bush wrote:

>>perhaps we'd produce higher quality and more relevant documents if 
>>we imposed some earlier review, and perhaps we'd get those 
>>documents done in a more timely fashion if the early reviews...
>
>[...]
>[...]

How about "the first bar is too high" or "we need a new lower  initial bar"?
-- 
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Keith Moore <[email-masked]>
Date: Thu, 22 May 2003 12:09:47 -0400

> effect this can have in a world moving at what used to be

then again, we have plenty of experience that

- security holes *will* be exploited, and
- vendors will avoid implementation of security if allowed to do so,
  even if it subjects their customers to greatly increased risk

(ask yourself: why is it so easy for viruses to propagate by email?)

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 May 2003 11:11:44 -0500

> Standard") -- or kill it completely as a bad idea.

Right now I think most people think of "Experimental" as "never going 
to be a standard."   I know that's not what it was originally intended 
to mean, but in practice that seems to be how people think of it.   I'm 
reluctant to provide a specific example here because the one I can 
think of is so controversial, but a lot of you probably remember the 
conversation anyway.

---

From: Randy Bush <[email-masked]>
Date: Thu, 22 May 2003 09:14:10 -0700

> How about "the first bar is too high" or "we need a new lower

[ i *really* don't like to post too much to the list.  but i guess
it's my day in the barrel.  so apologies. ]

i think the real issue may be the difference between what is
drafted and what is passed as PS.  that is perceived as "it is very
hard to get a document to PS."  i suspect this difference may be as
much or more caused by the input quality and relevance as it is by
the level of the bar.  if so, this is a core problem that clearly
affects the ietf's goals.

randy

---

From: "Charles E. Perkins" <[email-masked]>
Date: Thu, 22 May 2003 09:23:48 -0700

Hello Keith,

>   even if it subjects their customers to greatly increased risk

These considerations do NOT apply to the situation in Mobile IPv6.
This list is NOT the place for me to explain why.

Plus, your counterpoint has no relevance to anything I
tried to say, thus I view it purely as setting up a strawman.

> (ask yourself: why is it so easy for viruses to propagate by email?)

Because of certain operating systems features, mainly.
There is no point to a specification that says
viruses MUST NOT be included in e-mail attachments.
And, yes, I understand that further e-mail specification
_might_ prevent _some_ viral infections.

Regards,
Charlie P.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 May 2003 11:41:30 -0500

> (ask yourself: why is it so easy for viruses to propagate by email?)

Because of buggy programs that execute code in email messages, which 
has exactly zero to do with SMTP, POP or IMAP.   :'/   BTW, 
implementations of POP and SMTP security features in both open-source 
and vendor-supplied mailers seem to be getting pretty advanced at this 
stage, although there is still more work to do.

---

From: Randy Bush <[email-masked]>
Date: Thu, 22 May 2003 09:56:59 -0700

charlie,

i think there are at least two major factors in increased delay,
and you pointed out one, security.  we used not to even think about
it.  it was rude to run a unix host without a password-less account
named 'guest'.  but the world has changed.  and the <bleep>ing
security folk have not given us magic pixie dust to secure our
stuff, but instead we have a serious ramp-up to achieve even
reasonable security (for some value of reasonable).  this can be a
significant part of the work of producing a new protocol or piece
thereof.

but one we seem to miss is that the total protocol space is now
much larger than we were kids.  and the interactions have gone up
super-linearly with the number of features.  this has serious
impact in the intra-area and cross-wg and cross-area analysis and
operational evaluation of a proposal for a new protocol or feature.
and this only gets worse in time, and not linearly.  i think that
mip, being so basic, also hits this one seriously.

i think that these may be core problems.  sorry, but i am not sure
i have solutions.  the second may be somewhat ameliorated by the
restraint for which brian continually excoriates me.

randy

---

From: Keith Moore <[email-masked]>
Date: Thu, 22 May 2003 13:32:40 -0400

> _might_ prevent _some_ viral infections.

As a start, it might help if the existing ones were followed.

Keith

---

From: Pekka Savola <[email-masked]>
Date: Thu, 22 May 2003 20:48:35 +0300 (EEST)

> think of is so controversial, but a lot of you probably remember the 

Right.. which is why "Experimental" would be a good one.  What we want is 
a stable specification to get some limited testing and deployment, *NOT* 
widespread use.

---

From: Pekka Savola <[email-masked]>
Date: Thu, 22 May 2003 20:53:35 +0300 (EEST)

> How about "the first bar is too high" or "we need a new lower  initial bar"?

Completely disagree.

We need to be able to kill the bad ideas early so that we don't get to the
typical case:

"2 years has passed, people have spent a lot of work on this, so it
wouldn't be fair to kill it now.. so let's pass it through even if we
shouldn't."

.. of course, the alternative to killing bad ideas is correcting them 
early on.. but it's fallacy to believe that all the bad ideas can be 
corrected.

---

From: "Charles E. Perkins" <[email-masked]>
Date: Thu, 22 May 2003 10:59:21 -0700

Hello Randy,

Randy Bush wrote:

> and you pointed out one, security.

Someone reading your note might get the incorrect notion
that we didn't pay attention to Mobile IPv6 security.
That would be very much wrong.  To put it as briefly as
possible, the ADs demanded a key distribution mechanism
_in addition_ to the basic authentication scheme.  This is
in marked contrast to the situation with AH and ESP,  But
_nobody_ claims that AH and ESP were insecure because they
were standardized without key distribution.  I get steamed
when people perpetuate this serious misperception about
"the security problem".

I do NOT claim that insecure standards should be promulgated.
I _do_ claim that as we get more experience with protocols,
we can provide more ways to enable key distribution for
authorizing the use of those protocols.

<... discussion deleted about how nice it used to be ...>

> mip, being so basic, also hits this one seriously.

This was not the reason for delay of Mobile IPv6.   Besides the
demand for a key distribution mechanism (a bad reason for delay,
in my opinion), we did not exhibit any serious interaction problems
with other protocols.  There are some serious considerations
here, but discussion of them does not fit on this list.  It has to
do with how use of existing protocol specifications (you would think
this is a good thing) leads into some of the worst problems I can
remember.  Bottom line: at this point in time, I am convinced
that trying to use IPsec was my own worst mistake in the initial
designs of Mobile IPv6, and recent decisions have made the use
of RFC 2461 to be very, very questionable indeed.  Using someone
else's specification, with even the most microscopic refinements,
gives them carte blanche to destroy your protocol design in the future.

> restraint for which brian continually excoriates me.

There are no easy answers, and any solution we pick will have
problems that need further attention.  Rigidity and inability
to make change is only one of the worst answers, not the least
likely answer.

Regards,
Charlie P.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 May 2003 13:00:45 -0500

> a stable specification to get some limited testing and deployment, 

One slightly draconian way to fix this problem would be to *require* 
that a spec go through experimental on the way to proposed.   Then 
there can't be a stigma, because every spec does it.   This would be a 
small inconvenience for trivial specifications, but probably a big win 
for the bigger specifications.   I don't think it would make sense to 
put all the specs that are nearly at proposed through this extra step 
now, but it might be a good thing to do for new drafts.

---

From: Keith Moore <[email-masked]>
Date: Thu, 22 May 2003 14:09:23 -0400

> were standardized without key distribution. 

not insecure, just useless.  and the fallback to bare IP _is_ insecure.

I'd also claim that IPsec implementation in hosts is useless without an
API that allows apps to determine the validity of their peers'
credentials...

maybe this is another illustration that we don't really try to
understand the full scope of the problem until it gets to IESG.
and it's hardly surprising if IESG does a poor job of fixing that.

---

From: Pekka Savola <[email-masked]>
Date: Thu, 22 May 2003 21:11:24 +0300 (EEST)

On Thu, 22 May 2003, Charles E. Perkins wrote:
[...]
> It has to

Perhaps the problem here was trying to apply an apparent solution to an
apparent problem without considering the applicability of the protocol for
such a purpose.. or fleshing out the details.

Of course, I'm not sure how well the applicability of IPsec has/had been 
documented...

Early review from IPsec folks might have been useful; I'm not sure how 
much of that did indeed happen before the IESG noticed the issue.

IMO the problem was very real, and the action being the only responsible
thing to do.  Looking back, maybe MIPv6 could have been published as
Experimental (or the like) as is, but definitely not PS.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 22 May 2003 20:13:22 +0200

>>>>> Ted Lemon writes:

>> Right.. which is why "Experimental" would be a good one.  What we
>> want is a stable specification to get some limited testing and
>> deployment, *NOT* widespread use.

Ted> One slightly draconian way to fix this problem would be to
Ted> *require* that a spec go through experimental on the way to
Ted> proposed.  Then there can't be a stigma, because every spec does
Ted> it.  This would be a small inconvenience for trivial
Ted> specifications, but probably a big win for the bigger
Ted> specifications.  I don't think it would make sense to put all the
Ted> specs that are nearly at proposed through this extra step now,
Ted> but it might be a good thing to do for new drafts.

And then we kick out Draft (because Proposed is already pretty good
and few documents make two additional steps) and we end up with the
same three step process but just the steps have been renamed to match
today's common understanding. Since public perception is important,
this might actually not be such a bad idea and it is relatively cheap
to implement...

Of course, in a few years we will have raised the bar for Experimental.
So we need to introduce another new name. But luckily, since Draft has
then not been used for a while, we can simply introduce a new model
which has Draft->Proposed->Standard. :-)

/js

-- 
Juergen Schoenwaelder		International University Bremen
Phone: +49 421 200 3587		P.O. Box 750 561, 28725 Bremen, Germany
Fax:   +49 421 200 3103		<http://www.iu-bremen.de/>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 May 2003 13:37:50 -0500

> which has Draft->Proposed->Standard. :-)

Don't experimental RFCs have warnings all over them about not deploying 
them widely?   Anyway, that brings up another point, which I think is 
the real problem - it is not "Experimental" or "Proposed" or "Draft" or 
"Standard" that makes something a standard in the eyes of the wider 
Internet developer community.  Unfortunately, it is "RFC," despite the 
fact that "RFC" stands for Request For Comments.

What about using a different TLA for experimental than for other RFCs?  
  Like "XCR" - Experimental, Comments Requested.   XCRs would have the 
same degree of permanence in the IETF's RFC distribution as RFCs, but 
wouldn't have the much-coveted "RFC" TLA applied to them, and thus 
wouldn't be considered standards in the same way that something that 
has the RFC TLA is.   I-Ds could be IPD (Informational Protocol 
Description) instead of RFC, and so on.

---

From: "Charles E. Perkins" <[email-masked]>
Date: Thu, 22 May 2003 14:33:26 -0700

Hello Keith,

> not insecure, just useless.  and the fallback to bare IP _is_ insecure.

They weren't useless, because as Steve B. said, you could
use preconfigured keys.  Do you really claim that's useless?

> API that allows apps to determine the validity of their peers'

We can't boil the ocean all the time.

First the basic headers.

Then the key distribution.

Then the APIs.

If the basic headers were never published in a stable
document, I'll bet the whole process would stall and
_nothing_ would have gotten done.

> and it's hardly surprising if IESG does a poor job of fixing that.

It's equally well an illustration that we can't boil
the ocean over and over again -- especially with new
people every time.  We need to boil a few gallons first,
and then a few truckloads.  Failure to publish is a
distinct impediment in the process, and I think that
timely publication of a stable Proposed Standard should
be a good initial step.

Regards,
Charlie P.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 22 May 2003 17:24:05 -0500

> and then a few truckloads.

This is a _really_ good point.   One of the things that I've seen cause 
major hassle and pain for people who've been working on a draft for a 
while is that someone new comes along (it's been me! :') and asks the 
same damned questions that got answered *last* year.   And then someone 
else comes along a week later and does the same thing.   :'}

Incremental steps would really help with this (although of course in my 
case the questions I was asking were extremely pertinent. :')

---

From: Ralph Droms <[email-masked]>
Date: Fri, 23 May 2003 07:42:46 -0400

I agree with Keith about a need for baby steps.  We stumbled
across a sequence of baby steps entirely through serendipity
that seems to have worked well for the DHCPv6 specification (I
say "seems to" because the spec has only recently been
completed so we don't have a lot of history with the results).

The extra baby step we took was to perform two rounds of
interoperability testing between the acceptance of the DHCPv6
spec as a Proposed Standard and the publication of the RFC.
This interoperability testing allowed us to identify a dozen
or so places in the text that could be improved prior to
publication of the RFC: a couple of typos, some text that
required clarification, etc.  We published an I-D (and ran
a WG last call) summarizing the results of that
interoperability testing and the spec has been edited
based on the report in the I-D.

The result will be (I believe; we'll see how the spec fares
in practice) a better Proposed Standard spec.  The analogy
with the current usual path for a spec is, I think, that
draft-ietf-dhc-dhcpv6-28.txt (the I-D that was accepted as
the basis for the Proposed Standard) was roughly equivalent
to the definition (if not the practice) of a "Proposed
Standard".  The version of the spec that will be published
as an RFC is roughly equivalent to a "Draft Standard".  What
was different about the DHCPv6 spec process is that
draft-ietf-dhc-dhcpv6-28.txt had no acceptance and deployment
(which would have happened had it been published as Proposed).

I don't claim that this experience points out a specific
problem (and attendant solution) with the current process.  I
think the experience points out that there may be improvements
we could make to the current process that better achieves
the goal of "experience to prove correctness of a stable
specification" prior to widespread acceptance and deployment.

- Ralph

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 23 May 2003 16:05:19 +0200

Now, getting back from Mobile IP to the subject field...

> but that is not at all what the problem statement above says to me

Which I simply don't agree to, after my experience as a WG Chair
shepherding documents to PS. We should *not* be applying rubber
stamps more easily. There are enough bugs in PS documents to satisfy
anybody.

>   o there is insufficient differentiation between ps, ds, and fs

I agree there is enough differentiation - the problem is that
fs is too strongly differentiated, so nobody bothers. That's why
I propose to munge ds and fs into a single grade.

Make it simpler. Never make it more complicated.

   Brian

---

From: Keith Moore <[email-masked]>
Date: Thu, 22 May 2003 14:01:56 -0400

[list bcc'ed to encourage the thread to terminate.]

> has exactly zero to do with SMTP, POP or IMAP.  

It has a lot to do with the way MIME is implemented. 

Hint: there's a reason we didn't use filename suffixes as content-types
and expect operating systems to use the same dispatch tables as they
would do for local files.  There's also a reason we required a security
considerations section for content-type registration.