Subject: [Techspec] draft-mankin-pub-req-05.txt

From: Sandy Ginoza <[email-masked]>
Date: Fri, 14 Apr 2006 15:16:39 -0700 (PDT)

All,

I read through the tech spec document <draft-mankin-pub-req-05.txt> and
have the comments below.  While these are based on my experiences
working for the RFC Editor, the comments below are my personal comments
and do not represent the views of RFC Editor function.

Thanks,

Sandy Ginoza

-----------------------------------------------------------------

- Section 3 states

   Based upon this information we derive requirements on the IETF
   technical editor:

  6. Assignment of parameter values

This seems like incorrect phraseology (unless the functions are to be
combined), as the publisher would insert the assigned values, not
assign the values themselves.

- Section 3.1 states

  In many cases the technical publisher may provide a review or
  editing service to improve document quality prior to the approval of
  a document.  This review process would normally address issues such
  as grammar, spelling, formatting, adhrence to boilerplate, document 
  structure, proper use of keywords (RFC 2119), etc.

Prior to document approval, the document has different boilerplate
info.  The technical publisher is to check the boilerplate info of the
ID?  Are these functions to be combined?

- Section 3.1 states

  Discussion: Pre-approval review is not part of the normal process
  flow with the IETF but this concept has been explored with promising
  results in the early copy editing experiment.

I think the results of the experiment are inconclusive.  Thus far, only
one of six documents involved in this experiment has made it into the
RFC Editor queue.  It did not enter the queue until just recently
(maybe the week or two prior to IETF).  Alice has not yet reviewed the
diffs (as it has not made it to the front of the queue), so we do not
know whether the pre-editing has helped.

- Section 3.3 states

  The ambition level for cleanup can vary from:

  o Corrections to errors only,

  ...

  o Rewriting of all documents to the dictates of a style manual

We attempt only to correct errors, and we try to ensure that all
documents adhere to some form of grammar guide.  We try to make changes
that improve clarity and make the document more consistent within
itself, as well as across a document set.

- Section 3.3 states

  At times in the past year, stylistic editing has resulted in 40-100
  substantive changes in many documents.  These changes must then be
  vetted by all the authors followed by subsequent rounds of author
  acceptance and re-vetting.

I'm not sure what is considered a substantive change here, but
sometimes 40-100 substantive changes in one document may be necessary.
I believe this to be a very unusual case, and I am not sure if this
takes into account instances for which, although we did not change the
text correctly, the authors came back with a more clearly worded
alternate?

Also, an average size RFC is roughly 30 pages.  Assuming a middle
ground between the 40-100 changes gets us to an average of 70 changes
per document, which means an average of 2.33 changes per page.  This is
not a high rate of change considering the amount of work some of these
documents require.

Furthermore, there are often a large number of changes requested by the
authors themselves, many of which require AD review and approval.  Does
this 40-100 number differentiate the RFC Editor induced and the
author/wg chair/ad induced changes?

- Section 3.3 state
	
  o Potential Req-POSTEDIT-4 - The IETF Technical editor should
    refrain from making changes to improve readability that may change
    technical and consensus wording.

Do not make changes to improve readability??
We often find errors in documents after any number of revisions and
after the documents have been approved that require author/editor
clarification, e.g., cut and paste errors, missing text.

- Section 3.7 states

  o Potential Req-POSTCORR-3 - The IETF technical publisher should
    have the discretion to reject post-approval corrections as too
    late in the process and propose that it be handled as errata.

Errata maintenance will turn into an even bigger headache.  Why not
just make the correction instead of going through the process of
approving and posting an errata?

- Section 3.15 discusses the maintenance of errata... 

Who verifies this errata?

- Section 3.16 states

  When an permanent ID is allocated early, the technical publisher
  must provide a disclaimer and pointer in the index to resolve
  searches on that permanent ID.  The post approval version of the
  draft must be stored until the document is published since it could
  expire before publication.  

Again, this sounds as though the RFC Editor and ID functions will be
combined.  Or, does this just require that the technical publisher
request that the ID repository not expire a draft?

- Section 3.18 states

  Discussion: The RFC Editor does not maintain a document or database
  of terms or acronyms.

This statement is untrue.  However, our main goal is to be consistent
within a single document and across a set of documents.  If an author
has used terminology inconsistently, we will update the text with our
preferred usage.  If the author has already used the terminology
consistenly, we will leave the terms unaltered (unless they go against
some pre-specified usage, e.g., IPSEC would be changed to IPsec
regardless of whether the author used IPSEC consistently throughout the
document).

- Section 4 states

  Here are some metrics that could apply to the IETF technical
  publisher.

  ...

  3. Non author changes generated during publication

  4. Author changes generated during publication

This does not take into account changes requested by those other than
the editor and author.  For example, ADs and WG chairs.

- Section 4.1 states

  o Potential Req-TIMEFRMAES-1 - The IETF technical publisher should
    have a goal of an average publication time of no more than 4 weeks
    and at least 90% of all documents should be published within 8
    weeks. 

Does this time frame refer only to those documents that have been
through the early copy edit phase or all documents in general?

- Section 4.1 states

  o Potential Req-TIMEFRAMES-2 - For those documents for which early
    allocation of an identifier is request, The IETF technical
    publisher should have a goal of an average allocation time of no
    more than 1 week and at least 90% of early allocation documents
    should have a stable identifier allocated within 2 weeks of
    approval.  ...

  o Potential Req-TIMEFRAMES-3 - The IETF technical publisher should
    have a goal of an average time for completing the pre-approval
    editorial review of no more than 10 days and at least 90% of all 
    documents should be reviewed within 15 days.  ...

I guess since it says average, it's okay, but this does not take into
consideration document size and other variables associated with
document delay, such as the sudden influx of documents from the IESG
secretariat. 

- Section 4.3 states

  o Potential Req-EDITCHGSTATS-1 - The IETF technical publisher should
    provide monthly statistics on the types of editorial corrections
    being found during review as well as the percent of corrections
    which are rejected by the authors.

This will require even more time spent on each document, as it will
require stopping to make tally marks each time we mark a correction,
or going through the document a second time to count and tally the
marks inserted.  It will also require a third review of the document
to tally the changes accepted/rejected by the authors.  For a process
that is supposedly already too time consuming, this will make the
process even more lengthy.  It will also require formal definitions of
the varying types of errors that can be found, as often times, a
correction cannot be classified as purely
grammatical/editorial/technical. 

- Section 5 states

  o Exception Handling: If publication timelines can be reduced
    sufficiently or permanent identifiers allocated early, then
    expedited handling may no longer be needed.

According to section 3.13, exception handling refers to the reasons a
document can be withdrawn or put on hold.  The above seems to be
refering to early allocation requests.

_______________________________________________
Techspec mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/techspec

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 17 Apr 2006 07:31:15 -0700

At 3:16 PM -0700 4/14/06, Sandy Ginoza wrote:
>RFC Editor queue.

That document was in the WG I co-chair, so I feel comfortable speaking for it.

>It did not enter the queue until just recently
>(maybe the week or two prior to IETF).

The document went into the RFC Editor queue six weeks ago.

>Alice has not yet reviewed the

As we reported to Bert back in November, the experiment from the WG 
side went smoothly and quickly. As Jari Arkko said at the time:

At 4:41 PM +0200 11/2/05, Jari Arkko wrote:
>The change was really minor. Language in the document
>was fairly good already.
>
>Well, there were some dozens of changes as I recall, but
>none of them appeared to impact readibility in significant
>degree.

In other words, the early review step didn't impede us and hopefully 
helped the overall process.

--Paul Hoffman, Director
--VPN Consortium

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 17 Apr 2006 07:38:19 -0700

At 3:16 PM -0700 4/14/06, Sandy Ginoza wrote:
>marks inserted.

Not really. First, statistics could be collected on a representative 
sample of documents during the month, not every document. Second, the 
person tallying does not need to be the person correcting, and 
therefor there is no requirement of "stopping to make tally marks". 
Such tallying could be done by the management, not the workers, for 
example.

--Paul Hoffman, Director
--VPN Consortium

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 19 Apr 2006 14:04:09 +0200

>> one of six documents involved in this experiment has made it into the

I think it would be more accurate to say "the results aren't 
known yet." The fact is that the natural timescale for measuring
results is many months.
> As we reported to Bert back in November, the experiment from the WG side 
> went smoothly and quickly. As Jari Arkko said at the time:
> 

There's a question here which, I suggest, is actually out of
scope for techspec: should we push the onus of document
quality assurance back to the WGs? I suggest that discussing
that here and now would be a bit of a distraction.

     Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 19 Apr 2006 14:16:12 +0200

Sandy,

Sandy Ginoza wrote:
...
> This statement is untrue.

That's interesting. Where can authors find the glossary?

    Brian

> However, our main goal is to be consistent

---

From: Leslie Daigle <[email-masked]>
Date: Wed, 19 Apr 2006 16:13:16 -0400

I want to comment on one specific point:

> clarification, e.g., cut and paste errors, missing text.

The quoted requirement is not something that you can split
and interpret separately as you have done in your comment.

Yes, RFC Editor certainly routinely catches issues of readability,
greater and lesser.  AUTH48 is always (at least, for me :-) )
an exercise in learning how unclearly one writes ;-)   And,
there was a notable instance when the RFC Editor caught the fact
that my document had 2 non-sentences trailing at the end of
a section -- leftovers from previous edits.  I had not caught
that.  The IETF-wide last call had not caught that.  The IESG
review had not caught that.  Oops on all of us :-)

But.

"that may change technical ... wording" is key.  Too often
"clarity" breaks the intended meaning.

A simple sample, from my own experience:

ORIGINAL:
2.2.4  S-NAPTR and Successive Resolution

    As shown in the example NAPTR RR set above, it is possible to have
    multiple possible targets for a single application service+protocol
    pair.  These are to be pursued in order until a server is
    successfully contacted or all possible matching NAPTR records have
    been successively pursued to terminal lookups and servers contacted.
    That is, a client must backtrack and attempt other resolution paths
    in the case of failure.

AUTH-48:
2.2.4.  S-NAPTR and Successive Resolution

    As shown in the example set above, it is possible to have multiple
    possible targets for a single application service+protocol pair.
    These are to be pursued in order until a server is successfully
    contacted or all possible matching NAPTR records have been
    successively pursued to terminal lookups and contacted servers.  That
    is, a client must backtrack and attempt other resolution paths in the
    case of failure.

ISSUE:
The binding of "all" has failed, in rewriting "servers contacted"
to "contacted servers".

NEW:
2.2.4.  S-NAPTR and Successive Resolution

    As shown in the example set above, it is possible to have multiple
    possible targets for a single application service+protocol pair.
    These are to be pursued in order until a server is successfully
    contacted or all possible matching NAPTR records have been
    successively pursued through terminal lookup and server contact. That
    is, a client must backtrack and attempt other resolution paths in the
    case of failure.

Leslie.