Subject: [TOOLS-DEVELOPMENT] Thoughts on path forward on tools to support the new rfc format

From: Robert Sparks <[email-masked]>
Date: Tue, 15 Apr 2014 11:50:19 -0500

Trying to look at the horizon for a moment:

When we are at the point we are producing RFCs in XML, the workflow
will look like this:

- Authors create content in potentially many formats, one of which
   might be the new XML. (The others include text, word, nroff, and
   any other currently accepted formats).
   Authors need to be able to tell, and the submission point needs to 
check,
   whether the document meets the requirements for being accepted as input
   into publication process.

- The RFC Editor will work in the new XML format
   * Converting other input formats to the XML format
   * Refining the xml into a proposed final canonical RFC product

- The proposed canonical product will be rendered into several
   publication formats

- The authors and the RFC Editor will iterate on reviews of the proposed
   product. (We expect the authors will use the derived publication formats
   primarily for their review.) Both the authors and the rfc editor need to
   be able to quickly tell that the document is still acceptable for
   publication as they edit the proposed final result.

- The end result of that editing round will be published as the RFC
   (a frozen canonical XML document), and it's derived publication formats.

I think this flow would best be supported by these tools:

* A tool that converts text documents that are in close-to-internet-draft
   form into the new XML format. We can utilize the exisiting tools that
   render text from the other input formats for submission to get those
   formats.

* A tool that points out what needs to change in order for the xml document
   to be ready. This is likely a combination of the functionality (think in
   terms of functionality, not implementation for the moment) in id-nits,
   the first part of xml2rfc (that parses the document) and several of
   the standalone programs in the current production workflow,
   This should be usable by the rfc editor and document authors, and should
   be useful at any point in the development of the document, from initial
   version to published RFC.

* One or more tools that create the derived publication formats. These also
   need to be available to both the rfc editor and the authors.

* Diff tools to help compare versions.

I think these are the steps we should take towards acquiring those tools
(If you've seen Heather's email discussing threads of next steps, this
  is intended to incorporate them.)

1) The volunteer community (primarily Henrik and Julian if I understand 
correctly)
    will be prototyping the production of an example publication format 
(HTML)
    from a well-formed example input document in the canonical format. 
(A text
    format is likely to also fall out, but is not going to be the 
primary focus.) This
    work is expected to sanity check that the new grammar allows authors to
    express what we intended, and that it can be rendered as expected. 
Its focus
    won't be on solving rendering for arbitrary inputs however - they'll be
    concentrating on what needs to be done perform the sanity check and make
    the example publication format document so we can iterate with the
    community on those quickly. It may be that they discover that this
    production step is easy, and that it's essentially done when they're
    through, or it may be that they develop requirements for a larger
    effort.

2) We can start developing an SoW for the tool that would point out
    what needs to change. I see this as a single tool with switches
    to control the level of verification. It will at a minimum check
    for well-formed XML. It can be told to look for spelling errors and
    make grammar checks (such as duplicated words). It can be told to
    highlight where non-ascii UTF-8 is used. It _might_ also be the thing
    that can be told to verify any SVG satisfies our profile, and that
    and that other included things (like mibs or abnf) check out, but
    while developing the SoW we may decide to keep these as separate
    tools.

3) We can start developing an SoW for an xml diff tool (and continue
    discussing whether we really need such a thing as part of that
    development).

4) We can start developing an SoW for the tool that converts text
    documents that are in close-to-internet-draft form into the
    new XML format.

Early results from the prototyping effort in 1) will feed into 2).
Once the prototyping is fairly far along, we'll know if we need a
5) or not for specing the publication format converters.

What am I missing? Does anyone think this is the wrong way to
approach this?

RjS

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 15 Apr 2014 19:14:28 +0200

Hi Robert,

One point inline, and one at the end:

On 2014-04-15 18:50 Robert Sparks said the following:
>     correctly) will be prototyping the production of an example publication
>     format (HTML) from a well-formed example input document in the canonical
>     format.  (A text format is likely to also fall out, but is not going
>     to be the primary focus.)

This doesn't match my understanding -- needs discussion.  I think Julian
was looking explicitly at extending xml2rfc's text mode, as that's what
gets used most.

>     This work is expected to sanity check that
>     the new grammar allows authors to express what we intended, and that
>     it can be rendered as expected. Its focus

>From the viewpoint of xml2rfc, it currently can produce 3 of the 5
reading formats (paginated and unpaginated text, and html) but neither
pdf or epub is handled at all, and will be a substantial effort.  I think
we can safely say that we'll need a step 5) for that.

> What am I missing? Does anyone think this is the wrong way to

Seems fairly good overall.  I'm a bit concerned about the approach to things
like spell checking is to build it into a custom tool, instead of using
something like aspell in html mode.  Custom tools always cost more in
maintenance than an off-the-shelf tool.

Best regards,

	Henrik

---

From: Robert Sparks <[email-masked]>
Date: Tue, 15 Apr 2014 13:52:27 -0500

> was looking explicitly at extending xml2rfc's text mode, as that's what
Ack.
> something like aspell in html mode.
Understood - I was anticipating this code would _use_ aspell, not 
re-implement
it's functionality (perhaps via its library rather than it's 
command-line interface).

>   Custom tools always cost more in

---

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Apr 2014 21:24:38 +0200

On 2014-04-15 19:14, Henrik Levkowetz wrote:
> ...
> ...

Actually, I was mainly thinking about rfc2629.xslt's HTML generation 
(and maybe EPub).

Best regards, Julian