Subject: [TOOLS-DEVELOPMENT] Very early start to an SoW for building xml2rfc v3 documents from text documents

From: Robert Sparks <[email-masked]>
Date: Fri, 16 May 2014 17:21:20 -0500

This is far from complete, but I hope it will seed some useful discussion.

What else do you think this needs to say?

(I haven't included the existing potential components to a solution that 
folks have already pointed out).

RjS

---

From: Henrik Levkowetz <[email-masked]>
Date: Sat, 17 May 2014 17:51:52 +0200

Hi Robert,

> folks have already pointed out).

A first thought:  

I think the rfc-editor staff currently uses mac and linux,
but the detailed description say 'must run as a command-line program, under Unix-like shells or a Windows command prompt.'  I don't think it would be
a good thing to accept a solution which worked only on the windows command-
line prompt?

The following items all address aspects of what is 'good enough'.  Maybe
some thought should be given to how to express that, independent of how
successful the tool turns out to be in identifying and translating specific
constructs?  I think that it is likely that some constructs will turn out to
be easier to process than is believed at the outset, while other constructs
turn out to be very hard, much harder than initially thought.

Second:

Lossless is a strong word -- is the intention to say the same as 
'preserve all body text'?

To my mind, there are some differences, most of them being related to white-
space handling.  Indentation, list spacing, list symbols, pre-and post-
artwork spacing, and correct recognition of artwork comes to mind.

(I've given some thought to how to identify artwork correctly on and off
during the last half-year, and think I've come up with an approach that
is better than others I've seen proposed and implemented, but if 100%
perfection is required, I think the problem might be too hard.)

Thirdly:

'May remove boilerplate, headers, and footers':

  * Does this mean that correct parsing of author and other header content
    is desired but not required?  (I think yes)

  * Does this mean that it is desired, but not required, to remove boilerplate?
    (I think that it's either required or strongly desired to remove boilerplate
    that passes idnits).

Fourthly:

There are some constructs that can be rendered in different manner -- some
that comes to mind are tables and sparse lists, which may be hard to
distinguish from artwork.  This again comes down to what is good enough --
do we require that these be correctly identified, and tables and lists
generated in all cases?

Finally:

Would it make sense to include the documents used in xml2rfc testing in this
list?  In particular, in the repository you'll find 
  trunk/tests/input/draft-miek-test.xml and trunk/tests/valid/draft-miek-test.txt
which cover a lot of variations of draft constructs.

Best regards,

	Henrik

---

From: Robert Sparks <[email-masked]>
Date: Mon, 19 May 2014 09:33:02 -0500

> a good thing to accept a solution which worked only on the windows command-
It certainly wasn't my intent to suggest that as a possible outcome. I 
was focused on it not being a gui-only solution.
I'll change the text to be explicit that it must work as a linux and 
os/x command line argument and that working from a windows command 
prompt would be a nice to  have.

> constructs?
What I've come up with so far is the notion that completing the 
conversion takes minimal manual effort. Is there anything that would be 
useful to the implementer to say explicitly about what that means? Can 
you think of any other ways to characterize 'good enough'?
>    I think that it is likely that some constructs will turn out to

> 'preserve all body text'?
Yes, but as you note below, sometimes preserving whitespace turns out to 
be important too.
> perfection is required, I think the problem might be too hard.)
Exactly. We're not looking for something that's an AI project - rather 
something
that gets the bulk of the tedium out of the way, and lets a person solve the
classification problems with minimal hassle.
>      is desired but not required?  (I think yes)
Yes.
>      (I think that it's either required or strongly desired to remove boilerplate
Yes - and thanks for this kind of question and discussion - it teases 
out a requirement we might otherwise have missed (to _not_ remove 
boilerplate-like text that doesn't match.)
> do we require that these be correctly identified, and tables and lists
No, and I think it should fail to blocks of artwork when the decision 
isn't easy.
those will be easier to post-process than artwork that was incorrectly 
turned into
a table.

Relevant to a recent conversation on rfc-interest. Should these blocks 
of artwork be
captured in CDATA (rather than converting all the < to &lt; for 
example)? I think that would make understanding and working with the result
_much_ easier.

> which cover a lot of variations of draft constructs.
(For everyone else, here's a quick link to the txt version of that document:
<http://trac.tools.ietf.org/tools/xml2rfc/trac/browser/trunk/cli/tests/valid/draft-miek-test.txt>)

That's a good suggestion. Thanks!

Some of the rfc format design team discussions have pointed repeatedly 
to constructs like
a ladder diagram followed by PDU descriptions like this section:
<http://tools.ietf.org/html/rfc7118#section-8.2>
and, of course, packet bit descriptions like
<http://tools.ietf.org/html/draft-ietf-mip4-multiple-tunnel-support-07#section-4.2>

It would be good to make sure the tool does something sane with inputs 
like those.

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 19 May 2014 23:14:19 +0200

Hi Robert,

> was focused on it not being a gui-only solution.

Ah. Right :-)

> prompt would be a nice to  have.

Ack.

> you think of any other ways to characterize 'good enough'?

Not sure.  Maybe it would be possible to list constructs that must, should,
and would be nice to have, and those we expect to be hard (and not required)
and reserve final judgement after inspecting the results (maybe Alice and/
or Sandy should be in on that final inspection, too?)

> Yes, but as you note below, sometimes preserving whitespace turns out to 

Right.

> Exactly. We're not looking for something that's an AI project -
> rather something that gets the bulk of the tedium out of the way, and
> lets a person solve the classification problems with minimal hassle.
>>
>>
>> Thirdly:
>>

Right.

> No, and I think it should fail to blocks of artwork when the
> decision isn't easy. those will be easier to post-process than
> artwork that was incorrectly turned into a table.

Ack.

> Relevant to a recent conversation on rfc-interest. Should these
> blocks of artwork be captured in CDATA (rather than converting all
> the < to &lt; for example)? I think that would make understanding and
> working with the result _much_ easier.

Agreed.

> Some of the rfc format design team discussions have pointed
> repeatedly to constructs like a ladder diagram followed by PDU
> descriptions like this section: 

To me, this is indubitably artwork, and must be handled as such.

> <http://tools.ietf.org/html/draft-ietf-mip4-multiple-tunnel-support-07#section-4.2>

Ack.  Same comment.

> It would be good to make sure the tool does something sane with inputs 

Yes, definitely.

Best regards,

	Henrik

---

From: Robert Sparks <[email-masked]>
Date: Mon, 19 May 2014 16:39:47 -0500

Here's an iteration incorporating the below discussion with Henrik.
What else needs to change?

RjS

Overview

The RFC Editor intends to continue to accept submissions in many 
formats, but will be working primarily in xml, using the xml2rfc v3 
grammar. The most common submission format supplied now is 
Internet-Draft formatted text. Tools already exist to convert other 
formats to text.

The goal of this project is to simplify the creation of an initial 
xml2rfc v3 version of a document submitted in another non-xml format. A 
perfect automated conversion is not expected. Rather, the application 
should produce a document that is well-formed, and sufficiently close to 
correct that an editor can complete the conversion with minimal effort.

Deliverables/Tasks

This project will create an application to convert an Internet-Draft 
formatted text file to an xml2rfc v3 document. The development effort 
will include

  *

    Designing the command line interface

  *

    Demonstrating the conversion of a specified set of text documents

  *

    Providing an extensible test suite for the application

    Detailed Description and Requirements

    The application must run as a command-line program under Linux and
    os/x shells. Running at a Windows command prompt would be nice to
    have but is not required. The application must be easily adaptable
    to become part of a web-service offering the translation.

    The developer will work with the Program Manager to agree on an
    initial command-line interface before beginning development work.

    During development, the following documents will be used to refine
    the application. These documents were chosen to highlight difficult
    cases in determining the best conversion from text to xml (such as
    recognizing figures and tables, and correctly identifying references).

  *

    http://trac.tools.ietf.org/tools/xml2rfc/trac/browser/trunk/cli/tests/valid/draft-
    miek-test.txt

  *

    http://tools.ietf.org/html/draft-ietf-mip4-multiple-tunnel-support-07 (particularly
    section 4.2)

  *

    http://datatracker.ietf.org/doc/rfc7118/ (particularly section 8.2)

    The conversion must preserve the content of the input text file. In
    certain cases, some input text may be removed including

  *

    Headers, footers, and other content that would be generated
    algorithmically when otherwise expressed semantically in the xml format

  *

    Boilerplate that matches what is currently defined in RFC5741 (and
    would be accepted by id-nits). The appropriate attributes to the
    <rfc> element will replace such boilerplate. A section that appears
    where boilerplate is expected, but does not match currently
    acceptable boilerplate, must be preserved and converted the same as
    an ordinary section would be.

    In general, the conversion is not expected to preserve whitespace.
    Elements identified as artwork, however, must preserve whitespace.
    When it is ambiguous whether an input block is artwork, a table, or
    a list, the application should favor converting the block to
    artwork. Artwork should be captured in a CDATA construct rather than
    using xml- escaping to simplify the work of correcting a
    mis-classification or adjusting the boundaries of the artwork.

    To the extent possible, content that is explicitly represented in
    the xml2rfc v3 grammar should be converted to use that
    representation. For example, the application is expected to

  *

    Identify references and produce populated <reference> elements and
    appropriate <xref> elements

  *

    Parse any author information section and produce populated <author>
    elements

  *

    Identify special sections (such as the abstract) and represent them
    with the

    appropriate element

    Scratch/Questions

    Do we make our own single input and output document to test against?
    That combined with the expectation that the program behave similarly
    on similar inputs may be how we answer "good enough".

    Alternatively, should we only provide a set of test inputs, and
    define the project as iteratively converging on when the result is
    sufficient?

Microsoft Word - TextSubmissionConverter-01.docx
On 5/19/14, 9:33 AM, Robert Sparks wrote:
>> but the detailed description say 'must run as a command-line program, 
>> under Unix-like shells or a Windows command prompt.'  I don't think 
>> it would be
> conversion takes minimal manual effort. Is there anything that would 
> be useful to the implementer to say explicitly about what that means? 
> Can you think of any other ways to characterize 'good enough'?
> Yes, but as you note below, sometimes preserving whitespace turns out 
> to be important too.
>>

---

From: "Heather Flanagan (RFC Series Editor)" <[email-masked]>
Date: Tue, 20 May 2014 10:40:01 +0100

> Internet-Draft formatted text. Tools already exist to convert other

Suggested change:
The RFC Editor intends to accept submission in plain text and XML, but
will be working primarily in xml using the xml2rfc v3 grammar.

> xml2rfc v3 version of a document submitted in another non-xml format.
> A perfect automated conversion is not expected. Rather, the
> application should produce a document that is well-formed, and
> sufficiently close to correct that an editor can complete the
> conversion with minimal effort.
>

So this SoW does not cover conversion of v2 to v3?  This is just
text-to-XML? 

>     the application. These documents were chosen to highlight
>     difficult cases in determining the best conversion from text to
>     xml (such as recognizing figures and tables, and correctly
>     identifying references).
>
>  *
>
>     http://tools.ietf.org/html/draft-ietf-mip4-multiple-tunnel-support-07
>     (particularly section 4.2)
>
>  *
>
>     The conversion must preserve the content of the input text file.
>     In certain cases, some input text may be removed including
>
>  *
>
>     <rfc> element will replace such boilerplate. A section that
>     appears where boilerplate is expected, but does not match
>     currently acceptable boilerplate, must be preserved and converted
>     the same as an ordinary section would be.
>
>     When it is ambiguous whether an input block is artwork, a table,
>     or a list, the application should favor converting the block to
>     artwork. Artwork should be captured in a CDATA construct rather
>     than using xml- escaping to simplify the work of correcting a
>     Parse any author information section and produce populated
>     <author> elements
>
>  *
>
>     Identify special sections (such as the abstract) and represent
>     them with the
>
>     appropriate element
>
>     Scratch/Questions
>
>     Do we make our own single input and output document to test
>     against? That combined with the expectation that the program
>     behave similarly on similar inputs may be how we answer "good
>     enough".
>
I'd like to see inclusion of documentation of the tool and (possibly)
training on it's use for at least the RFC Editor, but perhaps also at a
tutorial for a future IETF meeting.  I think these things are going to
be common requirements for the SoWs developed for the pieces and parts
of the format work.

Thanks, Robert!
Heather