Subject: Re: [TOOLS-DEVELOPMENT] Thoughts on path forward on tools to support the new rfc format

From: Russ Housley <[email-masked]>
Date: Mon, 5 May 2014 10:45:05 -0400

Robert:

> I think this flow would best be supported by these tools:
> 
> * A tool that converts text documents that are in close-to-internet-draft
>  form into the new XML format. We can utilize the exisiting tools that
>  render text from the other input formats for submission to get those
>  formats.

Are you aware that nroffedit will create a .nroff file from an I-D text file.
Is there anything to leverage from nroffedit?

> 
> * A tool that points out what needs to change in order for the xml document
>  to be ready. This is likely a combination of the functionality (think in
>  terms of functionality, not implementation for the moment) in id-nits,
>  the first part of xml2rfc (that parses the document) and several of
>  the standalone programs in the current production workflow,
>  This should be usable by the rfc editor and document authors, and should
>  be useful at any point in the development of the document, from initial
>  version to published RFC.

This makes sense to me.  Do these existing tools offer a starting point?

> 
> * One or more tools that create the derived publication formats. These also
>  need to be available to both the rfc editor and the authors.

How the RFC Editor will use them is obvious.
Will the author use one format to do the review?
Do you envision the posting of XML, and the I-D submission tool making the other formats?

> 
> * Diff tools to help compare versions.

This is vital!

Russ

---

From: Tony Hansen <[email-masked]>
Date: Mon, 05 May 2014 11:23:51 -0400

> Is there anything to leverage from nroffedit?

I think there might be. However, the last time I looked, the author had 
not made any of the Java source available.

Another potential starting point is Henrik's rfcmarkup. He has spent 
considerable time getting the heuristics correct.

> Do you envision the posting of XML, and the I-D submission tool making the other formats?

I envision xml.resource.org (or possible replacements for it) being able 
to generate a tar/zip/tgz file with multiple output formats available.

I envision the I-D submission tool optionally accepting only the XML and 
generating the other formats.

People should still be able to upload I-Ds in only text format. The I-D 
submission tool should also start accepting the other forms in addition 
to the text format. I envision the I-D submission tool accepting a 
tar/zip/tgz file with all of the formats at once. (There are open 
questions about ensuring consistency across the formats, but it's no 
worse than the current state of affairs.)

In the future, I could see the I-D submission tool also accepting just 
the HTML form of a document without the text version. That would be a 
bigger transition. It's possible that an I-D html-to-text converter 
could be used to automatically extract a text version.

---

From: Russ Housley <[email-masked]>
Date: Mon, 5 May 2014 11:38:05 -0400

Tony:

> People should still be able to upload I-Ds in only text format.

I completely agree.

> The I-D submission tool should also start accepting the other forms in addition to the text format.

It already does, but the set is not sufficient.

> I envision the I-D submission tool accepting a tar/zip/tgz file with all of the formats at once. (There are open questions about ensuring consistency across the formats, but it's no worse than the current state of affairs.)

I do not think that there are significant checks today.

> In the future, I could see the I-D submission tool also accepting just the HTML form of a document without the text version. That would be a bigger transition. It's possible that an I-D html-to-text converter could be used to automatically extract a text version.

I do not think we should do this until there is a tool for the RFC Editor to get an XML file from the HTML.

Russ

---

From: Robert Sparks <[email-masked]>
Date: Mon, 05 May 2014 11:02:40 -0500

> I do not think that there are significant checks today.
At the moment we have a version (the txt) that we consider authoritative 
and we leave it to the community to complain about discrepancies in 
other versions.
We need, I think, to keep the notion of "what wins" for internet drafts. 
If we start allowing submission without the text form, what becomes 
authoritative?
I was expecting it to be the new xml, but this thread seems to be 
suggesting that we might allow someone to submit an internet draft as 
only HTML?
If someone submitted both HTML and PDF, what is the community supposed 
to review (and for an IETF stream document, what do you make a consensus 
call against) ?

> _______________________________________________
> TOOLS-DEVELOPMENT mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-development

---

From: Tony Hansen <[email-masked]>
Date: Mon, 05 May 2014 23:51:47 -0400

> It already does, but the set is not sufficient.

what I meant :-)

> I do not think that there are significant checks today.

I agree, but that should not be a blocking issue.

> I do not think we should do this until there is a tool for the RFC Editor to get an XML file from the HTML.

Add another item to the wish list. :-)

     Tony Hansen

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 06 May 2014 11:03:32 +0200

On 2014-05-05 18:02 Robert Sparks said:
>>> I envision the I-D submission tool accepting a tar/zip/tgz file
>>> with all of the formats at once. (There are open questions about
>>> ensuring consistency across the formats, but it's no worse than
>>> the current state of affairs.)

Correct; because the .txt version has been authoritative, and because the
v1 xml2rfc format permits a valid xml2rfc format .xml file to use
local include files, which means that the source .xml for a .txt document
cannot always be processed after upload, because of missing include files.

This was discussed when we wrote 4228, and when we issued the submission
tool SoW; and we decided that it was more important at that time to
encourage people to upload supplementary XML, rather than to require the xml
to be standalone and to run it and verify it against the .txt document.

> At the moment we have a version (the txt) that we consider
> authoritative and we leave it to the community to complain about
> discrepancies in other versions. We need, I think, to keep the notion
> of "what wins" for internet drafts.

Indeed.

> If we start allowing submission
> without the text form, what becomes authoritative? I was expecting it
> to be the new xml,

And I still expect that to be the case.  I cannot see that it's reasonable
to go any other way, at least at this point in time.  There are too many
people who have tooling that depend on having or being able to generate a
.txt version.

> but this thread seems to be suggesting that we
> might allow someone to submit an internet draft as only HTML? If
> someone submitted both HTML and PDF, what is the community supposed 
> to review (and for an IETF stream document, what do you make a
> consensus call against) ?

Good point.

>>> In the future, I could see the I-D submission tool also accepting
>>> just the HTML form of a document without the text version. That
>>> would be a bigger transition. It's possible that an I-D
>>> html-to-text converter could be used to automatically extract a
>>> text version.

I don't see that it's reasonable.  If the html version is produced from
xml2rfc-format xml, then the submitter could just as well submit the xml
too; if it is hand-written, the html markup may very well be so far outside
the envelope of what a html-to-txt tool is designed to handle that the
resulting txt would be rubbish.

>> Editor to get an XML file from the HTML.

Not even then, for the reason above.  We don't want to accept any HTML, and
if the html follows the format of the html generated from xml2rfc-format xml,
then (at least today) it's unlikely to have been produced from anything but
our xml format.  In which case we would want to have that source, directly.

Best regards,

	Henrik