Subject: [xml2rfc-dev] xml2rfc: use of <boilerplate> in preotool step

From: Julian Reschke <[email-masked]>
Date: Tue, 8 Oct 2019 06:37:47 +0200

So xml2rfc, when running the preptool step, apparently puts the TOC
inside the <boilerplate> element.

This is problematic for a variety of reasons:

- it uses <boilerplate> for something it was designed for (which would
require changes to RFC 7991 and RFC 7998)
- it makes automatic checking of <boilerplate> harder
- it requires the @pn attribute to become a full-blown link target

As far as I can tell, none of this is currently documented.

Now, as far as I can tell, the reason for this change is entirely an
implementation detail now leaking into the grammar and the canonical
XML. Namely, that xml2rfc internally used the prepped output to embed
information for the rendering backends. This is of course absolutely ok,
but it shouldn't affect the canonical format, nor the semantics of
<boilerplate>.

Best regards, Julian

PS: raised as <https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/440>

---

From: Heather Flanagan <[email-masked]>
Date: Wed, 9 Oct 2019 08:47:20 -0700

> On Oct 7, 2019, at 9:37 PM, Julian Reschke <[email-masked]> wrote:
> 

Hi -

Is there any reason not to create a separate <toc> element that would live in <front>?

-Heather

---

From: Julian Reschke <[email-masked]>
Date: Wed, 9 Oct 2019 18:14:59 +0200

On 09.10.2019 17:47, Heather Flanagan wrote:
> ...
> ...

That would be better, as it wouldn't conflate things with <boilerplate>.

That said, I still don't get why the Table of Contents needs to be part
of the canonical XML. I understand why it's desirable to be present in
an intermediate representation before rendering, but that doesn't mean
it needs to leak out into the canonical format.

Best regards, Julian

---

From: Heather Flanagan <[email-masked]>
Date: Wed, 9 Oct 2019 09:44:32 -0700

> On Oct 9, 2019, at 9:13 AM, Julian Reschke <[email-masked]> wrote:
> 

Ok. 

> it needs to leak out into the canonical format.

I feel fairly strongly that the final XML must be self-contained. While someone can derive the TOC based in section headers, deriving details is not as good as just have the thing available in one place. 

You don’t agree. I get that. But I am insisting on this one. 

Heather

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 9 Oct 2019 16:47:51 +0000

> it needs to leak out into the canonical format.

+1 to Julian's comment. That's one reason why the design team chose not to put it in the XML at all. Another reason is that people writing XML might think that they can edit the elements of <toc> and have those edits remain through the publication cycle.

The design in the RFCs describing v3 has the tables only being created during HTML / text / PDF / whatever output. This means that the TOC for HTML doesn't have page numbers, but the TOC for text and PDF can have the appropriate page numbers.

--Paul Hoffman

---

From: Julian Reschke <[email-masked]>
Date: Wed, 9 Oct 2019 18:50:45 +0200

> You don’t agree. I get that. But I am insisting on this one.

I don't see the point (and it contradicts the specs). But well, it won't
hurt either, except that it causes potential confusion. Just make sure
it's fully specified (for instance, what to do when it's present in the
input source and does not match the to-be-generated ToC).

Also, as a reminder, there are other places where text is generated, and
which is not reflected in the canonical XML (for instance, anything
generated by processing <u>, or the code that reformats the contents of
<postal>).

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 9 Oct 2019 18:55:50 +0200

On 09.10.2019 18:50, Julian Reschke wrote:
>> I feel fairly strongly that the final XML must be self-contained.
>> While someone can derive the TOC based in section headers, deriving
>> details is not as good as just have the thing available in one place.
>>

...or under which circumstances an <xref> can reference an element
identified by @pn instead of @anchor...

Best regards, Julian

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 9 Oct 2019 19:21:06 +0200

Hi Paul,

On 2019-10-09 18:47, Paul Hoffman wrote:
> +1 to Julian's comment. That's one reason why the design team chose
> not to put it in the XML at all. Another reason is that people
> writing XML might think that they can edit the elements of <toc> and
> have those edits remain through the publication cycle.
> 
> The design in the RFCs describing v3 has the tables only being
> created during HTML / text / PDF / whatever output. This means that
> the TOC for HTML doesn't have page numbers, but the TOC for text and
> PDF can have the appropriate page numbers.

The XML currently generated for the ToC is used as-is to generate entries
with page numbers for paginated text, and without them for other formats.
No conflict.

	Henrik

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 11 Oct 2019 17:37:08 +1100

On Thu, Oct 10, 2019, at 03:44, Heather Flanagan wrote:
> I feel fairly strongly that the final XML must be self-contained. While 
> someone can derive the TOC based in section headers, deriving details 
> is not as good as just have the thing available in one place. 

I'm struggling to understand this argument.  In what way does having a TOC in the XML make the document more self-contained?  It only seems to add redundancy, which only invites contradiction as far as I can see.  In my view, informed by long experience with DRY (https://en.wikipedia.org/wiki/Don%27t_repeat_yourself), derivations are always far more robust.

Removing the redundancy, all you have is a signal about the desired *location* of the TOC, but I don't see any value in encoding that information.

> You don’t agree. I get that. But I am insisting on this one. 

I get the appeal to authority bit (or assertion thereof really), but I don't think we should operate that way, and I hope that you don't expect us to accede to that either.

---

From: Julian Reschke <[email-masked]>
Date: Fri, 11 Oct 2019 08:58:46 +0200

> I'm struggling to understand this argument.  In what way does having a TOC in the XML make the document more self-contained?  It only seems to add redundancy, which only invites contradiction as fa…

Yes.

> Removing the redundancy, all you have is a signal about the desired *location* of the TOC, but I don't see any value in encoding that information.

Oh, that isn't supposed to be variable anyway.

> I get the appeal to authority bit (or assertion thereof really), but I don't think we should operate that way, and I hope that you don't expect us to accede to that either.

In particular, I would hope that the principle in question would be
applied everywhere, not just here. (Reminder: there is a least one far
more serious case where the canonical XML is not self-contained, and the
rendering depends on a concrete implementation (parts of which
undocumented), and a third-party database - Unicode - which is not
immutable).

Best regards, Julian

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 11 Oct 2019 11:16:17 +0200

Martin:

On 2019-10-11 08:37, Martin Thomson wrote:
> I'm struggling to understand this argument.  In what way does having
> a TOC in the XML make the document more self-contained?  It only
> seems to add redundancy, which only invites contradiction as far as I
> can see.  In my view, informed by long experience with DRY
> (https://en.wikipedia.org/wiki/Don%27t_repeat_yourself), derivations
> are always far more robust.
> 
> Removing the redundancy, all you have is a signal about the desired
> *location* of the TOC, but I don't see any value in encoding that
> information.
> 
> I get the appeal to authority bit (or assertion thereof really), but
> I don't think we should operate that way, and I hope that you don't
> expect us to accede to that either.

After a discussion about including the generated Table of Contents and
Index in the prepped XML output file last year, it was decided to keep
the ToC, but not include the Index.  Now Julian opposed having the ToC
as part of the <boilerplate> element, so it was moved to <toc> (which I
think is a good move).

Are you against moving the ToC to a separate element within <front>,
instead of making it a <section> within <boilerplate>, or are you aiming
to undo the decision from last year about including the ToC?

	Henrik

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 11 Oct 2019 21:52:58 +1100

> to undo the decision from last year about including the ToC?

I don't much see the point, but I don't want to re-open old discussions.  Where there more reasons in favour that contributed to the decision to include the (redundant) information?