Subject: Re: [TOOLS-DEVELOPMENT] Preview release of Text Submission Converter, id2xml

From: Megan Ferguson <[email-masked]>
Date: Fri, 16 Jun 2017 09:54:33 -0700

Hi Henrik,

Revisiting this file using the new release (v1.0.1):

>> Input file: draft-ietf-trill-directory-assist-mechanisms-12
>> Version: id2xml 1.0.0
>> Issues: File not originally generated with XML
>> Files available: 
>> https://www.rfc-editor.org/rfc/v3test/draft-ietf-trill-directory-assist-mechanisms-12v3.original 
>> https://www.rfc-editor.org/rfc/v3test/draft-ietf-trill-directory-assist-mechanisms-12v3.txt
>> https://www.rfc-editor.org/rfc/v3test/draft-ietf-trill-directory-assist-mechanisms-12v3-rfcdiff.html

Note - we edited the file somewhat to combat the previously discussed issues.

1) There were a few reference oddities and follow-up questions:

a) The following are the work in progress (WiP) references that could not parse.  

draft-ietf-trill-directory-assist-mechanisms-12v3.txt(2237): Warning: Failed parsing a reference:
[rfc6439bis] - D. Eastlake, Y. Li, M. Umair, A. Banerjee, and F. Hu,
     "Routing Bridges (RBridges): Appointed Forwarders", draft-ietf-
     trill-rfc6439bis, Work in Progress.

draft-ietf-trill-directory-assist-mechanisms-12v3.txt(2252): Warning: Failed parsing a reference:
[ARPND] - Y. Li, D. Eastlake, L. Dunbar, R. Perlman, I. Gashinsky,
     "TRILL: ARP/ND Optimization", draft-ietf-trill-arp-
     optimization, Work in Progress.

draft-ietf-trill-directory-assist-mechanisms-12v3.txt(2256): Warning: Failed parsing a reference:
[DirAsstEncap] L. Dunbar, D. Eastlake, R. Perlman, I. Gashingksy,
     "Directory Assisted TRILL Encapsulation", draft-ietf-trill-
     directory-assisted-encap, Work in Progress.

draft-ietf-trill-directory-assist-mechanisms-12v3.txt(2260): Warning: Failed parsing a reference:
[SmartEN] R. Perlman, F. Hu, D. Eastlake, K. Krupakaran, T. Liao,
     "TRILL Smart Endnodes", draft-ietf-trill-smart-endnodes",
     draft-ietf-trill-smart-endnodes, Work in Progress.

Some questions/notes:

The general format for WiP references should be:

   [I-D.ietf-sidr-bgpsec-protocol]
              Lepinski, M. and K. Sriram, "BGPsec Protocol                                                        
              Specification", draft-ietf-sidr-bgpsec-protocol-20 (work
              in progress), December 2016.

So we see that the first 4 references in the list above:

- are missing the date 
- have the draft string before the Work in Progress designation
- have draft strings that break across lines

We are curious which, or if all, of these things make parsing fail and the empty reference to be included?

b) With regard to this reference warning:

draft-ietf-trill-directory-assist-mechanisms-12v3.txt(2264): Warning: Failed parsing a reference:
[X.233] - ITU-T Recommendation X.233: Protocol for providing the
     connectionless-mode network service: Protocol specification,
     International Telecommunications Union, August 1997

It seems that the colon instead of a comma is to blame.  Once updated, this parsed fine.

c)  It seems using a 3-digit RFC number is not handled well.

In the xml2rfc text output from the id2xml xml file, we see:

  [RFC826]   - Plummer, D., "An Ethernet Address Resolution Protocol",
             RFC 826, November 1982.

  [RFC903]   - Finlayson, R., Mann, T., Mogul, J., and M. Theimer, "A
             Reverse Address Resolution Protocol", STD 38, RFC 903,
             June 1984.
The XML file contains:
<!ENTITY RFC0826 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.0826.xml">
<!ENTITY RFC0903 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.0903.xml”>

and it also contains:

	<reference anchor="RFC826"><front>
	<title>An Ethernet Address Resolution Protocol</title>
	<author fullname="D. - Plummer" initials="D." surname="- Plummer">
	</author>

	<date month="November" year="1982"/>
	</front>

	<seriesInfo name="RFC" value="826"/>
	</reference>

	<reference anchor="RFC903"><front>
	<title>A Reverse Address Resolution Protocol</title>
	<author fullname="R. - Finlayson" initials="R." surname="- Finlayson">
	</author>

	<author fullname="T. Mann" initials="T." surname="Mann">
	</author>

	<author fullname="J. Mogul" initials="J." surname="Mogul">
	</author>

	<author fullname="M. Theimer" initials="M." surname="Theimer">
	</author>

	<date month="June" year="1984"/>
	</front>

	<seriesInfo name="STD" value="38"/>
	<seriesInfo name="RFC" value="903"/>
	</reference>

2) List output variances.  There are several areas where the lists in this document did not translate well.

Original:
  The nature of dynamic distributed asynchronous systems is such that
  it is impossible for a TRILL switch receiving Push Directory
  information to be absolutely certain that it has complete
  information.  However, it can obtain a reasonable assurance of
  complete information by requiring two conditions to be met:
     1. The PDSS field is 3 in the ESADI zero fragment from the server
        for the relevant Data Label.
     2. In so far as it can tell, it has had continuous data
        connectivity to the server for a configurable amount of time
        that defaults to twice the server's CSNP time (PushDirTimer,
        see Section 2.7).
  Condition 2 is necessary because a client TRILL switch might be just
  coming up and receive an EASDI LSP meeting the requirement in
  condition 1 above but has not yet received all of the ESADI LSP
  fragments from the Push Directory server.

id2xml output:

  The nature of dynamic distributed asynchronous systems is such that
 it is impossible for a TRILL switch receiving Push Directory
 information to be absolutely certain that it has complete information.
 However, it can obtain a reasonable assurance of complete information
 by requiring two conditions to be met:

  1.  The PDSS field is 3 in the ESADI zero fragment from the server
      for the relevant Data Label.

  1.  In so far as it can tell, it has had continuous data connectivity
      to the server for a configurable amount of time that defaults to
      twice the server's CSNP time (PushDirTimer, see Section 2.7).
  Condition 2 is necessary because a client TRILL switch might be just

 coming up and receive an EASDI LSP meeting the requirement in
 condition 1 above but has not yet received all of the ESADI LSP
 fragments from the Push Directory server.

Original:
           Query Address: The query is asking for any other addresses,
              and the nickname of the TRILL switch from which they are
              reachable, that correspond to the same interface as this
              address, within the Data Label of the query of the
              address provided. A typically Query Address would be
              something like the following:
              (1) A 48-bit MAC address with the querying TRILL switch
                  primarily interested in either
                  (1a) the RBridge by which that MAC address is
                       reachable so that the querying RBridge can
                       forward an unknown (before the query)
                       destination MAC address native frame as a
                       unicast TRILL Data packet rather than flooding
                       it, or
                  (1b) the IP address corresponding to the MAC address
                       so that RBridge can locally respond to a RARP
                       [RFC903] native frame.
              (2) An IPv4 or IPv6 address with the querying RBridge
                  interested in the corresponding MAC address so it can
                  locally respond to an ARP [RFC826] or ND [RFC4861]
                  native frame [ARPND].
              But the query address could be some other address type
              for which an AFN has been assigned, such as a 64-bit MAC
              address [RFC7042] or a CLNS address [X.233].

id2xml output:
     Query Address: The query is asking for any other addresses,
        and the nickname of the TRILL switch from which they are
        reachable, that correspond to the same interface as this
        address, within the Data Label of the query of the address
        provided.  A typically Query Address would be something like
        the following: (1) A 48-bit MAC address with the querying TRILL
        switch primarily interested in either (1a) the RBridge by which
        that MAC address is reachable so that the querying RBridge can
        forward an unknown (before the query) destination MAC address
        native frame as a unicast TRILL Data packet rather than
        flooding it, or

             (1b) the IP address corresponding to the MAC address so
             that RBridge can locally respond to a RARP [RFC903] native
             frame.

             (2) An IPv4 or IPv6 address with the querying RBridge
                 interested in the corresponding MAC address so it can
                 locally respond to an ARP [RFC826] or ND [RFC4861]
                 native frame [ARPND].
             But the query address could be some other address type
                 for which an AFN has been assigned, such as a 64-bit
                 MAC address [RFC7042] or a CLNS address [X.233].

Original:
  There are a wide variety of strategies that a TRILL switch can adopt
  for making use of directory assistance. A few suggestions are given
  below.

     -  Even if a TRILL switch will normally be operating with
     information from a complete Push Directory server, there will be a
     period of time when it first comes up before the information it
     holds is complete.  Or, it could be that the only Push Directories
     that can push information to it are incomplete or that they are
     just starting and may not yet have pushed the entire directory.

D. Eastlake, et al                                             [Page 41]

INTERNET-DRAFT                       TRILL: Directory Service Mechanisms

     Thus, it is RECOMMENDED that all TRILL switches have a strategy
     for dealing with the situation where they do not have complete
     directory information. Examples are to send a Pull Directory query
     or to revert to [RFC6325] behavior.

     -  If a TRILL switch receives a native frame X resulting in
     seeking directory information, a choice needs to be made as to
     what to do if it does not already have the directory information
     it needs. In particular, it could (1) immediately flood the TRILL
     Data packet resulting from ingressing X in parallel with seeking
     the directory information, (2) flood that TRILL Data packet after
     a delay, if it fails to obtain the directory information, or (3)
     discard X if it fails to obtain the information. The choice might
     depend on the priority of frame X since the higher that priority
     typically the more urgent the frame is and the greater the
     probability of harm in delaying it. If a Pull Directory request is
     sent, it is RECOMMENDED that its priority be derived from the
     priority of the frame X with the derived priority configurable and
     having the following defaults:

id2xml output:
  There are a wide variety of strategies that a TRILL switch can adopt
  for making use of directory assistance.  A few suggestions are given
  below.

     -  Even if a TRILL switch will normally be operating with
     -

     -

     -

     -

     -

     Thus, it is RECOMMENDED that all TRILL switches have a strategy
     for dealing with the situation where they do not have complete

Eastlake, et al.        Expires September 3, 2017              [Page 41]

Internet-Draft   TRILL: Edge Directory Assist Mechanisms      March 2017

     directory information.  Examples are to send a Pull Directory
     query or to revert to [RFC6325] behavior.

     -  If a TRILL switch receives a native frame X resulting in
     -

     -

     -

     -

     -

     -

     -

     -

     -

     -

     -

     -

     -

3) Figure/table output

Original:
  The 4-bit Flags field of the message header for an Update Message is
  as follows:

        +---+---+---+---+
        | F | P | N | R |
        +---+---+---+---+

id2xml output:

  The 4-bit Flags field of the message header for an Update Message is
  as follows:

                            +---+---+---+---+
                            | F | P | N | R |
                            +---+---+---+---+
                            +---+---+---+---+

Original:

           Name         Default           Section   Note Below
    ------------------  -------           -------   ----------

    DirQueryTimeout     100 milliseconds  3.2.1           1
    DirQueryRetries       3               3.2.1           1
    DirGenQPriority       5               3.2.1           2

    DirRespMaxPriority    6               3.2.2.1         3

    DirUpdateDelay       50 milliseconds  3.3
    DirUpdatePriority     5               3.3.1
    DirUpdateTimeout    100 milliseconds  3.3.3
    DirUpdateRetries      3               3.3.3

    DirAckMaxPriority     5               3.3.2           4

  Note 1: Pull Directory Query client timeout waiting for response and
  maximum number of retries

id2xml output:

           Name         Default           Section   Note Below
    ------------------  -------           -------   ----------

    DirQueryTimeout     100 milliseconds  3.2.1           1
    DirQueryRetries       3               3.2.1           1
    DirGenQPriority       5               3.2.1           2

     DirRespMaxPriority      6 3.2.2.1 3

    DirUpdateDelay       50 milliseconds  3.3
    DirUpdatePriority     5               3.3.1
    DirUpdateTimeout    100 milliseconds  3.3.3
    DirUpdateRetries      3               3.3.3

     DirAckMaxPriority       5 3.3.2 4

  Note 1: Pull Directory Query client timeout waiting for response and
  maximum number of retries

4) Header vs. Authors’ Addresses section

Ideally, the Addresses section would show the full organization name.

Thank you.

RFC Editor/mf

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 16 Jun 2017 19:54:00 +0200

Hi Megan,

> and the empty reference to be included?

None of them.  I believe the patterns in place will catch all of those.
In 1.0.0 there was no pattern to handle ', work in progress' rather than
'(work in progress)', but I added that in 1.0.1.  What is throwing the
matching off is that in all cases above, all the non-ultimate names in
the author list are given as 'A. Nonymous' instead of 'Nonymous, A'.

> It seems that the colon instead of a comma is to blame.  Once updated, this parsed fine.

That's lucky, in that case -- there really should be double quotes around
the title.  Adding missing quotes around any title would be among the first
fixes to improve parsing.

> 	<seriesInfo name="RFC" value="903"/>

Oops.  Right.  That's a bug.  Will fix.

> this document did not translate well.

Yes.  Out of the 195 hours spent on id2xml so far, I would guess that almost
half has been spent on variations of list analysis and handling.  Lists turned
out to be difficult!

>  fragments from the Push Directory server.

Right.  Clearly not optimal.  I've not seen this issue, with the same
list number repeated, but I think that if you insert blank lines in the
original to make it more like the default list output, it should be parsed
better.

>               address [RFC7042] or a CLNS address [X.233].

Ouch.  That is going to be hard, particularly because of the non-standard
list markers (1) etc.  I don't think I'm even going to try to make it
handle this well.  Adding blank lines and using 1. instead of (1) is
probably the best approach, if you want this converted to something sensible.

>                  MAC address [RFC7042] or a CLNS address [X.233].

Right.  Sorry, but I've discovered that this is really hard for a program.
I don't have suggestions beyond those given above, for this.

>      having the following defaults:

Yup, the '-' is seen as a list bullet, but then the text doesn't have the
right following indentation.  When I tested this document, I edited the
indentation of the lines which follow the dash, and things were fine.

>   The 4-bit Flags field of the message header for an Update Message is

Hmm.  This is identified as a texttable, but since there's only the one
row, things turn out wrong.  I may be able to fix this.

>            Name         Default           Section   Note Below

I actually don't know for sure what this should be modelled as.  Do you
have a suggestion of how to handle this?

> Ideally, the Addresses section would show the full organization name.

Yees, agreed.  Could you send me an example of where it doesn't?

Best regards,

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 16 Jun 2017 21:37:14 +0200

Hi again,

Some more information on some particular points, below:

On 2017-06-16 19:54, Henrik Levkowetz wrote:

<<snip>>

>>      ...

This is fixed in my sources, will be in 1.0.2

<<snip>>

> row, things turn out wrong.  I may be able to fix this.

This is also fixed in my sources now, and will be in 1.0.2

>> Ideally, the Addresses section would show the full organization name.

Ok, I see the issue, and will fix in 1.0.2

Best regards,

	Henrik

---

From: Sandy Ginoza <[email-masked]>
Date: Mon, 19 Jun 2017 09:16:14 -0700

Hi Henrik,

Thanks for the quick response!  We’ll test this again and let you know if we run into any issues.

Thanks,
Sandy

> On Jun 16, 2017, at 12:37 PM, Henrik Levkowetz <[email-masked]> wrote:
> 
> Hi again,
> 
> _______________________________________________
> TOOLS-DEVELOPMENT mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-development