Subject: [xml2rfc-dev] End of support for xml2rfc on Python 2.x is coming soon

From: Russ Housley <[email-masked]>
Date: Tue, 8 Oct 2019 17:23:47 -0400

Heads up!

With the transition to xml2rfc vocabulary version 3, xml2rfc has gained
the ability to generate PDF output when the necessary system libraries
are installed.  However, xml2rfc runs on Python 2.7, but the library
needed for PDF generation ended support for Python 2.7 about 10
releases ago.  This means that the need to end support for xml2rfc on
Python 2.7 is becoming urgent.  Another factor is that bugfix support
for Python 2.7.x itself officially stops on 1 January 2020, so we need
to transition away from Python 2.7 soon in any case.

The latest xml2rfc release is 2.32.0.  There will most likely be one or
two additional xml2rfc releases in the 2.x series, but after that, the
plan is to transition to a 3.x release series, with two major changes:

(1) xml2rfc will no longer run under Python 2.7; it will require
    Python 3.5 or higher.  If you cannot install and run Python 3 on
    your system, the web service at xml2rfc.ietf.org can be used.

(2) The default output formatters will change to v3.  The v2 formatters
    will still be available by using a --legacy switch.

Expect the first xml2rfc 3.x series release before the end of the month.

On behalf of the Tools Team,
  Russ

---

From: Fred Baker <[email-masked]>
Date: Tue, 8 Oct 2019 14:47:08 -0700

That all fine, and as predictable as you say. What would very helpful would be a road map: if you’re using {windows X|Mac X|Linux X|whatever}, we think you should look at tools {D,E,F}.

Speaking personally, I am on a Mac and using XMLmind with Fenner’s tools. They mostly worked (note the past tense) except when they didn’t. Telling me “well, ABCDEF supports <IETF tools du jour if you can read Sanskrit>“ doesn’t quite work.

I used to write in NROFF. I’ll do what it takes. But really?

Sent using a machine that autocorrects in interesting ways...

> On Oct 8, 2019, at 2:31 PM, Russ Housley <[email-masked]> wrote:
> 
> Heads up!
> 
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 8 Oct 2019 23:48:15 +0200

>    will still be available by using a --legacy switch.

Please do this in a way that will not randomly break scripts and other programs that need to run xml2rfc.  (A calling script/program has no idea what version of xml2rfc is installed locally.)  [Actually, that is also true of people calling xml2rfc…]

Grüße, Carsten

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 9 Oct 2019 00:18:53 +0200

Hi Carsten,

On 2019-10-08 23:48, Carsten Bormann wrote:
> Please do this in a way that will not randomly break scripts and
> other programs that need to run xml2rfc. (A calling script/program
> has no idea what version of xml2rfc is installed locally.) [Actually,
> that is also true of people calling xml2rfc…]

Does it work for you if we say 'if you want v2 output, please add --legacy
to your scripts already now'?

The --legacy switch to force v2 output (for compatible input) has been
available for around 6 months, so even if you don't have the bleeding
edge version installed, this should work as a compatibility path, I think?

Best regards,

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 9 Oct 2019 00:23:09 +0200

Hi Fred,

On 2019-10-08 23:47, Fred Baker wrote:
> That all fine, and as predictable as you say. What would very helpful
> would be a road map: if you’re using {windows X|Mac X|Linux
> X|whatever}, we think you should look at tools {D,E,F}.
> 
> Speaking personally, I am on a Mac and using XMLmind with Fenner’s
> tools. They mostly worked (note the past tense) except when they
> didn’t. Telling me “well, ABCDEF supports <IETF tools du jour if you
> can read Sanskrit>“ doesn’t quite work.
> 

I'm sorry if the text wasn't clear enough.  The roadmap is this:  Please
install Python 3.5 or higher on your system, and install coming versions
of xml2rfc using the 'pip3' command which is part of that Python install.

When we got to the xml2rfc 3.0.0 release, I had planned to update the
release note with the information about using pip3, but I'm perfectly
happy saying it now, too.

Of course, if your default python is Python 3.5 or higher already, then
using plain 'pip' to install will continue to work.

Best regards,

	Henrik

> xml2rfc-dev mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc-dev
>

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 9 Oct 2019 00:46:32 +0200

On Oct 9, 2019, at 00:18, Henrik Levkowetz <[email-masked]> wrote:
> Signed PGP part
> to your scripts already now’?

It sure works for me, but I don’t know all the other users of my software.

> edge version installed, this should work as a compatibility path, I think?

6 months is very short in the grand scheme of things.

Generally people will upgrade tools like kramdown-rfc and xml2rfc on different timelines.
(And there are also a few hundred makefiles in some repositories somewhere that call xml2rfc.)

With that out of the way, I must admit I don’t even understand what this means:

  Format Options:
    --v3                                with --text and --html: use the v3
                                        formatter, rather than the legacy one.
    --legacy                            with --text and --html: use the legacy
                                        text formatter, rather than the v3
                                        one.

Does the choice of “legacy” and “v3” formatter have an influence on the accepted XML vocabulary?  On the output format?  Both?  How?

Grüße, Carsten

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 9 Oct 2019 01:15:46 +0200

Hi Carsten,

> It sure works for me, but I don’t know all the other users of my software.

Ack.

> Generally people will upgrade tools like kramdown-rfc and xml2rfc on
> different timelines. (And there are also a few hundred makefiles in
> some repositories somewhere that call xml2rfc.)
> 
> Does the choice of “legacy” and “v3” formatter have an influence on
> the accepted XML vocabulary?

Only indirectly.  If you use --v3, the new formatters will be used.
xml2rfc will accept both v2 and v3, and run the v2v3 converter internally
in order to not present any deprecated elements to the v3 formatters.

If you use --legacy, the old formatters will be used.  Since there is
no internal v3v2 converter, only v2 input will work.  

> On the output format?

Yes, that's the primary effect intended with these switches.

> Both? How?

Maybe a processing diagram will help:

http://tools.ietf.org/src/xml2rfc/trunk/cli/doc/xml2rfc3.html#processing-flow

Regards,

	Henrik

---

From: Job Snijders <[email-masked]>
Date: Wed, 9 Oct 2019 00:00:06 +0000

> using plain 'pip' to install will continue to work.

We should note that the potential for pip/pip3 confusion is a result
of how the python community approached this transition (acknowleding
what their options were in context of how the packaging eco system was
set up). Not ideal, but it is what it is.

I think it would be good to update public facing documentation about
xml2rfc that pip3 must be used, to make it very clear that any version
of xml2rfc is not expected to work correctly on python2 systems.

Perhaps the final update to xml2rfc 2.x series should be to add a
check at boot whether the python interpreter's major version is lower
than 3, and if so, exit the xml2rfc program with an informative
message and a non-zero exit code, inform the user that python3 must be
used? Sometimes it is better to just break fast & early.

Between the name of the tool (note the 2 in "xlm2rfc"), the industry's
transition from python 2 to python 3, and IETF's transition from the
v2 to the v3 RFC XML format, it is no surprise to me end users easily
become confused. A simple strong message that python2 can't be used
might be helpful, even if it appears somewhat unforgiving.

Kind regards,

Job

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 9 Oct 2019 08:09:08 +0200

Hi Job,

> of xml2rfc is not expected to work correctly on python2 systems.

Right.

> used? Sometimes it is better to just break fast & early.

I think that message will come through clearly on first attempt at
installing the first xml2rfc version requiring Python 3, as I expect
that to tell the user that 'this version of xml2rfc requires Python 3',
but I'll do some testing when I come to that point.  Better to fail
during that installation, but still leave the user and system with a
working (though out-of date) version of xml2rfc, than to install a
version that cannot be used to do work.

> might be helpful, even if it appears somewhat unforgiving.

Yes.  I'd still like to leave the user with a system which hasn't been
robbed of the xml2rfc functionality as a result of installing 'the last
2.x release of xml2rfc'.

Best regards,

	Henrik

---

From: Martin J. Dürst <[email-masked]>
Date: Wed, 9 Oct 2019 06:52:43 +0000

> might be helpful, even if it appears somewhat unforgiving.

What about changing xml2rfc's name to xml3rfc? That way, xml3rfc, python 
3, v3 of RFC XML, and pip3 would all align :-).

Regards,   Martin.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 9 Oct 2019 09:50:57 +0200

> http://tools.ietf.org/src/xml2rfc/trunk/cli/doc/xml2rfc3.html#processing-flow

Thank you!
That is really useful.

Grüße, Carsten

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 9 Oct 2019 16:22:40 +0200

Hi Martin,

> 3, v3 of RFC XML, and pip3 would all align :-).

True, but it would also break things for every person and script going to
the python package index to fetch the tool (https://pypi.org/project/xml2rfc/)
... ,:-)

Best regards,

	Henrik

---

From: Randy Bush <[email-masked]>
Date: Wed, 09 Oct 2019 13:42:13 -0700

[ bunch of cc:s removed ]

will xml2rfc.app, the mac gooey, be maintained in any way.  v3?  64-bit?

randy

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 9 Oct 2019 23:49:52 +0200

Hi Randy,

> will xml2rfc.app, the mac gooey, be maintained in any way.  v3?  64-bit?

I don't expect so.  It's not been updated in ages, and even if it runs
xml2rfc under the hood, it knows nothing of the new switches.

Best regards,

	Henrik

---

From: Robert Moskowitz <[email-masked]>
Date: Thu, 10 Oct 2019 08:49:35 -0400

> Does the choice of “legacy” and “v3” formatter have an influence on the accepted XML vocabulary?  On the output format?  Both?  How?

I have encounter:

draft-moskowitz-hip-new-crypto-02.xml(14): Warning: Setting 
consensus="true" for IETF STD document (this is not the schema default, 
but is the only value permitted for this type of document)
  Created file draft-moskowitz-hip-new-crypto-02.txt

when using the--v3 switch.

I have been told not to worry about this, but so far I have not found 
any documentation on "consensus".

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 10 Oct 2019 14:59:50 +0200

Hi Bob,

On 2019-10-10 14:49, Robert Moskowitz wrote:
> any documentation on "consensus".\

See:

	https://tools.ietf.org/html/rfc7749#appendix-A.4
	https://tools.ietf.org/html/rfc7991#section-2.45.2

Best regards,

	Henrik

---

From: Robert Moskowitz <[email-masked]>
Date: Thu, 10 Oct 2019 09:12:20 -0400

> 3, v3 of RFC XML, and pip3 would all align :-).

because it is suppose to be translated as "xml formatting TO RFC 
formatting"?

And there is some confusion?  It was xml2rfc back in v1.

Finally who remembers that originally it was called Blutooth after King 
Harald Blutooth?  (or somesuch spelling), and despite considerable 
documentation as to the origin of the name, the press won and it became 
Bluetooth?

:)

---

From: Julian Reschke <[email-masked]>
Date: Thu, 10 Oct 2019 15:21:42 +0200

On 10.10.2019 14:59, Henrik Levkowetz wrote:
> ...

...and...: <https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/420>

Best regards, Julian