Subject: [xml2rfc-dev] New xml2rfc release: v2.37.1

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 12 Dec 2019 05:23:55 -0800

Hi,

This is an automatic notification about a new xml2rfc release, 
v2.37.1, generated when running the mkrelease script.

Release notes:

xml2rfc (2.37.1) ietf; urgency=medium

  * Fixed a bug in the text formatter pagination code where it incorrectly 
    tried to annotate Comment and PI nodes with page number information.

  * Updated the v2v3 converter to do essentially what it did before v2.37
    with respect to unicode downcoding, but with more explicit calls.

  * Added a base writer method to downcode reference punctuation.

  * Moved the list of (tag, attr) combinations that permit unicode values 
    into util.unicode.  Rewrote docwncode_punctuation() to only touch 
    punctuation.

  * Restored lost trailing text after <contact> in <t> context for text 
    output.

 -- Henrik Levkowetz <[email-masked]>  12 Dec 2019 12:42:15 +0000

The preferred way to install xml2rfc is by doing 'pip install xml2rfc',
and 'pip install --upgrade xml2rfc' to upgrade.  If there are system-
installed python modules which pip will not upgrade, you may have to
use 'pip install --upgrade --no-deps xml2rfc' and install dependencies
manually.

The new version is also available through SVN checkout, with
  'svn checkout http://svn.tools.ietf.org/svn/tools/xml2rfc/tags/cli/2.37.1'

Regards,

	Henrik
	(via the mkrelease script)

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 13 Dec 2019 11:25:20 +1100

Hi Henrik,

One small fix is required for downcode_punctuation():

diff --git a/xml2rfc/util/unicode.py b/xml2rfc/util/unicode.py
index 51ffbb5..6f59e6e 100644
--- a/xml2rfc/util/unicode.py
+++ b/xml2rfc/util/unicode.py
@@ -218,21 +218,21 @@ unicode_quote_replacements = {
     u'\u201e': '"',
     u'\u2032': "'",
 }

 punctuation = {
     u'\u2026': '...',
 }
 punctuation.update(unicode_space_replacements)
 punctuation.update(unicode_dash_replacements)
 punctuation.update(unicode_quote_replacements)
-punctuation_re = re.compile(r'[%s]'%''.join(list(punctuation.keys())))
+punctuation_re = re.compile(r'([%s])'%''.join(list(punctuation.keys())))

 unicode_replacements = {
     # Unicode code points corresponding to (x)html entities, also in
     # rfc2629-xhtml.ent
     u'\x09': ' ',
     u'\xa0': ' ',
     u'\xa1': '!',
     u'\xa2': '[cents]',
     u'\xa3': 'GBP',
     u'\xa4': '[currency units]',

Aside from that, this meets my immediate needs.

On Fri, Dec 13, 2019, at 00:23, Henrik Levkowetz wrote:
> _______________________________________________
> xml2rfc-dev mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc-dev
>

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 13 Dec 2019 02:01:55 +0100

Hi Martin,

Please provide the reason for this change.

   Henrik

> On 13 Dec 2019, at 01:25, Martin Thomson <[email-masked]> wrote:
> 
> Hi Henrik,
>

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 13 Dec 2019 12:11:58 +1100

Sorry, I foolishly left that out.

The code crashes if you have unicode punctuation because of this:

def downcode_punctuation(str):
    while True:
        match = re.search(punctuation_re, str)
        if not match:
            return str
        str = re.sub(match.group(1), punctuation[match.group(1)], str)

The last line expects a grouping, but the regular expression lacks that grouping.  That's all.

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 13 Dec 2019 13:08:56 +0100

Hi Martin,

> The code crashes if you have unicode punctuation because of this:

Huh.  I guess this is a case of some Python 3 versions being stricter
than 2.7.  I did put in a test case for this, but in my rush to get the
fix out, and having to rush for an airport, I didn't run the full tox
suite.  Which Python version did you use?

	Henrik

---

From: "Martin Thomson" <[email-masked]>
Date: Sat, 14 Dec 2019 12:20:11 +1100

> suite.  Which Python version did you use?

Two different versions, maybe four. All python 3. 3.6.8 for sure, but that was before I upgraded my systems to fix the git vulnerabilities.