Subject: [TOOLS-DEVELOPMENT] Notes for 12Feb tools call

From: Robert Sparks <[email-masked]>
Date: Fri, 8 Feb 2019 10:51:16 -0600

Notes for Tuesday's (12 Feb) tools call:

# Datatracker #

The improvements to the group feature models have been released - see 
<https://datatracker.ietf.org/release>.

The next major focus will be on changing the way the datatracker sends 
email to stop appearing to spoof the From value. We have also increased 
the priority of shifting the datatracker to python 3. (Note that python 
2 will be officially deprecated and will no longer receive maintenance 
starting next January).

See <https://trac.tools.ietf.org/tools/ietfdb/browser/trunk/PLAN>
and 
<https://trac.tools.ietf.org/tools/ietfdb/wiki/MergePlan?action=diff&version=98&old_version=97>

# Mail Archives #

Turning off mhonarc has gone very smoothly.

There are a large number of private lists that have used or still use 
the built-in mailman archiver, pipermail. We are beginning an analysis 
of what it will take to stop generating those, and what redirects into 
the mailarchive system should look like. Currently, the thinking is to 
redirect each pipermail archive to the top level of the analogous 
mailarchive archive.

# IETF Website #

Torchbox has completed their last major deliverable (upgrading the 
website to a more modern version of wagtail). There are a small number 
of tickets related to documentation and requests for information we 
should have before shifting the code into our own public code 
repository. Note that the wagtail project continues to issue new 
releases. The project with torchbox brought us to wagtail 2.2.2. The 
current release now is 2.4.

However, we are blocked on deploying this latest deliverable. The way we 
are serving python applications via Apache right now (mod_wsgi) only 
allows the use of one version of python. Many of our applications are on 
Python 2.7 right now. This includes the datatracker and the mailarchive. 
Wagtail, on the other hand, dropped support for python 2.7 at Wagtail 
2.0. Waiting until we have all exisiting deployed applications ported to 
python 3 to deploy the deliverable from Torchbox is exceedingly 
suboptimal. It will delay the current plan for the tools-team to take 
over maintenance of the website code and start adding features the 
secretariat has asked for. We've started conversations about what other 
options we have besides waiting for everything to be ported to python 3.

# Yangcatalog #

We've shifted the plan for short-term deployment and development to 
occur using the existing externally hosted server.

# Format tools and Accepting v3 xml into the ID Repository #

All of the format tools have been released and have been in testing by 
the RPC.

We are evaluating a commercial PDF/A-3 conversion tool from a company 
that is willing to donate a license. We have found no open-source 
alternatives that are acceptable.

The IESG is ready to accept v3 documents into the ID Repository as soon 
as the tools team believes the infrastructure is ready to handle it. We 
have a small number of things to address before opening the repository:

* The preptool needs to be modified to inline SVG as artwork (instead of 
as a data URI) when there is already an ascii-art artwork alternative 
available. This is driven by a goal of accepting only a single prepped 
file into the repository, rather than solving the problem of accepting 
an xml file and a set of related SVG files.

* There are still features of the revised IDNits to complete, and it 
needs to be integrated with the submission tool on the datatracker.

* A v3 howto should exist when we open the ID repository. The RPC is 
working towards training materials that might have content that could be 
leveraged.

At the moment, other priorities make it unlikely that this will all be 
in place before Prague's ID-deadline (11 Mar), but I think it's likely 
this will be ready shortly after the Prague meeting.

# RFPs #

## Open / Out for Bid ##

The yangcatalog RFP is currently out for bid. Responses are due today 
(8Feb).

## In Development ##

The meeting application improvement SoW is essentially ready to issue.

I have text from Ekr to work into the RPC security code review SoW. The 
RPC has asked that we hold off issuing that RFP until some changes 
related to the new format are better integrated into their repository so 
that the review focuses more on the code that will actually be used.

The Inline-Errata SoW is still under review and has several unresolved 
comments. The lastest version is at 
<https://docs.google.com/document/d/1T-6eBrNPC7aIxOTChdIZE7fecbxUJ7kq-F6RhyCNUGI/edit>. 
One of the major observations in the comments is that this project is 
focusing on the existing (old-format) publication format, and is not 
targetting or taking advantage of the new format in any way. That's 
intentional. We need to have some run-time experience with the new 
format before we can sensibly ask a contractor to work on inlining 
errata into it. We plan to learn from this project, and apply what we 
learn into the new format with a future project.

The next SoW to be developed is for IANA expert review tracking.

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 8 Feb 2019 09:20:00 -0800

> options we have besides waiting for everything to be ported to python 3.

Why does the version of datatracker impact the version of Wagtail we run?
They seem to be on different hosts, so can't they run different versions of
Python?

> occur using the existing externally hosted server.

Can you provide some more background for this? Why can't this be moved to
an IETF server (or IETF-managed cloud)

-Ekr

> <
> https://docs.google.com/document/d/1T-6eBrNPC7aIxOTChdIZE7fecbxUJ7kq-F6RhyCNUGI/edit>.
>
> _______________________________________________
> TOOLS-DEVELOPMENT mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-development
>

---

From: Robert Sparks <[email-masked]>
Date: Fri, 8 Feb 2019 11:29:27 -0600

On 2/8/19 11:20 AM, Eric Rescorla wrote:
>
>
> On Fri, Feb 8, 2019 at 8:51 AM Robert Sparks <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     # IETF Website #
>
> Why does the version of datatracker impact the version of Wagtail we 
> run? They seem to be on different hosts, so can't they run different 
> versions of Python?
They are currently served from the same machine, even if they have 
different hostnames.
> Can you provide some more background for this? Why can't this be moved 
> to an IETF server (or IETF-managed cloud)
There were some difficulties trying to move this to an IETF server given 
the current level of access to internals (shell access to the machine) 
required by the application. Eric and the developers were experiencing 
connectivity problems that couldn't be reproduced from other locations, 
and it appears some trouble with outbound connections that Glen's team 
couldn't isolate. The goal is _definitely_ to be where we could deploy 
anywhere. The first task in the outstanding SoW includes the work needed 
to get the project to the point that shell access isn't required by the 
developers or application expert. It will also improve release packaging 
to make deploying wherever more feasible.
>     [email-masked] <mailto:[email-masked]>

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 8 Feb 2019 09:40:50 -0800

> They are currently served from the same machine, even if they have

This seems like a good opportunity to move one to a different machine, or
better yet, to some system (containers?) where hardware assignment doesn't
matter.

> connectivity problems that couldn't be reproduced from other locations, and
> it appears some trouble with outbound connections that Glen's team couldn't
> isolate. The goal is _definitely_ to be where we could deploy anywhere. The
> first task in the outstanding SoW includes the work needed to get the
> project to the point that shell access isn't required by the developers or
> application expert. It will also improve release packaging to make
> deploying wherever more feasible.''
>

Hmm.... So my instinct here would be to hold off on doing anything else
until that's happened. Alissa, WDYT?

-Ekr

---

From: Robert Sparks <[email-masked]>
Date: Fri, 8 Feb 2019 11:54:13 -0600

On 2/8/19 11:40 AM, Eric Rescorla wrote:
>
>
> On Fri, Feb 8, 2019 at 9:29 AM Robert Sparks <[email-masked] 
>>     On Fri, Feb 8, 2019 at 8:51 AM Robert Sparks
>>     <[email-masked] <mailto:[email-masked]>> wrote:
>>
>>         # IETF Website #
>>
>>     Why does the version of datatracker impact the version of Wagtail
>>     we run? They seem to be on different hosts, so can't they run
>>     different versions of Python?
> This seems like a good opportunity to move one to a different machine, 
> or better yet, to some system (containers?) where hardware assignment 
> doesn't matter.
Agreed - that one of the options we're talking about.
>>     Can you provide some more background for this? Why can't this be
>>     moved to an IETF server (or IETF-managed cloud)
>     There were some difficulties trying to move this to an IETF server
>     given the current level of access to internals (shell access to
>     the machine) required by the application. Eric and the developers
>     were experiencing connectivity problems that couldn't be
>     reproduced from other locations, and it appears some trouble with
>     outbound connections that Glen's team couldn't isolate. The goal
>     is _definitely_ to be where we could deploy anywhere. The first
>     task in the outstanding SoW includes the work needed to get the
>     project to the point that shell access isn't required by the
>     developers or application expert. It will also improve release
>     packaging to make deploying wherever more feasible.''
>
>
> Hmm.... So my instinct here would be to hold off on doing anything 
> else until that's happened. Alissa, WDYT?
>
> -Ekr
>
>>
>>     -Ekr
>>

---

From: Alissa Cooper <[email-masked]>
Date: Tue, 12 Feb 2019 09:09:34 -0500

Hi Robert,

> On Feb 8, 2019, at 11:51 AM, Robert Sparks <[email-masked]> wrote:
> 
> # RFPs #
> 
> ## Open / Out for Bid ##
> 
> The yangcatalog RFP is currently out for bid. Responses are due today (8Feb).
> 
> ## In Development ##
> 

Could you circulate the latest version? I had a question about the report generated for the IESG and I don’t think a final answer made it to the list.

Thanks,
Alissa

> 
> I have text from Ekr to work into the RPC security code review SoW. The RPC has asked that we hold off issuing that RFP until some changes related to the new format are better integrated into their repository so that the review focuses more on the code that will actually be used.
> 
> The Inline-Errata SoW is still under review and has several unresolved comments. The lastest version is at <https://docs.google.com/document/d/1T-6eBrNPC7aIxOTChdIZE7fecbxUJ7kq-F6RhyCNUGI/edit>. One of the major observations in the comments is that this project is focusing on the existing (old-format) publication format, and is not targetting or taking advantage of the new format in any way. That's intentional. We need to have some run-time experience with the new format before we can sensibly ask a contractor to work on inlining errata into it. We plan to learn from this project, and apply what we learn into the new format with a future project.
>

---

From: Alissa Cooper <[email-masked]>
Date: Tue, 12 Feb 2019 09:19:27 -0500

> On Feb 8, 2019, at 12:40 PM, Eric Rescorla <[email-masked]> wrote:
> 
> 
> 
> On Fri, Feb 8, 2019 at 9:29 AM Robert Sparks <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
>> On Fri, Feb 8, 2019 at 8:51 AM Robert Sparks <[email-masked] <mailto:[email-masked]>> wrote:
>> # IETF Website #
>> 
>> Why does the version of datatracker impact the version of Wagtail we run? They seem to be on different hosts, so can't they run different versions of Python?
> They are currently served from the same machine, even if they have different hostnames.
> 
> This seems like a good opportunity to move one to a different machine, or better yet, to some system (containers?) where hardware assignment doesn't matter.
> 
>  
>> # Yangcatalog #
>> 
>> Can you provide some more background for this? Why can't this be moved to an IETF server (or IETF-managed cloud)
> There were some difficulties trying to move this to an IETF server given the current level of access to internals (shell access to the machine) required by the application. Eric and the developers were experiencing connectivity problems that couldn't be reproduced from other locations, and it appears some trouble with outbound connections that Glen's team couldn't isolate. The goal is _definitely_ to be where we could deploy anywhere. The first task in the outstanding SoW includes the work needed to get the project to the point that shell access isn't required by the developers or application expert. It will also improve release packaging to make deploying wherever more feasible.''
> 
> Hmm.... So my instinct here would be to hold off on doing anything else until that's happened. Alissa, WDYT?

I think that sounds like what the SOW calls for.
Alissa

>> <https://docs.google.com/document/d/1T-6eBrNPC7aIxOTChdIZE7fecbxUJ7kq-F6RhyCNUGI/edit <https://docs.google.com/document/d/1T-6eBrNPC7aIxOTChdIZE7fecbxUJ7kq-F6RhyCNUGI/edit>>. 
>> https://www.ietf.org/mailman/listinfo/tools-development <https://www.ietf.org/mailman/listinfo/tools-development>

---

From: Robert Sparks <[email-masked]>
Date: Tue, 12 Feb 2019 08:44:04 -0600

On 2/12/19 8:19 AM, Alissa Cooper wrote:
>
>
>> On Feb 8, 2019, at 12:40 PM, Eric Rescorla <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>>
>>
>>>     Why does the version of datatracker impact the version of
>>>     Wagtail we run? They seem to be on different hosts, so can't
>>>     they run different versions of Python?
>> This seems like a good opportunity to move one to a different 
>> machine, or better yet, to some system (containers?) where hardware 
>> assignment doesn't matter.
>>
>>>         # Yangcatalog #
>>>
>>     There were some difficulties trying to move this to an IETF
>>     server given the current level of access to internals (shell
>>     access to the machine) required by the application. Eric and the
>>     developers were experiencing connectivity problems that couldn't
>>     be reproduced from other locations, and it appears some trouble
>>     with outbound connections that Glen's team couldn't isolate. The
>>     goal is _definitely_ to be where we could deploy anywhere. The
>>     first task in the outstanding SoW includes the work needed to get
>>     the project to the point that shell access isn't required by the
That's what I'm expecting of it too.

---

From: Robert Sparks <[email-masked]>
Date: Tue, 12 Feb 2019 08:54:28 -0600

Attached.

The report you mention is still in the text. The last I remember on the 
discussion of it was an outstanding question to Alexa.

RjS

---

From: Alissa Cooper <[email-masked]>
Date: Tue, 12 Feb 2019 10:03:21 -0500

Thanks.

Russ, could we add discussion of this to today’s agenda?

Alissa

> On Feb 12, 2019, at 9:54 AM, Robert Sparks <[email-masked]> wrote:
> 
> Attached.
> 
> The report you mention is still in the text. The last I remember on the discussion of it was an outstanding question to Alexa.
> 
> RjS
> 
> <sow_2019_meeting_v3.pdf><sow_2019_meeting_v3.docx>

---

From: Alexa Morris <[email-masked]>
Date: Tue, 12 Feb 2019 07:56:39 -0800

Apologies, I don’t remember this question. Can anyone remind me so that I can make sure I don’t hold up progress on this today?

Thanks,
Alexa

Sent from my iPhone

> On Feb 12, 2019, at 7:03 AM, Alissa Cooper <[email-masked]> wrote:
> 
> Thanks.
>

---

From: Robert Sparks <[email-masked]>
Date: Tue, 12 Feb 2019 10:02:41 -0600

I just forwarded the message to you directly

Sent from my iPhone

> On Feb 12, 2019, at 9:56 AM, Alexa Morris <[email-masked]> wrote:
>

---

From: Alexa Morris <[email-masked]>
Date: Tue, 12 Feb 2019 09:00:06 -0800

Ah, got it. I don't believe that I made a request for this report myself but I can say that we've been asked to provide this data in the past, and we have to manually tabulate it. So, insofar as the IESG (or others) may be curious about the number of requested sessions, sessions that met, number of BoFs, etc. in the future, the Secretariat would find it useful to have this information generated automatically rather than manually.

Alexa

> On Feb 12, 2019, at 8:02 AM, Robert Sparks <[email-masked]> wrote:
> 
> On Feb 12, 2019, at 9:56 AM, Alexa Morris <[email-masked] <mailto:[email-masked]>> wrote:
> 
>> On Feb 12, 2019, at 7:03 AM, Alissa Cooper <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>>> Thanks.
>>> 
>>>> On Feb 12, 2019, at 9:54 AM, Robert Sparks <[email-masked] <mailto:[email-masked]>> wrote:
>>>> 
>>>> Attached.
>>>> 
>>>>>> On Feb 8, 2019, at 11:51 AM, Robert Sparks <[email-masked] <mailto:[email-masked]>> wrote:
>>>>>> 
>>>>>> # RFPs #
>>>>>> 
>>>>>> ## Open / Out for Bid ##
>>>>>> 
>>>>>> The Inline-Errata SoW is still under review and has several unresolved comments. The lastest version is at <https://docs.google.com/document/d/1T-6eBrNPC7aIxOTChdIZE7fecbxUJ7kq-F6RhyCNUGI/edit <https://docs.google.com/document/d/1T-6eBrNPC7aIxOTChdIZE7fecbxUJ7kq-F6RhyCNUGI/edit>>. One of the major observations in the comments is that this project is focusing on the existing (old-format) publication format, and is not targetting or taking advantage of the new format in any way. That's intentional. We need to have some run-time experience with the new format before we can sensibly ask a contractor to work on inlining errata into it. We plan to learn from this project, and apply what we learn into the new format with a future project.
>>>>>>

---

From: Alexa Morris <[email-masked]>
Date: Tue, 12 Feb 2019 09:32:09 -0800

Robert has gently reminded me that I did, in fact, request this data. It's definitely the case that we've been asked to report on this in the past; however, if the current IESG doesn't find this useful, I think we should remove it for now. We can address it at a later time, if needed.

Perhaps I was conflating this with a slightly different request that we've received in recent years: to document the number of working groups and boFs formed leading into a meeting. But, either way, if the IESG doesn't find this sort of data useful, there's no reasons to keep this reporting section in the data.

Alexa

PS: It was just pointed out to me that we can now download raw data from the meeting statistics report (https://datatracker.ietf.org/stats/meeting/overview/). This was something I'd ask for previously, so thank you for that! 

> On Feb 12, 2019, at 9:00 AM, Alexa Morris <[email-masked]> wrote:
> 
>> On Feb 12, 2019, at 8:02 AM, Robert Sparks <[email-masked] <mailto:[email-masked]>> wrote:
>>