Subject: [TOOLS-DEVELOPMENT] Tools Call Agenda -- 10 Jan 2017 at 1:00 Eastern

From: Russ Housley <[email-masked]>
Date: Mon, 9 Jan 2017 15:54:13 -0500

Tools Call Agenda -- 10 Jan 2017 at 1:00 Eastern

1. Datatracker Projects
   - Expected Datatracker Releases -- Robert and Henrik
     -- http://trac.tools.ietf.org/tools/ietfdb/wiki/MergePlan
   - Contracts for Datatracker Enhancements -- Robert
     -- Author Statistics

2. Community & Other Projects
   - Improvements to the mail archive application -- Robert and Ryan
   - IETF Website Makeover -- Greg and Joe

3. RFC Services Projects
   - Contracts for Improvements to RFC Services -- Heather and Robert
     -- CSS for the RFCs
     -- IDnits
     -- Publication Formatter
     -- RFClint
     -- SVGcheck
     -- Text Submission
     -- XMLdiff

4. Server Infrastructure
   - IESG discussions of DMARC -- Robert and Ben
   - Migrate toward Django 1.9 and 1.10 -- Henrik
   - Libraries for HTML of I-Ds -- Henrik

5. Parking Lot
   - CDN support for HTML and PDF I-Ds -- Henrik
   - Migrate from MySQL to PostgreSQL -- Henrik
   - Discontinue MonArch email archives -- Robert
   - Improve infrastructure for finding and fetching artifacts -- Robert
   - Author information for very old I-Ds in the datatracker -- Robert
   - Replace I-Ds in proceedings with links to archive copy?
   - Performance improvements and transition to Postgres -- Henrik
   - VM Architecture for Servers -- Robert
   - Submit an I-D directly from GitHub
   - Move downref registry , and integrate with IETF LC generation
   - Add view of recent ballots by telechat to
   - Allow meetecho to associate recordings with sessions as they become available
   - Mailman support for internationalized email addresses
     -- Requires Mailman 3.1, which is expected in early 2017
   - Meeting registration support for internationalized email addresses
     -- Mailman must be done first; user is added to meeting mail lists

6. AOB

---

From: Robert Sparks <[email-masked]>
Date: Mon, 9 Jan 2017 15:18:50 -0600

Some notes to assist with call time tomorrow.

>       -- http://trac.tools.ietf.org/tools/ietfdb/wiki/MergePlan
As expected, developers were away for much of the holiday season so the 
delta between tools meetings isn't as high as it usually is.
But progress has been made, and the MergePlan has been updated.
>     - Contracts for Datatracker Enhancements -- Robert
This is well underway, with a significant part of the project checked in 
last week.
>     - Improvements to the mail archive application -- Robert and Ryan
Ryan can provide more detail, but we're targetting finishing this 
project within the next two weeks. There are still a couple of 
iterations that are not released.
>     - Contracts for Improvements to RFC Services -- Heather and Robert
The initial CSS created by the project has been community reviewed. 
Issues and suggestions are captured at the github instance for this 
document at https://github.com/rfc-format/draft-iab-rfc-css-bis/issues. 
Heather is iterating with the developer to decide which of the issues 
will result in a change to the css delivered by this project.
>       -- XMLdiff
These were awarded in late December. The developers are in the getting 
set up.
>     - IESG discussions of DMARC -- Robert and Ben
The IESG had a long discussion in a late December informal telechat. 
They are still weighing options. The proposed actions all involve 
waiting for a newer version of mailman to deploy (which will happen with 
the impending server upgrades). The proposal getting the most traction 
will also involve some development and configuration time if it is pursued.
> _______________________________________________
> TOOLS-DEVELOPMENT mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-development

---

From: Heather Flanagan <[email-masked]>
Date: Mon, 9 Jan 2017 20:31:04 -0800

...
> The initial CSS created by the project has been community reviewed. Issues and suggestions are captured at the github instance for this document at https://github.com/rfc-format/draft-iab-rfc-css-bis/issues. Heather is iterating with the developer to decide which of the issues will result in a change to the css delivered by this project.
>>      -- IDnits
>>      -- Publication Formatter
>>      -- RFClint
>>      -- SVGcheck
>>      -- Text Submission
>>      -- XMLdiff
> These were awarded in late December. The developers are in the getting set up.
>> 
> 

I don't have anything to add to Robert's update, and am tied up with something else during the call time. So, regrets from me but am happy to follow up and answer questions if they come up.

-Heather

---

From: Jari Arkko <[email-masked]>
Date: Tue, 10 Jan 2017 07:31:24 +0200

I will try to be on the call, but I’m also on my way to airport/in taxis/security around the time.

Jari

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 10 Jan 2017 11:37:59 +0100

A few additional notes:

>>     - Migrate toward Django 1.9 and 1.10 -- Henrik
 
Now working on 1.9.  I've gone through the list of incompatible changes
in the release notes and adjusted as needed, and dealt with immediate
deprecation warnings (or silenced them, for warnings in libs rather than
own code).  One very large batch of combined deprecation fixes and change
of idiom from an old convoluted form to a new straightforward form also
done.
 
One add-in field type we have been using (TimedeltaField) does not work any
more in 1.9, to the point that migrations to the new built-in DurationField
could not even be written.  This has been solved by doing a new trunk
release under 1.8 to migrate to DurationField, and merging the 1.9 branch
changes forward to a copy of that.
 
Now iterating through the test suite, which tickles occasional new
deprecation warnings or issues discovered by the stricter checks in 1.9.
 
When the test suite is clean, a final revisit of the release notes, item
by item, will be done.
 
>>     - Libraries for HTML of I-Ds -- Henrik
 
Now rescheduled as item 5 of the PLAN file, after the Django 1.9/1.10
upgrades and a bunch of small cleanup and schema chanage items.

---

From: Greg Wood <[email-masked]>
Date: Tue, 10 Jan 2017 13:23:01 -0500

Hi,

> On Jan 9, 2017, at 3:54 PM, Russ Housley <[email-masked]> wrote:
> 

As promised on today’s call, here is the link to the update timeline that aims to provide more detail, particularly around the community roll-out:

https://docs.google.com/spreadsheets/d/1dvVnCzVDRuyFoslwLBImLwMwVpjw8aSrd7zB6i_cM3k/edit?usp=sharing

-Greg

---

From: Robert Sparks <[email-masked]>
Date: Tue, 10 Jan 2017 12:47:26 -0600

Hi Greg -

Thanks for sending this update.

I'm concerned that there isn't time for the revampers to actually _do_ 
any review. With the current flow, you're expecting that review to 
happen within one week (and assuming that the teleconference will happen 
at the beginning of that week). My instinct is that the review (if done 
meaningfully) will take more like 2-3 full weeks (people are going to 
have to do this on top of their job and other volunteer work). Do we 
have a disconnect about what we're expecting the revampers to _do_? Do 
you have instructions for them put together yet?

RjS

---

From: Greg Wood <[email-masked]>
Date: Tue, 10 Jan 2017 14:31:03 -0500

Hi Robert,

> On Jan 10, 2017, at 1:47 PM, Robert Sparks <[email-masked]> wrote:
> 
> Hi Greg -
> 
> I'm concerned that there isn't time for the revampers to actually _do_ any review. With the current flow, you're expecting that review to happen within one week (and assuming that the teleconference will happen at the beginning of that week). My instinct is that the review (if done meaningfully) will take more like 2-3 full weeks (people are going to have to do this on top of their job and other volunteer work). Do we have a disconnect about what we're expecting the revampers to _do_? Do you have instructions for them put together yet?

I suspect we might have a disconnect. I don’t have a review/discussion guide yet, but that's probably a good tool to spark the conversation. I’ll work on this (as well as the review guide for the Key Audience groups), aim to have a conversation with Joe about them as a first step, and then plan to share next week.

-Greg

---

From: Jari Arkko <[email-masked]>
Date: Thu, 12 Jan 2017 10:41:28 +0200

I shared the request for more people to join the code sprints, and Mark Nottingham asked "So, when are tools going to GitHub?”

Thoughts?

Jari

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 12 Jan 2017 15:15:44 +0100

Hi Jari,

On 2017-01-12 09:41, Jari Arkko wrote:
> I shared the request for more people to join the code sprints, and
> Mark Nottingham asked "So, when are tools going to GitHub?”
> 
> Thoughts?

There are no current plans to migrate to GitHub, and personally I'm
happier working with subversion than git, for the tools codebases.

Git offers some features we haven't felt any need for, and while the
git engine and plumbing is more capable than that of subversion, the
user commands and the models one needs to understand to do advanced
stuff are much more convoluted.

For people who restrict themselves to interaction through GitHub, this
complexity has been very nicely papered over, and GitHub provides a very
easy work-flow to submit patches, through the pull requests.

However, the volume work on tools is not done as patches, and I don't
see an upside in moving to the git engine as opposed to subversion.

Now, that may not be exactly what Mark is after.  If what he is after
is what GitHub is good at, we might investigate supporting a copy of
the subversion repositories on GitHub (but keep the master in our own
subversion repository); this might make occasional patch submission easier
for the people more used to GitHub than raw git or subversion.  I have no
idea if that's actually going to be workable, and it probably would take
some effort to investigate, trial, and set things up for production.
But it's a thought.

Best regards,

	Henrik

---

From: Julian Reschke <[email-masked]>
Date: Thu, 12 Jan 2017 15:23:40 +0100

> some effort to investigate, trial, and set things up for production.

This is what many Apache projects do - they live in SVN, but have a 
mirror on github...

Best regards, Julian

---

From: Jari Arkko <[email-masked]>
Date: Thu, 12 Jan 2017 16:36:10 +0200

Thanks Julian and Henrik.

Before we embark to the mission of making mirrors, we should probably understand what Mark was thinking of, and what him and other people might be needing. It isn’t clear to me either, I just copied the message here. We could be talking about git being more familiar tool for some people, we could be talking about easier way to make bug fixes, or a number of other things.

Jari

---

From: Lou Berger <[email-masked]>
Date: Thu, 12 Jan 2017 09:41:13 -0500

Julian,

    How do they handle pull requests?

Lou

PS After much pain, I see real advantages with git over svn. It really
facilitates multiple contributors...

---

From: Julian Reschke <[email-masked]>
Date: Thu, 12 Jan 2017 15:59:00 +0100

>     How do they handle pull requests?

AFAIU, we don't (in case you expect something automatic)

Best regards, Julian

---

From: Tero Kivinen <[email-masked]>
Date: Thu, 12 Jan 2017 17:04:56 +0200

Jari Arkko writes:
> Before we embark to the mission of making mirrors, we should
> probably understand what Mark was thinking of, and what him and
> other people might be needing. It isn’t clear to me either, I just
> copied the message here. We could be talking about git being more
> familiar tool for some people, we could be talking about easier way
> to make bug fixes, or a number of other things. 

And it is completely unfamiliar with some people, who are much more
familiar with old tools... I for example have refused to do anything
with git, and will continue doing that... :-)

We are not going to be able to pick any tool so that everybody would
be familiar with it.

When making bug fixes or enhancements to the datatracker etc, the
issue is not about the svn or github or whatever, it is the setup
which you need to do to test your change. In every single codesprint
people usually use first few hours for setting up their system again
to be able to run their own code, before they can start making their
bug fixes or enhancments.

It is getting easier and easier all the time, so we are heading to the
good direction, but there is still some issues in that process
(copying the database, recovering your mysql password as you forgot
what you put there last time you recovered it during last codesprint,
importing the database, installing the missing pieces of requirements
which are new since last time, editing the local settings, getting
copy of the rfcs, drafts, iprs etc).
-- 
[email-masked]

---

From: Jari Arkko <[email-masked]>
Date: Mon, 16 Jan 2017 08:38:47 +0200

I got an answer from Mark:

> Jari Arkko Mark: can you expand on that a bit? Why, what advantages you believe it brings, what kind of work model did you have in mind? I'm not looking for this-version-control-system-is-better-than-the-other one response, but rather: did you have in mind that the user base is larger and people are more familiar with GitHub, or that it would be easier to use in some specific case, or what?
> 
> Mark Nottingham Github has significantly less friction for getting involved than SVN+trac; everything from the issues system to pull requests to the repo clone experience is better. The experience of getting productive with the tools system is friendly to insiders / "old-timers", and very unfriendly to newcomers -- IMO this is the single most helpful change you could make.

Thoughts?

Jari

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 16 Jan 2017 10:34:07 +0100

On 2017-01-16 07:38, Jari Arkko wrote:
>> Jari Arkko Mark: can you expand on that a bit? Why, what advantages
>> you believe it brings, what kind of work model did you have in
>> mind? I'm not looking for
>> this-version-control-system-is-better-than-the-other one response,
>> but rather: did you have in mind that the user base is larger and
>> people are more familiar with GitHub, or that it would be easier to
>> use in some specific case, or what?
>> 
>> Mark Nottingham Github has significantly less friction for getting
>> involved than SVN+trac; everything from the issues system to pull
>> requests to the repo clone experience is better. The experience of
>> getting productive with the tools system is friendly to insiders /
>> "old-timers", and very unfriendly to newcomers -- IMO this is the
>> single most helpful change you could make.
> 
> Thoughts?

This is pretty much covered by my earlier comment, but to reiterate and
expand some:

We could probably set things up so that GitHub has a copy of the repo,
and add glue to propagate pull requests back to the master svn.  Mark
is right about the friction for newcomers in getting involved, if they
already know GitHub, and don't know svn.  If we take the same approach
as the Apache projects, newcomers will then be able to suggest patches
through pull requests on GitHub.

Setting up the repo copy and production glue is a bit of work, I suspect.
One might have a look at what the Apache projects do, but if Julian is
right about them not having glue for feeding pull requests back, this is
still exploration, design, and coding work.

However, please note that in order to actually run the development
environment, newcomers will also need to set that up.  This used to be
a long process, but with the docker images I've made available this
should be much easier now, even if it's not a 1-minute thing:

   https://trac.tools.ietf.org/tools/ietfdb/wiki/Docker

(That page would have to be updated to describe cloning from GitHub
as an alternative to checking out from svn.  The repo contents and
docker commands would be the same whether the local working copy came
from svn or GitHub.)

The largest effort, which Mark does not mention, and which does not
necessarily come into play for trivial patches, is of course to start
getting familiar with the ~200 models/tables and ~75 000 lines of code.

Not having to deal with a barrier of unfamiliar tools in order to suggest
a small patch would of course make it easier to start contributing.

Best regards,

	Henrik

---

From: "Ben Campbell" <[email-masked]>
Date: Mon, 16 Jan 2017 13:03:06 -0600

In my opinion, any group, be it a work group, a team working in the 
hackathon, or the tools team in the code sprint should use whatever tool 
chain that team finds productive. Pushing a team to change tools gives 
the impression of an "official" tool chain, which I think we are trying 
to avoid.

In this particular case, I think the cost of the change would outweigh 
any benefit. Becoming productive on datatracker work is an intimidating 
learning curve, but the use of SVN is an extremely minor part of that. 
Switching to git would require rewriting a lot of automated processes.

Ben (who still remembers when subversion was the hot new tool for open 
source work.)

On 16 Jan 2017, at 0:38, Jari Arkko wrote:

>> you believe it brings, what kind of work model did you have in mind? 
>> I'm not looking for

---

From: Jari Arkko <[email-masked]>
Date: Mon, 16 Jan 2017 21:37:23 +0200

Switching is clearly out of question, but whether additional access would be worthwhile may have been more of a question.

(Btw I’m anyway trying to relay comments than making those myself.)

I tend to agree with Ben’s assessment that SVN/git is a minor part of the overall learning effort if one is to do tools. I think Mark’s point was that there’d be more potential volunteers with Github access; we haven’t really talked whether those people would materialise as actual tools volunteers though. And there’s a cost even with additional access, even if it less than with switching.

What say you, team? Am I correct in interpreting that you seem suspicious of actual net benefits? That’s a fine answer.

Jari

---

From: "Heather Flanagan (RFC Series Editor)" <[email-masked]>
Date: Mon, 16 Jan 2017 12:41:41 -0800

> What say you, team? Am I correct in interpreting that you seem suspicious of actual net benefits? That’s a fine answer.

Not being a developer, I don't have anything to add re: the merits (or
lack thereof) of either system. What I have observed, however, is that
this community is very focused on workflow and tools. Even though it's a
small part of the learning curve, it's going to loom large in people's
minds because, well, it's workflow and tools. If we can get this out of
the way, then we remove an attractive nuisance that gives people an
excuse not to participate.

-Heather

---

From: Lou Berger <[email-masked]>
Date: Mon, 16 Jan 2017 15:58:41 -0500

FWIW I bit the bullet and changed over to git about a year ago.  I
converted over multiple svn repos as well as our automation scripts. 
(~.5M lines of code, histories going back ~15 years.) The conversion was
less painful than I expected and we are definitely seeing benefits
(notably branch management,  upstream code integration, offline repo
access).  This said, there certainly is a learning curve and it's easy
to get caught up in (i.e. distracted by) some new capabilities that come
with git. 

This experience leads me to feel that converting everything over to git
*at some point* is probably worthwhile, but there is real cost to doing
so.  Perhaps wait until most developers have paid the learning cost on
other projects, and start with the shadow repo on github as initial steps. 

Clearly having an active developer/contributor on the project who is a
git advocate is also a prerequisite -- and it seems that none currently
fit this bill...

Lou

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 18 Jan 2017 20:30:03 +0100

On 2017-01-16 20:37, Jari Arkko wrote:
> Switching is clearly out of question, but whether additional access
> would be worthwhile may have been more of a question.
> 
> I tend to agree with Ben’s assessment that SVN/git is a minor part of
> the overall learning effort if one is to do tools. I think Mark’s
> point was that there’d be more potential volunteers with Github
> access; we haven’t really talked whether those people would
> materialise as actual tools volunteers though. And there’s a cost
> even with additional access, even if it less than with switching.
> 
> What say you, team? Am I correct in interpreting that you seem
> suspicious of actual net benefits? That’s a fine answer.

Mostly yes.  I'm not adverse to setting up a GitHub mirror and working
on feeding back and merging in pull requests, but I don't see it as a
high or even medium priority, given the overall learning effort, as
mentioned, and the amount of work I think it would take to do it properly.

Best regards,

	Henrik

---

From: Jari Arkko <[email-masked]>
Date: Wed, 18 Jan 2017 22:27:15 +0200

Ok.

Jari

---

From: Russ Housley <[email-masked]>
Date: Thu, 19 Jan 2017 12:23:00 -0500

> mentioned, and the amount of work I think it would take to do it properly.

It would be great is Mark or one of the other GitHub advocates could take the lead on that work.

Russ