Subject: [Tools-implementation] Fwd: Emacs lock files in rfc rsync

From: Glen <[email-masked]>
Date: Thu, 4 Feb 2021 15:40:14 -0800

Something we should review in our next meeting.   I'll provide the 
technical details at that time.  (Roman, sorry I forgot to cc you on the 
original reply, I missed your presence in the header until after I hit 
send, but this is the same email reply below.)

Glen

-------- Forwarded Message --------
Subject: Re: Fwd: Emacs lock files in rfc rsync
Date: Thu, 4 Feb 2021 15:38:02 -0800
From: Glen <[email-masked]>
Organization: AMS
To: Eric Rescorla <[email-masked]>
CC: Sandy Ginoza <[email-masked]>, Alice Russo <[email-masked]>, John 
R Levine <[email-masked]>

All -

In the interest of time, I am just grouping everyone together and 
replying to this thread directly.

>> From: Eric Rescorla <[email-masked]>
>> Subject: Emacs lock files in rfc rsync
>> Date: February 4, 2021 at 1:06:27 PM PST
>> To: RFC Editor <[email-masked]>, Roman Danyliw <[email-masked]>
>> Hi folks.
>> I just rsynced the rfc directory and I see that several of the files have Emacs lock files [0] associated with them:
>> lrwxrwxrwx  1 ekr  staff  39 Mar 16  2016 .#bcp-ref.txt -> [email-masked]:1457089460
>> lrwxrwxrwx  1 ekr  staff  39 Mar 27  2019 .#rfc-index.xml -> [email-masked]:1551880913
>> Aside from just creating problems when someone else tries to edit the files in Emacs, it suggests that you might have some kind of process problem, because this material should be machine generated, and people shouldn't be editing it in a way that the directories are just raw synced to the server.

Eric -

My apologies, but you are not using the authoritative source for your 
rsync.  You stated that you're using:

> rsync.ietf.org::rfc

That is the IETF's legacy mirror of RFCs.  I do not refer or recommend 
it.  It appears to be a legacy configuration, and to have some 
duplication in it.  Moreover, for reasons unknown to me, it is being 
rsynced without the --delete option, which means it will retain every 
file it ever saw during any prior rsync.  That is why you are seeing 
those files.  They existed at some interval in the past; they do not 
exist (on the RFC servers) now.

I will raise this with the Tools Implementation Team to see if (a) this 
behavior should change or (b) the IETF copy should be removed or 
referred.  Removing files from it, or making changes to it, are not 
things I can "just do" on my own in my role.  Note that they are working 
on other, larger projects right now, so this will need some time before 
it can get attention.

In the meantime, I recommend that you change your configuration to rsync 
your RFCs from rfc-editor.org:: , which is the authoritative source for 
RFCs.  They have a catalog of options, one of which will I hope suit:

# rsync rfc-editor.org::
everything-ftp  Everything FTP
refs            XML references for RFCs (for use with xml2rfc)
rfcs            Contents of in-notes including subdirectories std, bcp, 
fyi, and ien
rfcs-text-only  Only the text files from the directories in [rfcs]
rfc-ed-all      Entire repository (excluding internet-drafts)
internet-drafts Internet Drafts
ids-text-only   Only text files from the Internet Drafts mirror
rfcs-pdf-only   PDF versions of ASCII RFCs to ensure correct page 
breaks, etc
rfcs-exclude-json       Contents of [rfcs] excluding JSON files
rfcs-json-only  Only the JSON files from the directories in [rfcs]

In the meantime, the files you referred to, which may exist on the 
legacy IETF copy, will not be removed, nor will any other changes be 
made to the IETF copy, until the Tools Implementation Team can carve out 
time to review this and determine the correct solution.

I will send the relevant information to them now, to get this on their 
radar.

Thank you,
Glen

---

From: Roman Danyliw <[email-masked]>
Date: Fri, 5 Feb 2021 19:19:23 +0000

Hi Glen!
(looping in Greg)

Concur about adding this as an agenda item for the next meeting  A few related thoughts:

(1) During the community consultation on FTP, parties noted that we should improve our rsync documentation.  All I can find is:

https://www.ietf.org/standards/ids/internet-draft-mirror-sites/

Minimally, I was thinking maybe proving a list of the sync points and what they contain.

(2) https://www.ietf.org/standards/ids/internet-draft-mirror-sites/ currently says "All Internet-Drafts are available via ftp, http, and rsync. This page provided details about these services, including locations and instructions related to I-D mirrors."  

-- s/via ftp, http, and rsync/via ftp, https, and rsync/

-- This page only talks about rsync and not the others.  Should we be pointing to https://www.ietf.org/id/ as the reference for HTTPS?

(3) As you noted below that ::rfc is not authoritative, we might want to scrub the following descriptions to annotate which are mirrors of something else (i.e., ::rfc, ::iana, ::iana-timezone) -- say s/Repository of RFCs/Mirror of the repository of RFCs (see also rsync.rfc-editor.org)/

roman@ubuntu:~/ietf$ rsync rsync.ietf.org::
everything-ftp 	- The entire IETF FTP Archive
internet-drafts	- The Internet Draft Repository (currently active drafts)
id-archive     	- The Internet Draft Archive (both active and expired drafts)
iesg-minutes   	- IESG Minutes
proceedings    	- Repository of Proceedings
xml2rfc.bibxml 	The xml2rfc citation libraries
charter        	- Repository of WG Charters
concluded-wg-ietf-mail-archive	- Older list text archives
conflict-reviews	- Repository of Conflict Review documents
iana           	- IANA assignments
iana-timezone  	- IANA Time Zone Datatbase (see also http://www.iana.org/time-zones)
legacy-files   	- Legacy material supporting long-lived URLs
mailman-archive	- Repository of Mailing List Text Archives
rfc            	- Repository of RFCs
slides         	- Repository of Slide Documents
status-changes 	- Repository of Status Change Documents

(4) We likely also want to discuss the legacy status of ::rfc.

Roman

> -----Original Message-----
> From: Tools-implementation <[email-masked]> On
> Behalf Of Glen
> Sent: Thursday, February 4, 2021 6:40 PM
> To: [email-masked]
> Subject: [Tools-implementation] Fwd: Emacs lock files in rfc rsync
> 
> technical details at that time.  (Roman, sorry I forgot to cc you on the original
> reply, I missed your presence in the header until after I hit send, but this is the
> same email reply below.)
> 
> Glen
> 
> In the interest of time, I am just grouping everyone together and replying to
> this thread directly.
> 
> >> To: RFC Editor <[email-masked]>, Roman Danyliw
> >> <[email-masked]> Hi folks.
> >> Aside from just creating problems when someone else tries to edit the files
> in Emacs, it suggests that you might have some kind of process problem,
> because this material should be machine generated, and people shouldn't be
> editing it in a way that the directories are just raw synced to the server.
> 
> Eric -
> 
> My apologies, but you are not using the authoritative source for your rsync.
> You stated that you're using:
> 
> > rsync.ietf.org::rfc
> 
> That is the IETF's legacy mirror of RFCs.  I do not refer or recommend it.  It
> appears to be a legacy configuration, and to have some duplication in it.
> Moreover, for reasons unknown to me, it is being rsynced without the --delete
> option, which means it will retain every file it ever saw during any prior rsync.
> That is why you are seeing those files.  They existed at some interval in the
> past; they do not exist (on the RFC servers) now.
> 
> I will raise this with the Tools Implementation Team to see if (a) this behavior
> should change or (b) the IETF copy should be removed or referred.  Removing
> files from it, or making changes to it, are not things I can "just do" on my own
> in my role.  Note that they are working on other, larger projects right now, so
> this will need some time before it can get attention.
> 
> your RFCs from rfc-editor.org:: , which is the authoritative source for RFCs.
> They have a catalog of options, one of which will I hope suit:
> 
> # rsync rfc-editor.org::
> In the meantime, the files you referred to, which may exist on the legacy IETF
> copy, will not be removed, nor will any other changes be made to the IETF
> copy, until the Tools Implementation Team can carve out time to review this
> and determine the correct solution.
> 
> I will send the relevant information to them now, to get this on their radar.
> 
> Thank you,
> Glen
> 
> 
> 
> 
> --
> Tools-implementation mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-implementation

---

From: Robert Sparks <[email-masked]>
Date: Fri, 5 Feb 2021 15:50:01 -0600

Specifically this would be the next tools-implementation team meeting.

We don't have one of those scheduled (as we didn't have anything 
immediate to discuss). We can schedule one now.

1600 UTC had been working for us - is it still a reasonable time? If so, 
is Fri 12Feb open for folks.

RjS

---

From: Glen <[email-masked]>
Date: Fri, 5 Feb 2021 14:38:06 -0800

> is Fri 12Feb open for folks.

I cannot do anytime Wed-Fri this week.  My electric company has decided 
they need to dig up the street I live on, and I expect to be without 
power here on Wed, Thu and Fri (Fortunately only during the day, but, 
still.)  I've already had to "book the office" for Wednesday owing to 
critical events that day, not comfortable with later in the week either.

Monday or Tuesday of the following week would be better for me.

That said, our one item is not particularly urgent and we are still 
pushing on larger projects, so I did not mean to imply that a meeting 
was urgent.  I think we can wait until after 110 safely.

Glen

---

From: Robert Sparks <[email-masked]>
Date: Fri, 5 Feb 2021 16:46:06 -0600

On 2/5/21 4:38 PM, Glen wrote:
>> 1600 UTC had been working for us - is it still a reasonable time? If 
>> so, is Fri 12Feb open for folks.
>
> I cannot do anytime Wed-Fri this week.  My electric company has 
> decided they need to dig up the street I live on, and I expect to be 
> without power here on Wed, Thu and Fri (Fortunately only during the 
> day, but, still.)  I've already had to "book the office" for Wednesday 
> owing to critical events that day, not comfortable with later in the 
> week either.
>
Ok - will defer for the moment.

---

From: Robert Sparks <[email-masked]>
Date: Tue, 9 Feb 2021 12:02:03 -0600

> Minimally, I was thinking maybe proving a list of the sync points and what they contain.

Hi Roman - how is that not the list you retrieved below from 
rsync.ietf.org:: ?

Is the issue that the descriptions are not detailed enough?

> -- This page only talks about rsync and not the others.  Should we be pointing to https://www.ietf.org/id/ as the reference for HTTPS?

I think we should always point to the id archive unless we are 
_specifically_ trying to say "active drafts". So the link should be 
https://www.ietf.org/archive/id/

(which is a brutal beast to retrieve).

---

From: Robert Sparks <[email-masked]>
Date: Tue, 9 Feb 2021 12:53:37 -0600

For the short term, I propose that we delete and resync what's at our 
::rfc sync point so it matches the rfceditor exactly.

Does anyone know of a reason not to do that?

We can then discuss removing it as an exposed sync point.

RjS

On 2/4/21 5:40 PM, Glen wrote:
> technical details at that time.  (Roman, sorry I forgot to cc you on 
> the original reply, I missed your presence in the header until after I 
> hit send, but this is the same email reply below.)
>
> Glen
>
> CC: Sandy Ginoza <[email-masked]>, Alice Russo <[email-masked]>, 
> John R Levine <[email-masked]>
>
> All -
>
>>> <[email-masked]>
>>> I just rsynced the rfc directory and I see that several of the files 
>>> have Emacs lock files [0] associated with them:
>>> Aside from just creating problems when someone else tries to edit 
>>> the files in Emacs, it suggests that you might have some kind of 
>>> process problem, because this material should be machine generated, 
>>> and people shouldn't be editing it in a way that the directories are 
>>> just raw synced to the server.
>
> Eric -
>
> I will raise this with the Tools Implementation Team to see if (a) 
> this behavior should change or (b) the IETF copy should be removed or 
> things I can "just do" on my own in my role.  Note that they are 
> working on other, larger projects right now, so this will need some 
> time before it can get attention.
>
> In the meantime, I recommend that you change your configuration to 
> rsync your RFCs from rfc-editor.org:: , which is the authoritative 
> source for RFCs.  They have a catalog of options, one of which will I 
> hope suit:
>
> # rsync rfc-editor.org::
> rfcs            Contents of in-notes including subdirectories std, 
> bcp, fyi, and ien
> made to the IETF copy, until the Tools Implementation Team can carve 
> out time to review this and determine the correct solution.
>

---

From: Glen <[email-masked]>
Date: Tue, 9 Feb 2021 10:55:46 -0800

Agreed on all counts from my point of view.

Glen