Subject: [iola-conversion-tool] Incorrect time stamps in the History trail for recently-published RFCs

From: Cindy Morgan <[email-masked]>
Date: Thu, 1 Mar 2012 11:37:21 -0800

Hi,

I have a question and a bug.

Question: It looks like the datatracker now magically gets the information when a new RFC is published.  Previously, the Secretariat had to do part of that manually, including changing the IESG state to "RFC Published."  Does this now happen automatically, or does the datatacker sync with the RFC Editor database at set times?  In either case, it looks like the Secretariat no longer needs to take action here?

(I ask because there were RFCs published yesterday that weren't updated in the datatracker when I looked around noon Pacific yesterday, but are updated now.  There are also RFCs that were published around 0800 Pacific today that have already been updated in the datatracker.)

Bug: In the history trails of the RFCs that were published in the last two days, it shows the RFC as being published on 2012-02-01, rather than 2012-02-29 or 2012-03-01:

https://datatracker.ietf.org/doc/draft-lear-iana-timezone-database/history/
https://datatracker.ietf.org/doc/draft-irtf-hiprg-dht/history/
https://datatracker.ietf.org/doc/draft-ietf-behave-v4v6-bih/history/
https://datatracker.ietf.org/doc/draft-kucherawy-dkim-atps/history/
https://datatracker.ietf.org/doc/draft-stone-mgcp-vbd/history/

And strangely, for draft-ietf-6man-3627-historic, the history trail does not include an "RFC Published" message from the system at all (although the doc page itself does have a link to the RFC):

https://datatracker.ietf.org/doc/draft-ietf-6man-3627-historic/history/

[And if you look at draft-ietf-sieve-notify-sip-message, the history trail has the correct time stamp because Amy changed the state manually to RFC Published yesterday.  At some later point the datatracker got the correct RFC number information from somewhere else.]

Thanks,
Cindy

Cindy Morgan / Project Manager / IETF
48377 Fremont Blvd., Suite 117 / Fremont CA 94358
T: +1.510.492.4085 / F: +1.510.492.4001 / http://www.ietf.org
----------
Managed by Association Management Solutions (AMS);
Forum Management, Meeting and Event Planning
Website: http://www.amsl.com

---

From: Ole Laursen <[email-masked]>
Date: Thu, 1 Mar 2012 21:45:01 +0100

> Question: It looks like the datatracker now magically gets the information when a new RFC is published.  Previously, the Secretariat had to do part of that manually, including changing the IESG sta…

Yes, it gets the information from the RFC Editor. It did that before
too, but saved the information on the side instead of merging it in. I
couldn't do that with the new schema, had to merge, so in the end it's
now automatic I think. It will set it to be in the RFC state and add
the comment you've noticed. I hope that's okay?

There's a slight problem with the IESG state, it doesn't set that to
published, we need to get that fixed although I don't think you can
actually see it on the document pages.

> [And if you look at draft-ietf-sieve-notify-sip-message, the history trail has the correct time stamp because Amy changed the state manually to RFC Published yesterday.  At some later point the dat…

Odd, it's supposed to put in the publication date it gets from the RFC
Editor, but it's possible there's a bug somewhere. I'll take a look at
the above tomorrow.

Ole

---

From: Ole Laursen <[email-masked]>
Date: Fri, 2 Mar 2012 19:22:38 +0100

> actually see it on the document pages.

I've fixed that so if it is RFC Ed Queue, it'll set it to RFC
published automatically.

>> https://datatracker.ietf.org/doc/draft-ietf-6man-3627-historic/history/

A careful look at this reveals that it's using 2012-02-01 too, so the
entry ends up below in the stack.

For all of these, I've looked at the actual data, and although the
parser gives me a specific date, the data from the RFC Editor only
says "February 2012". So it just assumes it's the first day of the
month. Gah.

Does anyone know whether we could get the actual date out of the RFC
Editor? I can imagine a couple of other hacks to improve on the
first-day-of-month-assumption, but by far the most robust thing would
if the downloaded data just told us it.

Ole

---

From: Russ Housley <[email-masked]>
Date: Fri, 2 Mar 2012 15:09:33 -0500

> if the downloaded data just told us it.

This will be addressed in the future under another task order.  Currently, the RFC Editor send an email to the Secretariat, and the date comes from that email.  The goal is to have a message sent that the datatracker can consume in an automated fashion.

Russ

---

From: Ole Laursen <[email-masked]>
Date: Mon, 5 Mar 2012 13:52:49 +0100

> This will be addressed in the future under another task order.  Currently, the RFC Editor send an email to the Secretariat, and the date comes from that email.  The goal is to have a message sent t…

Aha. If that date could be put into the generated XML file here:

http://www.rfc-editor.org/rfc/rfc-index.xml

Then it's a <1 hour job to fix it in the Datatracker end. The XML file
already says

        <date>
            <month>March</month>
            <year>2011</year>
        </date>

so a <day>29</day> would fit the purpose nicely. Would it be feasible
to do this in a not too distant timeframe?

Ole

---

From: Russ Housley <[email-masked]>
Date: Mon, 5 Mar 2012 09:27:30 -0500

> to do this in a not too distant timeframe?

This data has never included the day.  I think we need to use the messages that are sent to the secretariat to get the day.

Russ

---

From: Ole Laursen <[email-masked]>
Date: Mon, 5 Mar 2012 15:49:43 +0100

> This data has never included the day.  I think we need to use the messages that are sent to the secretariat to get the day.

Hm, right, I was just wondering who's going to collect the information
in the future.

If there's a database at the RFC Editor's, then it may be easier to
put the information there, even if it requires someone to write an
importer to scan through all the messages (we could scan them and
provide a tab-separated file for the initial import).

But if that's never going to happen and the Datatracker database will
be the main source for this information, then I need to start thinking
about what the script we're currently using should do.

Ole

---

From: Cindy Morgan <[email-masked]>
Date: Mon, 5 Mar 2012 09:54:50 -0800

Please see below.

> This data has never included the day.  I think we need to use the messages that are sent to the secretariat to get the day.

And in fact, it's not messages that are sent directly to the Secretariat; we would get the information from the announcements that the RFC Editor sends to IETF-Announce (e.g. http://www.ietf.org/mail-archive/web/ietf-announce/current/msg09972.html).

Best regards,
Cindy

---

From: Ole Laursen <[email-masked]>
Date: Wed, 7 Mar 2012 16:54:27 +0100

Hi people!

An update on the publication dates of RFCs: I've talked to Henrik
about our options, and we'll go with a solution that doesn't require
any changes in the data provided by rfc-editor.org for the time being.
That's implemented now, so the "RFC published" entries should end up
at the proper spot in the future.

I have a dump of dates obtained from the RFC Editor announcements that
we can use to fix the existing entries.

Ole