Subject: [TOOLS-DEVELOPMENT] Fwd: Re: final migration

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 15 Feb 2012 17:26:31 +0100

Hi,

Here's what I sent Ryan yesterday about the final migration and
switch-over:

Best regards,

	Henrik

-------- Original Message --------
Subject: Re: final migration
Date: Tue, 14 Feb 2012 22:34:10 +0100
From: Henrik Levkowetz <[email-masked]>
To: Ryan Cross <[email-masked]>

Hi Ryan,

On 2012-02-14 19:11 Ryan Cross said the following:
> Hi Henrik,
> 
> I'm curious what your plans are for the final migration.  Some questions I have:
> 
> - what procedures will you use?
> 	- change "web" symlink to new release

Yes.  We also need to switch to a production version of settings_local.py

> 	- final db migration. In the past you mentioned running incremental
> 	  migrations. Will this result in an accurate copy of production?

I'm currently running only differential conversions, covering documents and
associated tables.  I plan on running one complete conversion every night
beween 01:00 and 02:30 (approximately) PST.  The differential conversions
will not (at the moment) catch stuff which isn't related to documents, but
looking at the daily changes, non-document-related changes are very few.

> 	  I am
> 	  thinking of ancillary objects that have been created in ietf_new,
> 	  like DocEvent objects, and other objects who's state has been
> 	  changed. Wouldn't these remain after an incremental import?

Yes, I guess they might, so just before the switch-over we'll need to run
one last full conversion.

> - do you expect downtime?

I'm a bit undecided on this, but leaning towards 'No'.  From the mysql's bin-log
we can see the exact events that might have been applied to the old tracker
during the last conversion run, and not been caught by it, and handle those
manually.  But my guess is that if we do the switch-over right after the
nightly run there won't be anything left which hasn't been handled by the
full conversion plus the differential document conversion.

> - Ole has provided me with an import script for the extra tables we have been
>   using for Interim Meetings (they reside in a separate database).  I've testing
>   them in my environment.  Would you like to add it to your migration process or
>   would you like me to run it after you have done your part (or before?).

I don't think it matters greatly, but it would maybe be safest to make that
script part of the complete conversion so we have some experience with running
that, too, in conjunction with the rest?  On the other hand, if we instead run
it after the last full migration that should work fine, too.  

> - what time of day / day of week are you planning on for the final cutover?

I'd prefer the time I mentioned above, during a Saturday.

> - there are various script outside of datatracker (cron, cgi-bin)
>   which access the database, some are secretariat tools that I am
>   familiar with, they should go away. Is there an inventory/migration
>   plan for these programs as well?

We've over the last 3 years run repeated inventorys of these, and there should
be no cgi-bin scripts left now, assuming you've taken care of what I think was
the last public one, the materials upload.

The cron scripts I know about have all been converted to Django/Python and should
continue to work (except that some of them, like some mirroring scripts, should
be meaningless).

I hope I've covered most, let me know if you think of more points related to
this :-)

Best regards,

	Henrik