Replicate live into a throw-away staging/dev db
Written by on in Tuxedo.

Problem that looks like it will be coming up soon... you are always wanting to be able to test your code against a (huge, PostgreSQL) LIVE DB before you release/promote the code. Loading a DB dump can take hours. Your test code can't all just run in a single transaction.

First approach: run the DB server in a virtual machine, sync the LIVE DB down to the virtual machine, snapshot just before you run tests, throw away the snapshot.

Second approach: run the DB server on real hardware, sync the LIVE DB down to a *template* DB on the real hardware, drop the database and recreate based on the template (note: you have to stop the sync before you create the new db). In my spike test the create-from-template approach still takes a few minutes, but it's *far* faster than loading an SQL dump.

create database moo with template moo_template;

Both approaches, of course, require a DB sync operation... that's the part I still need to research.

Comments

  1. T. Middleton

    T. Middleton on 08/31/2010 11:04 a.m. #

    That template db idea is pretty neat. I have to say it's getting pretty annoying downloading database backups and restoring them around here... they are around 25 gigs compressed these days. I don't do it very often though: i don't need up-to-date data for testing. And when I do restore I tend to run it overnight... and the test server is always sloooooooow compared to live.

    What I really have wanted to do for years is come up with a really good representative subset of data for testing against -- with lots of the edge cases. But we have a lot of relations... it's complicated. So I never get to it.

  2. Sabrina

    Sabrina on 09/08/2010 1:55 a.m. #

    Anyone who has developed a relatively complex site will be able to vouch that, while launching a new site is relatively simple, continuous integration becomes very tricky once the site has actually gone live. This problem becomes especially pronounced when there is more than one developer on a project, and when the project is being built using proper development methodology.

    <a href="http://www.adexim.com.pl/">przeprowadzki</a>

Comments are closed.

Pingbacks

Pingbacks are closed.