Yesterday we went out and bought two hard-disks to put in the server, just to make sure we don't run out of space.  We had plenty of extra space.  Didn't put them in, just set them on the desk.

So, of course, this morning we woke up to the server's root hard-disk mounted read-only and the physical media reporting errors... argh.  Spent the whole day doing backup onto the new drives and trying to rebuild the machine onto a new root drive.  No luck, in fact, while I've been booting a live-cd all day, it's now failing with ATA errors so I can't even try a fresh software install.  Looks like it's a hardware fault, possibly in the serial ATA adapter on the motherboard... if that's the case, may wind up having to replace the whole of the motherboard/processor/ram... which would eat up (another) whole day's productivity.

Very frustrating waste of a day.  Ah well.  Means I'm without email or internal bzr services (and file backups, of course), but hopefully I can get the work I need to get finished done without email.

[Update] Both the ReiserFS3 and the Ext3 disks were corrupt with what looks like physical errors.  The Reiser volume was recoverable, but the Ext3 disk is pretty much garbage at this point.  Even the one disk I have left from the original set is showing some SMART issues that mean I'll need to replace it ASAP (sigh).  In the meantime, I rebuilt the machine using a Karmic Beta this evening.  So, email is flowing again, I just need to plow through it all :) .