Surviving a Catastrophic IMAP (Cyrus) Crash
In a similar post I described what to do in case your IMAP server crashes and it still behaves well. This time, though, Cyrus crashed and burned beyond recognition, and there was nothing I could do to save the databases. All utilities in the books simply told me the db files were corrupt, and I should run recovery. I ran recovery, and nothing happened…
This time I found a two-way approach helped a lot: first, I instituted nightly backups of my email server, and keep the last three days, three weeks, and three months on file. I created a script (attached below) in TCL to do the trick.
Secondly, I figured that the only way to have a working database again was to uninstall and reinstall cyrus-imapd. I didn’t quite trust a blind reinstall (especially since it had cost me days to get the configuration just right). So I followed the following ’easy’ steps to get to a working system…
- FIRST AND FOREMOST: THIS IS NOT PROFESSIONAL ADVICE AND YOU SHOULD NOT RELY ON IT. CONSIDER IT A STARTING POINT FOR YOUR EXPLORATION, BUT IN NO CASE ACT UPON IT IF YOUR DATA IS IMPORTANT.
- STOP Cyrus: /etc/init.d/cyrus stop (maybe your distribution says cyrus-imapd or such)
- BACK UP YOUR DATA: cp -a /var/lib/imap /tmp/var_lib_imap
- FIND THE RPM: export rpmfile=
locate cyrus-imapd | grep rpm | tail -1 - UNPACK THE RPM IN A TEMPORARY LOCATION: rpm -i –relocate /=/tmp $rpmfile
- REMOVE THE DB FILES YOU ARCHIVED: rm -rf /var/lib/imap
- COPY THE DB FILES INTO THE DESTINATION: cp -a /tmp/var/lib/imap /var/lib/imap
- START Cyrus: /etc/init.d/cyrus start
That should have fixed your problems. Stop Cyrus, and import all your email using the script in {moscontentlink:Recovering from IMAP Crash}.
[Script: backupimapdb.tcl]
#!/usr/bin/tclsh
variables
set bkdir /root/backup
set imapdir /var/lib/imap
set today [clock format [clock seconds] -format %Y-%m-%d]
set monthly [string match *-01 $today]
set weekly [expr [clock format [clock seconds] -format %w] == 0]
perform today’s backup
exec tar cvzf $bkdir/imapdb-daily-$today.tgz $imapdir
and the other ones if appropriate
if {$weekly} {
file copy -force $bkdir/imapdb-daily-$today.tgz $bkdir/imapdb-weekly-$today.tgz
}
if {$monthly} {
file copy -force $bkdir/imapdb-daily-$today.tgz $bkdir/imapdb-monthly-$today.tgz
}
get lists of backups
set daily [glob -nocomplain /root/backup/imapdb-daily-*]
set weekly [glob -nocomplain /root/backup/imapdb-weekly-*]
set monthly [glob -nocomplain /root/backup/imapdb-monthly-*]
analyze what can go: start with daily
proc deleteus files {
if {[llength $files] > 3} {
foreach file [lrange $files 0 [expr [llength $files -4]]] {
file delete -force $file
}
}
}
deleteus $daily
deleteus $weekly
deleteus $monthly