The Best Way to Copy the Message Store

The Problem of Copying Live Data

Of course, we recommend stopping Kerio MailServer while doing a data migration. Some companys cannot afford downtime for such a long operation.

What can go wrong and how do sysadmins handle it?

  1. New mail can arrive during the migration. Stop the SMTP service. SMTP protocol involves mail queues on remote mailservers, so they should queue up the mail for you during your migration. After the new server comes online, you can enable SMTP and allow mail to come in again.
  2. Local mail can be sent via DAV or Webmail during the migration. This will be on the old store. Perhaps an incremental data migration of some kind can catch this later. Admins typically deal with this on a reactive basis. I.e., wait until users call and complain, then find the new mail in the old store and restore it for them.
  3. Folders can end up with corrupt indexes on the destination server. This is solved by reindexing the mail folders on the server. This is handled reactively. If a user calls to report a problem with strange looking folder listings, see knowledge base article 84 to re-index the folder.

Linux or Mac OS X

The trick to this is to preserve the timestamps on the .eml files in the mailstore. This is critical for proper indexing. In UNIX, probably the most reliable way to move data is with a tar pipeline. Under the store directory is a folder called "mail". This is the email that must be migrated.

sudo bash
cd /old/mail/store/mail
mkdir -p /new/mail/store/mail
tar cvpf - . | (cd /new/mail/store && tar xvpf - )

There are several things to note about these commands.

  1. The destination directory (your new store) must exist, hence the "mkdir" command.
  2. You must be root, hence the "sudo" command.
  3. You must be currently in the source directory, hence the "cd" command.
  4. The destination directory can be a network mounted drive. This is an excellent way to move data from one server to another.
  5. The double ampersands protects your source directory from overwriting itself and causing data corruption so do not use a semicolon which can cause this problem. Be sure the command is exactly as demonstrated in this document except for the paths.

Disaster situation: Suppose your old hard drive is failing, and you cannot copy the data due to device timeouts or corrupt blocks in the filesystem. This might be a very dire situation, but it still might be possible to copy the data with the following command:

tar cvpf - . --ignore-failed-read | (cd /new/mail/store && tar xvpf - )
Windows

For Windows there are several options

  1. Use cygwin. Install the cygwin tools, and use tar piplines such as the one in the UNIX section of this document.

    mkdir -p /cygdrive/f/store/mail
cd "/cygdrive/c/Program Files/Kerio/MailServer/store/mail/"
tar cvpf - . --ignore-failed-read | (cd /cygdrive/f/store/mail && tar xvpf - )

    The thing to notice with cygwin is that the drives are not specified the same way. The "C:" drive is specified by "/cygdrive/c" and similarly with the "F:" drive.

  2. Use xcopy. Xcopy is a powerful command line tool similar to tar. It is executed as follows (assuming F:\store\mail does not exist yet):

    3. C:
cd \program files\kerio\mailserver\store\mail
xcopy . F:\store\mail /SEIKO

    The switches are /SEIKO for recursive copy, include empty directories, create destination directory, preserve readonly and acl information. Modified timestamps on the individual files are preserved, but it seems timestamps on the directories are not. The tar using cygwin tools does preserve the modification times on the directories (which doesn't matter to my knowledge).

  3. Use ZIP. ZIP preserves timestamps which is good, but is prone to failure if there are any glitches in your hardware. Failing drives or memory or filesystem damage can cause ZIP to fail. When zipping gigabytes of data, there is a higher chance of running into such a hardware failure. It is certainly a viable way to migrate data.

Disaster situation: Suppose your source hard drive is failing and read errors and timeouts prevent copying from success. There are several ways to approach this:

  • Xcopy using the /C option, so the switches would be /SEIKOC. This option means to continue copying even if errors occur.
  • Using the cygwin tools to do a tar pipeline, try the --ignore-failed-read option as in the example above.
  • Some ZIP programs might have an ignore failed read option of some kind. I cannot find one in WinRAR, but perhaps I did not look hard enough. If I find such a ZIP utility, I will list it in here.
  • Scandisk repair. It might be tempting to run scandisk to repair the filesystem and see if that will allow successful copying of the store. The problem is, Scandisk might incorrectly reallocate data causing you to lose random amounts of email forever. To make matters worse, you don't even know if the failing drives are going to reallocate the blocks properly. If you know your hard drive is in the process of failing, use this as a last resort.