Move all your emails to Gmail

I've a lot of emails, about 30 thousands, and I like to have them all available in the same search-able place.

I also have a few email addresses to take care of (nothing special here). There are a few on @codesink.org and a couple on @inquare.com. Those mailboxes are hosted on cheap virtual server together with my personal website (www.codesink.org).

I use Mail (on Mac OS X [Leopard]) to handle all my emails and it does a pretty good job; it locks sometimes but it never crashed so badly to corrupt my data.

What I wanted was a safe place for all my ancient, old, recent and current emails :), webmail access and a powerful desktop client.

Google with your favourite IMAP client is all you need.

So the plan was:

move my domains to Google Apps (www.google.com/a/) and let the giant bring my email baggage.

This is how I did it:

1. create a Free account on Google Apps for your domain


2. be sure that emails for you get delivered to Google servers changing your DNS records


3. copy all emails to the new Gmail account (this is where things get tricky)

Google Apps account

Go to www.google.com/a/ page and sign-in for a free account. Insert your domain name when requested and create one mailbox for each mailbox you have in your current setup.

For example my personal email is [email protected] so, after creating an account to handle the domain "codesink.org" I created a user named "stefano".

Now log into your account and enable IMAP support, we will need that later.

DNS setup

The world must be notified that your mail server has changed otherwise they will keep sending emails to the old mail server.

To do that you've to modify the MX records of the DNS entry of your domain.

Google has many tutorials that explain how to do that based on your provider.

Unfortunately my hosting company did not provide any powerful DNS management interface so I moved to something better: EveryDNS. They have a powerful interface that lets you manage your DNS records via web (and it's free).

This is current codesink.org DNS setup:




The important bits here are those MX records that point to Gmail servers. The list of servers comes from Google docs.

Mail Migration

Now the tough part.

If you chose the Google Premier Edition (not free) AND you use Windows Outlook you may try the Google Email Uploader.

I chose the Free edition of Google Apps (the upload utility doesn't work for free accounts) and I'm on a Mac so I had to find another approach.

The net is full of tutorial on how to import messages into Gmail accounts using IMAP. In short you've to let your email client connect to your Gmail account using the IMAP protocol and then drag all messages into it.

That didn't work for me because transferring large folders on slow ADSL (like mine) takes forever and Google kicks you our after a while (less then an hour for me).

If you ever read this message you know what I mean:

Error: Unable to append message to folder (Failure)

You can start the upload over again but you don't know which messages have been already uploaded and which not so what you should do is split big folders in small chunks and upload one chunk at the time.

If you have a large number of emails splitting your folders in tens of smaller folders and upload them one by one gets you frustrated pretty soon.

So here's the trick I used:

• export all 30,000 emails and save the MBOX files in a single directory


• create a big compressed archive with all emails


• upload the compressed archive on a Linux server (with fast connection)


• sync the archive on Linux with the IMAP server on Gmail using mailsync

Export

First of all we need to export all email to MBOX files. With Mac OS X's Mail this is pretty easy, you've to CTRL-click on each mailbox and select "Archive Mailbox..." from the popup menu.

Save each mailbox in a temporary folder; let's call it MAILBOXES/.

Compress

Now compress the whole directory:

$ tar jvcf mailboxes.tar.tbz MAILBOXES/

Upload

Upload the archive to the Linux server and extract all *.mbox files in the 'mail' directory inside your home dir.

For example I used the 'scp' command to move the archive to my home dir (/home/tat) and then I extracted all mailboxes into /home/tat/mail/.

Sync with Gmail

Ok now you should have a remote Linux server that holds all your emails and an empty Gmail account.

With the next few steps we will move all messages to Gmail.

Login into your Linux server and install mailsync:

apt-get install mailsync

Then create a file named '.mailsync' in your home directory (change [email protected] and the password):

store gmail {
server  {imap.gmail.com/[email protected]/ssl}
ref     {imap.gmail.com}
pat     %
passwd  012345
}
store local-mbox {
pat     mail/%
prefix  mail/
}
channel default local-mbox gmail {
msinfo  mail/synchronization-info
}

Start syncing!

$ mailsync -t md5 -n default

That may take a few hours.

If it locks and Gmail kicks you out you can just restart it after a while, mailsync is smart enough to understand which messages still need to be uploaded and which messages are already in place.

Related links

codesink.org: my site

EveryDNS: free managed DNS services

mailsync: utility that syncs multiple email accounts

Amazon AWS: info about Amazon cloud computing and services

Ubuntu: Linux distro used

Google Apps: can handle emails for your own domains