Zimbra Mail Discoveries


Zimbra is a fantastic open source mail server. It basically wraps Postfix, and has a beautiful webmail interface including a great calendar system. Yes, AJAX, buzzwords, all apply.
I've been wanting to migrate my mail into an IMAP server with a nice webmail frontend for a while. The advantages: being able to use mail from more than one machine, including my laptop when away and disconnected, other members of the family being able to use mail, etc, etc. Zimbra seemed like the best fit I'd seen since my last set of investigations (which incorporated dovecot, Hula, etc, etc).
At first I thought I'd just try out moving my mail in and see what problems I hit. By the time I had that done, I thought it would be a waste of time to retreat so I advanced headlong into the fray... Thus eating up a week of time or so, in which I was mostly unable to do very much with mail or anything else. (I should have thought beforehand, "I'm not really a mail admin person").
Anyhow, at the end of the process I think its been very successful and am very happy with Zimbra. This post is basically a summary of what I've learnt in the hope that other people will try out Zimbra and I will feel relief at the sense of not having to go through that again...
I had around 270000 messages in my Mozilla folder hierarchy, some dating back to around 1996. Total size about 4GB. This meant that the actual process of copying the mails was fairly cumbersome, especially if I wanted to check that everything copied OK (don't want to lose mail in the process!)
I installed Zimbra on Fedora Core 4, on my normal development / server machine. Don't be too concerned about the admonition to use a separate server, you can run Zimbra happily alongside existing instances of MySQL, Apache etc (Zimbra installs its own, hopefully in the future you can integrate it into your existing apps). I just needed to set Zimbra to only serve https, then I can access webmail etc on that port and my normal Apache takes care of port 80.
I then ended up developing some Python code to be able to do the things I wanted to fix up my mailbox etc. I've called this pyzimbra. These don't really consist of bindings to Zimbra code yet, but rather code to interact with the database, the message store, etc.
Issues encountered included:
Messages that refused to be copied (like finding a needle in a haystack), - I still have to resolve this one, only a dozen or so.
Dates on messages that didn't have proper date headers came out as 1st Jan 1970. Dates in the webmail view showed the time the message was copied rather than its original time. To fix these problems I wrote modules to connect to the zimbra database, and find the message file corresponding to each message in the database. I then checked if the message lacked a Date header and gratuitously added one based on other headers like Received (I'm sure this is naughty...). I then update the received date in the database (and alter the content-length if I added a date header). It was quite fun finding my way around the database etc, and the modules for doing this are fairly generic.
I'd also like to retrieve the Mozilla Flags (replied to etc) but unfortunately they don't get stored in the headers on the IMAP server after copying so that would involve finding the original message in the Mozilla mailbox that corresponds to the one in the Zimbra store.
Migrating my Mozilla mail filters to Zimbra: Zimbra uses Sieve for mail filtering. There is a nice utility for converting Mozilla's msgFilterRules.dat to a Sieve script (Javascript in a web page - use "/" as a separator in the option): mozilla2sieve. The Sieve script is stored in an LDAP attribute that I couldn't seem to access. You can access it via the zmprov utility, but there are caching problems.
The more serious issues are that Zimbra's Sieve implementation uses regular expressions, which makes havoc when you try and download a few thousand messages that have been queuing up and none of them get filtered - it will only display errors with the script when it actually tries to filter them. Look out for [ and *!!! Interestingly the Webmail Filters page basically deals with an XML representation of the Sieve script that it sends back and forth to the server.
If you're just running a few accounts through Zimbra and your main domain is hosted elsewhere, the Split Domain article on the wiki is really helpful.
Since I'm on an ADSL connection with a changing IP address, I'm using fetchmail to retrieve my mail periodically. Refused SPAM messages give a 501 response to the SMTP instruction.
Finally I wrote some nice backup scripts and utilities. zimbra/scripts/linkfolders.py is quite neat - it creates a directory structure on disk like the Folder structure in Zimbra, and symlinks the messages to their home on the message store. Nice for working out disk allocation, and as an alternate backup mechanism.
Currently unrated


Recent Posts







RSS / Atom