Dovecot Full Text Search with Jetty and Solr

Want to archive all your email to a server that you control and have it searchable? Here’s how to configure Dovecot to support Full Text Search using Jetty and Solr.

Dovecot Full Text Search with Jetty and Solr

Warning

If you are going to be archiving all your email, make sure you are using a NFS mount (for IMAP) on a NAS that provides redundancy, in case of drive failure. Also have a process in place to regularly backup the data from that NAS to avoid data loss!

Getting Started

First you need to have a Dovecot IMAP email server setup. We’ll be using Debian, so Ubuntu will be very similar. If you are using another distro, please use the package manager for your distro, or feel free to install these packages from source.

Install Jetty and Solr

BASH

$ sudo apt-get install jetty8 solr-jetty

Configure Jetty and Solr

Copy the Dovecot Solr schema to the Solr configuration directory

BASH

$sudo cp /usr/share/dovecot/solr-schema.xml /etc/solr/conf/schema.xml

Restart jetty

BASH

$sudo /etc/init.d/jetty8 restart

Verify that Jetty is serving up Solr

Use your web browser and open http://your-dovecot-server:8080/solr

Click the “Solr Admin” link on this page and it should look like this:

Notice how it says dovecot!

Configure Dovecot Plugins

Modify Dovecot’s configuration files (I use vi, please use the text editor that you are comfortable with)

BASH

$ sudo vi /etc/dovecot/conf.d/10-mail.conf

Modify the “mail_plugins” line:

mail_plugins = $mail_plugins fts fts_solr

(note: add to end of existing plugins string)

BASH

$ sudo vi /etc/dovecot/conf.d/90-plugins.conf

plugin {
 fts = solr
 fts_solr = break-imap-search url=http://localhost:8080/solr/
 }

This should be the mail setup to configure Dovecot to perform Full Text Search of your email archive!

Modify your imap mailbox user’s crontab to do some housekeeping:

BASH

$crontab -e

5 * * * * doveadm fts rescan -u johnnyboy
0 1 * * * curl http://127.0.0.1:8080/solr/update?optimize=true
2 * * * * curl http://127.0.0.1:8080/solr/update?commit=true

Now you should be able to perform full text searches of your emails.

Please make sure that your email client is performing server side searches.

There may be additional helpful information at https://wiki.dovecot.org/Plugins/FTS/Solr

Warning Reminder!

If you are going to be archiving all your email, make sure you are using a NFS mount on a NAS that provides redundancy, in case of drive failure.Also have a process in place to regularly backup the data from that NAS to avoid data loss!

Now your Dovecot server should be fully able to perform a Full Text Search of your email archive! Give it a shot.

unixsamurai

unixsamurai

Dovecot Full Text Search with Jetty and Solr