Degoogling Part 1.2 - Self Hosted Email

Degoogling Part 1.2 - Self Hosted Email

This is the third part of a series focused on migrating from Google Workspaces. For Part 1.1, where I cover using ForwardEmail as an email go-between, go here.

Introduction

If you do any research on self-hosting email, you will very quickly find plenty of anecdotal evidence that it's not a recommended endeavour. This is for two reasons:

Email deliverability

Email is highly reliant on "sender reputation". When you send an email, it will contain information about where it comes from, such as the IP address of the server sending it. SPAM detectors rely on lists of server addresses known for reliably sending email that isn't SPAM. For example, Google's email sending (SMTP) servers will have a higher reputation than your own ISP's SMTP server (if they run one).

Imagine now that you are running an SMTP server on your home network. SPAM detectors have next-to-no information about your server's history, so your reputation will be low, and there is a much higher chance of any email sent through it being flagged as SPAM. Given this, it could take months of perfect deliverability to build a reputation good enough for SPAM detectors to accept it without extreme scrutiny. And if somehow your SMTP server is compromised and becomes a relay for a SPAM sender, then your reputation will immediately be trashed, maybe forever.

Add to that, most ISPs nowadays share one IP address among many customers, and regularly cycle IP addresses, so without a special arrangement with your ISP to reserve a single IP address for your connection, it's impossible for your email server to build a reputation.

Home Server Connection Reliability

When you set up a server to receive email, it announces that it is the final destination for your email address. So, other email servers handling email from other senders, that is addressed to you, will try to deliver it to your server. If your home internet connection is lost for a while, or you have a hardware failure, or you need to update or reconfigure your operating system or software, other email servers will try to deliver the email for a little while, but if they can't deliver it, they'll bin it, and you will never know that the email ever existed. The period that email servers wait before binning undeliverable email varies widely, so you really can't afford any downtime if you're hosting your own receiving server.

These two reasons are why I decided to use a hybrid approach, with forwardemail.net sitting in between the rest of the internet and my home server. The heavy lifting - receiving and sending email - is done by the email specialist - while my home server focuses on copying email delivered to forwardemail.net, and filtering, storing, archiving, and backing it up while providing a couple of fast ways to access it.

My Solution

While there are a number of all-in-one self-hosted email solutions (MailCow for example), many of them are bundled with Mail Transfer Agents (such as Postfix or Exim).

Since I am using a hybrid approach, I didn't need or want an MTA, and in fact having one bundled exposed a potential security vulnerability that I didn't want to deal with. I just needed a local pseudo-MTA between Fetchmail and Dovecot, for local delivery. For that reason, I ended up rolling my own docker image based on Dovecot atop Arch Linux, and wrote my own mail delivery agent.

The reason I chose Arch Linux instead of Alpine or Debian is that Arch had packages for the Dovecot plugins that I wanted, out of the box.

Here is the workflow I ended up settling on:

  1. An email is received by forwardemail.net and placed in a remote IMAP mailbox
  2. On my home network
    1. Fetchmail periodically checks the IMAP mailbox for new messages.
    2. When it receives a message, it downloads it and passes it through ClamAV and Spamassassin to modify it with relevant flags for later processing.
    3. It then passes the message to Dovecot, which routes it to the correct mail user.
    4. Dovecot applies some global and user-specific filters (using Sieve) to send the email to the correct folder in the user's mailbox (e.g. if it is marked as Spam, to the Junk folder).

There are a number of moving parts to this workflow, and I had to overcome a number of obstacles to get everything working reasonably well:

  1. How would I configure each user's mailbox in a docker environment?
  2. How could I simplify the process of mirroring forwardemail.net users and Dovecot users?
  3. Fetchmail's default configuration is to check a single mailbox. How would I configure it to check multiple mailboxes and route accordingly?
  4. Spamassassin and ClamAV need regular updates and maintenance to ensure they're effective. How would I accomplish this?

Going forward, you can refer to my Gihub Repo for the complete docker image configuration: https://github.com/logicalor/degoogle/tree/main/imap-dovecot

Dovecot Virtual Users

Dovecot's default configuration assumes that each user will have their own home directory on the server. This isn't ideal for a docker environment where I didn't want multiple actual users, so I opted to configure it using virtual users instead.

I created a directory to put the users' mailboxes on the host, and then used that as a volume in the docker configuration.

volumes:
      - ${DIR_MAIL_DOVECOT}:/var/mail

I then provided this directory in the Dovecot configuration like so:

mail_location = maildir:/var/mail/%u/Maildir

Where %u is the username.

I also needed a way to index each user, as I wouldn't be using Linux' native user system. I used a users file in passwd format to store users:

auth_mechanisms = plain login
disable_plaintext_auth = no

userdb {
  driver = passwd-file
  args = /opt/users/users
}

Entries in the users file look like this:

%%username%%@snj.au:{ENCRYPTED}%%encrypted-password%%:1000:1000::/var/mail/%%username%%@snj.au/Maildir::

I needed fetchmail to know each user's forwardemail.net IMAP password so that it could check the remote mailboxes, but I didn't want the passwords stored in plaintext in the users file, so I opted to use openssl encryption based on a passkey supplied in my docker's .env file. Fetchmail reads the users file, gets the encrypted password, runs it against openssl to decrypt it and then uses it to check the remote mailbox.

However, this presented another problem. When I connect to the Dovecot server to check my mail, I authenticate using the password in plaintext. Dovecot doesn't have a way out-of-the-box to check the plaintext password against the encrypted version stored in users, so I need to provide Dovecot with a custom authentication script to validate the password:

passdb {
  driver = checkpassword
  args = /usr/local/bin/dovecot-auth
}

This tells Dovecot to pass the password to a custom script, and let the script verify the password matches. I created a custom script to decrypt the password in the users file using openssl, and then check it against the supplied password.

Doing this meant that I could create users with the same credentials as forwardemail.net, and it should just work.

Finally, I created a script, make-user, to create and modify users in the users file.

Automatically Updating / Training ClamAV and Spamassassin

This was actually pretty straightforward since I had set up deck-chores in Part 0. I just needed to create some scheduled jobs as labels in the docker-compose.yml file:

    labels:
      # Update CA Certs
      deck-chores.dovecot-update-ca-certs.cron: "0 0 0"
      deck-chores.dovecot-update-ca-certs.command: sudo update-ca-trust
      # Update ClamAV DB
      deck-chores.dovecot-update-clamav.cron: "0 0 0"
      deck-chores.dovecot-update-clamav.command: sudo freshclam --daemon-notify=/dev/null
      # Update Spamassassin rules from remote sources
      deck-chores.dovecot-spamassassin-update.cron: "0 0 0"
      deck-chores.dovecot-spamassassin-update.command: sudo /usr/bin/vendor_perl/sa-update -D
      # Update Spamassassin rules via learning
      deck-chores.dovecot-spamassassin-learn.cron: "0 0 0 15"
      deck-chores.dovecot-spamassassin-learn.command: sa-learn

To train Spamassassin on a per-user basis, I did need to add some Spamassassin configuration, and create a script to iterate over each user and run the Spamassassin training command.

For the Spamassassin configuration, I copied the default local.cf and added some lines to the end to ensure Spamassassin knew where to find each user's training database.

bayes_path /var/mail/%u/.spamassassin/bayes
user_scores_dsn /var/mail/%u/.spamassassin/user_prefs
allow_user_rules 1

The script sa-learn iterates over the directories in /var/mail, and for each user found specifies the 'Spam' and 'Ham' folders (Ham in email parlance is the opposite of Spam - legitimate email). The Spamassassin learner then uses email in those folders to fine-tune its rules.

Dovecot Plugins

I am using Dovecot's Sieve plugin for global and user-specified filtering. I also configured Dovecot to expose a managesieve service, which allows compatible email clients to synchronise its rules with the IMAP server. This means that theoretically you could set up your mail filtering rules in one place, and they would work all the time, without relying on your email client to do the filtering. In practice, it's a bit difficult to find email clients these days that work well with managesieve, so I do it via Snappymail, a web-based email client, which will be covered a bit later.

For indexing, I am using the Xapian plugin. Xapian is a full-text indexing service that plugs into Dovecot and works via the standard IMAP full-text search protocols. The upshot of this is that basic text searching via an email client should be much faster with Xapian than without.

I won't go over configuration of these plugins too much as all of the config is provided in the github repo.

Multi-user Fetchmail

In its default incarnation, Fetchmail is intended to connect to a single mailbox and fetch mail from that single mailbox for one user. So, to have it work on a multi-user, multi-mailbox basis, I used a wrapper script.

Initially I leaned pretty heavily on this post which outlines how to have Fetchmail check multiple accounts and route it to different folders for the same users, but my needs were a bit different so I made a lot of alterations to the point of it pretty much being a new script. Many of the other principles are the same though.

Essentially, I pull the users file, iterate over it, and instantiate Fetchmail for each user using a generated fetchmailrc for each, then check the mailbox using their decrypted password, run it through ClamAV and Spamassassin, and deliver it via Dovecot to the correct mailbox. This is repeated for each user every 30 seconds.

Spamassassin / DNS

When Spamassassin checks email against its Spam rules, it runs a DNS check to check the originating IP address against the purported sending domain. Out of the box, it uses a third-party DNS provider to do this. I quickly discovered that the service imposes a cap, and my check requests were being rejected afterward. To solve this I needed to implement my own DNS resolver and make sure my docker instance was using it. To do that I ended up installing Unbound in my docker image, and overwriting my /etc/resolv.conf to use the local Unbound resolver rather than a third party.

Dovecot SSL Certificates

To correctly serve IMAP over SSL, Dovecot needs to be configured with the SSL certificates supplied for the domain you wish to serve on. In my case, the domain is mail.snj.au.

I already had an installation of Snappymail webmail (see below) running on mail.snj.au, and I had configured its domain / SSL using Nginx Proxy Manager, so I had Letsencrypt certificates available in that container's volumes.

I ended up finding where these certificates are located on my Nginx Proxy Manager container, creating a shared certs directory on the host which I mounted as a volume, then and adding a deck-chores job to copy them periodically to the shared directory.

    volumes:
...
      - ./shared-ssl-certs:/opt/shared-ssl-certs
    labels:
      deck-chores.imap-ssl-copy.cron: "*/15 0" # Every 15 minutes
      deck-chores.imap-ssl-copy.command: /bin/sh -c "cp /etc/letsencrypt/live/npm-23/* /opt/shared-ssl-certs/snj.au/ && chmod -R 777 /opt/shared-ssl-certs"

I then mounted this shared directory in my Dovecot container, and configured Dovecot to use them:

# /etc/dovecot/conf.d/10-ssl.conf
ssl = required
ssl_cert = </etc/letsencrypt/fullchain.pem
ssl_key = </etc/letsencrypt/privkey.pem
ssl_dh = </opt/dovecot/dh.pem

Port Forwarding

I already had a DNS entry for mail.snj.au pointing to my home network's public IP address. However, I needed to configure my router to port forward ports 993 (IMAP over SSL) and 4190 (ManageSieve) to point to the correct IP address in my internal network. This allows remote requests to directly connect to the internal services.

Using the Docker Stack

You can clone the github repo as-is, which contains all of the scripts and custom configuration. You will need to create a .env file based on .env.example with your own values. You will also need to figure out your own method of generating or syncing a LetsEncrypt SSL cert for your dovecot service.

The most basic procedure is

  1. Run docker compose build imap-dovecot

    This will build the image. It takes some time because it installs all the relevant packages, installs a series of Perl modules that Spamassassin uses, and runs through a few other preliminary jobs to get the environment set up.
  2. Run docker compose up -d. This will boot the container. To view its logs you can run docker compose logs -f
  3. To create users, you can run docker compose exec imap-dovecot sudo create-user username@your.domain your-plaintext-password. This will add the user to the users file, and prompt the fetchmail wrapper to pick up the new user and start checking its email at forwardemail.net. You can also use the same command to edit an existing user, but you may need to restart the container for the changes to be correctly picked up. This is docker compose down && docker compose up.

Backups

I use the Borg container that I set up in Part 0, and add a new volume and deck-chores command to it, so that it backs up to Hetzner.

    volumes:
...
      - ${SRC_DIR_IMAP_BACKUPS}:/sources/imap-backups
...
    labels:
...
      #### IMAP BACKUP ####
      # Daily at 1:15am
      deck-chores.borg-backup-imap-backups.cron: "1 15 0"
      deck-chores.borg-backup-imap-backups.command: borg create --compression lz4 ${BORG_REPO}::\"data-imap-backups-{now:%Y-%m-%d}\" /sources/imap-backups
      # Daily at 1:30am
      #deck-chores.borg-prune-imap-backups.cron: "1 30 0"
      #deck-chores.borg-prune-imap-backups.command: borg prune --list --glob-archives data-imap-backups- --keep-daily=14 --keep-weekly=8 --keep-monthly=12 ${BORG_REPO}
...

Connecting via an Email Client

Using Thunderbird, you will leave your SMTP settings the same (that is, continue using forwardemail.net). However you update your IMAP settings to use your Dovecot service's domain (in my case - mail.snj.au). IMAP over SSL port, and user credentials, remain the same.

Web Mail - Snappymail

Snappymail is a well-maintained fork of the now-defunct Rainloop. It is an efficient webmail client with just the right number of features. It's also fairly easily configurable.

As mentioned earlier, I already had Snappymail installed and reverse-proxied via Nginx Proxy Manager, and was using it to connect to forwardemail.net's IMAP services directly. All I needed to do was reconfigure it to connect to mail.snj.au 's IMAP services (that is, my home network) instead.

For the docker-compose configuration I use for Snappymail, refer to my Github repo: https://github.com/logicalor/degoogle/blob/main/snappymail/docker-compose.yml.

Snappymail has an admin interface where you can configure your allowed domains. That is, domains that users can use to log into your webmail.

There are 3 parts to each domain's configuration - the IMAP server that will be used to check email for the domain, the SMTP server that will be used to send email on behalf of the domain, and the ManageSieve server that will be used to store filtering rules for the IMAP server.

I added a white listing for the bare snj.au domain.

Here the IMAP server is mail.snj.au. The username format should be the same as what the user enters when logging into the webmail interface (someuser@snj.au).

I'm using forwardemail.net for my SMTP, and it uses the same user credentials as my local IMAP server. So I add the correct settings to connect to smtp.forwardemail.net here.

I connect to the ManageSieve service at mail.snj.au here. The authentication mechanism is the same as for the IMAP service.

Once the domains are set up, I can then log into Snappymail using my ForwardEmail email address and password combination.

Snappymail Login

Mail Filters

In your SnappyMail settings is a Filters section. From here you can manage your simple filters to add manual filtering rules. In my case I have a couple of spam rules:

Simple Snappymail filters

Conclusion

Hosting your own email is certainly not trivial. Getting all of these services working well together in a self-contained docker image was a pretty exhaustive process. I had to overcome a number of configuration issues for which there is not a lot of documentation to be found. However, the end result is a fairly reliable stack that gives me local email that I have total control over for filtering and backup, and optionally migration to another email provider should the need ever arise.