2020/12/14: Reimplementing forward synchronisation of email

Since quite I while I use my simple scripts to forward-synchronise emails and they still do their job properly. Nevertheless, to use them, there are quite a few individual components to set it up; in fact, I know of several people who got interested in this form of synchronisation, but then decided that the setup, especially the initial one, was too complicated.

Recently, Christian Lütke-Stetzkamp and myself reimplemented the method of forward synchronising mails. However, a couple of things are done differently. (The description of those changes assumes that the description of my old method of forward synchronisation is read first, or known to the reader.)

Single binary, also taking care of the transfer

To have less loose parts, everything is handled by a single binary. Moreover, this binary also takes a more active role in the data transfer; the program is still parametric in the transport (which in most cases will be ssh anyway), but the task of queuing and and keeping track of what still needs to be transferred is handled by the program directly and not delegated. In this way, we can do a couple of simplifications.

Initial sync

We also added something that was lacking with my old scripts: a good story for adding a new peer. To do so, the peer already in the network creates an initial message bundle for the new peer.

The initial sync is usually done either over a reliable link or in a store-and-forward fashion. In any case, it can be repeated without a problem, should any crash occur on new peer. So, to speed things up (actually quite a lot!), the new peer handles the initial messages without caring about on-disk consistency in between. The state of record is only written to disk at the end, and only a final sync(2) is done at the end—instead of the usual write-sync-rename dance for every mail when delivering it to a maildir.

uux

Given that we queue tasks anyway, it was easy to also add a feature to request that one particular peer execute a given command (provided they whitelist it) with given stdin. The intended commands to be whitelisted are obviously rsmtp and/or sendmail. In this way, the binary becomes a one-stop shop for mail handling on, e.g., a laptop.

Sources

The program is written in Haskell. The sources are available under terms and conditions of a BSD-style license. They can be downloaded from the mail-sync home page.

There is also an isilmar port for mail-sync that was successfully tested on Debian/10. The packages mentioned in haskell-depends.default are taken form Debian, libghc-async, libghc-typed-process, libghc-unliftio, libghc-rio, and mail-sync are taken (in that order) from ports.