Tony Finch – blog

One of the weaknesses of the current Hermes architecture is the comparative lack of data security for the message transport queues on ppswitch. These store messages in the process of being delivered, or which have been delayed because we can’t reach the destination host, or because of quota problems for a user on the Cyrus store. The ppswitch machines have SCSI RAID controllers with battery-backed caches, so they’re resistant to single disk failures, power outages, etc. but are not safe from filesystem bugs or multiple hardware failures. It would be nice to augment them with some kind of replication technology so that they have comparable reliability to the Cyrus machines.

At the start of last year I was thinking about a new feature called “early delivery”, which would allow you to get the MTA to deliver a copy of the message before confirming to the sending machine that it has been safely delivered (between the “.” at the end of the message data and the server’s response). This copy would be kept on another machine, therefore safe in case of some nasty failure, while the original would be delivered as usual. I didn’t really work out any good way of preserving all the message’s metadata (client IP address, authentication, etc.) or a way of removing copies of messages that have been delivered so are no longer needed.

David Woodhouse suggested another use for early deliveries, which would be an early attempt at normal delivery rather than a separate copy of the message. His scenario was a border MTA delivering to MTAs inside an organization which might have different security policies - similar to ppswitch, in fact. By trying to deliver early you can pass success or failure directly back to the sender, instead of accepting the message then generating a bounce if there’s a problem. As such, it’s somewhat like souped-up SMTP call-forward address verification, but applied to the whole message.

http://www.exim.org/pipermail/exim-users/Week-of-Mon-20040105/064531.html

I’ve recently been thinking a bit about using standard email protocols for communication within the parts of an MTA. For example, rather than the sendmail program being the MTA itself, and having to be setuid in order to switch from the user’s security context to the MTA’s, it should be a standard SMTP client. It can’t quite be as simple as that because sendmail has a number of management functions as well as message submission, so it might use a second protocol for those, or extensions to SMTP. You can preserve some of its security properties (such as knowledge of the user that ran sendmail) by doing SMTP over a Unix domain socket (instead of over TCP), and using AUTH EXTERNAL to tell the SMTP server to use SCM_CREDS to find out the user at the other end of the socket.

Another example would be to detach local delivery from the core of the MTA, again so that the MTA doesn’t have to be setuid in order to switch from the MTA’s security context to the user’s. The MTA could send messages to the local delivery agent using LMTP. Cyrus already does this, but it’s less normal to do so for traditional Unix message stores. (Postfix is structured like this but AFAICT its internal protocols are undocumented.)

It occurred to me yesterday that you can combine these ideas in an interesting way, using early delivery to implement content-scanning callouts. In this case the early delivery isn’t a delivery as such (the message isn’t kept by the content scanner), but rather it’s just a means of getting the data from the SMTP server to the scanner. In this respect it’s different from other common email filtering architectures where the scanner acts as an SMTP proxy that receives the message from the MTA and re-injects it back into the MTA:

Lots of ideas to play with if I ever do some work on Postmaster…