Message header manipulation by mailing lists

In my previous entry I made a couple of comments about mailing lists which 👤james_r asked me about IRL, so here’s a follow-up.

There has been a very blurry line between mailing lists and multi-recipient aliases almost forever: the ambiguity is referred to in RFC 821 (dated August 1982). The earliest clear and prominent explanation of the difference is in RFC 1123 (dated October 1989), but that doesn’t say anything about what lists do to the message header. Traditionally, during list expansion the Sender: field has been replaced with an address related to the list (often the same one used to replace the return path). Lists may also make other changes (about which more below) but it’s this abuse of Sender: that I would like to improve.

As RFC 821 hints, mailing lists have been implemented with varying degrees of integration with the MTA. Ten years ago a common setup was to have Majordomo managing the contents of an aliases file, which was then used directly by Sendmail to do the list expansion. Nowadays it’s more common to run something like Mailman which is mostly independent of the MTA. This leads to arguments about what architectural label to put on the list manager (which are probably aggravated by the fact that the labels were invented for X.400): is it part of the Message Transmission System alongside the MTA, or is it an MTS client, i.e. an MUA?

There’s another historically ambiguous area, around the word “forwarding”. In fact it overlaps with the list/alias blurriness because “forwarding” is sometimes used to mean “aliasing”, especially when it is set up with a .forward file or an equivalent such as the Sieve redirect action (as opposed to something like the /etc/aliases mechanism). It’s best to call this “aliasing” or “redirecting”, not “forwarding” (even though RFC 2822 describes this meaning of “forwarding”). “Forwarding” should be reserved for sending a new message with the forwarded message attached.

There is a third operation which is called “forwarding” by RFC 822. Some MUAs (including Pine) describe it as “bouncing” which is also confusingly ambiguous terminology that should be avoided. RFC 2822 clarified that the old 822 terminology should not be used, so I call this “resending” after the header fields it uses. From the SMTP point of view, resending is very similar to mailing list expansion in that the message gets a new set of recipients, its return path is changed to refer to the resender, and the message header and body mostly remain the same. The difference is that resending is usually more manual, and the Resent- header fields include more information than mailing lists’ traditional Sender: manipulation.

One of the improvements that RFC 2822 makes compared to RFC 822 is that it allows a message to be resent multiple times, with an unambiguous record of what happened in the message header. The top of a message includes a sequence of Received: fields which record in blog order (most recent at the top) the MTAs that a message went through. RFC 2822 says that (like Received: lines), Resent- fields should be added to the top of the message, so that they appear at a point in the Received: chain corresponding to the point that the message was resent. (RFC 822 didn’t say anything in particular about where Resent- fields should go, so you could only resend a message once unambiguously.)

So that’s the background. What I propose is that:

Mailing list managers should be considered MUAs, i.e. that the result of list explosion should be considered a new message submission rather than anything like relaying or redirecting;
They should use the Resent- convention to indicate that the message was resent via a list, and at what point in the chain of MTAs this occurred;
They also should add their List-ID: and other List- fields at the top of the header, next to the Resent- fields.

The advantages of this are that it’s much clearer which system is responsible for doing what to a message; the original Sender: field isn’t obliterated; and it’s much more friendly to cryptographic signature schemes like DKIM which don’t like the message header to be arbitrarily modified.

Things it doesn’t improve are the controversies over adding list name tags to the message’s Subject: and overriding the original sender’s choice of Reply-To:. On these matters I’m of the opinion that this mangling is only done because MUAs still don’t have decent user-interfaces for the List- fields.