Re: [apps-discuss] Review of draft-melnikov-smtp-priority-14
Ned Freed <ned.freed@mrochek.com> Mon, 11 June 2012 21:48 UTC
Return-Path: <ned.freed@mrochek.com>
X-Original-To: apps-discuss@ietfa.amsl.com
Delivered-To: apps-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C84B821F852A; Mon, 11 Jun 2012 14:48:03 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.655
X-Spam-Level:
X-Spam-Status: No, score=-1.655 tagged_above=-999 required=5 tests=[AWL=-0.125, BAYES_00=-2.599, DATE_IN_PAST_06_12=1.069]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OS-wd-cpVM2o; Mon, 11 Jun 2012 14:48:02 -0700 (PDT)
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by ietfa.amsl.com (Postfix) with ESMTP id CBE7121F8464; Mon, 11 Jun 2012 14:48:01 -0700 (PDT)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OGKA8FQ5FK003CJ7@mauve.mrochek.com>; Mon, 11 Jun 2012 14:42:56 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OGIRBMUA6O0006TF@mauve.mrochek.com>; Mon, 11 Jun 2012 14:42:52 -0700 (PDT)
Message-id: <01OGKA8DLA0Y0006TF@mauve.mrochek.com>
Date: Mon, 11 Jun 2012 07:29:50 -0700
From: Ned Freed <ned.freed@mrochek.com>
In-reply-to: "Your message dated Tue, 05 Jun 2012 07:23:15 -0400" <6828E9C8-3C2A-46C9-8BD1-1308000CD91B@g11.org.uk>
MIME-version: 1.0
References: <6.2.5.6.2.20120521130747.0c219ab0@elandnews.com> <4FBDF199.2050300@isode.com> <4FC722A2.2050905@dcrocker.net> <4FC89931.5060201@isode.com> <4FC914DB.4000806@dcrocker.net> <4FCA6BFE.3050609@isode.com> <4FCD175D.30307@dcrocker.net> <01OGAJ8GBR2Q0006TF@mauve.mrochek.com> <F6882C013F7272CED4D345A9@PST.JCK.COM> <E503581B-E89A-4E09-B06C-CF18263F7376@g11.org.uk> <1E3DCEC5990AF898F1E3582D@PST.JCK.COM> <6828E9C8-3C2A-46C9-8BD1-1308000CD91B@g11.org.uk>
To: John C Klensin <john-ietf@jck.com>, Ned Freed <ned.freed@mrochek.com>, apps-discuss@ietf.org, draft-melnikov-smtp-priority.all@tools.ietf.org, iesg@ietf.org
Cc: ken carlberg <carlberg@g11.org.uk>
Subject: Re: [apps-discuss] Review of draft-melnikov-smtp-priority-14
X-BeenThere: apps-discuss@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion of application-layer protocols <apps-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/apps-discuss>, <mailto:apps-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/apps-discuss>
List-Post: <mailto:apps-discuss@ietf.org>
List-Help: <mailto:apps-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/apps-discuss>, <mailto:apps-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Jun 2012 21:48:04 -0000
This entire discussion has been growing increasingly abstract, and that's rarely a good thing in standards work. Rather than continue in that direction, I decided to do something I should have done a lot sooner: I implemented the draft. <disclaimer> The fact that I have implemented this draft in Oracle Messaging Server does not imply any commitment on the part of Oracle to actually support this extension. And should Oracle decide to support it, this doesn't mean it will be done in any particular release or in any particular time frame. </disclaimer> (Sorry about having to include that, but rules are rules.) Implementation work is usually very instructive, and this time was no exception. I'll list the simple things I learned first: (1) The draft says that the extension is valid for LMTP, but doesn't give any guidance as to what that means. This could be interpreted as saying that an implementation that uses LMTP has to support the MT-PRIORITY extension there to be compliant. That would in turn mean that prioritization has to be done on the LMTP server side, which may be a bit difficult as there's no queue there. Our implementation, and I suspect many others, handles prioritization on the LMTP client side. That said, it's entirely possible for the MT-PRIORITY to affect the LMTP server by changing processing or I/O handling in some way, or affect subsequent store access behavior, or may be needed just so it can be logged. So the extension should be allowed over LMTP, but some clarifying text is needed to say an implementation MAY choose to handle prioritization on the client side of the LMTP connection only. (2) The draft also allows MT-PRIORITY on submission, which of course makes complete sense. However, given the overall state of MUAs in the world today, it is next to certain that clients are going to be used that don't support it. (And please don't try the line that this can always be dealt with by requiring the use of certain clients. I'm talking about the real world, not fantasyland.) As such, there are going to be cases where the MSA needs to attach an MT-PRIORITY to messages that don't have one. It would be good to mention this, but even if it's not discussed there needs to be an enhanced status code defined to indicate when it has been done, and more generally to indicate when the MT-PRIORITY has been upgraded. (3) Implementations that support Sieve need to provide a way for Sieves to test the current MT-PRIORITY value. I implemented this as an environment item because that's a place that allows for implementation-specific values. The alternative would be to do it as a full-blown Sieve extension and add it to the envelope test. Given that MT-PRIORITY is an SMTP extension the envelope test is the natural place to do it, but a problem with that may be it restricts the test to implementations and situations where the envelope test itself is supported. This is not a problem for our implementation, but it may be a problem for others. Another issue is the lack of a defined comparator that will work with MT-PRIORITY values. i;ascii-numeric doesn't handle signed comparisons. An i;ascii-integer comparator is needed, so I added one and I think I'll go ahead and add the definition to the comparator registry. It may also be a bad idea to add all this in the current draft, especially given where this draft is in the process. But it needs to be done somewhere. (4) The draft doesn't say anything about logging. The current MT-PRIORITY value does appear in Received: fields, but that's not the same thing. Logging of how MT-PRIORITY is used is going to be a requirement in some environments, so a general suggestion along the lines of "MT-PRIORITY values SHOULD be logged as part of any logging of message transactions" is in order. (5) The suggested text for the X.3.TBD3 error code doesn't conform to the requirement that the text begin with the new priority value. (Note that the new error code suggested under (2) should have the same requirement.) (6) The draft talks about greylisting, but fails to mention that connection-level and SMTP HELO/HELO greylisting options exist, and when these options are used there's an issue if the trust relationship is established through the use of SASL. This needs to be pointed out. I also spotted a minor typo in the Introduction: sattelite -> satellite. The draft also spells out numbers in some places and writes them as actual numbers in others, making it hard to search for things. I don't care which approach is used as long as it is consistent. Now on to the more difficult stuff. I first need to say a few words about how our product is implemented. We have separate entities we call "channels" that messages can be routed to based on various criteria. Each channel has it's own queue for messages, and can be separately configured as to how many delivery threads to use, how frequently to retry, what IP address and port to use, and so on and so forth. Additionally, each queue supports three priorities, basically meaning higher priority messages get tended to first and will be retried more frequently. I realized right away that not only does a single channel approach not suffice because a single channel doesn't have enough available priorities (the draft requires six; see section 5), but also because even if we were to add additional queue priorities, it's unlikely that that would actually create a useful distinction between all the different priorities. For a useful distinction to exist, especially across so many different levels, you need access to things like different connections that shape traffic differently, and in our world that means different channels. So what I did was provide various hooks so that routing can be adjusted based on priority. The obvious way to configure things would be to have two channels and route three MT-PRIORITY levels to each one, but of course many other arrangements are possible, up to and including ones that fullly support 19 distinct levels. But there's no way I can guarantee that things will be configured this way. I can't even test for it, since I have no way of testing external conditions. And this is true no matter what implementation strategy I use, because any sufficiently flexible configuration mechanism can be used to defeat priority handling just as easily as it can be used to enforce it. What this means is that the six level business is really an operational requirement not an implementation requirement. What the implementation needs to do is provide the *means* to support at least six priority levels. That's all that can be reasonably expected, and the draft really needs to make that clear. This then raises the question of whether or not we should make use of this extension contingent on operational support of six levels. I think the answer to that is a rather emphatic "no" - it should be left to the service profile to decide how many levels they actually need. And that also needs to be made very clear - in particular, statements as to supporting six levels being a requirement, which sure sound like an operational requirement to me, need to either be qualified or removed. But this in turn raises another question: Should there be some indication of what service profile is being provided? Proposals have been made previously to do things like include the available priorities in the EHLO response. There are two problems with that: (1) It doesn't actually identify the profile, and (2) There's nothing a client can usefully do with the information. A better alternative would be to include some sort of profile name in the EHLO response. This wouldn't be for operational use, but rather for auditing purposes. Of course an implementation like ours, and I suspect most others, would not be done in a profile-specific way, so this would have to be a configuration option. And I'm not sufficiently fond of the idea to say it should be done. I'm only offering it as a suggestion. Notifications next. The draft says that notifications SHOULD be processed at the same priority as the message they are responding to. Having now implemented this, I'm somewhat tempted to make that a MUST, because if you try and do it any other way it's like opening a pandora's box of failure cases. Perhaps a warning to the effect that, "Regardless of the approach chosen, schemes which cause notifications to be completely lost MUST NOT be used." But the draft then goes on to say that: For delivery reports received by an MTA, processing rules specified in Section 4.1 apply. But note that while it might be tempting to handle all delivery reports (a.k.a. DSNs) at their stated priority, under the assumption that failure notices need to get through quickly in some situations, but such a policy creates an exposure to fake-DSN attacks if sources of DSNs can't be reliably established. This seemed to make sense until I tried to implement it, at which point I realized it makes no sense at all. The problem is fundamental: There is in general no way to tell whether or not you're dealing with a DSN. Yes, we have defined format for DSNs, but there are still plenty of things out there that generate DSNs in a different format. Even if we were to require support for NOTARY (and thus for our DSN format), we need to be able to make priority decisions during envelope processing, and the only indication there is the empty MAIL FROM. But while DSNs are required to use an empty MAIL FROM, the converse isn't true: Plenty of things other than DSNs use empty MAIL FROM. And if that wasn't enough, the implication here seems to be that DSNs are intrinsicly harder or more difficult to authenticate. Huh? That would seem to imply that DSNs arrive from some sort of alternate source, or we're supposed to base authentication decisions on the MAIL FROM. I don't know of any of the former and the latter is a *really* bad idea. DSNs are just messages that arrive from some source, and that source is either appropriately authenticated or it isn't. Yes, you can make make additional choices once you've received the message based on the content being a DSN, but that's no different than any other sort of content-based decisions. This isn't X.400, where DSNs aren't actually messages. I therefore strongly recommend dropping the entire paragraph except for the first sentence. And that's about it. I will say that having implementing the draft in what I think is is a useful way, I'm a lot more confident that this extension makes sense. I'll try to follow up on some of the other traffic on this topic in a bit. Ned
- [apps-discuss] APPSDIR review of draft-melnikov-s… S Moonesamy
- Re: [apps-discuss] APPSDIR review of draft-melnik… Barry Leiba
- Re: [apps-discuss] APPSDIR review of draft-melnik… Barry Leiba
- Re: [apps-discuss] APPSDIR review of draft-melnik… S Moonesamy
- Re: [apps-discuss] APPSDIR review of draft-melnik… t.petch
- Re: [apps-discuss] APPSDIR review of draft-melnik… Barry Leiba
- Re: [apps-discuss] APPSDIR review of draft-melnik… Alexey Melnikov
- Re: [apps-discuss] APPSDIR review of draft-melnik… S Moonesamy
- Re: [apps-discuss] APPSDIR review of draft-melnik… Pete Resnick
- Re: [apps-discuss] APPSDIR review of draft-melnik… Pete Resnick
- Re: [apps-discuss] APPSDIR review of draft-melnik… John C Klensin
- Re: [apps-discuss] APPSDIR review of draft-melnik… John C Klensin
- Re: [apps-discuss] APPSDIR review of draft-melnik… Alexey Melnikov
- Re: [apps-discuss] APPSDIR review of draft-melnik… Barry Leiba
- Re: [apps-discuss] APPSDIR review of draft-melnik… Barry Leiba
- Re: [apps-discuss] APPSDIR review of draft-melnik… S Moonesamy
- [apps-discuss] Review of draft-melnikov-smtp-prio… Dave Crocker
- [apps-discuss] Review of draft-melnikov-smtp-prio… Dave Crocker
- Re: [apps-discuss] APPSDIR review of draft-melnik… Alexey Melnikov
- Re: [apps-discuss] APPSDIR review of draft-melnik… S Moonesamy
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Alexey Melnikov
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Ned Freed
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Dave Crocker
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Alexey Melnikov
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Ned Freed
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Ned Freed
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Alexey Melnikov
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Dave Crocker
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Ned Freed
- Re: [apps-discuss] Review of draft-melnikov-smtp-… John C Klensin
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Pete Resnick
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Pete Resnick
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Claudio Allocchio
- Re: [apps-discuss] Review of draft-melnikov-smtp-… John C Klensin
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Dave Crocker
- Re: [apps-discuss] Review of draft-melnikov-smtp-… ken carlberg
- Re: [apps-discuss] Review of draft-melnikov-smtp-… ken carlberg
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Martin J. Dürst
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Alexey Melnikov
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Dave Crocker
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Ned Freed
- Re: [apps-discuss] Review of draft-melnikov-smtp-… Alexey Melnikov