Subject: [lemonade] SMTP-over-IMAP harmful (especially to Lemonade)

From: Marc Perkel <[email-masked]>
Date: Thu, 18 Aug 2005 16:26:09 -0700

I don't know how to reply to this because he's making an argument 
against what he thinks I proposed and not about what I proposed. Mark 
clearly doesn't understand what I'm talking about. I suggest Mark that 
you calm down and just read what I proposed and then if you don't like 
it - fine. But this argument here doesn't seem in any way related to 
what I suggested.

Mark Crispin wrote:

> The proposed SMTP-over-IMAP has the following flaws.  I recommend 
> that, because of these flaws, the proposal receive no further discussion.
>
>
> 1. It is out of charter and accomplishes no purpose of Lemonade.
>
> The proposal accomplishes none of the charter items of the Lemonade 
> Working Group.  It is a degenerate form of the "Push" proposal, which 
> did accomplish a charter item but was ultimately passed over in favor 
> the "Pull" proposal.
>
> Unlike "Push", the proposal does not address forward-without-download.
>
>
> 2. It is unnecessary and duplicates existing functionality.
>
> The proposal does not add any new functionality.  Instead, it is a 
> shortcut for a subset of existing posting cases; the case *if* and 
> *only if* your posting credentials are the same as your access 
> credentials.
>
> The existing submission protocol is general and accomplishes 
> everything that this proposal purports to accomplish.
>
>
> 3. It is based upon a myth.
>
> It's been two decades since Phil Karn debunked the myth that "TCP 
> connections are expensive."  A Google search on his work is highly 
> recommended.
>
> In spite of the fact that TCP *is* the multiplexing protocol, we 
> continue to see people urging the addition of multiplexing options 
> such as this into other protocols in order to avoid the mythical 
> "expensive TCP connection."
>
>
> 4. It adds complexity.
>
> For this mechanism to be used, it must be implemented in both client 
> and server.  This adds code to both client and server.
>
> The proponent of this proposal minimizes the added complexity to the 
> protocol and to server implementations, and claims that "all that is 
> needed" is a simple pass-through tunnel.
>
> That claim ignores the fact that entering the tunnel is a state 
> change. The underlying protocol engines are very different in IMAP and 
> SMTP.  It is necessary to have well-defined mechanisms to enter and 
> exit that state in both client and server.  A pass-through lacks such 
> mechanisms.
>
> This requires an encapsulation mechanism, which in turn requires a 
> quoting mechanism and an escape mechanism.  This is quite a bit more 
> than a pass-through.
>
> It is also necessary for the IMAP server, acting as SMTP client, to 
> know how to negotiate authentication (and probably also TLS) with the 
> SMTP server.  A pass-through does not do this; this is a job for a proxy.
>
> There are additional SMTP protocol details which the IMAP server must 
> also understand.  The 421 response code comes immediately to mind.  It 
> may also be necessary to deal with authentication-related fields in 
> the MAIL FROM and RCPT TO commands.  This entails the IMAP server 
> knowing the state of the SMTP protocol.  This also is proxy work.
>
> A further level of complexity occurs in the question of how the IMAP 
> server authenticates the user to the SMTP server.  It does not suffice 
> to copy the credentials used by the IMAP client in authenticating to 
> the IMAP server; that may work for userid/password authentication but 
> it does not work with authentication protocols which authenticate to a 
> service without giving that service the information needed to 
> authenticate to a third party.
>
> Presumably, this can be addressed by creating a trust relationship 
> between the IMAP server and the SMTP server, so that if the IMAP 
> server says that it is "fred" then the SMTP server believes it without 
> making any authentication check on its own.  This involves a long and 
> elaborate security considerations section in the specification for 
> this proposal, which in turn will have to go to the Security AD (and 
> perhaps some security working groups) for review.
>
>
> 5. It does not reduce complexity, unless...
>
> The specifications state that the existing subsmission mechanism must 
> be implemented.  In fact, use of the existing submission mechanism is 
> required in order to accomplish certain Lemonade charter items, such 
> as forward-without-download.
>
> This proposal does not directly remove this requirement, and alleges 
> that it is an "option."  However, the implication of this proposal is 
> that it does remove this requirement, otherwise the proposal is 
> pointless.
>
> Specifically, why would a client implement all the code to do this 
> (including the quoting and escape mechanisms required by IMAP) as a 
> separate code path?  How does this benefit the client?
>
> Then there is the question on how it benefits the server to implement 
> all the code to do the encapsulation and proxy.  The only benefit that 
> I can see is as a marketing tool, to ensnare customers who are stuck 
> with a client that requires the SMTP-over-IMAP feature.
>
> "Options" in Internet protocol add functionality that is not already 
> there.  Over time, either they become de jure requirements in the base 
> specification (only STARTTLS has done this), or become de facto 
> requirements for a quality implementation (a few), or languish in 
> well-deserved obscurity (most).
>
> This proposal duplicates functionality that already exists, and is 
> pointless unless it becomes a de-facto requirement.  It is therefore 
> not an option.  The appropriate forum for this proposal is one which 
> is considering a revision to the IMAP base protocol to add new 
> requirements to that base.  That forum is not Lemonade.
>
>
> 6. It creates a long-term support burden.
>
> SMTP and SUBMISSION are not static protocols.  They change over time. 
> The idea of a pass-through is that the pass-through does not need to 
> know about any of these changes; but in fact since we have to have a 
> proxy then it is necessary to know about the changes.
>
> For example, an SMTP extension for a multi-line client command is 
> something that the IMAP server would need to know about, otherwise it 
> would not analyze the SMTP command/response interaction correctly.
>
> What's worse, it is possible that the existance of a proxy facility in 
> another protocol would be broken by a proposed SMTP extension.  That, 
> in turn, would impede the development of SMTP.  Cross-protocol 
> dependencies are a Very Bad Thing.
>
>
> 7. Multiple code paths to do the same thing create multiple security 
> risks.
>
> If, as I suspect, this proposal languishes in well-deserved obscurity, 
> the code paths in those implementation that support it are unlikely to 
> be much exercised or regression tested.  Such code paths are a proven 
> fertile breeding ground for security bugs.
>
>
> 8. It has extensive, and unexplored, security implications.
>
> Just off the top of my head...:
>
> I spoke above about the presumed creation of a trust relationship 
> between IMAP server and SMTP server.
>
> There is an additional trust relationship created between IMAP client 
> and IMAP server (how does the client know that it's an honest message 
> submission service?).
>
> What if the IMAP server is actually a proxy to your real IMAP server?
>
> I have little doubt that we can come up with more security issues that 
> the proposal would have to address, and that's even before it gets to 
> the Security AD and the security-related working groups.
>
>
> This is all a lot of trouble and work for a proposal that is out of 
> charter, and accomplishes none of the charter goals, for Lemonade.
>
> -- Mark --
>
> http://staff.washington.edu/mrc
> Science does not emerge from voting, party politics, or public debate.
> Si vis pacem, para bellum.
>
> _______________________________________________
> lemonade mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/lemonade
>

-- 
Marc Perkel - [email-masked]

Spam Filter: http://www.junkemailfilter.com
   My Blog: http://marc.perkel.com

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Tony Hansen <[email-masked]>
Date: Fri, 19 Aug 2005 08:03:35 -0400

So, here's one: if all you're doing is "tunnelling SMTP", how do you
know when to stop? There are several things in the SMTP protocol that
can cause the stream to stop, including the QUIT command or one of the
sides closing the connection. So you either have to add intelligence to
know about the SMTP protocol, base everything on timing out
"eventually", or add some sort of protocol on top of SMTP.

If you're looking for QUIT commands, you ARE building what Mark calls a
"proxy". The problem with looking *only* for QUIT commands on the
sending side is that you can be fooled by data that's sent as part of
AUTH exchanges or the DATA command, or that's obscured by the protocol
having switched to TLS. Any attempt to get around these problems by
adding more intelligence about the SMTP protocol is walking further down
the path of building a "proxy". I'll defer to all of Mark's other
arguments against building proxies.

Obviously the sending side can't close the connection, because that
would close the IMAP connection as well. So the SMTP sender can't use
that way of signalling the end.

One problem with noticing a broken subconnection on the server side is
knowing what to do with the data that's sent on the main connection
around the time of the broken subconnection: how do you sync back up so
that you know that subsequent data is once again part of the IMAP stream
and not supposed to be shunted off to the SMTP subconnection. The
workaround here is to add time outs. But how big a timeout is big
enough? 1 minute? no. 2 minutes? no. These are both within normal
timeout limits of SMTP.

If you say that you need to add some sort of meta-protocol that
encapsulates the SMTP protocol, **ouch**.

"Simply tunnelling" is not so simple after all, is it?

	Tony Hansen
	[email-masked]

---

From: Marc Perkel <[email-masked]>
Date: Fri, 19 Aug 2005 08:00:53 -0700

Tony Hansen wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>know when to stop? 
You stop when either end closes the connection. The IMAP layer doesn't 
need to monitor the traffic.

---

From: Dave Cridland <[email-masked]>
Date: Fri, 19 Aug 2005 18:46:30 +0100

On Fri Aug 19 02:00:53 2005, Marc Perkel wrote:
> You stop when either end closes the connection. The IMAP layer 
> doesn't need to monitor the traffic.

For any kind of useful efficiency, yes, it does.

Otherwise this could only be used with a fresh connection each time, 
and the additional round trip involved is especially wasteful with 
mobile devices. As I've stated before, that round-trip cannot be 
pipelined. Submission connections are sufficiently rarely used that 
they tend to be used for a single message submission. I'm unaware of 
a client which does not.

So in order to save yourself the effort of using one of the several 
existing mechanisms for client configuration, you've added a 
round-trip on every send. I don't see this as a good trade-off.

On the other hand, you have to pass credentials from the IMAP server 
to the submission server securely, which will give the security folk 
heart failure. You keep proposing that this can be skipped, without 
appreciating that mechanisms like BURL rely on the submission server 
knowing these credentials. So you've also added a huge amount of 
complexity at the security level. Again, not a good trade-off.

In short, you've traded a one-time problem in for a set of serious 
ongoing issues. I can clearly see the benefit, I just can't see why 
you think it would be worthwhile.

Dave.

---

From: Marc Perkel <[email-masked]>
Date: Fri, 19 Aug 2005 11:16:38 -0700

> For any kind of useful efficiency, yes, it does.

No - it doesn't.

> they tend to be used for a single message submission. I'm unaware of a 
> client which does not.

If you read what I proposed - yes - you would establish a separate 
connection to send the message. If you use SMTP you have to establish a 
separate connection to send email so there is not overhead difference.

> complexity at the security level. Again, not a good trade-off.

Again - I've said this over and over. Because IMAP has authenticated you 
there is no need to pass credentials on. The idea is to eliminate the 
need to authenticate to SMTP because you have authenticated to IMAP and 
therefore you are already trusted.

> you think it would be worthwhile.
Dave, It would be helpful if you read what I proposed before you 
criticize it.

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 19 Aug 2005 14:23:57 -0400

Hi Marc,

> therefore you are already trusted.

I think that is way to general a statement. For example. I can easily 
imagine a scheme in the mobile environment where users get charged for each 
message they send say over a certain limit (e.g. 100 free emails a month, 
anything more you pay $0.05 per message). In that case it is important for 
the credentials of the user to be passed to the SMTP server for 
accounting/billing purposes.

-- 
Cyrus Daboo

---

From: Marc Perkel <[email-masked]>
Date: Fri, 19 Aug 2005 11:34:38 -0700

Cyrus Daboo wrote:

> imagine a scheme in the mobile environment where users get charged for 
> each message they send say over a certain limit (e.g. 100 free emails 
> a month, anything more you pay $0.05 per message). In that case it is 
> important for the credentials of the user to be passed to the SMTP 
> server for accounting/billing purposes.
>
Does the billing company also control the IMAP connection? If it does 
then you can use data from the IMAP server for billing or insert billing 
code in the process of handing the connection from IMAP to smtp.

If the user is connecting directly with their own IMAP server then you 
would have a problem billing for sent email. As a user I consider that a 
feature, not a problem. ;)

This protocal isn't about just cellphones and blackberrys is it? I'm 
thinking in terms of end users with real computers.

---

From: Dave Cridland <[email-masked]>
Date: Fri, 19 Aug 2005 19:49:55 +0100

On Fri Aug 19 05:16:38 2005, Marc Perkel wrote:
>> Otherwise this could only be used with a fresh connection each 
>> time, and the additional round trip involved is especially 
>> wasteful with mobile devices. As I've stated before, that 
>> round-trip cannot be pipelined. Submission connections are 
>> sufficiently rarely used that they tend to be used for a single 
>> message submission. I'm unaware of a client which does not.
> 
> connection to send the message. If you use SMTP you have to 
> establish a separate connection to send email so there is not 
> overhead difference.
> 
> 
Please read the paragraph below:

>> So in order to save yourself the effort of using one of the 
>> several existing mechanisms for client configuration, you've added 
>> a round-trip on every send. I don't see this as a good trade-off.
>> 
>> 
That is a significant overhead diference to a client on almost any 
connection, but most especially on a high latency mobile connection.

>> On the other hand, you have to pass credentials from the IMAP 
>> server to the submission server securely, which will give the 
>> security folk heart failure. You keep proposing that this can be 
>> skipped, without appreciating that mechanisms like BURL rely on 
>> the submission server knowing these credentials. So you've also 
>> added a huge amount of complexity at the security level. Again, 
>> not a good trade-off.
> 
> Again - I've said this over and over. Because IMAP has 
> authenticated you there is no need to pass credentials on. The idea 
> is to eliminate the need to authenticate to SMTP because you have 
> authenticated to IMAP and therefore you are already trusted.
> 
> 
And I've drawn your attention to BURL several times. The submission 
server requires the credentials in order for BURL to operate.

>> ongoing issues. I can clearly see the benefit, I just can't see 
>> why you think it would be worthwhile.
>> 
>> 

Likewise.

Dave.

---

From: Tony Hansen <[email-masked]>
Date: Fri, 19 Aug 2005 23:25:04 -0400

OK, I see the disconnect. You want to replace

	open connection to submission port
	EHLO
	AUTH
	rest of SMTP protocol

with

	open connection to imap port
	CAPABILITIES
	AUTHENTICATE
	START SMTP TUNNEL
	SMTP protocol starting at EHLO, but evidently not needing to do
		another AUTH

The gain is you don't need to configure another authentication string
and the location of the submission server.

But you still need to write your client software to handle the case
where the imap server doesn't support the START-SMTP-TUNNEL extension,
and be able to configure for that situation.

I'm sorry, but the gain doesn't appear to me to be a big enough win.

	Tony Hansen
	[email-masked]