Subject: [lemonade] SMTP-over-IMAP harmful (especially to Lemonade)

From: Mark Crispin <[email-masked]>
Date: Thu, 18 Aug 2005 15:59:05 -0700 (PDT)

The proposed SMTP-over-IMAP has the following flaws.  I recommend that, 
because of these flaws, the proposal receive no further discussion.

1. It is out of charter and accomplishes no purpose of Lemonade.

The proposal accomplishes none of the charter items of the Lemonade 
Working Group.  It is a degenerate form of the "Push" proposal, which did 
accomplish a charter item but was ultimately passed over in favor the 
"Pull" proposal.

Unlike "Push", the proposal does not address forward-without-download.

2. It is unnecessary and duplicates existing functionality.

The proposal does not add any new functionality.  Instead, it is a 
shortcut for a subset of existing posting cases; the case *if* and *only 
if* your posting credentials are the same as your access credentials.

The existing submission protocol is general and accomplishes everything 
that this proposal purports to accomplish.

3. It is based upon a myth.

It's been two decades since Phil Karn debunked the myth that "TCP 
connections are expensive."  A Google search on his work is highly 
recommended.

In spite of the fact that TCP *is* the multiplexing protocol, we continue 
to see people urging the addition of multiplexing options such as this 
into other protocols in order to avoid the mythical "expensive TCP 
connection."

4. It adds complexity.

For this mechanism to be used, it must be implemented in both client and 
server.  This adds code to both client and server.

The proponent of this proposal minimizes the added complexity to the 
protocol and to server implementations, and claims that "all that is 
needed" is a simple pass-through tunnel.

That claim ignores the fact that entering the tunnel is a state change. 
The underlying protocol engines are very different in IMAP and SMTP.  It 
is necessary to have well-defined mechanisms to enter and exit that state 
in both client and server.  A pass-through lacks such mechanisms.

This requires an encapsulation mechanism, which in turn requires a quoting 
mechanism and an escape mechanism.  This is quite a bit more than a 
pass-through.

It is also necessary for the IMAP server, acting as SMTP client, to know 
how to negotiate authentication (and probably also TLS) with the SMTP 
server.  A pass-through does not do this; this is a job for a proxy.

There are additional SMTP protocol details which the IMAP server must also 
understand.  The 421 response code comes immediately to mind.  It may also 
be necessary to deal with authentication-related fields in the MAIL FROM 
and RCPT TO commands.  This entails the IMAP server knowing the state of 
the SMTP protocol.  This also is proxy work.

A further level of complexity occurs in the question of how the IMAP 
server authenticates the user to the SMTP server.  It does not suffice to 
copy the credentials used by the IMAP client in authenticating to the IMAP 
server; that may work for userid/password authentication but it does not 
work with authentication protocols which authenticate to a service without 
giving that service the information needed to authenticate to a third 
party.

Presumably, this can be addressed by creating a trust relationship between 
the IMAP server and the SMTP server, so that if the IMAP server says that 
it is "fred" then the SMTP server believes it without making any 
authentication check on its own.  This involves a long and elaborate 
security considerations section in the specification for this proposal, 
which in turn will have to go to the Security AD (and perhaps some 
security working groups) for review.

5. It does not reduce complexity, unless...

The specifications state that the existing subsmission mechanism must be 
implemented.  In fact, use of the existing submission mechanism is 
required in order to accomplish certain Lemonade charter items, such as 
forward-without-download.

This proposal does not directly remove this requirement, and alleges that 
it is an "option."  However, the implication of this proposal is that it 
does remove this requirement, otherwise the proposal is pointless.

Specifically, why would a client implement all the code to do this 
(including the quoting and escape mechanisms required by IMAP) as a 
separate code path?  How does this benefit the client?

Then there is the question on how it benefits the server to implement all 
the code to do the encapsulation and proxy.  The only benefit that I can 
see is as a marketing tool, to ensnare customers who are stuck with a 
client that requires the SMTP-over-IMAP feature.

"Options" in Internet protocol add functionality that is not already 
there.  Over time, either they become de jure requirements in the base 
specification (only STARTTLS has done this), or become de facto 
requirements for a quality implementation (a few), or languish in 
well-deserved obscurity (most).

This proposal duplicates functionality that already exists, and is 
pointless unless it becomes a de-facto requirement.  It is therefore not 
an option.  The appropriate forum for this proposal is one which is 
considering a revision to the IMAP base protocol to add new requirements 
to that base.  That forum is not Lemonade.

6. It creates a long-term support burden.

SMTP and SUBMISSION are not static protocols.  They change over time. 
The idea of a pass-through is that the pass-through does not need to know 
about any of these changes; but in fact since we have to have a proxy then 
it is necessary to know about the changes.

For example, an SMTP extension for a multi-line client command is 
something that the IMAP server would need to know about, otherwise it 
would not analyze the SMTP command/response interaction correctly.

What's worse, it is possible that the existance of a proxy facility in 
another protocol would be broken by a proposed SMTP extension.  That, in 
turn, would impede the development of SMTP.  Cross-protocol dependencies 
are a Very Bad Thing.

7. Multiple code paths to do the same thing create multiple security 
risks.

If, as I suspect, this proposal languishes in well-deserved obscurity, the 
code paths in those implementation that support it are unlikely to be much 
exercised or regression tested.  Such code paths are a proven fertile 
breeding ground for security bugs.

8. It has extensive, and unexplored, security implications.

Just off the top of my head...:

I spoke above about the presumed creation of a trust relationship between 
IMAP server and SMTP server.

There is an additional trust relationship created between IMAP client and 
IMAP server (how does the client know that it's an honest message 
submission service?).

What if the IMAP server is actually a proxy to your real IMAP server?

I have little doubt that we can come up with more security issues that the 
proposal would have to address, and that's even before it gets to the 
Security AD and the security-related working groups.

This is all a lot of trouble and work for a proposal that is out of 
charter, and accomplishes none of the charter goals, for Lemonade.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Dave Cridland <[email-masked]>
Date: Fri, 19 Aug 2005 00:37:04 +0100

On Thu Aug 18 09:59:05 2005, Mark Crispin wrote:
> The proposed SMTP-over-IMAP has the following flaws.  I recommend 
> that, because of these flaws, the proposal receive no further 
> discussion.
> 
> 
Agreed. A couple of comments below:

> In spite of the fact that TCP *is* the multiplexing protocol, we 
> continue to see people urging the addition of multiplexing options 
> such as this into other protocols in order to avoid the mythical 
> "expensive TCP connection."

I'm not sure that's been advanced as a benefit here.

However, setting up a new TCP connection, starting TLS, and 
authenticating *does* have a relatively high cost at higher latencies.

This proposal, however, doesn't help with that aspect, and indeed 
adds a new round-trip to submission - one that cannot be pipelined, 
since its success not only changes state, but changes protocol. I 
have no idea if the intent is that you can change back, but I doubt 
many clients would find that easy to implement.

True, but:

> This requires an encapsulation mechanism, which in turn requires a 
> quoting mechanism and an escape mechanism.  This is quite a bit 
> more than a pass-through.

I think you're reading too much into the proposal. I believe the 
intent of the proposal is to literally switch protocol mid-flow. It's 
far worse than you previously thought.

Dave.

---

From: Mark Crispin <[email-masked]>
Date: Thu, 18 Aug 2005 17:16:27 -0700 (PDT)

On Fri, 19 Aug 2005, Dave Cridland wrote:
> However, setting up a new TCP connection, starting TLS, and authenticating 
> *does* have a relatively high cost at higher latencies.

True.

Hence the attraction of a new all-in-one protocol that is specifically 
oriented towards (1) the mobile device class in question and (2) its 
server being a client to IMAP and SUBMISSION servers.

Although this may surprise some people, I'm neutral on that idea.  The 
working group haven't chosen to go that route, so the question hasn't come 
up.  Although that idea certainly its problems(!), done right the result 
could be preferable to excessive baggage in IMAP and SMTP.

To be honest, that's what I would want if I was developing email for a 
mobile phone.  I'd look with horror at the chattiness of SMTP and 
(especially) IMAP, and want something stripped down.  I'd probably do a 
good deal of the formatting work in the server and not in the phone; 
building menus, etc.

You can see where this is going.  Gee, I'd think, do I really need a new 
protocol?  No, I can do a webmail, then the phone doesn't need any code 
since it already has a perfectly good browser.  It also simplifies my 
software distribution problem; I can upgrade every phone instantly...

> This proposal, however, doesn't help with that aspect, and indeed adds a new 
> round-trip to submission - one that cannot be pipelined, since its success 
> not only changes state, but changes protocol. I have no idea if the intent is 
> that you can change back, but I doubt many clients would find that easy to 
> implement.

As presented, there is no change back in the proposal.  That is likely to 
be the first thing that would be amended if this proposal got serious 
consideration.  As you point out, without a change back there's an extra 
non-pipelineable RTT.

Your counter-proposal for a capability that advertises a usable submission 
server name and port is much better.  I think that it's silly -- there are 
other means to obtain configuration -- but it's not harmful.  I'd abstain
on it.

> I think you're reading too much into the proposal. I believe the intent of 
> the proposal is to literally switch protocol mid-flow. It's far worse than 
> you previously thought.

To be sure; but you can't assume that problem would not be amended when 
it's raised as a show-stopper.

Put another way, it's necessary to consider the obvious evolution of the 
proposal if it is to have a snowball's chance in hell of coming to 
fruition.

I glossed over this, but the only true benefit of the proposal (which your 
counter-proposal does not address) is as a way to get around SMTP port 
blocking.  That problem is solved by the SUBMISSION port (port 587).

Even without SUBMISSION, using IMAP as a way to get around SMTP port 
blocking is another step down the slippery slope of using port 80 
tunnelling for all services to get around port blocking.  That is 
tantamount to fighting ill-considered countermeasure with ill-considered 
countermeasure.

-- Mark --