SMTP Email - live server edits vs relay

[sbulen]

Description

We have seen a couple of instances where SMTP servers are performing edits when SMF connects to queue up an email for sending. I.e., it is validating the recipient in real time, while adding the entry to the mail queue on the SMTP server.

Some SMTP servers apparently allow this behavior, and do not act as a relay.

The far more common behavior is to queue it up, and later, upon sending, if it fails, it fails.

The problem with this behavior, though, is that the request fails, the entry remains in the mail queue. So, it gets processed again, the next time the SMF mail queue gets processed.

So... If the recipient does not exist, the email can stay in the SMF queue forever...

Additional information/references

From the forum: https://www.simplemachines.org/community/index.php?topic=586346.0

[JetzeMellema]

To provide a bit more context, there can be all kind of reasons why an SMTP server would not accept the message. In all cases the SMTP server will respond with an error code that consists of three fields, separated by a dot. Temporarily failures return an error code that starts with a 4, e.g.:

4.2.1.: Your message was temporarily deferred by the recipient server. This is usually a result of too many connections in a short timeframe or too many messages.
4.5.1.: Requested action aborted: This mail account has sent too many messages in a short amount of time. Please try later.

As the text indicates, these are typically issues where we should retry to send the email at a later time. Permanent errors return an error code that starts with a 5. Some samples:

5.1.5 Recipient address reserved by RFC 2606
5.7.0 Message Size Violation

These errors are permanent, hence the next time we will try to send we should expect it to fail again.

If this leads to modifying the SMF mail queue behavior, we probably should evaluate that error message to determine the appropriate action. In case of a 5xx error the message probably should be removed from the queue, or at least not be processed again. When the error is of the 4xx type we may want to try again later, but the question is if that would be immediately or maybe hold the message a few minutes before trying to send it again.

Hope this helps. I'm not a developer but I do like to think that I understand email.

[jdarwood007]

550 can also occur, seems from exim validating the recipient domain and it not existing.
I'm doing some testing for manipulating SMF's mail queue logic to handle this in a more graceful way.

@JetzeMellema,
If your getting this, your welcome to try the PR I am working on. This is a slow PR as I need to test it in production to ensure it works as expected. The code I am testing is not quite this (its more verbose for checking), but this should be the code I end up with hopefully.

[jdarwood007]

Incase anyone wants to know, we can add more errors. But the errors we don't log I think should be for remote server issues. Any local mail server issues should be logged, allowing the admin to know to correct that. The admin can't fix a remote mail server.
https://en.wikipedia.org/wiki/List_of_SMTP_server_return_codes

[m4z]

What about "user doesn't exist"? Wouldn't that be useful info for the admin? (Or is that a "local" error?)

[JetzeMellema]

But the errors we don't log I think should be for remote server issues.

Can you explain what you'd consider local or remote server issues?

[jdarwood007]

Errors received from the recipients mail server are remote. Errors from the sending server are local.

[JetzeMellema]

We are talking about error messages during the SMTP conversation, to stick with your terminology this would be local error messages. If the smarthost server accepts the email and fails to deliver the message to the recipients mail server, it will generate a non-delivery report message and send this email back to the from-address.

So in the context of this change, all error messages we receive during the SMTP conversation are local. We have no visibility in any delivery issues that happen on downstream servers.

Let me know if this doesn't make sense.

[jdarwood007]

If your mail server says something like you have exceeded your sending limits or DNS failure. That is more than likely a configuration issue that you can resolve. If the error message received is that the recipient doesn't exist, that is something the admin can't fix and shouldn't log. Some mail systems seem to be verifying before accepting the mail during the transmission.

Either way, the email will be deleted over time. Just we don't want to delete messages which may assist the admin in understanding that their sending server (I avoid using local as it could be a remote server not local to the web server) has a configuration issue that needs addressed.