Skip to content

Broken MTAs

In this text we use the term FreeMail and @FreeMail provider broadly, it's an alias for all free e-mail providers - eg. @yahoo.com, @aol.com, etc

Over the last few years we've run into a common deliverability problem in which senders from some of our hosted domains would send to recipients at certain FreeMail providers with poor results. 

Received e-mails @FreeMailProvider would be moved directly to a "Junk" or "Spam" folder at the FreeMail destination (@yahoo.com), and in some cases be delayed for hours.  This has been most common with the destination yahoo.com (and the Yahoo affiliates - aol.com, verizon.com, sbcglobal.net, etc).  The sending domains we'll keep redacted for our customers privacy, but it does seem to happen more commonly from just a handful of domains with larger send counts.

Depending on seemingly random times, these otherwise normal e-mail communications would aggressively be sorted to the Junk folders at the FreeMail destination without any indication or explanation as to why this is happening.  This could be mid-communication in an e-mail reply conversation, or it could be starting randomly some weekday morning and going back to normal the following day.  The events appeared random, and unexplainable.

Initially, we were thinking the recipient provider is detecting something from our end as a misconfiguration or invalid message content, or possibly abuse if we were maybe sending too many sends per second.  We do have a fairly complex setup of mail routing to maximize deliverability and performance, so the initial assumption was this complexity was a bit abnormal for these providers and they didn't know exactly how to handle it properly, so they mis-identified and sorted to Junk.

We went to great lengths and burnt countless hours testing possible solutions.  We modified configurations settings, spam analysis, outbound signing, and routing, to help mitigate false identification and this mail-sorting-to-junk effect that appeared randomly.  We went so far as to signup our mail servers as registered mailing list servers through appropriate parties.  We became members of dozens of feedback loop notification lists for our hosted domains. Still no luck identifying the cause...

We went further with testing:

  • we modified reverse DNS settings a number of times with attempts to solve the situation

  • sent e-mail through proxy servers, throttled sending rates

  • allocated a pool of round-robin IP addresses on outbound sends just to see the effects

  • cloaked and replaced some of the header content in outgoing mail messages

  • probably a dozen or more other changes and tests...

Frustrated...

All of these tests were completed with a hope to come to some conclusion as to why these providers, at seemingly random times would send e-mails from our customers over to the "Junk" or "Spam" folder on the recipient location.

Well, it turns out the whole process is much simpler than we were considering, and has had zero to do with our infrastructure or setup.  Most recently, in voice discussion with Company(TM)'s "Small Business Elevated Technical Support", we've come to the conclusion that these are anti-spam techniques being employed by these companies which are conflating the definitions of UBE and Spam to detrimental effects for all of their users.

If you haven't had a minute yet, please read our Defining UBE article to fully understand what follows.

Hang in there, it's pretty simple...

Defending junk e-mail requires

A) a bunch of resources (spam identification systems, big databases, fast DNS servers and fast mail servers to parse e-mail on inbound, etc) to perform,

as well as

B) a reasonable amount of knowledge to not fall into the pit traps that can cause deliverability errors. 

One or both of these two items are lacking with these providers, as they are instead using

C) using user defined spam identification for a global weight against sending domains and disregarding the very important subjective analysis required to accurately identify spam

Example please...

When user1@mybusiness.example.com sends an e-mail to user2@freemail-provider.domain.com user2 has options provided to them to: "Mark as Junk", or possibly "Mark as Spam". 

It's assumed at this point by users of @freemail-provider.domain.com that when user2 performs this "mark", that any future e-mails of similar types of content (or possibly even sender address) would be moved to the Junk or Spam folders.  For most normal providers, or mail user agents, this assumption would be correct. 

But what freemail-provider.domain.com is doing is a bit more sinister.  They are matching "@mybusiness.example.com" and then weighting this as "likely spam".  Now, ALL e-mails sent from @mybusiness.example.com to ANY destination @freemail-provider.domain.com are being sent to the Junk folder for a period of time until this has been determined as not spam.

TLDR;

FreeMail providers have a problem in which they have to protect against spam for millions of clients. They cut some serious corners to provide spam protection to their users in a very broad way. This leads to the end users having to 'un-junk' lots of e-mail that is mis-identified. The end users aren't provided the education they need to use e-mail effectively, so they have a tendency to junk lots of e-mail they don't like to see.

Message for end users of FreeMail providers

Be very careful what you mark as "Junk" or "Spam".  It's possible you are doing more damage than good.

Your subscription to popular science, or home depot, or amazon, those aren't spam.  Don't mark these as spam you are doing all of society and the tech industry a disservice if you mark an e-mail that isn't spam as spam. Understand what Spam is firstly, and then selectively and carefully mark something as spam/junk only after you are certain. If you are uncertain, don't mark as spam.