Thursday, May 28, 2009

Protecting Your Reputation: How Blocking Spam Helps Your Mail Get Delivered

1 comments
Many years in the past, it was possible to set your Pobox account to not do any spam filtering, at all. As much time and effort as we put into having fantastic spam protection, we knew that some people would rather wade through heaps of junk mail to be sure they didn't miss anything.

Oh, how the times have changed! Today, more than 90% of the mail received is spam, and total spam volume grows by leaps and bounds every month. A few years ago, Pobox servers were blocked from delivering mail to Comcast, as they said we were delivering too much spam. After more analysis, the problem was traced to fewer than 3% of our customers forwarding to Comcast, who weren't using spam filtering. At that point, we no longer allowed accounts to turn off their spam filters, but we didn't require people currently going without to turn it on.

Well, Comcast represented the wave of the future. In the last several months, Yahoo, Gmail and more have been getting more aggressive with legitimate email providers, insisting that the total amount of spam coming through be throttled, or we would face the possibility of getting blocked. As such, existing accounts without spam filters have been being converted to our weak spam filters weekly.

Even weak spam filters do a lot! We estimate that they catch over 70% of all spam, and are wrong once in every 10,000 messages. But don't stop there -- trade up! Standard filters catch over 85% of all spam, and only misidentify 1 in 1350 messages. Aggressive filters (which is what I use) catch 95% of all spam, and, even better, bounce the messages from the super-accurate filters, which means I only review about a tenth of all the spam I receive (hundreds a day -- I've had my address since 1994!) And, if you send your email out through Pobox, we auto-build your Trusted Sender list for you, so you further reduce the chances of mail from one of your legitimate correspondents getting caught. (Our slider help page details all these numbers, too.)

Even better, when we released the new recommendations, we updated what we meant by them. The old Standard and Aggressive preset groups were static lists of specific conditions. The new Pobox recommendations have statistic-based averages to define aggressiveness, and conditions can be moved to new preset groups if their individual stats stray beyond their categories' acceptable numbers. This way, your preset group's level of protection stays consistent, even as the spamming world turns and wobbles.

Since we began the process of adding spam protection for unfiltered accounts, we've seen an almost 5% drop in the amount of spam forwarded on to other services. (In the anti-spam world, that is HUGE -- if 10% of your total mail volume from a source is spam, you start thinking about blocking.) As we finish this transition, we hope to reduce the amount of spam forwarded by another few percent.

For our affected customers, thank you for your understanding on this policy change. As the anti-spam universe moves, sometimes old policies have to be updated. For all our other customers, please feel free to contact us for recommendations if you feel like you're getting too much spam; we'd love to help you customize your settings!

-----

Fun fact: Pobox customers receive 39% less mail over the weekends, but only 9% less spam. I guess spammers love their work!

Wednesday, May 13, 2009

All About Email: Why can't I always send messages?

1 comments
You may sometimes find that you can't send mail from home computer or workstation at the office. Or, if you have a laptop, you can send messages from your office or an Internet cafe, but you can't send them from home. What causes these problems sending mail? In many cases, it's actually your ISP blocking you!

Bryan Allen, Pobox Operations Head, is back with a look into the spam-fighting world, and how it can spill into your world, blocking your attempts to send mail.
In the last decade, the spam industry has garnered most of its resources through compromising and controlling your standard home PC. Most home PCs are not kept up-to-date by their owners (software updates are frequently made to fix security problems), aren't secured from network connections in any way, and are thus easy targets for takeover by spammers.

A computer being controlled in this manner is called a zombie, or bot. When an individual or group controls enough machines (almost always without the owners' knowledge), you may hear it referred to as a "botnet."

In the old days of the Wild West Internet, nefarious computer enthusiasts would utilize botnets to stage attacks against servers they didn't like, or each other. Nowadays, spamming is big money (it is, in every sense, an industry), so that's what most bots end up being used for.

Sending spam in volume is extremely problematic for ISPs and other providers. Bandwidth costs money, other users trying to utilize the network resources being consumed by bots relaying spam are impacted, and the provider's reputation is hurt, so it is more difficult for them to send legitimate mail to other service providers.

To try to prevent spammers from abusing their networks, network administrators will block outbound mail to everywhere except their own outbound mail servers. This way, they can control the total amount of mail you're sending, and verify using their own antispam that your mail isn't spam -- before it leaves their network.

Mail is sent using the Simple Message Transfer Protocol (SMTP), which is run across TCP port 25. So when network admins block mail, they're actually dropping any outbound connections to port 25.

The Pobox SMTP servers require customers to authenticate using their Pobox account. We also run our own antispam suite against any mail going out through our servers, and we limit the number of messages that can be sent over a given period of time. We do all of this for the same reason ISPs do: To protect our IP reputation and ensure we can always send legitimate mail to other providers.

Given that we take care in relaying customer mail in this manner (and that there are a few antispam features we provide that require mail be relayed through our servers), we provide extra ports to work around ISPs blocking the default SMTP port out of their network. Those are defined in our help section.

So while it can be something of an inconvenience to have to do some extra configuration in your mail client to send mail through us, your ISP has some very good reasons for blocking that traffic at their border.

As an aside, the outbound SMTP block is very similar to another issue which was very common about a decade ago. There were a swath of vulnerabilities in the NetBIOS/CIFS/SMB services on the Windows platform, and to stop systems getting infected, most providers and institutions blocked inbound and outbound traffic to those services. Those ports are still blocked everywhere, as those services are still common vectors for attack. For instance, the Conficker worm, which has gotten a lot of press recently, uses them.

Once a vulnerability is identified, it is almost always going to be abused by someone as long as the platform or service continues to exist. For certain platforms, even N iterations and years down the line, problematic services will continue to be problematic. Nothing on the Internet ever dies. Spammer botnets and blocked outbound port 25 are here to stay.
Thanks, Bryan!

----

Network admins try to fight email crimes. Now the Detroit police is trying to use email to fight real-world crimes.

Tuesday, May 5, 2009

What if "This is spam" really isn't?

4 comments
Most ISPs and email providers will let you declare any piece of email spam, using a "This is spam." or "Junk Mail" button. Sometimes, this will delete the message; sometimes it will just flag or mark it. But what happens behind the scenes?

Spam is the bane of all email providers' existence, so most of them use a variety of methods to block mail. If that's the case, your spam report is probably processed to improve the accuracy of all those methods.

For content filters, your message is processed for:
  • URLs and email addresses - these can't change, or they won't work. They're very popular to filter on.
  • Checksums - the whole message is processed down into a short string. This works because many pieces of spam are identical, and are sent millions of times.
  • words and phrases - one of the more unreliable methods, but still used.
Most other methods look at the computer that sent the email to the ISP. They assume it's sending spam because:
  • it's infected with a virus.
  • it hasn't been properly secured, and someone is taking advantage of it.
  • it's owned by a spammer.
However, sometimes people click the "This is spam." button for things that aren't really spam. Like:
  • A mailing list they subscribed to, but don't want to be on anymore.
  • A message from a friend that they didn't want to receive.
  • As an alternative to the Delete button. (At least, that's the only thing I can assume, seeing some of the messages that people have submitted as spam. Maybe they're just from people they really hate.)
However, your ISP doesn't care why you clicked the Spam button. They always assume the worst. So, what do they do?

If the message was sent by a legitimate email provider, they'll let you know that an anonymous someone reported a message as spam, and show you the message so that you can take action to prevent these messages from being sent in the future.

If you don't prevent it from being sent in the future, or they don't know who you are, they treat you like a spammer. This means they treat all the mail coming from that computer like it's probably spam.

I know the conventional wisdom is, "Never try to unsubscribe from spam." That is true. However, it is not only OK, but you should try to unsubscribe from lists from legitimate companies and/or providers that you've subscribed to in the past, but don't want to receive anymore. Companies like the Gap, American Airlines, Expedia, etc., are more than happy to remove you from their email lists -- they don't want their messages treated like spam, so they want to make sure that only people who want to receive them do!

When your friend is CCing you on mail you don't want to receive anymore, just tell them, "I'm really trying to get my email under control. Would you mind leaving me off your joke of the hour messages from now on?" And if they won't? Well, delete is always available, too (unless you think your friend really has crossed over into the realm of spamming.) If you are a Pobox Plus or Mailstore user, you can use email filters to automatically send messages matching "Joke of the Hour" to the Spam section. And most email programs will also let you set up filters that will automatically move messages from your Inbox to another folder, or the trash.

What should you do if you accidentally report a legitimate message as spam? Most ISPs are looking for multiple reports, so don't worry too much about any one message. But, if you accidentally selected 20 messages in your mailbox, and instead of clicking Move to store them in your "all-time greatest messages" folder, you accidentally hit "This is spam." instead, well... you might want to shoot an email off to your ISP's customer support, to let them know that your friends aren't really spammers.

-----

The New York Times suggests a list of 10 questions we should all ask our mothers this Mother's Day. Let me suggest one more: "Mom, is there anything I can help you out with for your computer or cell phone?"