Linode Forum
https://forum.linode.com/

Anti-SPAM Revisited
https://forum.linode.com/viewtopic.php?f=11&t=2122		 Page 1 of 2
Author:  schmichael [ Thu Feb 23, 2006 11:35 am ]
Post subject:  Anti-SPAM Revisited
I know there are a number of old forum discussions about preferred anti-spam measures, but I thought I'd just see what the current anti-spam flavor of the month is.

Currently I use Postfix 2.1.5 with ClamAV (via clamsmtp) and the only spam blocking I have is two RBL services: sbl-xbl.spamhaus.org & bl.spamcop.net.

So far this has gotten 100% of e-mail viruses, but the spam blocking has become less and less effective. I'm running a Linode 160, but as I fear I'm running low on resources - so I haven't tried a big bulky anti-spam program like SpamAssasin.

What does everyone else use? Should I simply add more RBL services?

Is there a relatively light-weight and safe (few false-positives) anti-spam program I could plug into Postfix on a Debian Sarge Server?

Basically: What's the spam blocking flavor of the month?
Author:  mikegrb [ Thu Feb 23, 2006 11:47 am ]
Post subject: 
I use cbl.abuseat.org, bl.spamcop.net, dynamic.spamlists.tqmcube.com and a return code of 127.0.0.10 from dnsbl.sorbs.net. If you want to accept direct to MX mail from dynamic IPs remove the last two. cbl.abuseat.org would be a good list to add to your current checks either way.

Another thing to concider is greylisting. This gives a message a temp failure the first time delivery is attempted and will let it pass on reattempts after a configurable delay (eg 10, 15, 20 minutes or so). Many spam trojans on infected machines will not reattempt at all. Those that do will have given the blocking lists you use an extra period of time to list their IPs. This is as easy to implement in your setup as "apt-get install postgrey".

Lastly, configure your mailsever to be more strict about what it accepts for a HELO value and in enforcement of other RFC standards. For example quite a few pieces of spam get rejected by mail server for failing to wait for the SMTP banner before saying HELO.
Author:  drware [ Thu Feb 23, 2006 12:43 pm ]
Post subject:  TMDA
I know a lot of people hate it, because they feel that it is to restrictive, and breaks the free flowing nature of email, but I have been using TMDA (Tagged Message Delivery Agent) for several years with great results. TMDA uses whitelists. When I send an email to someone they are added to my whitelist and can send me email unhindered. If someone sends me email out of the blue, they get a reply asking for confirmation. When they confirm they are added to the whitelist and the original message is released.

I can also create dated email address with TMDA on the fly which only work for several weeks, then require confirmation. These addresses work well say when I am registering for a website and want to receive the password, but not be bothered later with marketing crap.

TMDA may not be for everyone, but it works well for me and the few users that I support.
Author:  mikegrb [ Thu Feb 23, 2006 12:58 pm ]
Post subject:  Re: TMDA
drware wrote:
I know a lot of people hate it, because they feel that it is to restrictive, and breaks the free flowing nature of email, but I have been using TMDA (Tagged Message Delivery Agent) for several years with great results. TMDA uses whitelists. When I send an email to someone they are added to my whitelist and can send me email unhindered. If someone sends me email out of the blue, they get a reply asking for confirmation. When they confirm they are added to the whitelist and the original message is released.

I can also create dated email address with TMDA on the fly which only work for several weeks, then require confirmation. These addresses work well say when I am registering for a website and want to receive the password, but not be bothered later with marketing crap.

TMDA may not be for everyone, but it works well for me and the few users that I support.


I would suggest rereading the Linode Terms of Service and Acceptable Use Policy. Since most spam uses forged from addresses, and since some of these addresses do exist, you are sending unsolicited bulk mail. We treat reports about Challenge/Response systems just as we treat any other spam report, account termination is not out of the question.
Author:  schmichael [ Thu Feb 23, 2006 1:31 pm ]
Post subject: 
I added cbl.abuseat.org after my other two RBLs. No hits to that one yet, but its new. :)

I installed the postgrey package, but I haven't enabled it yet. I'm extremely conservative with my mail server because I have some clients that won't put up with e-mail not working (or bounces or whatever).

Should I build a pretty comprehensive whitelist_clients.local file for postgrey?

Should I enable auto-whitelisting? It seems like a good idea to me, but it doesn't seem to be enabled by default.

(I've always thought TMDA was excellent in theory, but the inability to determine an e-mail's true sender with 100% accuracy renders it not just useless, but harmful. I'd disable it unless you manually monitor a lot of its activities which kind of defeats its purpose anyway.)
Author:  mikegrb [ Thu Feb 23, 2006 2:11 pm ]
Post subject: 
untitled9 wrote:
I added cbl.abuseat.org after my other two RBLs. No hits to that one yet, but its new. :)


Yeah, I have it checked first in my case, it will have entries from your other dnsbls but in my tests seemed to catch stuff others miss.

untitled9 wrote:
I installed the postgrey package, but I haven't enabled it yet. I'm extremely conservative with my mail server because I have some clients that won't put up with e-mail not working (or bounces or whatever).

Should I build a pretty comprehensive whitelist_clients.local file for postgrey?

Should I enable auto-whitelisting? It seems like a good idea to me, but it doesn't seem to be enabled by default.


I use greylistd with exim personally. It has a feature that acts what I assume auto-whitelisting is in postgrey enabled by default. I think it is worthwhile, especially if old data is removed from the whitelist when it hasn't been used in some amount of time.

untitled9 wrote:
(I've always thought TMDA was excellent in theory, but the inability to determine an e-mail's true sender with 100% accuracy renders it not just useless, but harmful. I'd disable it unless you manually monitor a lot of its activities which kind of defeats its purpose anyway.)


Right, I think comunism/socialism is great in theory... ;-)
Author:  sednet [ Thu Feb 23, 2006 5:46 pm ]
Post subject:  Re: Anti-SPAM Revisited
untitled9 wrote:
Basically: What's the spam blocking flavor of the month?


sbl-xbl.spamhaus.org on the mail server. Just that one DNSBL seems to catch about 60% of my spam.
Plus SpamAssassin with a few custom rules from rulesemporium.com.
That catches about 60% of whats left.

SpamAssassin is a resource hog, but if you don't get thousands of mails a day it should not be a problem.

I'd stay well clear of TMDA, it's a solution thats as bad as the problem.
Author:  schmichael [ Thu Feb 23, 2006 6:18 pm ]
Post subject: 
Quote:
SpamAssassin is a resource hog, but if you don't get thousands of mails a day it should not be a problem.


And therein lies the problem. My server handles over 2,000 e-mails a day, over 60% of which are spam along with about a dozen relatively low traffic web sites. (So I guess SpamAssassin would only see < 1,000/day.)

Spam bombs and the occassional 5mb e-mail to 30 co-workers bring the server to a crawl while ClamAV scans everything. I'm afraid adding another resource hog that scans every e-mail onto the chain will really bring down the server.

Should I just quit whining and install SpamAssassin?
Author:  sednet [ Fri Feb 24, 2006 12:12 am ]
Post subject: 
untitled9 wrote:
Quote:
SpamAssassin is a resource hog, but if you don't get thousands of mails a day it should not be a problem.


And therein lies the problem. My server handles over 2,000 e-mails a day, over 60% of which are spam along with about a dozen relatively low traffic web sites. (So I guess SpamAssassin would only see < 1,000/day.)

Spam bombs and the occassional 5mb e-mail to 30 co-workers bring the server to a crawl while ClamAV scans everything. I'm afraid adding another resource hog that scans every e-mail onto the chain will really bring down the server.

Should I just quit whining and install SpamAssassin?


A very rough estimate of how much mail a linode 80 could handle:
Mail that gets rejected by the MTA due to a DNSBL wastes almost no time. I'm ignoring those.
A big HTML mail thats comes in, gets scanned by spamassassin, and gets forwared externally takes a total of 13 seconds end to end to process. Lets say 13 seconds is average for all mail.
Thats 4.6 mails a minute, 276 an hour, 6624 a day.

I have never run ClamAV so don't have any idea how long that takes so I'm guessing it doubles the processing time. I don't think it does anything on mails without attachments so lets say it averages out at another 13 seconds
per mail. Thats 3312 mails a day.

I'm not taking into account the fact that you can process two mails at once with very little slowdown, or the fact that your mail isn't going to come in evenly spaced. The bulk of it will likely come in during the daytime. It's a very rough estimate, but your linode should be able to handle running spamassassin as well.

The only real way to tell is to try it and see what happens. You can always turn spamassassin off if your linode starts swap thrashing.
Author:  pclissold [ Sat Feb 25, 2006 7:08 am ]
Post subject:  Re: TMDA
mikegrb wrote:
I would suggest rereading the Linode Terms of Service and Acceptable Use Policy. Since most spam uses forged from addresses, and since some of these addresses do exist, you are sending unsolicited bulk mail. We treat reports about Challenge/Response systems just as we treat any other spam report, account termination is not out of the question.


Your policy, or rather this application of it, seems a little harsh. What happens about out-of-office autoresponders replying to spam that leaks through the filters? They seem to fit your criteria for ubm, yet they are a legitimate business practice that has been screwed up by spammers. Half the crap in my postmaster inbox is the result of this sort of thing.
Author:  mikegrb [ Sat Feb 25, 2006 7:12 am ]
Post subject:  Re: TMDA
pclissold wrote:
Your policy, or rather this application of it, seems a little harsh. What happens about out-of-office autoresponders replying to spam that leaks through the filters? They seem to fit your criteria for ubm, yet they are a legitimate business practice that has been screwed up by spammers. Half the crap in my postmaster inbox is the result of this sort of thing.


Out of office autoresponders aren't purposefully exploited by spammers like challenge response systems are though. That said, if we received a legitimate complaint about an autoresponder, we would suggest that it be disabled.
Author:  pclissold [ Sat Feb 25, 2006 7:22 am ]
Post subject:  Re: TMDA
mikegrb wrote:
Out of office autoresponders aren't purposefully exploited by spammers like challenge response systems are though.


Point taken.

Note to self: Think before posting :? .
Author:  tony [ Wed Mar 01, 2006 4:52 am ]
Post subject:  Re: Anti-SPAM Revisited
untitled9 wrote:
What does everyone else use?


A LOT of procmail and perl and shellscripts centered around a several years old version of bogofilter.

(I use procmail on outgoing e-mails also.)

Due to the spamtraps and all the automatic whitelisting it's been working way beyond great, for a number of years.

Now I'm moving/porting it all to my server here, so I've started to rewrite it all; starting with adding a cpl of rbls (relays.ordb.org sbl-xbl.spamhaus.org) at smtp-level.
Author:  schmichael [ Wed Mar 01, 2006 10:55 am ]
Post subject: 
Thanks to everyone for the great responses. cbl.abuseat.org is now catching a respectable amount of spam every day that the other 2 DNSBLs miss, but its obvious that DNSBLs can only do so much.

I have postgrey installed and configured but not enabled because I really don't like the idea of breaking SMTP to block spam. I know technically it shouldn't break anything, but a quick glance at the long list of exceptions in postgrey's whitelist was enough to make me nervous.

I think its time to bite the bullet and install SpamAssassin. If it starts thrashing my swap too much, I guess I'll just have to upgrade my Linode 160. I've also been thinking about moving our non-critical web sites to a super cheap host like GoDaddy, but I'm not sure that would free up very many resources.

Thanks again for your help!
Author:  tony [ Wed Mar 01, 2006 1:44 pm ]
Post subject: 
1. Do as much rejecting as possible at SMTP-level; using DNSBLs is probably the easiest way to do this with most mailservers. Be very careful about what DNSBLs you use, you rather want 10'000 spam extra to pass than 1 ham get rejected here. (Remember, you don't even have a spam-folder where you can find the ones that got rejected.)

2. Whitelist as much as possible; check the outgoing e-mails and automagically whitelist every person that someone at your company writes to. Don't forget to also keep track of all message-ids, that way you can also whitelist all replies to all sent e-mails even if they don't come from the address it was sent to; you also whitelist e-mails (e-mailaddresses and/or message-ids) that are replies to sent e-mails... that way you whitelist whole threads on mailinglists even if you don't whitelist the mailinglist itself.

3. Blacklist as much as possible... Here you probably want to use something like procmail, making it possible for you to block exactly the kinds of e-mails that you _NEVER_ get as ham, like e-mails written in chinese and things like that.

4. Use the white- and blacklisted e-mails to train whatever bayesian(ish) filter it is that you're using, let it be something like bogofilter or spamassassin.

5. Here's where you take what's left and let the software "guess" if it's spam or not. Just don't forget that if the mail is for a whole company there are some people into things which others are not (marketingpeople getting lots of flashy HTML-emails, techguys might only get HTML in spam)... So if you're filter of choice is bayesian(ish)oriented you problably want individual databases, or at least one for each group/department.


Just my 2 c...
Page 1 of 2 All times are UTC-04:00
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/