Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic
Author Message
PostPosted: Mon Jan 04, 2010 8:28 pm 
Offline
Senior Member

Joined: Wed May 13, 2009 1:32 pm
Posts: 737
Location: Italy
Hi,
I followed this guide
http://wiki.centos.org/HowTos/postfix_restrictions
to configure my spam filter.

Anyway I have the feel that there are some false positive that never reach my spam folder.
Is there a way to test for false positive?

Some site that send us dozens of email and let us checks how many email arrived on our inbox?

Please help.

/etc/postfix/main.cf
Quote:
# HELO restrictions:
smtpd_delay_reject = yes
smtpd_helo_required = yes
smtpd_helo_restrictions =
permit_mynetworks,
permit_sasl_authenticated
reject_non_fqdn_helo_hostname,
reject_invalid_helo_hostname,
permit
# Sender restrictions:
smtpd_sender_restrictions =
permit_mynetworks,
permit_sasl_authenticated
reject_non_fqdn_sender,
reject_unknown_sender_domain,
permit
# Recipient restrictions:
smtpd_recipient_restrictions =
permit_mynetworks,
permit_sasl_authenticated
reject_unauth_pipelining,
reject_non_fqdn_recipient,
reject_unknown_recipient_domain,
reject_unauth_destination,
check_sender_access
hash:/etc/postfix/sender_access,
reject_rbl_client zen.spamhaus.org,
reject_rbl_client bl.spamcop.net,
# check_policy_service unix:postgrey/socket,
permit


master.cf
Quote:
amavisfeed unix - - n - 2 lmtp
-o lmtp_data_done_timeout=1200
-o lmtp_send_xforward_command=yes
-o disable_dns_lookups=yes
-o max_use=20
127.0.0.1:10025 inet n - n - - smtpd
-o content_filter=
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=reject_unauth_pipelining
-o smtpd_end_of_data_restrictions=
-o smtpd_restriction_classes=
-o mynetworks=127.0.0.0/8
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters,no_address_mappings
-o local_header_rewrite_clients=
-o smtpd_milters=
-o local_recipient_maps=
-o relay_recipient_maps=
smtp inet n - n - - smtpd




Last edited by sblantipodi on Mon Feb 15, 2010 1:12 pm, edited 1 time in total.

Top
   
 Post subject:
PostPosted: Tue Jan 05, 2010 1:40 am 
Offline
Senior Member
User avatar

Joined: Sun Feb 08, 2004 7:18 pm
Posts: 562
Location: Austin
I don't have an answer to your specific question, but I would like to recommend against having a spam folder.

In my experience it's far better to simply reject borderline mail at SMTP time. That way, the sender gets an immediate notice that it didn't go through and why. If such mail instead went into a spam folder, it could languish unnoticed for days or weeks or forever, while the sender believes it got through. Such silent failure is the worst-case scenario.


Top
   
 Post subject:
PostPosted: Tue Jan 05, 2010 6:34 am 
Offline
Senior Member

Joined: Wed May 13, 2009 1:32 pm
Posts: 737
Location: Italy
Xan wrote:
I don't have an answer to your specific question, but I would like to recommend against having a spam folder.

In my experience it's far better to simply reject borderline mail at SMTP time. That way, the sender gets an immediate notice that it didn't go through and why. If such mail instead went into a spam folder, it could languish unnoticed for days or weeks or forever, while the sender believes it got through. Such silent failure is the worst-case scenario.


Thanks for your reply.
My problem is that I think that some mail doesn't reach the spam folder at all because the mail is discarded before they reach the folder.


Top
   
 Post subject:
PostPosted: Tue Jan 05, 2010 9:37 am 
Offline
Senior Member
User avatar

Joined: Mon Dec 10, 2007 4:30 pm
Posts: 341
Website: http://markwalling.org
Don't think. Know. Check your SMTP logs for the mails you "think" are being rejected. Find out why. Adjust your rules accordingly.


Top
   
 Post subject:
PostPosted: Tue Jan 05, 2010 2:56 pm 
Offline
Senior Newbie

Joined: Wed Jun 18, 2008 10:44 am
Posts: 15
Quote:
Don't think. Know. Check your SMTP logs for the mails you "think" are being rejected. Find out why. Adjust your rules accordingly.

Having faced exactly this question--is something I *don't* want to miss being rejected?--I created the following script to dump in /etc/cron.daily. It parses /var/log/maillog to provide a daily summary of different events, so I can see what's being rejected. It's already helped me find a misconfigured Postfix virtualhost.

I still need to install logtail, so I can ONLY check new errors each day, instead of getting all errors since last rotation.

I'd also recommend using warn_if_reject for a while when adding new restrictions.

Code:
#!/bin/bash
#
# Script to parse postfix logs for issues to report
# Created: 2009-12-30
# Changed: 2009-12-31 Added more detail: relaying, RBLs

LOGFILE=${logfile:-/var/log/maillog}
echo "Checking for relaying"
relay=`egrep "postfix\/smtpd?\[[0-9]*\]: [NOQUA-F]+:"  ${LOGFILE} | egrep -v "due to listing in|Sender address rejected|Client host rejected|Recipient address rejected" | sed '/Relay access denied/s/^\(.*\) postfix\/smtpd.*from=\([^ ]*\) to=\([^ ]*\) proto=.*/From: \2 To: \3 On: \1/' | sed -e 's/ To:/\nTo:/g' -e 's/ On:/\nOn:/g'`

echo "Relaying denied from:"
echo "$relay" | grep "^From: " | sed 's/^From: //g' | sort | uniq -c | sort -rn
echo "Relaying denied to:"
echo "$relay" | grep "^To: " | sed 's/^To: //g' | sort | uniq -c | sort -rn

rbl=`egrep "postfix\/smtpd?\[[0-9]*\]: [NOQUA-F]+:"  ${LOGFILE}\
   | egrep -v "Relay access denied|Sender address rejected|Client host rejected|Recipient address rejected" \
   | sed '/due to listing in/s/.*due to listing in \([^:]*\):.*from=\([^ ]*\).* to=\([^ ]*\).*/From: \2 To: \3 RBL: \1/g' \
   | sed -e 's/ To:/\nTo:/g' -e 's/ RBL:/\nRBL:/g'`

echo -n "Total RBL blocks: "
echo "$rbl" | grep "^To: " | wc -l
echo "RBL blocked email to:"
echo "$rbl" | grep "^To: " | sed 's/^To: //g' | sort | uniq -c | sort -rn
echo "RBLs:"
echo "$rbl" | grep "^RBL: " | sed 's/^RBL: //g' | sort | uniq -c | sort -rn

echo "Checking for new postfix errors"
egrep "postfix\/smtpd?\[[0-9]*\]: NOQUEUE:" ${LOGFILE} | egrep -v "Relay access denied|due to listing in|Sender address rejected|Client host rejected|Recipient address rejected" || echo "      none."

echo "Statistics"
egrep 'postfix\/smtpd' ${LOGFILE} | egrep -v 'NOQUEUE:|connect from|client=' | sed -e 's/.*smtpd\[[0-9]*\]: //' -e 's/lost connection.*/lost connection/' -e 's/warning.*/warning/' -e 's/timeout.*/timeout/' -e 's/too many errors.*/too many errors/' -e 's/.*reject.*/other reject/' | sort | uniq -c | sort -rg


Top
   
 Post subject:
PostPosted: Tue Jan 05, 2010 3:02 pm 
Offline
Senior Newbie

Joined: Wed Jun 18, 2008 10:44 am
Posts: 15
Here's an example output. The script also checks for NOQUEUE reasons other than the ones I already know about. I can also see which RBLs are catching spammers. If I'm concerned about whether an RBL is too aggressive, I can follow up to see what it's rejecting.

Code:
Checking for relaying
Relaying denied from:
      3 <spamery@tiscali.it>
      1 <mytestaddress@mydom.ain>
Relaying denied to:
      3 <spamery@tiscali.it>
      1 <anothertest@mydom.ain>
Total RBL blocks: 372
RBL blocked email to:
    212 <200403130913.36483.lfs-user@mydom.ain>
     57 <mailinglistuser@mydom.ain>
     29 <info@mydom.ain>
     28 <bikedc@mydom.ain>
     20 <user@mydom.ain>
      6 <200403140926.46641.lfs-user@mydom.ain>
      3 <jumanji@choicemedical.be>
      2 <zz@mail2000.com.tw>
      2 <sseenndd0622@yahoo.com.hk>
      2 <fedora-user@mydom.ain>
      1 <superedm001@yahoo.com.tw>
      1 <servicemagic-customer@mydom.ain>
      1 <poi@mail2000.com.tw>
      1 <lfs-user@mydom.ain>
      1 <myself@mydom.ain>
      1 <53363.lfs-user@mydom.ain>
      1 <41.lfs-user@mydom.ain>
      1 <200403141011.00740.lfs-user@mydom.ain>
      1 <200403122212.03348.lfs-user@mydom.ain>
      1 <200403122137.06415.lfs-user@mydom.ain>
      1 <200403122118.53363.lfs-user@mydom.ain>
RBLs:
    371 zen.spamhaus.org
      1 dul.dnsbl.sorbs.net
Checking for new postfix errors
      none.
Statistics
    163 lost connection
     89 warning
      2 timeout


Top
   
 Post subject:
PostPosted: Tue Jan 05, 2010 9:17 pm 
Offline
Senior Member

Joined: Wed May 13, 2009 1:32 pm
Posts: 737
Location: Italy
Dahak wrote:
Quote:
Don't think. Know. Check your SMTP logs for the mails you "think" are being rejected. Find out why. Adjust your rules accordingly.


Using your script I can't find any false positive.
I will check for a week to see better if some false positive is catched.

Thanks for your help.

PS: What do you think about zen.spamhaus.org ???
Is it good to use it?


Top
   
 Post subject:
PostPosted: Wed Jan 06, 2010 8:43 am 
Offline
Senior Newbie

Joined: Thu Dec 17, 2009 2:26 pm
Posts: 10
Take a look at Logwatch. It provides detailed information re: postfix from your syslog.


Top
   
 Post subject:
PostPosted: Wed Jan 06, 2010 8:49 am 
Offline
Senior Member

Joined: Wed May 13, 2009 1:32 pm
Posts: 737
Location: Italy
node_tux wrote:
Take a look at Logwatch. It provides detailed information re: postfix from your syslog.


I often seen my logwatch but I never founded a false positive.
The strange things is that I often loss forums thread reply notifications also if there is no trace on logwatch of flase positive.


Top
   
 Post subject:
PostPosted: Wed Jan 06, 2010 11:53 am 
Offline
Senior Member

Joined: Fri May 02, 2008 8:44 pm
Posts: 1121
Could this be what's causing the false positives?

https://issues.apache.org/SpamAssassin/ ... gi?id=6269

SpamAssassin will bump up the spam score of any email dated this year.


Top
   
 Post subject:
PostPosted: Wed Jan 06, 2010 9:51 pm 
Offline
Senior Newbie

Joined: Wed Jun 18, 2008 10:44 am
Posts: 15
node_tux wrote:
Take a look at Logwatch. It provides detailed information re: postfix from your syslog.


Unfortunately, I haven't figured out how to fix logwatch's parsing of postfix logs. I get a megabuttload of Unmatched Entries like
Code:
 
NOQUEUE: reject: RCPT from 118-168-138-33.dynamic.hinet.net[118.168.138.33]: 554
5.7.1 Mail from 118-168-138-33.dynamic.hinet.net[118.168.138.33] rejected based on
Client host due to listing in zen.spamhaus.org:
http://www.spamhaus.org/query/bl?ip=118.168.138.33; from=<t8.t8@msa.hinet.net>
to=<poi@mail2000.com.tw> proto=SMTP helo=<70.85.16.115>

instead of a useful summary of errors; hence the quick'n'dirty script as a stopgap. BTW, I've updated it some more if anybody is interested.


Top
   
 Post subject:
PostPosted: Wed Jan 06, 2010 9:56 pm 
Offline
Senior Newbie

Joined: Wed Jun 18, 2008 10:44 am
Posts: 15
sblantipodi wrote:
What do you think about zen.spamhaus.org ???
Is it good to use it?

When I was looking into blacklists, that's the one that appeared the best, and the stats seem to show it. Zero false positives so far (3 months), and very few get past it to be caught by one of the other BLs I have configured (dul.dnsbl.sorbs.net, and warning on dsn.rfc-ignorant.org & bl.spamcop.net).


Top
   
 Post subject:
PostPosted: Thu Jan 07, 2010 10:02 am 
Offline
Senior Member
User avatar

Joined: Mon Dec 10, 2007 4:30 pm
Posts: 341
Website: http://markwalling.org
Code:
smtpd_client_restrictions = warn_if_reject reject_rbl_client 1634435237.geobl.spameatingmonkey.net, 
  permit_mynetworks,
  reject_unauth_pipelining,
  reject_rbl_client bl.spameatingmonkey.net,
  reject_rbl_client zen.spamhaus.org


I get maybe 1 or 2 false negatives, and an undetectable number of false positives.


Top
   
 Post subject:
PostPosted: Sun Feb 14, 2010 7:40 am 
Offline
Senior Member

Joined: Wed May 13, 2009 1:32 pm
Posts: 737
Location: Italy
I catched a FALSE Positive in my log,
I don't want to disable spam filter but I don't want to lose email in this way.

Can you help me with a lighter settings?


Top
   
 Post subject:
PostPosted: Mon Feb 15, 2010 10:38 am 
Offline
Senior Member

Joined: Wed May 13, 2009 1:32 pm
Posts: 737
Location: Italy
I also noticed that with the suggested configuration from the guide I posted on the first thread I lost many mail from various forum thread subscriptions.

What is the incriminated value of this false positive?


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
cron
RSS

Powered by phpBB® Forum Software © phpBB Group