This is not really a linode question, but I'm hoping that a smart person here can point me in the right direction.

I'd like to build an open source spam "appliance" -- something that sits on the network as an email gateway, and that processes incoming mail to tag our spam for the MTA, which is running Groupwise.

The problem with this plan is backscatter. I need to be able to reject email that has bad recipient addressees at the gateway, without generating a bounce message.

The only way I can see to do this is to keep a list of valid addresses on the spam "appliance".

But in a perfect world, the spam server would keep a list of email addresses it's seen before, and query the actual mail server via SMTP to see if new ones are valid. If the real mail server would accept mail for an address, then the spam server would know that it's real. But if it won't, it would know that it isn't, and it could reject it at the perimeter, and not create backscatter.

Does anyone know if it's possible to do this with a system built out of an open source MTA like postfix, and tools like spamassassin and clamAV?

Do you just have to maintain that list of users twice?

How do people solve this?

I think LDAP would be a nice solution in this case:

http://www.postfix.org/LDAP_README.html#config

Much nicer if you already have an LDAP server setup of course. Can also do MySQL for a similar result.

Dunno about postfix directly, but sendmail should be able to do this in conjunction with "milter".

_________________
Rgds
Stephen
(Linux user since kernel version 0.11)

First off, postfix supports sendmail's milter protocol, so anything a milter can do in sendmail, it can do in postfix.

Secondly, your ability to do this really depends on how Groupwise stores and allows access to valid addresses. I haven't administered Groupwise in a very, very long time, but I believe the latest versions use LDAP as the config info storage with a customized schema.

Regardless, if you can figure out a way to get the list of valid email addresses from Groupwise, it should be pretty simple to set up a recurring job that transfers that list from Groupwise to postfix. Something like python or perl should be up to the task.

One thing to think about is where that process is going to run. If you can get direct access to a Groupwise system from outside your network, you could run it on the Linode and have it pull the list for you. If, like most networks these days, you have a firewall that's locking things down tightly, you could use an old PC behind the firewall to pull the data from Groupwise and use SSH to push the data to the Linode whenever it changes. SSH/SCP and RSA/DSA keys are magic for things like this.

Basically, I'd say it's definitely possible to do what you're asking, the trick is just finding the right way to get the data out of Groupwise--Google can probably help you with that.

--James

[quote="astrashe3"]This is not really a linode question, but I'm hoping that a smart person here can point me in the right direction.

I'd like to build an open source spam "appliance" -- something that sits on the network as an email gateway, and that processes incoming mail to tag our spam for the MTA, which is running Groupwise.

The problem with this plan is backscatter. I need to be able to reject email that has bad recipient addressees at the gateway, without generating a bounce message.

The only way I can see to do this is to keep a list of valid addresses on the spam "appliance".

But in a perfect world, the spam server would keep a list of email addresses it's seen before, and query the actual mail server via SMTP to see if new ones are valid. If the real mail server would accept mail for an address, then the spam server would know that it's real. But if it won't, it would know that it isn't, and it could reject it at the perimeter, and not create backscatter.

Does anyone know if it's possible to do this with a system built out of an open source MTA like postfix, and tools like spamassassin and clamAV? yes

Do you just have to maintain that list of users twice?
no

I am using sendmail, spamassassin and clamav. I use my address book for a white list (along with other known good mail) and create a large blacklist from email that gets through spamassassin (which isn't much after a while). This works very well. I am using clamav as a milter and also run a sendmail milter to reject dynamic ip's as well as some rbl checks too. I ususally get about one or two spam messages a week. Without all this I get dozens of spam email's, daily.

I am sure you can do the same with postfix if you want to use that MTA.

Look at some of the howto's at SourceForge.net.

First of all, thanks for all of the help.

Groupwise does use eDirectory, which can export information via LDAP. I think that to configure Groupwise, you usually make a change to eDirectory, and the data propagates to another tree that's private to Groupwise. So ideally, the eDirectory and the Groupwise information is in sync.

But I don't know if LDAP is a perfect solution. There are different kinds of groupwise objects that act as email addresses -- distribution lists, LDAP accounts that also allow people to login to the system, Groupwise accounts that don't have login accounts associated with them, etc.

Finally, I didn't really explain my goal. We're trying to get out of Groupwise and into something else. So I'd like a solution that's fairly agnostic with respect to mailers. I'd like it to work with Domino, or Exchange, or whatever.

This is what I'm asking, I guess.

In theory, it ought to be possible to write a script that would tell you if a given recipient was valid by querying the real mailserver. It ought to be able to do this entirely within the SMTP protocol. And it ought to work no matter what software the mailserver is running.

Because that's obviously going to put a lot of wear and tear on the mail server, it ought to also be possible to cache recipient queries so you don't keep looking up the same addresses again and again.

Such a thing running on the relay host would allow it to reject incoming mail with bad recipients, and it would work with any mail server via SMTP.

Does that exist?

http://www.postfix.org/ADDRESS_VERIFICATION_README.html

I think the Recipient address verification section is what you might be thinking about.

Get one of the o'reilly or no starch postfix books and read it. It doesn't sound like you want anything thats difficult to setup once you understand postfix.

You don't need external programs for this. Sendmail can use an access database:

define(`_RELAY_FULL_ADDR_', `1')
FEATURE(`access_db', `hash -T<TMPF> /etc/mail/access', `relaytofulladdress')dnl

Postfix can do it with a relay_recipient_map.

Wouldn't "access" maps and relay_recipient_maps require pushing the user list to the spam "appliance"? That's what the original question was trying to avoid, by querying the backend SMTP server directly.

_________________
Rgds

Stephen

(Linux user since kernel version 0.11)

Pushing the user lists to the internet facing mailer is the only good way to do this.

If this mailer does not operate independently it can't queue mail when the internal mailer, or the network in between them goes down.

Also, If you perform a SMTP session from the internet to an internal mailer to check every single receipient ( i.e. a custom milter ) you are going to flood your internal mailer with unnecessary connections and cause SMTP delays to all incomming internet mail. You can't be sure how long mailers will wait for you to respond, not everyone follows the RFC's and you do have to comunicate with those that don't.

If you perform any kind of caching you will get cache sync issues and backscatter every time you delete or rename a user. There is also a risk of bugs and security holes in your custom milter.

I use ASSP for this type of things.

_________________
==: terii :==