[kwlug disc.] Greylisting for spam purposes

[Andrew Kohlsmith]

On Thursday 30 November 2006 16:09, Paul Kopacz wrote:
> The caveats listed here (http://en.wikipedia.org/wiki/Greylisting)
> turned me off, specifically where they mention that mail sent from a
> "farm" of servers with different IP addresses may not get delivered if a
> different mail server sends the retry email.  I certainly wouldn't

This *can* happen, yes, but in practice (I have been running Postgrey for the 
past year or so on what I'd consider a high-volume mail server) this is the 
least of your worries.  Postgrey already has a number of the server farms 
whitelisted, and you can easily add more (I've had to add only one).

I actually *like* the initial delay in mail; it helps drive home the point to 
the office types that no, SMTP is not a guaranteed protocol and no, you 
cannot guarantee that a message will get to the destination "as soon as you 
hit send."  Some people just do NOT get it, just like some people here like 
to send 300M attachments through email and blame me when some SMTP server 
between here and there rejects it.

> different mail server sends the retry email.  I certainly wouldn't
> recommend this where I work, as we receive a large volume of email from
> all over and it is important that it gets through with minimal hassles
> (People calling to say that they received a warning email about a
> message not being delivered counts as hassle).  I'm not sure how it

I run Postgrey on a mail server for a general ISP without issue.  It's also 
running where I work, where we get email from all over the world (lots of .kr 
and .cl and .cn stuff, too.)  I'd certainly count that as "mail from all 
over".

Also, if someone else's SMTP server is sending back warnings to 450s... then 
their server is broken and you will likely end up with complaints about a 
number of other things from users (or victims) of such software.

> would be possible to "test" this without just doing it and dealing with
> the consequences.  I would also be interested in hearing anyone's actual
> experiences, as people I work with  have noticed an increase in spam
> lately.

As I've already mentioned, I run Postgrey, have been for quite some time 
(uptime on the box is 420 days, and I've been running Postgrey longer than 
since the last reboot), and while it's not a perfect solution, every little 
bit helps.

A brief rundown on how my mail system is organized:

the one and only MX record for my domain is what I'd call a "front-line" box 
-- it knows *nothing* about mail users, only domains.  This "front-line" box 
also does this same service for 13 other domains.

(Why only one MX?  Because spammers typically try to contact lower-priority 
MXes as they may have less/no filtering software, as the domain owners are 
expecting the high-priority MX to take 99% of the mail.  If you wanted to 
cluster the front-line boxes you could do it with a shared Postgrey db and 
some locking/caching magic inserted into Postgrey.)

All this box does (in order) is the following:
- basic, basic sender HELO and recipient checks
- check the sender IP against two blackhole lists (cbl.abuseat.org, 
list.dbsl.org)
- check against the greylister

If these tests pass, it accepts the mail and queues it up for delivery to the 
"real" SMTP server.

The real server knows about the individual users and performs more intensive 
checks for spam and viruses.  Mail passing those checks gets put in the 
user's inbox, and they access it via POP3 or IMAP4 as required.

I use two different boxes for the simple reason that the head-end box takes a 
LOT of shit.  If I did this all on one box (which I was doing before), we 
would have legitimate SMTP connections getting rejected and general slowness 
of email to and from our own offices due to the network system being too busy 
dealing with garbage connections.  The load on the front-line box is next to 
zero; all of these checks are fast and take next to no CPU time.  The load on 
the back-end box is higher since it's doing a LOT more intensive checking, 
but on FAR less email.

Some stats on the front-line box:
The average week sees 650,000+ SMTP connection alone.  240k messages are 
rejected because I can't find their domain.  300k messages are rejected by 
the greylister with 450 (come back later).  16k messages are blocked using 
the two blackhole lists.

After all is said and done, 22,783 messages make it through and are sent on to 
the appropriate mail server.

Does it work?  I think so.  Is it worth an initial delay until the 
sender-recipient pair is auto-whitelisted by the greylister?  Absolutely.  
Will the greylister eventually become less effective?  Yes, but until then, 
this is a VERY simple, VERY light, and VERY effective countermeasure.

-A.