[rbldnsd] regular expression support for rbldnsd

[Steven Champeon]

on Wed, Aug 12, 2009 at 06:21:11PM +0200, Per Jessen wrote:
> Steven Champeon wrote:
> 
> > We have a patch against rbldnsd 0.996b that provides support for
> > regular expression-based fast lookups of HELO and PTR strings, in an
> > rbldnsd zone, that return our classifications for hostnames for use in
> > scoring or blocking bot-originated email.
> 
> Interesting idea.  We have a list of such patterns which is evaluated by
> Postfix.  I can't immediately see if a DNS-based solution instead would
> improve things.   

It depends on whether your list is short or long; sendmail handled
inline regex maps just fine until we hit around 10K-15K, at which point
it became a matter of avoiding the hassle of recompiling the .cf file
every time there was an update. The DNSBL approach simplified the process
or managing updates tremendously. I've had reports that Postfix with a
policy daemon works rather well, but again you're just shifting the load
from one server to another, and the policy daemon needs to have a local
copy of the patterns, etc. Exim, at least anecdotally, fell over quite
hard when dealing with large flat files containing the patterns.

We used to use a set of "compact" (left-anchored) hostname-only (not
including domain) patterns for a while, but there were too many idiot
setups sending mail from hosts named "^host[0-9]+\." and the like, so
we have stuck with just fully qualified patterns and "right anchor"
strings (such as "dynamic.example.net"), but we're thinking of even
abandoning the latter as we see an occasional mail server set up for
use by residential customers, for example, that uses the residential
keyword/token as part of its name :-/
 
> In our current setup, it would shift the CPU-load (for the matching)
> from the individual mailserver to the rbldnsd server, which wouldn't be
> good. 
> 
> > This has proven very useful in the context of dealing with botnets, as
> > it's vanishingly rare to see hosts with generic names trying to send
> > legitimate email. 
> 
> I guess that depends - I see a lot every day.  There are a lot of
> less-than-competent mail-admins out there, and forgetting to ask your
> provider for a reverse entry is quite common around here,
> unfortunately. 

That's why we recommend scoring on certain classes; rejecting on static
generic PTR would see FPs, but it depends on your mailflow, customer or
user base, etc. and legitimate generic HELO is *very rare*, almost
certainly a bot trying to make sure its IP's PTR matches its HELO to
appear more legitimate. We've seen places where rejecting at HELO time
on dynamic HELO alone kills about 1/7 to 1/4 of all incoming mail, with
zero FPs. YMMV, of course.

Steve

-- 
hesketh.com/inc. v: +1(919)834-2552 f: +1(919)834-2553 w: http://hesketh.com/
antispam news, solutions for sendmail, exim, postfix: http://enemieslist.com/