[rbldnsd] regular expression support for rbldnsd

[Steven Champeon]

Hi -

Summary:

We have a patch against rbldnsd 0.996b that provides support for regular
expression-based fast lookups of HELO and PTR strings, in an rbldnsd
zone, that return our classifications for hostnames for use in scoring
or blocking bot-originated email.

It's very small, just a few hundred lines, but requires the presence of
Apache Portable Runtime (any recent version) and PCRE (which any modern
Unix/Linux system has already; we're running our mirrors under OS X and
various Linux distros). On my MacBook 2.0Ghz, the underlying library can
do 45K lookups/second; on better hardware, it is closer to 60K/s.

It may be found here:

 http://enemieslist.com/dnsbl/rebl-0.01-rbldnsd-0.996b.tgz

Details:

Enemieslist is an antispam project in its sixth year, which attempts to
classify reverse DNS (PTR) naming conventions by assignment type and
technology, so given a hostname such as

cable-89-216-241-237.dynamic.sbb.rs

we'd say it is a dynamic/cable connection.

$ dig +short -ta cable-89-216-241-237.dynamic.sbb.rs.g.enemieslist.com.
127.0.0.3

$ dig +short -ttxt cable-89-216-241-237.dynamic.sbb.rs.g.enemieslist.com.
"cable"

This has proven very useful in the context of dealing with botnets, as
it's vanishingly rare to see hosts with generic names trying to send
legitimate email. In addition, hosts with generic PTRs seldom HELO with
their PTR name, so that in itself is a very reliable bot indicator.

All patterns are fully qualified; no foolishness such as regex groups
like "(cable|dialup|dsl|host|user)". But the EL patterns data are not
necessary for the patch to work; you simply need to provide a zone file
in the following format, so it could be used without licensing the
existing EL patterns data if you so desire:

key:regex:class:tech

where class is a dotted quad, and tech is a string. Key in this case
is based on the domain, so for sbb.rs above you'd see RSSbb_01 or
equivalent.

For more information about the various classes and techs, and how to
decode A record lookup response codes, see:

 http://enemieslist.com/how/use.html

We license our data (41K patterns in 23K domains worldwide) to ISPs,
state governments, reputation service providers, and antispam appliance
vendors, and are branching out to others; we've recently released a beta
plugin for SpamAssassin which uses the DNSBL lookups for HELO string and
PTR, and we're looking to see wider support in the core rbldnsd package
for lookups of this sort, so as to make it easier for folks wishing to
provide mirrors to do so without fear. We're working with the folks at
Spamhaus to see about incorporating it into their rbldnsd infrastructure
as well, and have worked with Alex Broens at URIBL and other folks at
other blacklists to deploy EL as part of their detection and listing
infrastructure. We've been running our rbldnsd mirrors for over two 
years in production; they're very stable and reliable (and much faster
than the perl-based hacks we had working for the two years prior).

The coverage is very good; in a recent run against a CBL list.txt EL
matched well over 99.5% of the hosts with PTRs; most of the rest were
vulnerable/compromised legitimate mail servers.

What would it take to incorporate support for this patch into the
official rbldnsd source distribution? The library it relies on is under
a BSD-style license, with the caveat that the name of the programmer not
be mentioned if it is re-used elsewhere, but with no other legal
restrictions on use.

Please note that the DNS zones, while not restricted, are for commercial
or evaluation purposes only, that's why we're looking to get the patch
into rbldnsd proper, so we can expand our mirror network and open it up
for non-commercial use via SpamAssassin or other means (there is a
package for sendmail, as well as various versions for use with exim and
postfix, etc.)

Thanks,
Steve

-- 
hesketh.com/inc. v: +1(919)834-2552 f: +1(919)834-2553 w: http://hesketh.com/
antispam news, solutions for sendmail, exim, postfix: http://enemieslist.com/