[rbldnsd] Problems with exemptions

[Michael Tokarev]

Karl Maftoum wrote:
> Hi,
> 
> We've just implmented rbldnsd-0.992 for hosting a local mirror of the 
> sorbs.net RBL, and I have been unable to get exemptions to work 
> correctly. We are combining all the lists into a single zone, and have 
> the exemptions file which should exempt a number of class B addresses.
> 
> We are starting rbldnsd as follows:
> 
> /usr/local/rbldnsd/rbldnsd -a -u rbldns -b dnsbl.canberra.edu.au 
> dnsbl.canberra.
> edu.au:ip4set:/usr/local/rbldnsd/exemptions,/usr/local/rbldnsd/smtp.dnsbl.sorbs. 
> net,/usr/local/rbldnsd/http.dnsbl.sorbs.net,/usr/local/rbldnsd/misc.dnsbl.sorbs. 
> net,/usr/local/rbldnsd/socks.dnsbl.sorbs.net,/usr/local/rbldnsd/web.dnsbl.sorbs. 
> net,/usr/local/rbldnsd/zombie.dnsbl.sorbs.net

Ugh..  Save yourself from typing and possible source of errors,
use -w option:

  /usr/local/rbldnsd/rbldnsd ... -w /usr/local/rbldnsd


and use short file names after that... ;)

> The exemptions file is in the format:
> 
> !123.456.789.0/16 etc, but individual entries within that range are not 
> exempted.
> 
> What are we doing wrong?

Nothing.

Rbldnsd "exception mechanism" isn't supposed to work this way
in the first place.  It was meant to be used to excempt a small
netrange from larger listing.  Classical example of this is
a dynablock (DUL, DUHL, whatever): you list large dialup pool,
and want to excempt some individual IP addresses in that range
(e.g. IPs of known mailservers etc).  Basically (but not exactly),
the "idea" is that smaller, more specific entry wins.  If you
have large listing range and small exception range, the exception
wins.  If the reverse, the listing wins.

We talked with Matthew on IRC yesterday about this very issue
(referencing Telstra net ranges).  Basically some wants to
"whitelist" all the Telstra net range, which is listed as spam
support/whatever in SORBS, and Mattew suggested including a huge
exception entry for all Telstra networks "together" with the
SORBS data.

There's no way to do so in rbldnsd currently, without resorting
to whitelisting EVERY SINGLE IP of the netrange in question (but
in this case you may simply run out of memory).  One possible
solution will be to modify rbldnsd to support such a "super
exceptions", by changing the code.  But after thinking about
this more, I see no good way to do it either.

Let's look at this very example again.  The zone is build up
from "spam" and "http" datsets (amongst others).  We want to
have a "super-exception" for spam data only, but still want
to list http proxies.  Ie, we don't want to globally whitelist
this net range, but "remove" only some data from some dataset,
without affecting others.  (Well, in this particular case it
may be done by querying individual subzones, and using the
"super exception" for a single zone only; yet, the issue is
more generic than this particular example).

I can't think of a good solution currently.  So far, the
stuff looks clumsy at best, and is very difficult to explain
in the documentation (I found that if behaviour of a program
is difficult to document, 99,99% chances are that this is
because the behaviour itself is incorrect in one way or
another).

There are 2 ways to solve this issue (code modifications are
reqd anyway):

  a) always consider exceptions, regardless of the size of the
     netrange, i.e., ignore all, even smaller, listings, if we
     have a larger exception entry.  I guess this may conflict
     with some current usages of exceptions, where one depends
     on current "more specific entry wins" rule.  This should
     work in a single dataset only, not affecting other datasets
     for the zone in question.

  b) introduce "super-exception" (e.g. prefixed with double-!,
     !!127.0.0.0/8), that "propagate" to other datasets for a
     zone in question too, and made the order of datasets to
     be important (so that if we have 3 datasets for a zone,
     first and 3th have listings for a given query, and 2nd
     have "super-exception", the result will be the listing
     from first dataset only, and processing will stop after
     finding the super-exception (not even consulting 3th
     dataset).

The b) case is clumsy.  The a) case may break compatibility,
which I want to avoid as much as possible.  Maybe a combination
of the two will work, after all....   Ideas?

/mjt