[rbldnsd] Problems configuring BIND 9 with rbldnsd

[Michael Tokarev]

Chris Gabe wrote:
> We would like to use rbldnsd to help scrape urls for our zone file
> providers.   Our rbldnsd implementation services several DNSBLs and url
> block lists at once.  It is problematic to use a log of *all* the
> queries to obtain the list in question.  It would be better if rbldnsd
> provided the following capabilities:
> 
> - log each query that goes to one of a specified set of block lists,
> which is not present in any of them.  We don't want to include those
> already in any of the lists; but queries will go to specific lists.  We
> also want to exclude queries to those lists that are not appropriate,
> like ip4sets.
> 
> - support very frequent updates, required to fight quick DNS games by
> spammers.  E.g. log to a separate file each minute.  wrap after, say, 60
> minutes to the first file.
> Or some equivalent that lets us efficiently process the output every
> minute, i.e. sort | uniq, and submit it to the url block list server for
> examination and potential inclusion.
> We could get around this by keeping track of the position in the syslog,
> I suppose.  Or newsyslog every minute (blech)

Oh well.

The issue with all this is - everyone want different things wrt logging,
statistics and the like.  For example, NJABL wants to log queries coming
from certain IP addresses, logging only the query IP which is not yet
listed.  Spamhaus wanted to see ALL realtime statistics per client IP.
And so on.

So it's impossible to implement it all in generic way, the only way I
see is to use some programming locally.

Rbldnsd has compile-time support for user-defined hooks in certain
places.  See rbldnsd_hooks.h and rbldnsd_hooks.c for examples.
There's no documentation about how to use it all, and I never tried
to actually implement a real example, don't even know if it will
work.  But I see no other way elegant way.

Maybe some dynamically-loaded object implementing similar hooks,
but this places unnecessary overhead for all the rest of users
(and believe me, high load is quite common for public DNSBL
mirrors, so every extra instruction counts).

> It is both inefficient and problematic to do this using the existing -l
> feature.  It would require re-querying the server to see if it's in any
> of the url lists, which creates a loop.
> 
> Is this something others are interested in?
> I know we could run multiple rbldsnd's to get what we need, but that
> would get pretty ugly.
> 
> Michael Tokarev - does this have merit or do you see it as unacceptably
> beyond scope?

I'd say it's beyond scope.  Not completely - because I did think about
something like that before, when doing that hook interface.

We probably can try to roll something together, as a working example.
Maybe I can create a dll-calling hook example as well.

The thing is - I don't have spare time now.  I can describe how it
supposed to work, but not implement things.

/mjt