[rbldnsd] Option -d (Bind dump) and wildcards: a small problem

[Michael Tokarev]

Michael Tokarev wrote Sun, 10 Oct 2004 17:29:14 +0400:
> furio ercolessi wrote:
> 
>  > Small problem with -d dumps.
>  >
>  > Let us consider the following (real) case, from SBL.
>  > SBL contains the following records related with 222.65.0.0/16:
>  >
>  > 222.65.0.0/16 http://www.spamhaus.org/SBL/sbl.lasso?query=SBL19307
>  > 222.65.20.170/32 http://www.spamhaus.org/SBL/sbl.lasso?query=SBL19565
>  >
> []
> 
>  > 170.20.65.222   A       127.0.0.2
>  > *.65.222        A       127.0.0.2
> 
> Oh well.  This isn't a *small* prob, it's a large problem
> (in the context of -d option anyway).  I'll think about this
> more - the whole stuff is getting to be quite ugly... :(

Well, this is really getting very ugly.  I tried to sort this prob
out several times already, but can't figure out how.

Here's an example of some fun stuff:

The following entries in rbldnsd source:
  127.0.0.0/8   x1
  127.0.0.0/8   x1
  127.0.0.2     y
  127.1.0.0/24  z
  128.0.0.0/8   n

should result in (optimal) the following master-file records:

       *.127 x1
       *.127 x2
     *.0.127 x1
     *.0.127 x2
   *.0.0.127 x1
   *.0.0.127 x2
   2.0.0.127 y
     *.1.127 x1
     *.1.127 x2
   *.0.1.127 z
       *.128 n

Add some more stuff with exclusions, and.. you're screwed... ;)

So, unless something changes and I will have an idea about what
to do with all this mess... i'm considering declaring master-format
dump as unsupported, aka "use on your own risk", option.  I have
some code here but it is fat, ugly, and.. wrong.

In case someone will want to step further and think about how
to deal with all this, here's some implementation details.
There are 4 arrays in ip4set: /32 entries, /24 entries (with
last byte equal to 0), /16 entries (with last 2 bytes being 0)
and /8 entries (3 bytes are zero).  Each array element is a
pair of (IP address, RR-pointer), where RR-pointer is a ptr
to the RR template for the given entry, or NULL in case of
exclusion.  So, for the above 3 entries, we will have the
following content of the arrays:

  /8 (2 entry)   /16     /24 (1 entry)  /32 (1 entry)
  127.0.0.0=>x1  (empty) 127.1.0.0=>z   127.0.0.2=>y
  127.0.0.0=>x2
  128.0.0.0=>n

All IP addresses are 32bit integers in *host* byte order.
All 4 arrays are sorted by IP in increasing order.  Repeated
entries (with different RRs - for multiple TXTs for example)
are stored as shown above for /8 array.  If there's an exclusion
entry (where RR pointer is NULL), it is guaranteed the IP
in question is stored in the array only once (it is not
possible for an IP to be both excluded and included, and
it can only be excluded once).

BTW, I found a prob with -d for ip4trie too - a typo in fact, but
a typo which may prevent significant portion of data from being
included into master-format file too, somewhat similar to the
current problem.

/mjt