[rbldnsd] Netblocks vs. Ranges - different behavior

[Michael Tokarev]

Scott Knight wrote:
> I've noticed some differing behavior regarding the use of netblocks vs. 
> ranges in RBLDNSD.  Particularly when combined with another single IP 
> that is included in that netblock range. 
> 
> For example:
>  
> If I have the following two entries in my zone file (fake IPs):
>  
> 1.2.3.0/24:127.0.0.3
> 1.2.3.5:127.0.0.2
>  
> If I do an nslookup on 5.3.2.1.zonename.com, I get 127.0.0.2 *only* as 
> the response (this is the desired effect, the single IP is overriding 
> the netblock)
>  
> But, if I have the following two entries in my zone file:
> 1.2.3.0-1.2.3.255:127.0.0.3
> 1.2.3.5:127.0.0.2
>  
> An nslookup on 5.3.2.1.zonename.com here returns 2 results, 127.0.0.3 
> and 127.0.0.2 (and order seems to matter... if the single IP was listed 
> first, then 127.0.0.2 would be returned before 127.0.0.3).

This behaviour looks strange.  In my tests, and as designed, rbldnsd
should behave exactly the same, whenever you specify xx.0/24 or xx.0-xx.255
or xx.0-255.  Here's the output when it loads a "zone" with 127.0.0.0/24
(all 3 variants are the same):

rbldnsd: ip4set:testdata: 20041119 170435: e32/24/16/8=0/1/0/0

ie, there's one /24 entry only.

Please tell me the output (the line like the above) rbldnsd
gives you in your case.

> For cases like the above, it doesn't matter much because I can just use 
> netblocks instead of ranges, but for smaller ranges that don't 
> necessarily map to a netblock, I run into this issue.

It is (or should be -- at least as it is designed) no difference
whenever you're using CIDRs, x-y or x.y.z.a-b notation for ranges,
in all cases it is a pair of addresses -- first and last addresses
that forms the range (in case of /32, first = last).

> Is there anyway to force a single IP to overide a range?

Yes and no.

And this very question, in different forms, becoming more and more
frequent.

Internally, in ip4set, rbldnsd keeps 4 arrays of addresses, one array
for /32 "ranges", one for /24 "ranges", one for /16 and one for /8.
Everything >= /8 goes into /8 array, >= /16 into /16 array and so
on.  Eg, if you list this, and result will be:

  x/32  -- one entry inserted into /32 array
  x/31  -- two entries inserted into /32 array
  ...
  x/25  -- 128 entries into /32 array
  x/24  -- one entry into /24 array
  x/23  -- 2 entries into /24
  ...
  x/17  -- 128 into /24
  x/16  -- one into /16
....

Each array element only contain an address (significant octets)
and a pointer to resulting values (A and TXT), not the "size"
of original line caused the entry.

A search (lookup) is performed in /32 array first, in /24 if
not found, in /16 if not found etc, first match wins.  So,
eg /32 takes precedence over /24 or /16, but /30 and /32,
as they're both in the same array, are "equal in weight".

I understand full well it is a limitation and a source of
confusion.  But the bad thing is that in almost all cases,
this variant just works, because there's just no overlapping
ranges like this, and/or because it is ok to return all
relevant data (note also i received exactly opposite questions
too: ie in the case of one /32 and containing /24 or /16,
how to force rbldnsd to return ALL entries instead of the
most specific /32).  But this is the way how it is implemented.
Keeping more information about every IP address means more
memory (eg add another 4 bytes, mem requiriment will grow
to 3/2 from current) and slower operations (more difficult
sorting as more bytes has to be moved etc), and, most important,
for... unclean reason.  That is, yes there are some specific
cases which -- seems to be -- require such a "finer control",
but even in most of those cases the "finer control" isn't
in fact a requiriment but just a wish... ;)

There's another dataset type, ip4trie, which will return
most specific entry ONLY (and disallow repeated listings).
But it isn't as good as ip4set for a large number of single
ip addresses (the best here is ip4tset); while ip4set is
much worse than ip4trie when you have alot of large CIDRs
most aren't one of /32, /24, /16 and /8.  Maybe you can
use ip4trie instead..

...if not, please tell me whenever you really *require* rbldnsd
to return only most specific entry... ;)

/mjt