[rbldnsd] ip6set ?

[Jeff Dairiki]

On Sat, Jun 29, 2013 at 04:29:47PM +0400, Michael Tokarev wrote:
> 29.06.2013 16:14, Alex Lasoriti wrote:
> > A question pops out: will ip6set (equivalent of ip4set) be added
> > at some point, or we better forget about it ? At present, the SBL
> > uses ip4set, but the only real reason for not using ip4trie is the 
> > presence of duplicates - which are not useful at all from the operational 
> > point of view but are used to point people to different issues sitting
> > on a same IP that are documented in the database under different records.  
> > This is not a big issue, but if duplicates are impossible we would have 
> > to do some structural changes.
> 
> This is interesting.  I was thinking about *removing* ip4set (instead
> of adding ip6set) - because this one is the most difficult to understand
> and awkward dataset out of all the other datasets supported by rbldnsd,
> due to the way how it handles "uneven" ranges (where the cutoff is between
> octets).
> 
> With v6 this become even more awkward than v4.
> 
> Maybe it is better to allow multiple entries to be listed for *trie
> instead of implementing and using ip[46]set?
> 
> I think it should not be very difficult to implement this in a compact
> form, especially since we have another bit to (ab)use in the trie code
> (iirc) -- this bit can be used to distinguish between single data and
> a linked list of data items, so that in common case (which is the
> single data) we don't need to expand data structures.  Jeff?

I haven't yet had enough coffee this morning to be functioning with
full coherency, but I'm going to be off the net for the next couple
of weeks, so make my best attempt at an interim answer.  (More when
I return in a couple of weeks...)

I think it's probably doable without too much pain, though it's not
completely trivial.  (I don't think there are any unused bits in the
trie nodes right now, unless one wants to start (ab)using the low bit
of the data pointer.)

There are some semantic questions, e.g.:

Does one want to combine data from different prefixes?  E.g. suppose
the data includes:

  dead::/16        :1:some data
  dead:beef::/32   :2:other data
  dead:beef::/32   :3:still more data

Do we want to return three answers or just two for dead:beef::1 ?
Returning all three answers seem most "correct" to me, but also
seems more likely to cause confusion.

(From examining the code, I think the ip4set dataset currently does
some combination of these two behaviors.  It combines data from
prefixes which all are in the same list (/8, /16, /24 or /32), but not
from prefixes in other lists.)

It does seem like this (no matter how the semantic details are
resolved) is somewhat confusing and awkward.  It might be wise not to
include this in the default behavior of ip6trie.  Either introduce an
option to enable it, or introduce a new dataset type (which would
presumably share most/all of its code with ip6trie) to support
multiple data per prefix.

Jeff