[rbldnsd] RFC: Data expire support

[Michael Tokarev]

Recently I received several questions/requests about supporting
"data expire" thing in rbldnsd.  The scenario is rather common:
some nameserver of a large/widely used blocklist, sich as dsbl.org,
for whatever reason stops updating the data and rbldnsd happily
continues to answer queries with old, stale data => people start
screaming "we've been out of the blocklist for a while but our
mail is still being rejected!".

So, after several small discussions here and there, we come across
this simple idea: a data file may contain an 'expire marker', ie,
a timestamp indicating when the data becomes invalid.

Yes there is `expire' field in the SOA record, which is not used
currently, but I'm for another way to specify this expire time,
because of several reasons:

  o normally (according to the DNS specs), SOA expire field is
    relative, counting since last update time.  Since rbldnsd does
    not know when the file has been updated (with external update
    mechanism, as opposed to eg named with named-xfer), it can not
    use this field directly in 'relative' meaning.  Re-defining
    it may be somewhat confusing, even if it's not used in rbldnsd.

  o there may be only one SOA record per zone, and a zone may be
    composed of several datasets and datafiles (first SOA encountered
    is used, the rest if any are ignored), each with its own expire
    time (some data may be more dynamic/volatile than other, eg.
    lists of proxies as opposed to lists of dynamic network ranges).
    So, every expire time should be counted/considered.

  o while it is common practice to specify 'current date' in the SOA
    'serial' field, it's not mandatory and different people/sites use
    their own formats for the timestamp (yyyymmddn is one common usage,
    unix time_t is another etc).  With current usage of rbldnsd, this
    field has different meaning (if at all) in every place.

So, I think it's best to introduce another $-special directive, like
$EXPIRE, to specify expire time.

The only (probably quite serious) problem here is that local time may
be inaccurate/wrong.  Since rbldnsd can only use absolute expire time
(again, without integrating update mechanism directly into rbldnsd,
which I want to avoid at all costs because current mechanism is most
flexible), it can only compare that expire time with local machine time,
and if the latter is not accurate, the whole thing will not work at all.

So, after some more thoughts, current version of this feature looks like
this:

  o there's a new directive introduced,
      $TIMESTAMP when-created when-expires [...probably some more fields]
    where both timestamps are either in unix time_t format (secounds since
    Epoch) or in form yyyy-mm-dd[:hh24[:mi[:ss]]] (hours, minutes and
    secounds are optional), in GMT/UTC.  When-created must be the current
    timestamp, and when-expires must be a time when the data will expire
    and must be reloaded.  When-created is here in order to detect the
    situation when local time is set to the past, before when-created.

  o data expiration is checked during a time when rbldnsd checks for
    data updates (normally every minute, controlled by -c option).
    So if you disabled automatic checking for new data (-c0), it will
    not verify whenever the data has expired during runtime, only during
    reloads when data has actually changed, which is sorta pointless.

  o (optional, I like it but it's somewhat clumsy in implementation,
    when there are several timestamps per dataset) when expire time
    is about to come (eg, 90% of time between 'created' and 'expire'
    has passed), rbldnsd produces a warning on reload.

  o If there are several expire dates per dataset/zone, the smallest one
    will be used.

  o there's an option introduced, to tell rbldnsd to ignore all those
    'expire' directives, in order to be able to force it to run "in case
    of emergency" (current time is wrong, etc).  I dislike introducing
    this option, but unfortunately there WILL be alot of demand for it
    once major dnsbls will start using this feature and people upgrade
    their rbldnsd installs to the version supporting this stuff.

  o Or maybe, an ability to turn this feature on/off per-dataset, by
    specifying `$TIMESTAMP 0 0'.  Also ugly but demanding...

Comments?

/mjt