[rbldnsd] RFC: Data expire support

[Amos Jeffries]

----- Original Message ----- 
From: "Michael Tokarev" <mjt at tls.msk.ru>


> Amos Jeffries wrote:
>> Comments after each point.
>>
>> ----- Original Message ----- From: "Michael Tokarev" <mjt at tls.msk.ru>
>>
<snip>
>>> So, I think it's best to introduce another $-special directive, like
>>> $EXPIRE, to specify expire time.
>>>
>>> The only (probably quite serious) problem here is that local time may
>>> be inaccurate/wrong.
>>
>> Not that serious from a maintainers viewpoint, if you prohibit/error on 
>> zones created in the future. Zones expiring early only cause a 
>> default-accept in the end-systems, which in this case is ironicly better 
>> than default-deny.
>
> Think about exclusions... ;)
>
> I don't want to go "if there's no single 'exclude' entry in the file, we
> will treat it differently" route.  It will be non-deterministic.
>

I'm not sure we're thinking the same thing here at all.
I'm thinking 'default-exclude' on errors.
Unless your thinking a zone with two files; one of includes and one with 
excludes and an error ??

<snip>
>>
>>>  o If there are several expire dates per dataset/zone, the smallest one
>>>    will be used.
>>
>> Likewise, multiple created lines the _latest_ will need to be used.
>> So that mixed zones 1hr + 48hr data will be based on the latest 1hr data, 
>> not the older 48hr by chance ordering.
>
> Hmm.
> Now I'm confused.
> /me *thinks*
>

think of a file with this:
$TIMESTAMP 2005-12-01 2005-12-31
01/8
$TIMESTAMP 2005-12-29 2006-01-31
203/8

The joint file was likely created 29th Dec 2005.
thus the latest_create date as beginning of actual period and 
earliest_expire as ending.

<snip yuck about partial loading from above.>
>
> Ok.
>
> I don't think it's currently possible to derive SOA and TTL values
> from file timestamps, at least not now.
>
> What I want is something like:
>
>  new directive `$TIMESTAMP when-created when-expires' (when-expires
>  may be a relative time, like +3d).
>
>  a command-line option to set 'local time delta' (see below).
>  Set it to, say, 10m or 1h by default.
>
>  when loading a data file, process every $TIMESTAMP line and:
>
>   o compare when-created with local time - when-created should not
>     be greather than now.  If it is, warn or better abort loading.
>
>   o compare when-expires with current time - if when-expires is less
>     than current time, declare the file 'expired'.
>
>   o when doing the above comparisions, take 'local time delta' into
>     account, and allow `now' to be 'a bit' (to the 'local time delta')
>     less than when-created, or 'a bit' greather than when-expires.

which will screw up delayed mirrors.

>
>   o (probably) compare file timestamp with when-created, and require
>     the two to be the same (+/- local time delta again).  Abort loading
>     if not.

um, error is a bad idea. a warning would be good though.

>
>  As usual, if there was an error loading any part of a dataset, all zones
>  based on it will be non-functional (returning SERVFAIL).
>
>  The $TIMESTAMP directive works per-datafile, not per-dataset or per-zone.
>  This is the main reason we can't use real $SOA stuff.

Here is where you solve my problems. per-datafile!

>
>  Each zone has when-expires value computed as a minimum of all 
> when-expires
>  values from each datafile of each dataset the zone is based on.  On every
>  reload-check cycle, current time is compared with that when-expires 
> timestamp
>  and if (again, with `local time delta' taken into account) the zone is 
> expired,
>  it will be marked as such and will stop processing queries, returning 
> SERVFAIL
>  (with proper logging ofcourse).  On next reload (if we detect a file has 
> changed),
>  per-zone timestamps are recalculated and all 'expired' zones which are ok 
> now
>  are released.
>
>  If datafilecheck is disabled (-c0), no zone expiration takes place.
>
>  Timestamps are specified as unix_time, or as yyyy-mm-dd-hh-mi-ss UTC
>  (or GMT?  I mean, to use gmtime() and friends), with ':' accepted in
>  place of '-', and [-hh-mi-ss] part is optional.  When-expires may be
>  relative, starting with a plus sign (+), with optional unit specifier
>  like +2d, +10h etc.
>
>  It's probably a good thing to omit seconds altugether here.
>
> Or something like that anyway.
>
> Any objections to this?
>
> Also, how about comparing file timestamp with $TIMESTAMP's when-created 
> value --
> does it look ok?
>
> Thanks.
>
> /mjt

I was preparing the example to show my problem here and saw how the problem 
zones would work nicer with a few processing changes.
So my main complications just disappeared for now :-) yay!
It does still depend on the TIMESTAMP per-data file though.

Happy New Years.
AYJ