[rbldnsd] TTLs and negative caching

[Jeff Chan]

On Tuesday, August 3, 2004, 4:16:30 AM, Michael Tokarev wrote:
> Jeff Chan wrote:
>> We run SURBL which has RBLs containing spam URI domains.
>> They are used to block spam based on URI domains contained
>> in message bodies.
>> 
>> Currently some of our zone files have fairly long TTLs
>> of several hours.  This appears to be causing new entries
>> to take several hours to become active.

> Hmm....  That does not seem to be right.  What's your
> zone and any test url/domain, for me to check?

One zone with a long TTL is ob.surbl.org, with a record
like ezherbalbuy.com.ob.surbl.org

with the SOA record:

  $SOA 21600 origin zone.surbl.org 1091531965 1200 600 604800 21600

>> 1.  Does that sound right?  In other words does TTL apply
>> to negative caching or only to positive caching (or to both).
>> By observing the behavior of an rbldnsd installation, TTL
>> does seem to affect negative caching.

> According to DNS specifications, negative TTL is taken from
> the SOA record's minttl field (the last number in the $SOA
> line).  So no, the TTL (either -t option or $TTL value) does
> not affect negative TTL.

Perhaps we should be using a finer-grained approach to setting
these.  Suggestions would be definitely appreciated. 

>> 2.  Can we expect a lot more DNS traffic if we lower our
>> TTLs to say 10 minutes?

> In a usage like this (spammer's url/domain db), I don't expect
> any significant increase of traffic after lowering *positive*
> TTL, because positive hit ratio should be rather low anyway.

Yes, most queries result in NXDOMAIN.  The lists have only 400
to 50,000 records, which is only a tiny subset of domains and
addresses on the net (but the most spammy ones we could find).

> But playing with negative TTL should make much more difference.
> IMHO ofcourse.

Long negative caches are ok if new entries (positive hits)
can resolve quickly.  If we have a long negative TTL
and a short positive TTL do new records become resolvable
quickly?

SBL has a very short minimum TTL of 5 minutes:

> sbl.spamhaus.org.       0S IN SOA       need.to.know.only. hostmaster.spamhaus.org. (
>                                         2004080311      ; serial
>                                         1H              ; refresh
>                                         15M             ; retry
>                                         1W              ; expiry
>                                         5M )            ; minimum


Jeff C.
-- 
Jeff Chan
mailto:jeffc at surbl.org
http://www.surbl.org/