[rbldnsd] regular expression support for rbldnsd

[Per Jessen]

Steven Champeon wrote:

> on Wed, Aug 12, 2009 at 02:45:51PM -0400, Jon Lewis wrote:
>> On Wed, 12 Aug 2009, Per Jessen wrote:
>>
>>> Interesting idea.  We have a list of such patterns which is
>>> evaluated by
>>> Postfix.  I can't immediately see if a DNS-based solution instead
>>> would improve things.
>>
>> The benefit is centralized management of the regex's and a plug-in
>> standard (DNSBL query) that just about everything supports.
> 
> I'd wager that our regex processing library is faster than Postfix's;
> it's based on the idea that the string being matched against is a
> hostname and so the regexes are searched according to the TLD, then
> SLD, then domain, only then are any regexes checked against the
> string.
> 
> As I recall, Postfix uses a simple brute-force "check all patterns
> until something matches" approach. 

Yep.

> Depending on the size and coverage of the patterns file (ours is over
> 41K patterns in 23K domains), this would probably represent a pretty
> severe performance hit.  

My list only has about 2000 entries, with a few fairly generic patterns
taking care of most. The non-generic entries are sorted according to
daily frequency, i.e. the more hits, the further up the list.  I have
no idea how postfix might cope (or not) with 41K.  

> The library also lowercases all queries, to avoid the overhead of a
> case-insensitive regex scan.

postfix's regex matching is also case-insensitive by default, but I
don't know if strings are lowercased before matching.  Judging by the
contents of a policy daemon request, I'd say they are though.


/Per Jessen, Zürich