[rbldnsd] Idea for cleaning up forked child when reloading parent dies

[Michael Tokarev]

Victor Duchovni wrote:
> On Wed, Apr 27, 2005 at 12:31:10PM +0400, Michael Tokarev wrote:
> 
> 
>>Hmm.  In which situation that can happen?  I mean, why the parent should
>>ever exit?  Crash?  Or just SIGTERM?
>>
>>If it's the latter, and granted, I don't remember how this situation is
>>handled currently, it should be easy to fix (modulo the stupid linux
>>behaviour).  If it's the former.. well, yes, the problem is real.
> 
> This is based on comments in the code, either crash or SIGTERM, anything
> really that makes the parent process go away.

I'll take a look at it again.  When I first wrote that forking-reload
stuff, I thought about creating the pipe as you suggested, but something
prevented me from using it, maybe the reason was that I was too lazy to
perform that select-based stuff.  After all, it should be possible to
start a timer in child in the same way as it is running in parent,
and check whenever the parent is still alive (using pipe or whatever)
each time after receiving SIGALRM or something like that.

>>I'm thinking about using a sort of binary on-disk format like DJB's
>>rbldns does, this should eliminate the need for forking altugether.
> 
> CDB seems like a natural candidate, unless performance requirements
> necessitate something faster... I think that atomic update to a
> database snapshot makes a lot of sense.

No, I want to use the data structure which is already used internally (for
ip4set, it's 4 sorted arrays), but just move the reading/parsing/sorting
stuff out of main process.  That is, one process prepares the data in
the way very similar to current one (with the obvious difference - changing
pointers into indexes or offsets), and the parent mmap()s the whole thing,
reading it in one go and verifies that the whole stuff looks ok (it is fast
operation, especially since the file should still be in filesystem cache).

And yes, atomic updates it a must.  I was thinking about this scenario:
somewhere in the file header, have a flag, say, "active".  Main rbldnsd
process checks the flag (mmaped) on each query, and peforms the reload
if it's zero.  And the "builder" process does the following:

   prepare_new_file(file.tmp);
   fd = open(file);
   rename(file.tmp, file);
   pwrite(fd, 0, 1, offsetof(active_flag));

so when main process sees "active" is zero, it will be able to reload
the whole dataset in almost one go.  And it even can keep old file stll,
if something is wrong with new one...

But all this is a theory for now.  We're having quite.. significant probs
with dsbl nameservers for example, -- reloading 6M+ entries even as
ip4tset takes quite some time and requires quite some memory, ...
Ofcourse this whole prepare-binary-image process isn't needed at 99%
sites using rbldnsd, -- the ones who're dealing with smaller amounts
of data...

/mjt