[rbldnsd] OT: Advise needed on what RBL operators supposed to do with logs

[furio ercolessi]

On Sun, Sep 25, 2005 at 10:40:00PM -0700, William Leibzon wrote:
> 
> On Sun, 25 Sep 2005, Jon Lewis wrote:
> 
> >But seriously, what do you need from the logs?  Either condense out the 
> >data you want from them, and store just that, or don't log at all.
> 
> I try to use them to get summaries, i.e. number of hits, number of 
> individual query sources, etc. I don't have a very good summary generation 
> program so it
> is useful to have logs in case I want to see what it was like few days ago.
> In case somebody is reporting an error, I may also sometimes try to see what
> logs show for those queries for that time, etc.

There is another possibility, which may be useful for instance to
extract the big users without using too much disk: log by sampling,
write one log line every L.
Some time ago I did a quick-and-dirty patch to do this, that I enclose
below.  This was for 0.993.1.  Added an extra switch -L to specify
the periodicity.  Implementation boils down basically to a new variable
and an if statement containing a modulus operation.

I would be happy to see this incorporated into the mainstream
distribution, although I admit that it's a bit weird and I might be
the only one using it..

furio

===========================================================================

--- rbldnsd.c.ORIG	2004-07-12 18:55:14.000000000 +0200
+++ rbldnsd.c	2004-10-22 20:22:23.000000000 +0200
@@ -87,6 +87,8 @@
 static unsigned recheck = 60;	/* interval between checks for reload */
 static int initialized;		/* 1 when initialized */
 static char *logfile;		/* log file name */
+static int logcount = 0;	/* counter of queries for sampling in log file */
+static int logsamplingstep = 1;	/* sampling step for logging queries in log file */
 #ifndef NOSTATS
 static char *statsfile;		/* statistics file */
 static int stats_relative;	/* dump relative, not absolute, stats */
@@ -210,6 +212,7 @@
 "  during reload (may double memory requiriments)\n"
 " -q - quickstart, load zones after backgrounding\n"
 " -l [+]logfile - log queries and answers to this file (+ for unbuffered)\n"
+" -L step - log a query every step queries in logfile (default is 1: log all)\n"
 #ifndef NOSTATS
 " -s [+]statsfile - write a line with short statistics summary into this\n"
 "  file every `check' (-c) secounds, for rrdtool-like applications\n"
@@ -382,7 +385,7 @@
 }
 
 static void init(int argc, char **argv) {
-  int c;
+  int c, c1;
   char *p;
   const char *user = NULL;
   const char *rootdir = NULL, *workdir = NULL, *pidfile = NULL;
@@ -400,7 +403,7 @@
 
   if (argc <= 1) usage(1);
 
-  while((c = getopt(argc, argv, "u:r:b:w:t:c:p:nel:qs:h46dvafH:")) != EOF)
+  while((c = getopt(argc, argv, "u:r:b:w:t:c:p:nel:L:qs:h46dvafH:")) != EOF)
     switch(c) {
     case 'u': user = optarg; break;
     case 'r': rootdir = optarg; break;
@@ -459,6 +462,12 @@
         logfile = NULL, flog = stdout;
       break;
 break;
+    case 'L':
+      if ( (c1 = satoi(optarg)) > 0)
+        logsamplingstep = c1;
+      else
+        error(0, "invalid sampling step for -L option: `%.50s'", optarg);
+    break;
     case 's':
 #ifdef NOSTATS
       fprintf(stderr,
@@ -968,7 +977,8 @@
   if (!r)
     return;
   if (flog)
-    logreply(&pkt, (struct sockaddr *)&sa, salen, flog, flushlog);
+    if ( ! ( ++logcount % logsamplingstep ) )
+      logreply(&pkt, (struct sockaddr *)&sa, salen, flog, flushlog);
 
   /* finally, send a reply */
   while(sendto(fd, pkt.p_buf, r, 0, (struct sockaddr *)&sa, salen) < 0)