[rc5] personal proxy stats

Magnus Hagander mha at edu.sollentuna.se
Thu Aug 21 12:11:14 EDT 1997


On Wednesday, August 20, 1997 4:45 PM, etd at st-andrews.ac.uk
[SMTP:etd at st-andrews.ac.uk] wrote:
> Thus spoke the keyboard of Randy McCaskill (20/08/97):
> 
> >There was a program that someone had in perl that could give you
stats on
> >your machines based on the output of the personal proxy version 1 log
> >files.  Has someone done a similar thing for the output files of the
> >version 2 personal proxy?
> 
> The Sollentuna folks have been (are??) wrestling with this one. I've
not
> seen any results yet, but maybe they can help you out.
> 
> See:
> http://www.des.sollentuna.se
> and onwards.

Almost, but not quite :-)
First we had a script that handled the v1 logs (back when we only had
keymaster.des.sollentuna.se running personal proxy, nothing else). Then
I wrote a script to handle the logs from the _full v2 proxy_
(rc5proxy.des.sollentuna.se). This statistics program broke when the
logging method changed in one of the full proxy builds. And I have not
had the time/motivation to fix it.
I have not extensivly checked what the pproxies log with, but I beleive
it is at least very similar to what the full ones did back then.
For those interested, I used the following perl script to pull that
information into a SQL database (not. This is pretty ugly code. But I
was in a hurry...):
The program will first parse through what is in the log and replace the
SQL contents with that. After that, it will poll the log for changes and
use a SQL stored procedure to update the actual database. That way, the
databaseserver and webserver could run on the same machine as the
keyproxy, and use even less CPU (which totals to <1% on a P-133).


//Magnus
(former) SolNET DES Team

open(LOGFILE,"success.txt") || die "Could not open logfile!\n";
while (<LOGFILE>) {
   ($dat,$timocli,$at,$ip,$email, at rest) = split(" ");

   $email = lc substr($email,1,length($email)-2);

   $ipblocks{"$ip"}++;
   $emailblocks{"$email"}++;
}

open(EMF,">emails.sta") || die "Could not write to emails.sta!\n";
foreach $email (keys %emailblocks) {
   printf EMF "$email;%s;%s\n",$emailblocks{$email},time();
}
close(EMF);
system("isql -E -d rc5stat -Q \"delete from emails\"");
system("bcp rc5stat.dbo.emails in emails.sta -T -f email.fmt");

&Read_Cache();
open(IPF,">ip.sta") || die "Could not write to ip.sta!\n";
foreach $ip (keys %ipblocks) {
   $hn = &dns_resolve($ip);
   printf IPF
"$ip;%s;%s;%s;%s\n",$ipblocks{"$ip"},$hn,time(),length($hn);
}
close(IPF);
&Save_Cache();
system("isql -E -d rc5stat -Q \"delete from ips\"");
system("bcp rc5stat.dbo.ips in ip.sta -T -f ip.fmt");

while (1) {
   ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) =
localtime(time);
   $hour = "0$hour" if length($hour) < 2;
   $min  = "0$min" if length($min) < 2;
   $sec  = "0$sec" if length($sec) < 2;
   printf "$hour:$min:$sec  Now sleeping 30 seconds in wait for new
lines...\n";
   sleep(30);
   $i=0;
   while (<LOGFILE>) {
      $i=1;
      printf(".");
      ($dat,$timocli,$at,$ip,$email, at rest) = split(" ");
      $email = substr($email,1,length($email)-2);
      $thetime = sprintf("%s",time());
      $hname = &dns_resolve($ip);
      $hlen = length($hname);
      system("isql -E -d rc5stat -Q \"increment_blocks
'$ip','$hname','$email',$hlen,$thetime\"");
   }
   printf "\n" if ($i > 0);
   &Save_Cache();
}
exit(0);

sub dns_resolve {
    local ($ip) = @_;
    if (length($dns{"$ip"}) > 1) {
        $hostname = $dns{"$ip"};
    }
    else {
        $hostname = gethostbyaddr(pack('C4',split(/\./,$ip)),AF_INET);
        if (length($hostname) <= 1) {
           $hostname = $ip;
        }
        $dns{"$ip"} = $hostname;
        $dnstime{"$ip"} = time();
        printf("Resolved $ip -> $hostname\n");
    }
    lc $hostname;
}

sub Read_Cache {
    open(CF,"cache.dns") || die "Could not open DNS cache!\n";
    while ($line = <CF>) {
        chomp($line);
        ($ip,$hostname,$lasttime) = split(';',$line);
        $dns{"$ip"} = $hostname;
        $dnstime{"$ip"} = $lasttime;
    }
    close(CF);
}
sub Save_Cache {
    open(CF,">cache.dns") || die "Could not write to DNS cache!\n";
    foreach $ip (keys %dns) {
        $hn = $dns{"$ip"};
        $t = $dnstime{"$ip"};
        printf CF "$ip;$hn;$t\n";
    }
    close(CF);
}


----
To unsubscribe, send email to majordomo at llamas.net with 'unsubscribe rc5' in the body.



More information about the rc5 mailing list