[stats-dev] log loader

Jeff Lawson jlawson at bovine.net
Fri Apr 22 04:03:04 EDT 2005


> A few words about the pre-processor; If we use a C 
> pre-processor, it's obviously faster, but at a cost of 
> portability. 

The performance difference might not be that bad if you ensure that the
limiting rate is the I/O of the database insertions, and not your other
pre-processing activities.  For example, by using threading to continue to
parse log lines while you're waiting for the database to do the bulk insert
statement that is executing in another thread.  Of course threading in Perl
is a rather rarely used feature and some consider it to still be a little
experimental.



More information about the stats-dev mailing list