[plans] distributed.net .plan update

plans at nodezero.distributed.net plans at nodezero.distributed.net
Fri Oct 6 21:00:02 EDT 2000


distributed .plan updates in the last 24 hours: 
---

decibel :: 06-Oct-2000 07:22 (Friday) ::

With some of the bad things that have been happening with stats lately,
I thought it'd be nice to report something good for a change. :)

First, I've made a few changes to the old rc5 statsrun code recently that
have had a dramatic impact on the run time. Retroactive team joins used
to take an hour; it's now under a minute. Email ranking for yesterday used
to take over an hour; it's down to a few minutes. The overall run is back
down to a 'reasonable' amount of time (it was back in the 6-10 hour range,
depending on when it started running (the heavier activity during the US
day would slow it down)).

Here's the real good news though:  I've been running some rc5 logs through
the new stats system. Here's a sample...

[10:46:56] Spawning daily.pl for rc5 [10:47:01] Beginning daily processing
routines [10:51:30] retire.sql 5 completed successfully (276 seconds)
[10:53:24] dy_appendday.sql 5 completed successfully (100 seconds)
[10:56:39] em_update.sql 5 completed successfully (183 seconds) [10:58:03]
em_rank.sql 5 completed successfully (85 seconds) [11:02:30] tm_update.sql
5 completed successfully (274 seconds) [11:02:45] tm_rank.sql 5 completed
successfully (14 seconds) [11:03:02] dy_dailyblocks.sql 5 completed
successfully (16 seconds) [11:03:56] Looking for new logs, last log
processed was 20001005-14 [11:04:09] ogr20001005-15.log.gz received: 35,857
bytes in 1.5 seconds (23.9 KB/s) [11:04:16] ogr20001005-15.log.gz
successfully decompressed (82.8% compression) [11:04:18]
ogr20001005-15.log.gz successfully filtered through ./logmod_ogr.pl.
[11:04:47] ./workdir/ogr20001005-15.log.filtered successfully BCP'd; 2,275
rows at 58.33 rows/second.  [11:05:04] ogr20001005-15.log.gz successfully
processed.  [11:05:31] audit.sql 5 completed successfully (141 seconds)
[11:05:38] clearday.sql 5 completed successfully (15 seconds) [11:11:46]
backup.sql 5 completed successfully (368 seconds) [11:11:51] Daily
processing for 20000902 has completed [11:12:02] daily.pl complete for
rc5

This means that the rc5 run took less than half an hour, even though a
OGR hourly run snuck in the middle. Compare that to the 3+ hours that the
statsrun takes with the old system!  Granted, there are only 3 days worth
of rc5 data in the new system at this time, but due to the design of the
system, that should not have a large impact on the amount of time the run
takes.

We are still in the process of creating a version of the new php code for
rc5, but we should be ready to drop the old system entirely in the very
near future.


bovine :: 06-Oct-2000 12:25 (Friday) ::

I've updated my wormfree utility with a few more heuristics and strategies
for helping to clean and secure Win9x machines from an infection of one
of these replicating worms.  You can download this new version (both source
and compiled binary) from the same URL:
http://www1.distributed.net/~bovine/wormfree.zip


--
To unsubscribe, send 'unsubscribe plans' to majordomo at lists.distributed.net



More information about the plans mailing list