[RC5] Lookin' at stats

Jim C. Nasby jim at nasby.net
Sat Jul 6 12:50:03 EDT 2002


This is close to what we're actually doing, and it's why it's so slow.
All of the main statistics are pulled out of 'secondary' tables
(Email_Rank, Team_Members, and Team_Rank for anyone who wants to look at
the code). These tables are incrementally updated every stats run with
the new data for the day, as well as with changes to accomodate retires
and changes to which participants are blocked/hidden. Because these
tables take retires into account, it's fast and easy to do lookups on
that information.

There is no equivalent secondary table for participant history, though.
The history page hits the main table that contains all the statistical
information for participants. This table is over 1.6G, so even with
indexing, lookups in this table can be slow, especially with RC5 where
many participants have over 1000 days of history. But because of how
retires work, 1000 days can be 1000 rows or 10,000 rows. No matter what
you do, reading 10,000 rows is going to take some effort.

This problem really took effect when we put RC5 online in the new format
because we could no longer keep this table cached entirely in memory.
This meant that every time we generated participant history (which used
to be happening on every participant summary page), we'd have to hit the
disk. This really bogged everything down horribly.

On Sat, Jul 06, 2002 at 10:51:36AM -0400, Scott Dodson wrote:
> Would it be possible to create some form of a linked list?  Each account
> detailing account ids of those accounts that have been 'retired' to it?
> Then when you bring up an account which has a retired acount beneath it
> it would then also retrieve the data for the old account and display that
> as well.  This will keep retires from actually reassigning blocks, but would
> allow for easy (maybe not) representation, and keep the data available 
> historically.
> 
> 
> On 29-Jun-2002, Jim C. Nasby wrote:
> > Yes, maybe, eventually. The problem is that the code used to generate
> > that nice little stat couldn't handle having all the RC5 data in the new
> > system, so we had to yank it. If nothing else it will be back once we
> > figure out how to do away with retires.
> > 
> > On Wed, Jun 19, 2002 at 01:48:28PM -0700, Jason Hartzell wrote:
> > > And one of the things I am actually missing from the previous stats box, is
> > > the little blurb on the bottom of my user summary page that says 
> > > 
> > > this is x% of the participants best day ever which was mm/dd/yyyy
> > > 
> > > Any chance that's coming back?
> > > 
> > > 
> > > --
> > > To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
> > > rc5-digest subscribers replace rc5 with rc5-digest
> > > 
> > 
> > -- 
> > Jim C. Nasby (aka Decibel!)                    jim at nasby.net
> > Member: Triangle Fraternity, Sports Car Club of America
> > Give your computer some brain candy! www.distributed.net Team #1828
> > 
> > Windows: "Where do you want to go today?"
> > Linux: "Where do you want to go tomorrow?"
> > FreeBSD: "Are you guys coming, or what?"
> > --
> > To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
> > rc5-digest subscribers replace rc5 with rc5-digest
> > 
> 
> -- 
> Scott Dodson  		PGP KEY id 0x5F9A9E5E 
> sdodson at sdodson.com	
> 
> --
> To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
> rc5-digest subscribers replace rc5 with rc5-digest
> 

-- 
Jim C. Nasby (aka Decibel!)                    jim at nasby.net
Member: Triangle Fraternity, Sports Car Club of America
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"
--
To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
rc5-digest subscribers replace rc5 with rc5-digest



More information about the rc5 mailing list