[RC5] client versions - AIX

Jonathan Smith jsmith at cygnacom.com
Wed Oct 2 11:42:41 EDT 2002

Actually since the all the blocks are a collection of keys, a fairly large
collection of keys, in the case of rc5-64, IIRC the smallest block was 2^28
keys, then as long as you don't split a block through a key you can hand out
blocks smaller than the pproxy received.

In fact in the current build of the pproxy I believe it can request block up
to 2^60 but can only hand out blocks of 2^32 (I could be off, but the basic
idea is right).  
The problem comes with recombining the blocks.  Because every block returned
by a client is tagged with an email address, client version, operating
system, and machine architecture, if the pproxy were to recombine the
returned blocks it would have to choose only one set of returned information
to send back the main proxies.  So data would be lost unless every machine
that talked to the pproxy was crunching for the same person/email address,
was the same client version, OS, and architecture.

Since some teams or groups run their own pproxy for the team to get better
stats that wouldn't work well.  Also by lumping the client version info
together it would prevent d.net from blackballing a client version that was
discovered to return bad data.

Having the pproxy calculate the stats has 2 problems.  First of all since
users can control pproxies and the machines they run on it would be possible
to generate fake stats and send them to d.net, so the stats generated by the
pproxy would have to be regenerated by d.net to ensure they were correct.
The 2nd problem with pproxies doing stats is that they can't see enough of
the block space to check for dupes.  Because you don't get credit for a
returning a block that has already been submitted the main proxy servers
need to compare returned blocks against a list of previously returned
blocks.  This list becomes huge, far to large to push to all the pproxies.
But without that data the pproxies can't compute correct stats.  I know that
towards the end of rc5-64 I usually have 3 to 4% difference between the
number of blocks my pproxy saw and the number I was credited for.   
This should be less of a problem in rc5-72 because it seems more care is
being taken to prevent random blocks from fragmenting the keyspace, but it
is still a problem.

-----Original Message-----
From: waldo kitty [mailto:wkitty42 at alltel.net]
Sent: Wednesday, October 02, 2002 9:25 AM
To: rc5 at lists.distributed.net
Subject: Re: [RC5] client versions - AIX

Jack Beglinger wrote:
> > As was noted earlier, it is typical for OGR workunits to take much
> > longer than RC5-64 workunits.  If you are using the most current
> > clients, there is no reason why you should not be able to process OGR
> > workunits until RC5-72 becomes available.  You are of course entitled
> > to ignore the OGR project if it isn't appealing to you.
> First off, a week is too long for any work unit.  

well, i guess we don't have to worry about you participating in any
distributed medical research studies, then... i'm in one where some
"units" have taken my machine 18 and 19 hours to complete... there are
10 "units" per block and one works on a block at a time... oh yeah, and
there's no checkpointing other than between "units"... so... 190 hours
or so... that works out to right at 8 days...

> It was also the most current
> version available.   OGR has no appeal to me - for this very reason.

why? the length of time it takes to process a unit?

i've left the rest of what you wrote below... it seems to be a decent
idea but it puts more on the perproxies than just proxy work.. it also
may not be possible for the perproxy to split a unit down for different
machines and then to recombine it before sending it off to the master
keyservers... if it was, wouldn't it have been done with the 64bit
stuff? i don't think that it can really be done due to the nature in
which the numbers are manipulated... when you shift or rotate numbers in
a register, you have to pull other bits in or push them out... if you
break a number into smaller parts and then try to rotate those smaller
parts, you don't get the same results because the bits being pulled in
or pushed out aren't the same... once you get past that very first
rotate and/or shift, you've already left the trail and can't get back to
it... it may be possible to do but the clients will have to talk amongst
themselves so that they can hand the other bits around to each other as
needed... at that point, we're into beowolf clusters and such...
> > While I'm mentioning long workunits, I'd like to point out to
> > everyone that RC5-72 workunits will be longer than RC5-64 workunits
> As I have mention prior too...  Give out bigger work units to personal
> Let them handout smaller usable ones (2^28 if required, maybe memory only
> clients or older machines) to the local machines.  Then the personal
> can re-gather the information and hand back the big work unit to the
server in
> sky. If a part is missing - it can locally rehand it out to get it fixed
and the
> block big block returned.  This way, some of the large farms out there,
> could hand out 2^48 or larger and let them do the sub work.  This way your
> server are off loaded.
> Also if the personal proxy runs out of work units... Allow it to pick the
> random block to use.  This way it will "create" a block and hand it out to
> local crunchers.  In the end the personal proxy starts to appear to be the
> center of AMP machine, with one DNET client on each processor... no
> matter the type.
> At the same time let the personal proxies hand back pre processed stats.
> So checking a completed blocked from a personal proxy, also gives back
> tallies for the stats.
> --
> To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
> rc5-digest subscribers replace rc5 with rc5-digest

      (@@)                      Waldo Kitty, Waldo's Place USA
__ooO_( )_Ooo_____________________ telnet://bbs.wpusa.dynip.com
_|_____|_____|_____|_____|_____|_____ http://www.wpusa.dynip.com
____|_____|_____|_____|_____|_____|_____ ftp://ftp.wpusa.dynip.com
_|_Eat_SPAM_to_email_me!_YUM!__|_____|_____ wkitty42 (at) alltel.net
To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
rc5-digest subscribers replace rc5 with rc5-digest
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.distributed.net/pipermail/rc5/attachments/20021002/b3952e53/attachment-0001.htm

More information about the rc5 mailing list