[RC5] Re: Block sizes

Mike Spann mikes at gammalink.com
Mon Jul 20 13:40:41 EDT 1998

> Date: Fri, 17 Jul 1998 14:45:53 -0700 (PDT)
> From: Jason Untulis <untulis at netgate.net>
> Subject: Re: [RC5] d.net
> Joe Zbiciak <j-zbiciak1 at ti.com> wrote:
> : 'GMH13 at aol.com' said previously:
> : | 4)  During the past week someone on this list pointed out that when you
> : | request a 2^31 block there are several different combinations you might
> : | receive, making choosing a buffering size difficult. Would it be possible to
> : | setup the proxies such that they have four buffers (2^28, 2^29, 2^30, and
> : | 2^31)? 
> :
> : On its face, this seems like a good idea, and it might be made to work.
> : There are a couple problems though.
> :
> : First, there are 8 valid block sizes currently, if I understand things
> : correctly:
> :
> :  Block Size     Number of 2^28 blocks
> :  ----------     ---------------------
> :      2^28                1
> :      2^29                2
> :    3*2^28                3
> :      2^30                4
> :    5*2^28                5
> :    3*2^29                6
> :    7*2^28                7
> :      2^31                8
> :
> :
> : So, you'd actually need 8 lists.  You'd also still need the clients to
> : accept odd-sized blocks in larger blocks are simply unavailable.  (See
> : explanation below.)
> You still only need four because the four odd sizes are only created
> because the proxy wants to hand out the "partial" block created when a
> <2^31 block is created. With four queues for the four different
> requestable sizes, none of these funky sized "partials" ever need to be
> created.

You are correct that you only need 4 lists from the point of view of
handing them out.  Unfortunately there are additional problems when it
comes to reporting blocks.

Let's assume that the pproxy gets large chunks of blocks from the main
proxies.  They break them into smaller pieces for clients to process.  I
do not know the exact protocol used to communicate results to the main
proxies but summize it is one record per key range checked (insert the
usual blurb of how source code would help reduce suspicion and confusion)
which the main proxies use both to mark the range of keys as checked and
to update the by-cpu, by-os and by-version statistics. 

The current generation of pproxies try to combine completion messages when
sending them to the main proxies.  In doing so, they discard at least the
version information and possibly the cpu and os information.  This reduces
bandwidth between the pproxy and main proxy but also looses information in
the process.  Again, supposing that there is only one message and that
this one message must be used to report both key range checked as well as
configuration information, then there is no way to combine blocks from
different configuration clients.  Given this supposition the number of
lists required is less than or equal to 4 * number of configurations (cpu,
os, version) if you want to both minimize the traffic to the main proxies
and retain configuration information.  This may, or may not, be
complicated by whether or not the client sends configuration information
the proxy (or pproxy) when it requests blocks to check.  If the client
does NOT send configuration information, then the proxy (or pproxy) must
retain a memory of the configuration based on some information available
at the request time. This seems a non-trivial task given the number of
machines in the effort and the dynamic nature of some of the IP addresses. 

So, while I would love to see the pproxies behave in a manner I would
consider 'the right way', I doubt it will happen until V3 (or hell freezes
over and all the little devils go ice scatting) when we hopefully have
source code and the protocol is open for peer review and improvement. 

To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
rc5-digest subscribers replace rc5 with rc5-digest

More information about the rc5 mailing list