[RC5] If you were writing our next client...
Robert A. Rosenberg
Bob.Rosenberg at digitscorp.com
Mon May 3 23:20:14 EDT 1999
I feel that I might as well throw my 3 cents (2 cents adjusted for
inflation) into this discussion. Some of these comments may duplicate
(or expand on) other suggestions or things that already are
implemented (such as in the 440 level clients).
In some of these descriptions I will use the term KR (Key Range)
which is a shorthand way of saying a 2^28 key block. Thus the current
clients handle 1-8 KR blocks (1-32 KR for 440+ Clients).
FIFO support - If/When new KRs are added to a non-empty BUFF-IN
file, the new KRs should be added so that they get processed AFTER
any KRs that were in the file prior to the add (ie: The sequence of
checking will be the same as currently occurs if the Fetch/Add was
delayed until after the BUFF-IN file was totally empty).
Split Support - Provide the client with a command that will read
the current BUFF-IN file and copy the first x (as selected by the
user) KRs and move them to a split file (for transfer to another
non-internet connected client [see Sneakernet Support below]). The
result to the BUFF-IN file will be the same as occurs when the
removed KRs have been checked. Also provide a similar command to copy
the BUFF-OUT file for transfer from the Internet Connected Machine.
Note1: I know this latter result can be done by moving/renaming the
BUFF-OUT file but this provides the locking needed to insure that the
Client does not try to update the file while it is being
Note2: The client should suffix the file name with a number if the
current name is already in use (this allows the user to do a number
of split requests before distributing the files).
Sneakernet Support - Allow for the Client to read Split Files and
merge them into its BUFF-IN/BUFF-OUT files (as currently occurs if
you rename a BUFF-IN/BUFF-OUT from another client to the Checkpoint
file name and then start the client). The add to BUFF-IN should be
subject to the FIFO support in that the split files are added as if
they were the result of a FETCH. This action would be performed by
the user copying the split files into a separate directory and the
client reading all the files that are there and deleting them as
KR Support - When doing a FETCH or SPLIT allow the user to
designate the amount of data in (as at current) max-sized blocks OR
in KR blocks. Thus if the client asks for 100 2^31 blocks it will get
either 100 blocks (comprising from 100 to 800 KRs) or enough blocks
so the total count is 800-807 KRs (for split if the last BUFF-IN
block will go over 800 copy enough KRs to total 800 and reset the
block in BUFF-IN to be the rest of the original range). The count by
Blocks or KRs would be controlled via a switch on the Buffers Menu
and the menu used for Split.
KR Return Support - For cases where the Client is going to be
removed from service and the user does not want to (or can not)
transfer the BUFF-IN/BUFF-OUT files to another client, have a way to
flag the KRs in BUFF-IN as "not checked yet" and move them to
BUFF-OUT (to be uploaded before the Client is removed from service).
In the case of the in-progress block being more than one KR long and
one or more of these KRs having been checked already, slit the block
into two blocks during the transfer (one marked checked and one
marked to-be-ckecked). As an extra, if it is possible, preserve the
checked %-age for that in-progress KR so that when it is re-Served
the new client will pick up where the old client left off (as now
happens after a shut-down or recovery from the checkpoint file). Any
"Returned" KRs should be placed in a special queue on the Key Server
so that they get reassigned in preference to the KRs that have not
yet been Served. IOW, as long as there are Returned KRs in the queue
they used to service FETCH requesters until the queue is empty at
while time the current scheme is used. This will recycle any KRs that
were served and returned instead of the current method of leaving
them as gaps to be re-Served once all the ranges have been served at
KR Checked Validation Support - Allow the Client to optionally
supply the Key Server with a List of all KRs in its BUFF-IN file and
receive a list of all KRs that are marked as "already checked"
(presumably due to some other client having run out of KRs and
grabbed it as a "random" KR). The response will be "all your KRs
still need to be checked" or a list of KRs to bypass since they have
been checked. This will avoid the client wasting its time rechecking
KRs that will not be credited to the user.
I'm not sure if I'm missing any other suggestions that I came up with
but have temporarily forgotten but this should work as a start to add
to the list, draw fire, or trigger others to enhance my list <g>.
To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
rc5-digest subscribers replace rc5 with rc5-digest
More information about the rc5