[RC5] If you were writing our next client...

Robert A. Rosenberg Bob.Rosenberg at digitscorp.com
Mon May 3 23:20:14 EDT 1999


I feel that I might as well throw my 3 cents (2 cents adjusted for 
inflation) into this discussion. Some of these comments may duplicate 
(or expand on) other suggestions or things that already are 
implemented (such as in the 440 level clients).

In some of these descriptions I will use the term KR (Key Range) 
which is a shorthand way of saying a 2^28 key block. Thus the current 
clients handle 1-8 KR blocks (1-32 KR for 440+ Clients).


   FIFO support - If/When new KRs are added to a non-empty BUFF-IN 
file, the new KRs should be added so that they get processed AFTER 
any KRs that were in the file prior to the add (ie: The sequence of 
checking will be the same as currently occurs if the Fetch/Add was 
delayed until after the BUFF-IN file was totally empty).

   Split Support - Provide the client with a command that will read 
the current BUFF-IN file and copy the first x (as selected by the 
user) KRs and move them to  a split file (for transfer to another 
non-internet connected client [see Sneakernet Support below]). The 
result to the BUFF-IN file will be the same as occurs when the 
removed KRs have been checked. Also provide a similar command to copy 
the BUFF-OUT file for transfer from the Internet Connected Machine.
  Note1: I know this latter result can be done by moving/renaming the 
BUFF-OUT file but this provides the locking needed to insure that the 
Client does not try to update the file while it is being 
moved/renamed/duplicated.
  Note2: The client should suffix the file name with a number if the 
current name is already in use (this allows the user to do a number 
of split requests before distributing the files).

   Sneakernet Support - Allow for the Client to read Split Files and 
merge them into its BUFF-IN/BUFF-OUT files (as currently occurs if 
you rename a BUFF-IN/BUFF-OUT from another client to the Checkpoint 
file name and then start the client). The add to BUFF-IN should be 
subject to the FIFO support in that the split files are added as if 
they were the result of a FETCH. This action would be performed by 
the user copying the split files into a separate directory and the 
client reading all the files that are there and deleting them as 
processed.

   KR Support - When doing a FETCH or SPLIT allow the user to 
designate the amount of data in (as at current) max-sized blocks OR 
in KR blocks. Thus if the client asks for 100 2^31 blocks it will get 
either 100 blocks (comprising from 100 to 800 KRs) or enough blocks 
so the total count is 800-807 KRs (for split if the last BUFF-IN 
block will go over 800 copy enough KRs to total 800 and reset the 
block in BUFF-IN to be the rest of the original range). The count by 
Blocks or KRs would be controlled via a switch on the Buffers Menu 
and the menu used for Split.

   KR Return Support - For cases where the Client is going to be 
removed from service and the user does not want to (or can not) 
transfer the BUFF-IN/BUFF-OUT files to another client, have a way to 
flag the KRs in BUFF-IN as "not checked yet" and move them to 
BUFF-OUT (to be uploaded before the Client is removed from service). 
In the case of the in-progress block being more than one KR long and 
one or more of these KRs having been checked already, slit the block 
into two blocks during the transfer (one marked checked and one 
marked to-be-ckecked). As an extra, if it is possible, preserve the 
checked %-age for that in-progress KR so that when it is re-Served 
the new client will pick up where the old client left off (as now 
happens after a shut-down or recovery from the checkpoint file). Any 
"Returned" KRs should be placed in a special queue on the Key Server 
so that they get reassigned in preference to the KRs that have not 
yet been Served. IOW, as long as there are Returned KRs in the queue 
they used to service FETCH requesters until the queue is empty at 
while time the current scheme is used. This will recycle any KRs that 
were served and returned instead of the current method of leaving 
them as gaps to be re-Served once all the ranges have been served at 
least once.

   KR Checked Validation Support - Allow the Client to optionally 
supply the Key Server with a List of all KRs in its BUFF-IN file and 
receive a list of all KRs that are marked as "already checked" 
(presumably due to some other client having run out of KRs and 
grabbed it as a "random" KR). The response will be "all your KRs 
still need to be checked" or a list of KRs to bypass since they have 
been checked. This will avoid the client wasting its time rechecking 
KRs that will not be credited to the user.


I'm not sure if I'm missing any other suggestions that I came up with 
but have temporarily forgotten but this should work as a start to add 
to the list, draw fire, or trigger others to enhance my list <g>.

--
To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
rc5-digest subscribers replace rc5 with rc5-digest



More information about the rc5 mailing list