[RC5] Mergebuf?

James Mastros root at jennifer-unix.dyn.ml.org
Sat Apr 4 11:40:18 EST 1998

The basic reason that I don't like the ideas presented in the "Really useful
feature request" thread is that it seems overcomplex to me: instead of being
able to use multiple buff-* files, why not use LARGE buff-* files.  This
seems to me to be a much cleaner solution.  So, my plan (NB - This is a plan
for The Way Things Should be, not how to make them that way):

1)  Make the buff-in file be FIFO instead of FILO.  (Note:  It just seems
    cleaner that way -- for reasons brought up near the end of desII-1, this
    makes a minimal (but still present) effect on total time to completion.)
2)  Remove the size limitation on buff-* files.  Sugested format:
	\0\0\0#\0\0\0 -- Magic (note: differs from current \0\0\0@\0\0\0 magic)
	1 byte	      -- more magic 
				- first hex digit is contest:
					0 = RC5-64
					1 = DESII-2
					2 = whatever we do next
				- second hex digit is type of buffer:
					0 = in
					1 = out
					2 = checkpoint
	1 byte	      -- Size.  The value zero is magic, it means that there
			 are 256 more blocks in the file, followed by
		         another size byte.  (buff-* files that really have
			 zero size end right after the magic.)
	data	      -- As at present.
	(if size[0]==0) -- Another size byte.  If the buff-* file is exactly 
			256 blocks long, then we end right after the last block.

This format should be usable at least until we run out of contest numbers.
The only limit on the number of blocks storable in one file is the size of
the device it is on.  (Note: we could just rely on the filesize to determine
the number of blocks, but I think that would be more coruptable. This format
will be readable if random garbage is appended to it, and it is detectable
if the file is truncated.) Note, BTW, that this is very close to the current
format.  Since the code isn't released, I have no way of telling.  I do know
that the current format is \0\0\0@\0\0\0<size byte>, from examination, but I
don't know what happens when the size is over 255.  (In the current scheme,
empty files are 0x00 00 00 64 00 00 00 00 -- that is, the size byte is 0,
not non-existant, so it can't use 0 as a magic number.  BTW, the 64 dosn't
indicate rc5-64 -- des buffers use the same magic.)

3)  Create a utility to merge buff-* files.  This shouldn't be to
    difficult to do.

OK, with that done, running a client without any network IO is fairly
simple.  You give it a in-buffer.  Every now and again, you go to it with a
new buff-in, and merge the existing buffer file with the new one.  (IE mv
buff-in.rc5 old-buff-in.rc5; rc5cat /3.5/buff-in.rc5 old-buff-in.rc5 >
buff-in.rc5.  You want the new buff in to be on the bottom of the stack, so
you don't get stagnant blocks.  This does create a tiny window where the
in-buffer is invalid or non-existant, but if the client happens to need a
block during that window, it should just generate a random block, and
complain about having to do a random block because the buff-in is corrupted
or missing.) When you do that, you take it's out-buffer with you to a client
that can do network IO, and flush it from there.  (Just merge it with that
client's buff-out file, and that client will flush it on the next connect.)

Viola!  My plan may be harder to implement, but I think that it is far

	-=- James Mastros

To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
rc5-digest subscribers replace rc5 with rc5-digest

More information about the rc5 mailing list