[PROXYPER] Personal Proxy Questions

Andreas Beckmann andreas.beckmann at student.uni-halle.de
Sat Feb 13 21:48:20 EST 1999


Hi,

I'm running several sneakernet clients (buffer transfer on floppies) and
installed lately the 304 WIN PerProxy (ProxyPer?) to handle them offline on
my local machine - especially to protect the telephone monopolist from
earning too much money ;) So there is only one client (the proxy) that needs
a connection to fetch/flush blocks.
Using the proxy I 've got some questions:
1. The proxy requests always 64*2^28 blocks (short: 64 blocks). It gets
assigned the next available contiguous blocks from the full proxy which may
be less than 64. OK. When the in-buffer is nearly filled, it will need less
than 64 blocks (in most cases) to fill the remaining empty space. But the
proxy requests again 64 blocks. At least it writes this into the logfile,
see below. The full proxy either sends only the needed amout of blocks or
the overhead is thrown away and only the needed amout is reported in the
logfile. Logfile extract:
... fetching of 9967 blocks snipped ...
02/11/99 20:36:42,server: Queueing rc564 request, iter 64.
02/11/99 20:36:43,server: Assigned us rc564 827fe00f:10000000, iter 7
02/11/99 20:36:43,server: Queueing rc564 request, iter 64.
02/11/99 20:36:45,server: Assigned us rc564 827fe012:20000000, iter 26
02/11/99 20:37:13,Server 205.149.163.214 is no longer connected (reading).
02/11/99 20:37:13,server: Closing server connection.
02/11/99 20:37:21,rc564 r=10000/10000, d=0/10, 0.0 Mkeys/sec, tot=644
...

2. Why does the personal proxy report DES statistics even when DES is closed
(which is most of the year)? This adds lots of absolutely useless
information of the logfile. There might be added a switch to enable this to
be compatible to some log analyzers.
Logfile extract:
...
02/13/99 01:28:25,rc564 r=5041/10000, d=13/10, 5.6 Mkeys/sec, tot=9
02/13/99 01:28:25,desII r=0/20, d=0/100, 0.0 Mkeys/sec, tot=0
02/13/99 01:29:26,rc564 r=5041/10000, d=13/10, 4.9 Mkeys/sec, tot=9
02/13/99 01:29:26,desII r=0/20, d=0/100, 0.0 Mkeys/sec, tot=0
...

3. I like producing large buff-in.rc5 files (up to 500*8*2^28 blocks,
depending on processor etc.) for my sneakernet clients. So I don't need to
update them more than once per month. Because blocks wont be reissued for at
least 90 days, there is no problem. But buff-in's fetched from the personal
proxy contained always less 2^28 blocks than buff-in's fetched from a full
proxy. So I looked at the proxies logfiles and recognized the following: the
proxy got assigned lots of blocks with iter 64. But they are split in an
inefficient way. Example: one iter 64 block from the logfiles:
02/11/99 20:36:07,server: Assigned us rc564 827ec2f7:50000000, iter 64
02/11/99 21:00:55,127.0.0.1: assigned rc564 827ec2f7:50000000, iter 8
02/11/99 21:00:55,127.0.0.1: assigned rc564 827ec2f7:d0000000, iter 3
02/11/99 21:00:55,127.0.0.1: assigned rc564 827ec2f8:00000000, iter 5
02/11/99 21:00:56,127.0.0.1: assigned rc564 827ec2f8:50000000, iter 8
02/11/99 21:00:56,127.0.0.1: assigned rc564 827ec2f8:d0000000, iter 3
02/11/99 21:00:56,127.0.0.1: assigned rc564 827ec2f9:00000000, iter 5
02/11/99 21:00:56,127.0.0.1: assigned rc564 827ec2f9:50000000, iter 8
02/11/99 21:00:56,127.0.0.1: assigned rc564 827ec2f9:d0000000, iter 3
02/11/99 21:00:56,127.0.0.1: assigned rc564 827ec2fa:00000000, iter 5
02/11/99 21:00:56,127.0.0.1: assigned rc564 827ec2fa:50000000, iter 8
02/11/99 21:00:56,127.0.0.1: assigned rc564 827ec2fa:d0000000, iter 3
02/11/99 21:00:56,127.0.0.1: assigned rc564 827ec2fb:00000000, iter 5
I know, clients take blocks up to 8*2^28 keys (iter 8). Then I see from the
above: the higher 32 bit of these keys must be the same in all of the 8*2^28
keys.
Furthermore I see, the personal proxy splits all the iter 64 (or less)
internal into iter 16 blocks. Such a iter 16 block will be split most of the
time into three blocks when a client requests a 8*2^28 block. (two blocks
only in case of the first key being of form xxxxxxxx:00000000 or
xxxxxxxx:80000000). These three blocks will be (assumed: 0 < m < 8):
xxxxxxxi:m0000000 iter 8
xxxxxxxi:n0000000 iter (8-m)    (n = m+8)
xxxxxxxj:00000000 iter m    (j = i+1)
(in case 16 > m > 8 the result will be similar, the iter 8 block will be the
second block).
Wouldn't it be better to split the large_block assigned to the personal
proxy (up to iter 64) using following algorithm:
- take up to 15 blocks (block = 2^28 keys) from the large_block, so the
remaining_large_block starts with the following key: xxxxxxxx:00000000
- split the remaining_large_block into iter 16 internal_blocks
- there is a last_block yyyyyyyy:00000000 iter <16
depending on the actual large_block, not all three points must occur.
The result will provide more blocks of the actually requested size (for all
sizes in {2^29, 2^30, 2^31}) and less smaller blocks. After all, more
efficient block splitting into better aligned blocks will reduce bandwith.
I don't know the splitting algorithm used in the keymaster and the full
proxies. But perhaps this might be an idea there too.

If I see this wrong, please someone correct me.


BTW, is there some utility to merge two buffer files into one? This may be
helpful updating not yet empty buffers on sneakernet clients. Older clients
were able to execute another instance while the client was running. So I
could run
    rc5des -ckpoint newbuff-in.rc5
and stop the second instance after "recovering" from newbuff-in.rc5. But
since 432 (perhaps earlier) the client detects itself and exit
automatically, except when I used command line parameters
like: -bench*, -fetch/-flush/-update.
Or perhaps another command line parameter might be added:
-recover    recovers blocks from current checkpointfile (might be used
with -ckpoint <filename> to set the filename) or
-merge <file>    merges contents from file into the correct buff-in.xxx
Not to mention operations that put the new data below the current stack
(buff-in.xxx) ...


Who will read such a long email ? ;))

Waiting for StatsBox II,
Andreas

-----
CPU time is a terrible thing to waste!
Join the world's fastest computer - http://www.distributed.net - Team #11292


--
To unsubscribe, send 'unsubscribe proxyper' to majordomo at lists.distributed.net



More information about the proxyper mailing list