[rc5] distributed.net: a Generalised Compute Resource?

Matt J. Gumbley csa49 at keele.ac.uk
Fri Jun 20 19:33:03 EDT 1997


At 05:54 20/06/97 -0500, Adam wrote:
>I'm gonna hit the current issues in what's happening.

<snip>

>--- What's next:
>This seems to be a hot topic. Read the FAQ. The version 3 clients will allow
>us to dynamicly send out new modules for new projects. I'm implementing a
>small part of one of my OS protocols to do this. Hopefully by the time we
>crack RC5 these new clients should be available and we will be able to "hot
>swap" over to the mersenne prime work. From there the possibilities are very
>interesting. The goal however is to keep distributed.net together, building
>a faster and faster virtual machine working on interesting challenges. We
>passed an estimated Tera-operations per second several weeks ago, with the
>addition of the DES folks we should hit 5Tops/sec relatively soon.

Just a couple of thoughts... they've probably been thought of already, but
here goes...

The idea of a "generalised compute resource" is very interesting: that the
machines we currently use give up some cycles to *whatever job we throw at
them*.

Here's a (rather rambling) Friday-afternoon thought for you:

Each machine runs a shell. This provides the following:
1) The user interface, a la Bovine Win32 client, perhaps.
2) The network I/O connection to the proxies.
Using the network I/O routines, the shell downloads:
   a) The computational "engine" which may be:
      the RC5 algorithm, or
      the DES algorithm, or
      the Mersenne Primes algorithm, or
      a neural net algorithm (why restrict ourselves to cryptography?)
      <insert Your Favourite Computational Task here>
   b) Some range of the problem space (keyspace) to work with.
      I'll call this range "keys" for brevity.

Can these two downloadable elements be generalised, such that any
computational task we want to perform can be distributed? The mention above
of the v3 clients' "hot swapping" ability seems to suggest this.

Java seems to be one "way forward". It may be inefficient for the
computational part, but would be ideal for the shell. The shell could use
the "os.name" property to provide complete portability for different OSs.
[Gos97] The computational part could be written using native methods. The
shell requests, say, a Pentium/Win32 native computational "engine", or a
SPARC/Solaris engine, etc. This "plugs in" to the shell, and off we go :-)

What are the possibilities for generalising the problem-space server end?
My rationale for generalising this comes from my research area: synchronous
groupware, although there is some overlap between this and distributed
computation.
   'Some ... applications come with their own "private"
application-specific server.
    The technical advantage of such a server is that it can have better
performance
    and integration with the clients than a general- purpose server. ...
Each new
    application requires a new server to be supported. However ... such
single-
    application servers are a nightmare. Each new application requires a new
    server to be supported. ... We believe that [these applications] are
simply
    one instance of a general class ... and we would like to support them
    accordingly.' [Day96]

So, create a general-purpose problem-space server, coupled to a set of
proxies, using perhaps the round-robin DNS approach used by
rc5proxy.distributed.net. This server *does not* know about the problem at
hand, i.e. RC5, Mersenne Primes. It talks to "the application", which
handles the problem, and the problem-space. 

We need a protocol for distributing "keys" which is not specific to the
problem.

I'm not sure how the client authors are currently getting round the
authentication problem, namely, how to prevent malicious clients from
taking parts of the keyspace and returning them as "nope, the key isn't in
this block".

I've seen mention of this in today's rc5 mail deluge :-)

This protocol would be used between:
1) "the application" and the server, 
2) the server and the proxies
3) the proxies and the clients

Call it, uh, DCTP: Distributed Computation Transfer Protocol. It would
allow transfer of the computational engines, "keys" to work on, and control
messages to flow between client and server (via the proxies). The server
knows nothing of the problem space and the format of the "keys" - they're
just undifferentiated bytestreams as far as the server is concerned. The
server passes completed "keys" to the application, and receives new "keys"
for the client to try. The proxies are similarly ignorant of the "keys",
but they do know about the computational engines, and send the appropriate
one to a client when requested. Similarly, "the application" knows nothing
about the computational engines - it just talks "keys". The application
should be able to tell the server to distribute new clients, for example,
when we switch over to Mersenne Primes, the new MP application tells the
server that any new requests for "keys" should return a new computational
engine.

Difficult to see without a picture perhaps. You're not looking at my
whiteboard :-)

Imagine distributed.net as a computational bureau, where customers ask us
to "run *this* application for a few months". We plug it into the server,
and all clients are (eventually) updated with new computational engines,
and parts of the problem space.

This could be one viable way of justifying the bizzare economics of today's
Internet. Your workstation is sitting idle on your desk at night. If it
could be "hired out" to perform computation, the customers who require this
would start to foot the bill for the infrastructure we have.

Comments please!
Matt.


References
[Day96] "What Synchronous Groupware needs: Notification Services", Mark
Day, Lotus Development Corp. Available from http://nstp.research.lotus.com
(Probably)
[Gos97] "The Feel of Java", James Gosling, IEEE Computer, June 1997

Matt Gumbley        | Email:  | 
Research Assistant  | csa49 @ |                Happy?
Rm 106, Comp. Sci.  | keele . | 
Dept., Keele Univ.  | ac . uk |  [These are my views, not necessarily
+44 (0)1782 583438  |         |    those of .cs.keele.ac.uk -- MJG]
----
To unsubscribe, send email to majordomo at llamas.net with 'unsubscribe rc5' in the body.



More information about the rc5 mailing list