[RC5] d.net project: indexing the web

Matt.Wilkie Matt.Wilkie at gov.yk.ca
Thu Jul 8 14:59:58 EDT 1999


Prompted by the "Distributed Indexing thread on /.:
(http://slashdot.org/article.pl?sid=99/07/08/136255&mode=thread)

In a nutshell, current search engines only index about 16% of the total
web. There's simply too much data to sift through from a centralized
computer. Harnessing individual computers to index as they browse 
could potentially  greatly improve search engine's indices.

I think d.net could help craft a solution to this problem because: 

1) (to the best of my knowledge) most well developed distributed 
computing  network in existence today

2) d.net developers and participants are well aware of privacy 
issues, and anything they build in this line will be privacy 'compliant'

3) a proprietary version of this is doomed to failure, nobody will
trust it and it wouldn't get broadly enough used.

What does everybody else think?

-matt

--
To unsubscribe, send 'unsubscribe rc5' to majordomo at lists.distributed.net
rc5-digest subscribers replace rc5 with rc5-digest



More information about the rc5 mailing list