Brian Warner [Fri, 8 Jun 2007 04:47:21 +0000 (21:47 -0700)]
move almost all hashing to SHA256, consolidate into hashutil.py
The only SHA-1 hash that remains is used in the permutation of nodeids,
where we need to decide if we care about performance or long-term security.
I suspect that we could use a much weaker hash (and faster) hash for
this purpose. In the long run, we'll be doing thousands of such hashes
for each file uploaded or downloaded (one per known peer).
robk-org [Wed, 6 Jun 2007 19:49:03 +0000 (12:49 -0700)]
change the 'ez_setup.py' script to have distinct desired & minimum required versions of setuptools
and change zfec/setup.py's invocation of ez_setup to require 0.6a9 (which happens to be the default
version installed by apt-get on dapper machines) while leaving the default (desired) version at 0.6c5
Brian Warner [Sat, 2 Jun 2007 01:48:01 +0000 (18:48 -0700)]
move validation data to thingA, URI has storage_index plus thingA hash
This (compatibility-breaking) change moves much of the validation data and
encoding parameters out of the URI and into the so-called "thingA" block
(which will get a better name as soon as we find one we're comfortable with).
The URI retains the "storage_index" (a generalized term for the role that
we're currently using the verifierid for, the unique index for each file
that gets used by storage servers to decide which shares to return), the
decryption key, the needed_shares/total_shares counts (since they affect
peer selection), and the hash of the thingA block.
This shortens the URI and lets us add more kinds of validation data without
growing the URI (like plaintext merkle trees, to enable strong incremental
plaintext validation), at the cost of maybe 150 bytes of alacrity. Each
storage server holds an identical copy of the thingA block.
This is an incompatible change: new messages have been added to the storage
server interface, and the URI format has changed drastically.
switch from FieldStorage.value to FieldStorage.file
Unfortunately this doesn't make the O(n) memory usage go away. It might reduce the constants -- I'm not sure. I look forward to enhancement #54 -- memory usage tests!
Brian Warner [Wed, 23 May 2007 19:48:52 +0000 (12:48 -0700)]
node.py: use 'node.pem' for all nodes
Rather than use separate client.pem and introducer.pem files, use 'node.pem'
for all nodes regardless of what type it is. This is slightly cleaner, but
introduces a compatibility. Users who upgrade to this change should do
'mv client.pem node.pem' to avoid generating a new certificate and thus
changing their TubID.
refactor iputil and make it return addresses in descending order of goodness instead of in a set
Actually of course iputil can't tell exactly how good they are, and a wise user
of iputil will try all of them. But you can't try all of them simultaneously,
so you might as well try the best ones first.
don't check for existence of portnum file and then try to open it -- instead try to open it and catch exception
This avoids a race condition, also known as time-of-check-to-time-of-use.
don't test for existence of certfile and then try to open it -- instead try to open it and catch exception
This avoids a race condition, also known as time-of-check-to-time-of-use.
enable private upload, in which the file is inserted at the grid layer but not at the vdrive layer
This patch is actually by Faried Nawaz, as per ticket #33:
Brian Warner [Tue, 8 May 2007 02:10:24 +0000 (19:10 -0700)]
UNDO: forget about old peers (closes #26)
Add a new method to RIIntroducer, to allow the central introducer node to
remove peers from the active set after they've gone away. Without this,
client nodes accumulate stale peer FURLs forever. This introduces a
compatibility break, as old introducers won't know about the 'lost_peers'
message, although the errors produced are probably harmless.
Brian Warner [Tue, 8 May 2007 02:10:24 +0000 (19:10 -0700)]
forget about old peers (closes #26)
Add a new method to RIIntroducer, to allow the central introducer node to
remove peers from the active set after they've gone away. Without this,
client nodes accumulate stale peer FURLs forever. This introduces a
compatibility break, as old introducers won't know about the 'lost_peers'
message, although the errors produced are probably harmless.
Brian Warner [Fri, 4 May 2007 07:45:23 +0000 (00:45 -0700)]
test_introducer.py: turn off the defer.setDebugging(True) call that was left over from a troubleshooting session: it slows down the test suite by a factor of 2
bump test timeout even higher
It does indeed take longer than 2400 seconds to run test_upload_and_download on a virtual windows machine when the underlying real machine is heavily loaded down with filesystem analysis runs...
re-re-factored the download and install instructions
Now the instructions about how to download debian packages live on a separate
page of the wiki instead of on the front page or in the README. The README is
only about building from source. The front page contains pointers to those two
other pages (the debiandownloadpage and the README).