tahoe-lafs/tahoe-lafs.git
17 years agoiputil.py: remove unused import
Brian Warner [Tue, 17 Apr 2007 00:08:00 +0000 (17:08 -0700)]
iputil.py: remove unused import

17 years agodownload: verify that bad blocks or hashes are caught by the download process
Brian Warner [Mon, 16 Apr 2007 23:30:21 +0000 (16:30 -0700)]
download: verify that bad blocks or hashes are caught by the download process

17 years agostorageserver: ignore files in verifierdir whose filenames aren't of the right form...
Zooko O'Whielacronx [Wed, 18 Apr 2007 14:41:56 +0000 (07:41 -0700)]
storageserver: ignore files in verifierdir whose filenames aren't of the right form for shares

17 years agopyutil: iputil: fix netbsd, irix, sunos
Zooko O'Whielacronx [Wed, 18 Apr 2007 14:40:26 +0000 (07:40 -0700)]
pyutil: iputil: fix netbsd, irix, sunos

17 years agotest_system: bump up timeout again, the new extra download forms take more time on...
Brian Warner [Mon, 16 Apr 2007 22:53:25 +0000 (15:53 -0700)]
test_system: bump up timeout again, the new extra download forms take more time on poor overloaded slave1

17 years agotest_iputil: improve error message
Brian Warner [Mon, 16 Apr 2007 22:05:25 +0000 (15:05 -0700)]
test_iputil: improve error message

17 years agodownload: log more information when hashtree checks fail
Brian Warner [Mon, 16 Apr 2007 20:08:19 +0000 (13:08 -0700)]
download: log more information when hashtree checks fail

17 years agodownload: improve test coverage on our IDownloadTarget classes, make FileHandle retur...
Brian Warner [Mon, 16 Apr 2007 20:07:36 +0000 (13:07 -0700)]
download: improve test coverage on our IDownloadTarget classes, make FileHandle return the filehandle when its done so that it is easier to close

17 years agodon't include test code itself in the test-coverage numbers
Brian Warner [Mon, 16 Apr 2007 19:32:45 +0000 (12:32 -0700)]
don't include test code itself in the test-coverage numbers

17 years agofec: add test for mathutil
Brian Warner [Mon, 16 Apr 2007 19:01:58 +0000 (12:01 -0700)]
fec: add test for mathutil

17 years agofec: remove unused mathutil.linear_fit_slope
Brian Warner [Mon, 16 Apr 2007 18:51:36 +0000 (11:51 -0700)]
fec: remove unused mathutil.linear_fit_slope

17 years agofec.util.mathutil.permute: fix docstring
Brian Warner [Mon, 16 Apr 2007 18:51:04 +0000 (11:51 -0700)]
fec.util.mathutil.permute: fix docstring

17 years agointerfaces: use explicit TupleOf and ChoiceOf constraints, since the upcoming version...
Brian Warner [Sat, 14 Apr 2007 02:04:38 +0000 (19:04 -0700)]
interfaces: use explicit TupleOf and ChoiceOf constraints, since the upcoming version of Foolscap changes the meaning of bare tuples (from ChoiceOf to TupleOf)

17 years agoencode.py: remove an unused import
Brian Warner [Fri, 13 Apr 2007 03:09:32 +0000 (20:09 -0700)]
encode.py: remove an unused import

17 years agotest_hashtree.py: get full coverage for hashtree.py
Brian Warner [Fri, 13 Apr 2007 02:58:13 +0000 (19:58 -0700)]
test_hashtree.py: get full coverage for hashtree.py

17 years agohash trees: further cleanup, to make sure we're validating the right thing
Brian Warner [Fri, 13 Apr 2007 02:41:48 +0000 (19:41 -0700)]
hash trees: further cleanup, to make sure we're validating the right thing
hashtree.py: improve the methods available for finding out which hash nodes
 are needed. Change set_hashes() to require that every hash provided can
 be validated up to the root.
download.py: validate from the top down, including the URI-derived roothash
 in the share hash tree, and stashing the thus-validated share hash for use
 in the block hash tree.

17 years agodownload: always validate the blockhash, and don't let the bucket trick us into not...
Brian Warner [Thu, 12 Apr 2007 22:18:46 +0000 (15:18 -0700)]
download: always validate the blockhash, and don't let the bucket trick us into not validating hashes

17 years agohashtree.py: reindent from 2-spaces to 4-spaces. No functional changes.
Brian Warner [Thu, 12 Apr 2007 21:24:11 +0000 (14:24 -0700)]
hashtree.py: reindent from 2-spaces to 4-spaces. No functional changes.

17 years agorename chunk.py to hashtree.py
Brian Warner [Thu, 12 Apr 2007 20:13:25 +0000 (13:13 -0700)]
rename chunk.py to hashtree.py

17 years agoverify hash chains on incoming blocks
Brian Warner [Thu, 12 Apr 2007 20:07:40 +0000 (13:07 -0700)]
verify hash chains on incoming blocks
Implement enough of chunk.IncompleteHashTree to be usable.
Rearrange download: all block/hash requests now go through
a ValidatedBucket instance, which is responsible for retrieving
and verifying hashes before providing validated data. Download
was changed to use ValidatedBuckets everywhere instead of
unwrapped RIBucketReader references.

17 years agooops -- the previous commit of iputil wasn't the right version
Zooko O'Whielacronx [Mon, 16 Apr 2007 22:12:01 +0000 (15:12 -0700)]
oops -- the previous commit of iputil wasn't the right version
Too bad synchronizing pyutil and allmydata.util includes a manual step.

17 years agoport iputil to Windows (and Irix, and NetBSD, and Solaris 2, ...)
Zooko O'Whielacronx [Mon, 16 Apr 2007 21:59:13 +0000 (14:59 -0700)]
port iputil to Windows (and Irix, and NetBSD, and Solaris 2, ...)

17 years agopyfec: bump version number to 1.0.0a1-2-STABLE
Zooko O'Whielacronx [Sun, 15 Apr 2007 19:08:12 +0000 (12:08 -0700)]
pyfec: bump version number to  1.0.0a1-2-STABLE

17 years agopyfec: add mention of sha256sum to README
Zooko O'Whielacronx [Sun, 15 Apr 2007 19:07:35 +0000 (12:07 -0700)]
pyfec: add mention of sha256sum to README

17 years agopyfec: bump the performance measurement bragging up higher in the README
Zooko O'Whielacronx [Sun, 15 Apr 2007 01:03:54 +0000 (18:03 -0700)]
pyfec: bump the performance measurement bragging up higher in the README

17 years agopyfec: bump version number to 1.0.0a1-1-STABLE
Zooko O'Whielacronx [Sun, 15 Apr 2007 00:50:40 +0000 (17:50 -0700)]
pyfec: bump version number to 1.0.0a1-1-STABLE

17 years agopyfec: add -f option to fec, add more user-friendly handling of filesystem errors...
Zooko O'Whielacronx [Sun, 15 Apr 2007 00:48:32 +0000 (17:48 -0700)]
pyfec: add -f option to fec, add more user-friendly handling of filesystem errors and user errors

17 years agopyfec: add bragging about how fec is way faster than "par2"
Zooko O'Whielacronx [Sun, 15 Apr 2007 00:47:48 +0000 (17:47 -0700)]
pyfec: add bragging about how fec is way faster than "par2"

17 years agopyfec: more progress indicators, handling of already-existent outfile
Zooko O'Whielacronx [Sat, 14 Apr 2007 23:00:59 +0000 (16:00 -0700)]
pyfec: more progress indicators, handling of already-existent outfile

17 years agopyfec: argparse: add a feature of exclusivecreate to the argparse FileType
Zooko O'Whielacronx [Sat, 14 Apr 2007 23:00:33 +0000 (16:00 -0700)]
pyfec: argparse: add a feature of exclusivecreate to the argparse FileType

17 years agopyfec: argparse: simplify
Zooko O'Whielacronx [Sat, 14 Apr 2007 22:43:41 +0000 (15:43 -0700)]
pyfec: argparse: simplify

17 years agopyfec: fix up docs, version numbers, bump version to 1.0.0a1-0-STABLE
Zooko O'Whielacronx [Sat, 14 Apr 2007 19:54:51 +0000 (12:54 -0700)]
pyfec: fix up docs, version numbers, bump version to 1.0.0a1-0-STABLE

17 years agopyfec: update README and bump version number to 0.9.9-0-STABLE
Zooko O'Whielacronx [Sat, 14 Apr 2007 19:02:18 +0000 (12:02 -0700)]
pyfec: update README and bump version number to 0.9.9-0-STABLE

17 years agopyfec: add bin/fec and bin/unfec, do better handling and reporting of various errors
Zooko O'Whielacronx [Sat, 14 Apr 2007 19:00:10 +0000 (12:00 -0700)]
pyfec: add bin/fec and bin/unfec, do better handling and reporting of various errors

17 years agopyfec: new filefec with compressed metadata, better error handling, much better unit...
Zooko O'Whielacronx [Sat, 14 Apr 2007 18:19:24 +0000 (11:19 -0700)]
pyfec: new filefec with compressed metadata, better error handling, much better unit tests

17 years agopyfec: import a copy of the argparse module
Zooko O'Whielacronx [Sat, 14 Apr 2007 18:05:54 +0000 (11:05 -0700)]
pyfec: import a copy of the argparse module

17 years agopyfec: import a copy of mathutil.py from the pyutil library
Zooko O'Whielacronx [Sat, 14 Apr 2007 17:54:43 +0000 (10:54 -0700)]
pyfec: import a copy of mathutil.py from the pyutil library

17 years agopyfec: import a copy of fileutil.py from the pyutil library
Zooko O'Whielacronx [Sat, 14 Apr 2007 17:54:02 +0000 (10:54 -0700)]
pyfec: import a copy of fileutil.py from the pyutil library

17 years agopyfec: add easyfec.Decoder(), which is easier to use than fec.Decoder()
Zooko O'Whielacronx [Sat, 14 Apr 2007 17:52:45 +0000 (10:52 -0700)]
pyfec: add easyfec.Decoder(), which is easier to use than fec.Decoder()

17 years agopyfec: tighten internal datatype from int to short and undo attempt to make it unsign...
Zooko O'Whielacronx [Sat, 14 Apr 2007 17:51:48 +0000 (10:51 -0700)]
pyfec: tighten internal datatype from int to short and undo attempt to make it unsigned (Python doesn't do unsigned short)

17 years agogive it a version number -- v0.9
Zooko O'Whielacronx [Wed, 11 Apr 2007 17:49:33 +0000 (10:49 -0700)]
give it a version number -- v0.9
I intend to bump it to 1.0 after adding the cmdline tools.

17 years agoimport Version from pyutil
Zooko O'Whielacronx [Wed, 11 Apr 2007 17:48:44 +0000 (10:48 -0700)]
import Version from pyutil

17 years agomerge changes and fix wrong type -- k and m need more than 8 bits (because they are...
Zooko O'Whielacronx [Wed, 11 Apr 2007 17:34:27 +0000 (10:34 -0700)]
merge changes and fix wrong type -- k and m need more than 8 bits (because they are the count rather than the index, i.e. they are 1-indexed)

17 years agopyfec: make unit test failures print out a few useful bytes instead of kibibytes
Zooko O'Whielacronx [Thu, 1 Feb 2007 02:02:57 +0000 (19:02 -0700)]
pyfec: make unit test failures print out a few useful bytes instead of kibibytes

17 years agopyfec: add variant of randomized test which happens to match the interface to benchutil
Zooko O'Whielacronx [Thu, 1 Feb 2007 05:55:28 +0000 (22:55 -0700)]
pyfec: add variant of randomized test which happens to match the interface to benchutil

17 years agopyfec: make randomized unit tests more comprehensive
Zooko O'Whielacronx [Thu, 1 Feb 2007 02:02:24 +0000 (19:02 -0700)]
pyfec: make randomized unit tests more comprehensive

17 years agopyfec: tighten internal C types
Zooko O'Whielacronx [Wed, 31 Jan 2007 20:55:31 +0000 (13:55 -0700)]
pyfec: tighten internal C types

17 years agopyfec: fix precondition checks on k and m to actually check the value before coercing...
Zooko O'Whielacronx [Thu, 1 Feb 2007 02:01:40 +0000 (19:01 -0700)]
pyfec: fix precondition checks on k and m to actually check the value before coercing it into a smaller type (oops)

17 years agopyfec: loosen preconditions -- you can have up to 256 total shares, not up to 255...
Zooko O'Whielacronx [Wed, 31 Jan 2007 23:53:16 +0000 (16:53 -0700)]
pyfec: loosen preconditions -- you can have up to 256 total shares, not up to 255 total shares

17 years agosilence warning when building
Zooko O'Whielacronx [Wed, 11 Apr 2007 15:50:59 +0000 (08:50 -0700)]
silence warning when building

17 years agoport GNUmakefile to Windows, cygwin, Linux, MacOSX/PPC, MacOSX/Intel
Zooko O'Whielacronx [Tue, 10 Apr 2007 19:24:32 +0000 (12:24 -0700)]
port GNUmakefile to Windows, cygwin, Linux, MacOSX/PPC, MacOSX/Intel
And of course it may well work on lots of other modern unixes, too, especially the more GNUish ones.

17 years agomerge TODO into roadmap.txt
Zooko O'Whielacronx [Tue, 10 Apr 2007 17:33:38 +0000 (10:33 -0700)]
merge TODO into roadmap.txt

17 years agoadd TODO: port to Windows
Zooko O'Whielacronx [Tue, 10 Apr 2007 16:27:58 +0000 (09:27 -0700)]
add TODO: port to Windows

17 years agoupdate TODO (add operational TODOs)
Zooko O'Whielacronx [Mon, 9 Apr 2007 23:20:34 +0000 (16:20 -0700)]
update TODO (add operational TODOs)

17 years agotweak README
Zooko O'Whielacronx [Mon, 9 Apr 2007 23:14:44 +0000 (16:14 -0700)]
tweak README

17 years agotest_util: add more coverage for assertutil.py
Brian Warner [Sun, 8 Apr 2007 20:02:13 +0000 (13:02 -0700)]
test_util: add more coverage for assertutil.py

17 years agotest_util: add full coverage for mathutil.py
Brian Warner [Sun, 8 Apr 2007 19:43:01 +0000 (12:43 -0700)]
test_util: add full coverage for mathutil.py

17 years agodownload: retrieve share hashes when downloading. We don't really do much validation...
Brian Warner [Sat, 7 Apr 2007 05:51:19 +0000 (22:51 -0700)]
download: retrieve share hashes when downloading. We don't really do much validation with them yet, though.

17 years agocodec: remove more unused debugging code
Brian Warner [Sat, 7 Apr 2007 03:56:39 +0000 (20:56 -0700)]
codec: remove more unused debugging code

17 years agonode: enable Tub.logLocalFailures now that we require foolscap-0.1.2
Brian Warner [Sat, 7 Apr 2007 03:55:59 +0000 (20:55 -0700)]
node: enable Tub.logLocalFailures now that we require foolscap-0.1.2

17 years agocodec: remove some unused debugging code
Brian Warner [Sat, 7 Apr 2007 03:55:37 +0000 (20:55 -0700)]
codec: remove some unused debugging code

17 years agodebian: remove the lines that install the now-removed .tac files
Brian Warner [Sat, 7 Apr 2007 03:51:32 +0000 (20:51 -0700)]
debian: remove the lines that install the now-removed .tac files

17 years agomisc cleanup: remove old .tac files, move old stuff into misc/
Brian Warner [Sat, 7 Apr 2007 03:37:38 +0000 (20:37 -0700)]
misc cleanup: remove old .tac files, move old stuff into misc/

17 years agotest_system: bump up timeouts for the sake of slow slave1, give each test a separate...
Brian Warner [Sat, 7 Apr 2007 03:34:32 +0000 (20:34 -0700)]
test_system: bump up timeouts for the sake of slow slave1, give each test a separate base directory

17 years agoinclude pyfec in the trial tests
Brian Warner [Sat, 7 Apr 2007 01:17:23 +0000 (18:17 -0700)]
include pyfec in the trial tests

17 years agoencode: add more logging to investigate occasional test failures
Brian Warner [Sat, 7 Apr 2007 01:04:38 +0000 (18:04 -0700)]
encode: add more logging to investigate occasional test failures

17 years agofigleaf output: include a stats.out for parsing by a munin plugin
Brian Warner [Sat, 7 Apr 2007 00:41:01 +0000 (17:41 -0700)]
figleaf output: include a stats.out for parsing by a munin plugin

17 years agotest_util.py: sigh, one last minor python-2.5 issue
Brian Warner [Sat, 7 Apr 2007 00:21:25 +0000 (17:21 -0700)]
test_util.py: sigh, one last minor python-2.5 issue

17 years agotest_util.py: fix another minor python-2.5 issue
Brian Warner [Sat, 7 Apr 2007 00:12:26 +0000 (17:12 -0700)]
test_util.py: fix another minor python-2.5 issue

17 years agotest_util.py: fix problems
Brian Warner [Fri, 6 Apr 2007 23:36:22 +0000 (16:36 -0700)]
test_util.py: fix problems

17 years agomore allmydata.util test coverage, rename the test case to be more correct
Brian Warner [Fri, 6 Apr 2007 23:33:22 +0000 (16:33 -0700)]
more allmydata.util test coverage, rename the test case to be more correct

17 years agoadd test coverage for stuff in allmydata/utils
Brian Warner [Fri, 6 Apr 2007 23:29:25 +0000 (16:29 -0700)]
add test coverage for stuff in allmydata/utils

17 years agoencode/upload: add more logging, to understand the test failure on a slow buildslave
Brian Warner [Fri, 6 Apr 2007 22:45:45 +0000 (15:45 -0700)]
encode/upload: add more logging, to understand the test failure on a slow buildslave

17 years agochunk: add IncompleteHashTree for download purposes, plus tests
Brian Warner [Fri, 6 Apr 2007 16:09:57 +0000 (09:09 -0700)]
chunk: add IncompleteHashTree for download purposes, plus tests

17 years agoencode: clean up some weirdness that was there to make unit tests easier to write
Brian Warner [Fri, 6 Apr 2007 05:36:18 +0000 (22:36 -0700)]
encode: clean up some weirdness that was there to make unit tests easier to write

17 years agorename encode_new.py to encode.py, now that there isn't an old one anymore
Brian Warner [Fri, 6 Apr 2007 04:17:42 +0000 (21:17 -0700)]
rename encode_new.py to encode.py, now that there isn't an old one anymore

17 years agoupdate roadmap: add download-peer-selection
Brian Warner [Thu, 5 Apr 2007 19:34:22 +0000 (12:34 -0700)]
update roadmap: add download-peer-selection

17 years agoadd more useful text to the README
Brian Warner [Thu, 5 Apr 2007 00:55:35 +0000 (17:55 -0700)]
add more useful text to the README

17 years agoadd .deb targets for edgy
Brian Warner [Wed, 4 Apr 2007 23:39:25 +0000 (16:39 -0700)]
add .deb targets for edgy

17 years agotests: clean up tearDown to use flushEventualQueue instead of hacking fixed-time...
Brian Warner [Wed, 4 Apr 2007 23:09:13 +0000 (16:09 -0700)]
tests: clean up tearDown to use flushEventualQueue instead of hacking fixed-time delays

17 years agoallmydata.interfaces: remove some of the placeholders now that we require foolscap...
Brian Warner [Wed, 4 Apr 2007 22:59:36 +0000 (15:59 -0700)]
allmydata.interfaces: remove some of the placeholders now that we require foolscap-0.1.2

17 years agouse the "binary" flag on open() for files that shouldn't have line-endings automatica...
Zooko O'Whielacronx [Wed, 4 Apr 2007 23:12:30 +0000 (16:12 -0700)]
use the "binary" flag on open() for files that shouldn't have line-endings automatically converted

17 years agoport the makefile to Windows and update the README
Zooko O'Whielacronx [Wed, 4 Apr 2007 18:18:38 +0000 (11:18 -0700)]
port the makefile to Windows and update the README

17 years agoadd Operations/Deployment for first open source release
Zooko O'Whielacronx [Tue, 3 Apr 2007 17:32:47 +0000 (10:32 -0700)]
add Operations/Deployment for first open source release
As per discussion I had with Peter this morning, we should do these three things and then release the first open source version.

17 years agoroadmap.txt: editing
Zooko O'Whielacronx [Tue, 3 Apr 2007 17:32:34 +0000 (10:32 -0700)]
roadmap.txt: editing

17 years agomark multiple shares per verifierid as done
Zooko O'Whielacronx [Tue, 3 Apr 2007 17:16:44 +0000 (10:16 -0700)]
mark multiple shares per verifierid as done

17 years agoroadmap: updated who is working on what
Zooko O'Whielacronx [Tue, 3 Apr 2007 17:16:20 +0000 (10:16 -0700)]
roadmap: updated who is working on what
Namely that we don't know who is working on what at the moment...

17 years agoroadmap.txt: mark LFE as done
Zooko O'Whielacronx [Tue, 3 Apr 2007 17:14:34 +0000 (10:14 -0700)]
roadmap.txt: mark LFE as done

17 years agotidiness: return res from logging of progress
Zooko O'Whielacronx [Mon, 2 Apr 2007 17:23:24 +0000 (10:23 -0700)]
tidiness: return res from logging of progress

17 years agoremove util.ring, no longer used. also remove test_ring.py
Brian Warner [Sun, 1 Apr 2007 03:38:43 +0000 (20:38 -0700)]
remove util.ring, no longer used. also remove test_ring.py

17 years agopyfec: tidy-up: #include the appropriate header
Zooko O'Whielacronx [Mon, 9 Apr 2007 23:16:18 +0000 (16:16 -0700)]
pyfec: tidy-up: #include the appropriate header

17 years agopyfec: trial-ize the unit tests, making sure to keep working if trial is unavailable
Brian Warner [Sat, 7 Apr 2007 01:16:50 +0000 (18:16 -0700)]
pyfec: trial-ize the unit tests, making sure to keep working if trial is unavailable

17 years agopyfec: rename and clarify -- "blocks" are the units of input/output of the codec...
Zooko O'Whielacronx [Fri, 30 Mar 2007 18:52:43 +0000 (11:52 -0700)]
pyfec: rename and clarify -- "blocks" are the units of input/output of the codec, "shares" are sequences of blocks (used to process arbitrary-length files)

17 years agofix BucketWriter to not create a finalhome until it is complete, and to clean up...
Zooko O'Whielacronx [Sat, 31 Mar 2007 01:01:56 +0000 (18:01 -0700)]
fix BucketWriter to not create a finalhome until it is complete, and to clean up the empty $VERIFIERID dir under incoming/ when it moves the last share out of it

17 years agolog a running total of how much of your file has been uploaded
Zooko O'Whielacronx [Sat, 31 Mar 2007 01:01:37 +0000 (18:01 -0700)]
log a running total of how much of your file has been uploaded

17 years agoassert that only dicts get passed to _got_response()
Zooko O'Whielacronx [Sat, 31 Mar 2007 01:00:40 +0000 (18:00 -0700)]
assert that only dicts get passed to _got_response()

17 years agofix storage server to handle the case that it has no directory at all when someone...
Zooko O'Whielacronx [Sat, 31 Mar 2007 00:12:07 +0000 (17:12 -0700)]
fix storage server to handle the case that it has no directory at all when someone asks for buckets

17 years agofix bug in storage-server: yes, "0" is a number, Mr. storage server
Zooko O'Whielacronx [Sat, 31 Mar 2007 00:07:04 +0000 (17:07 -0700)]
fix bug in storage-server: yes, "0" is a number, Mr. storage server

17 years agohandle foolscap's annoying tendency to turn everything into a sets.Set, which are...
Brian Warner [Fri, 30 Mar 2007 23:55:04 +0000 (16:55 -0700)]
handle foolscap's annoying tendency to turn everything into a sets.Set, which are not interoperable with 2.4's builtin 'set' type

17 years agofinish making the new encoder/decoder/upload/download work
Brian Warner [Fri, 30 Mar 2007 23:50:50 +0000 (16:50 -0700)]
finish making the new encoder/decoder/upload/download work