Zooko O'Whielacronx [Thu, 19 Apr 2007 00:30:08 +0000 (17:30 -0700)]
iputil.list_async_addresses now "works" on cygwin
Zooko O'Whielacronx [Thu, 19 Apr 2007 00:28:36 +0000 (17:28 -0700)]
copy testutil from pyutil
Zooko O'Whielacronx [Thu, 19 Apr 2007 00:27:33 +0000 (17:27 -0700)]
copy repeatable_random from pyutil
Zooko O'Whielacronx [Thu, 19 Apr 2007 00:25:16 +0000 (17:25 -0700)]
finish renaminf "fec" to "zfec" in tahoe
Zooko O'Whielacronx [Thu, 19 Apr 2007 00:18:11 +0000 (17:18 -0700)]
zfec: bump version number to 1.0.0a2-1-STABLE
Zooko O'Whielacronx [Thu, 19 Apr 2007 00:11:13 +0000 (17:11 -0700)]
zfec: finish up some renaming of pyfec to zfec
Zooko O'Whielacronx [Wed, 18 Apr 2007 23:09:54 +0000 (16:09 -0700)]
GNUmakefile: clean up better
Zooko O'Whielacronx [Wed, 18 Apr 2007 23:01:03 +0000 (16:01 -0700)]
zfec: update docs, metadata, version number to 1.0.0a2-0-STABLE
Brian Warner [Wed, 18 Apr 2007 19:23:33 +0000 (12:23 -0700)]
rename bin/fec and bin/unfec to zfec and zunfec
Brian Warner [Wed, 18 Apr 2007 19:18:12 +0000 (12:18 -0700)]
rename top-level package from 'fec' to 'zfec'
Zooko O'Whielacronx [Wed, 18 Apr 2007 17:11:23 +0000 (10:11 -0700)]
rename "pyfec" to "zfec" within tahoe build system and source code
Brian Warner [Wed, 18 Apr 2007 04:22:32 +0000 (21:22 -0700)]
encode.py: remove unused pad() code
Brian Warner [Wed, 18 Apr 2007 04:11:20 +0000 (21:11 -0700)]
download: remove unused import
Brian Warner [Wed, 18 Apr 2007 03:37:51 +0000 (20:37 -0700)]
download: oops, NotEnoughHashesError comes from hashtree, not hashutil
Brian Warner [Wed, 18 Apr 2007 03:29:08 +0000 (20:29 -0700)]
encode: fix multi-segment uploads: lambdas inside for loops require special attention to make sure you are capturing the *value* of the loop variable and not just the slot it lives in
Brian Warner [Wed, 18 Apr 2007 03:28:19 +0000 (20:28 -0700)]
test_system: minor comment
Brian Warner [Wed, 18 Apr 2007 03:27:56 +0000 (20:27 -0700)]
factor out the tagged hash function used for subshares/blocks
Brian Warner [Wed, 18 Apr 2007 03:25:52 +0000 (20:25 -0700)]
comment out some verbose log messages, add commented-out new ones
Brian Warner [Wed, 18 Apr 2007 03:21:05 +0000 (20:21 -0700)]
storageserver: assert that blocks are written in-order, clean up tests a bit
Zooko O'Whielacronx [Wed, 18 Apr 2007 16:49:47 +0000 (09:49 -0700)]
GNUmakefile: rm -rf ./src/pyfec/build on clean
Zooko O'Whielacronx [Wed, 18 Apr 2007 16:48:24 +0000 (09:48 -0700)]
GNUmakefile: don't stop clean if clean-pyfec fails
Zooko O'Whielacronx [Wed, 18 Apr 2007 16:19:00 +0000 (09:19 -0700)]
pyfec: rename pyfec to zfec
It turns out that "pyfec" turns off people who aren't Python hackers.
"allmyfec" is too long for a low-level core utility library.
"fec" is too generic (it is already used by Luigi Rizzo's library which this
library is based on, for one thing).
I'm open to other naming suggestions, especially before we widely announce this
library, which I expect will happen within a few days.
Zooko O'Whielacronx [Wed, 18 Apr 2007 03:14:44 +0000 (20:14 -0700)]
don't clobber existing storefile every put block
Zooko O'Whielacronx [Wed, 18 Apr 2007 03:14:26 +0000 (20:14 -0700)]
pyutil: fileutil.open_or_create()
Brian Warner [Wed, 18 Apr 2007 03:03:44 +0000 (20:03 -0700)]
test_storage: add (failing) test of the BucketWriter/BucketReader implementation
Brian Warner [Tue, 17 Apr 2007 20:41:45 +0000 (13:41 -0700)]
storageserver: the last segment is not always going to be the same size as the rest, so don't assert such a thing
Brian Warner [Tue, 17 Apr 2007 20:40:47 +0000 (13:40 -0700)]
system_test: exercise multiple segments
Brian Warner [Tue, 17 Apr 2007 20:39:35 +0000 (13:39 -0700)]
download.py: don't truncate tail segments that are the same size as all the others
Brian Warner [Tue, 17 Apr 2007 19:57:55 +0000 (12:57 -0700)]
test_encode.Roundtrip: cover more combinations of data size relative to segment size and number of block hash tree leaves
Brian Warner [Tue, 17 Apr 2007 19:29:56 +0000 (12:29 -0700)]
test_encode.Encode: cover more combinations of data size relative to segment size and number of block hash tree leaves
Brian Warner [Tue, 17 Apr 2007 02:55:03 +0000 (19:55 -0700)]
test_encode: test filesizes which are an exact multiple of the segment size. This test fails right now.
Brian Warner [Tue, 17 Apr 2007 02:29:57 +0000 (19:29 -0700)]
encode: make MAX_SEGMENT_SIZE controllable, to support tests which force the use of multiple segments. Also, remove not-very-useful upload-side debug messages
Brian Warner [Tue, 17 Apr 2007 00:21:37 +0000 (17:21 -0700)]
download: more test coverage
Brian Warner [Tue, 17 Apr 2007 00:17:57 +0000 (17:17 -0700)]
download: remove some leftover (and not very useful) debug logging
Brian Warner [Tue, 17 Apr 2007 00:15:44 +0000 (17:15 -0700)]
download: validate handling of missing sharehashes too
Brian Warner [Tue, 17 Apr 2007 00:08:00 +0000 (17:08 -0700)]
iputil.py: remove unused import
Brian Warner [Mon, 16 Apr 2007 23:30:21 +0000 (16:30 -0700)]
download: verify that bad blocks or hashes are caught by the download process
Zooko O'Whielacronx [Wed, 18 Apr 2007 14:41:56 +0000 (07:41 -0700)]
storageserver: ignore files in verifierdir whose filenames aren't of the right form for shares
Zooko O'Whielacronx [Wed, 18 Apr 2007 14:40:26 +0000 (07:40 -0700)]
pyutil: iputil: fix netbsd, irix, sunos
Brian Warner [Mon, 16 Apr 2007 22:53:25 +0000 (15:53 -0700)]
test_system: bump up timeout again, the new extra download forms take more time on poor overloaded slave1
Brian Warner [Mon, 16 Apr 2007 22:05:25 +0000 (15:05 -0700)]
test_iputil: improve error message
Brian Warner [Mon, 16 Apr 2007 20:08:19 +0000 (13:08 -0700)]
download: log more information when hashtree checks fail
Brian Warner [Mon, 16 Apr 2007 20:07:36 +0000 (13:07 -0700)]
download: improve test coverage on our IDownloadTarget classes, make FileHandle return the filehandle when its done so that it is easier to close
Brian Warner [Mon, 16 Apr 2007 19:32:45 +0000 (12:32 -0700)]
don't include test code itself in the test-coverage numbers
Brian Warner [Mon, 16 Apr 2007 19:01:58 +0000 (12:01 -0700)]
fec: add test for mathutil
Brian Warner [Mon, 16 Apr 2007 18:51:36 +0000 (11:51 -0700)]
fec: remove unused mathutil.linear_fit_slope
Brian Warner [Mon, 16 Apr 2007 18:51:04 +0000 (11:51 -0700)]
fec.util.mathutil.permute: fix docstring
Brian Warner [Sat, 14 Apr 2007 02:04:38 +0000 (19:04 -0700)]
interfaces: use explicit TupleOf and ChoiceOf constraints, since the upcoming version of Foolscap changes the meaning of bare tuples (from ChoiceOf to TupleOf)
Brian Warner [Fri, 13 Apr 2007 03:09:32 +0000 (20:09 -0700)]
encode.py: remove an unused import
Brian Warner [Fri, 13 Apr 2007 02:58:13 +0000 (19:58 -0700)]
test_hashtree.py: get full coverage for hashtree.py
Brian Warner [Fri, 13 Apr 2007 02:41:48 +0000 (19:41 -0700)]
hash trees: further cleanup, to make sure we're validating the right thing
hashtree.py: improve the methods available for finding out which hash nodes
are needed. Change set_hashes() to require that every hash provided can
be validated up to the root.
download.py: validate from the top down, including the URI-derived roothash
in the share hash tree, and stashing the thus-validated share hash for use
in the block hash tree.
Brian Warner [Thu, 12 Apr 2007 22:18:46 +0000 (15:18 -0700)]
download: always validate the blockhash, and don't let the bucket trick us into not validating hashes
Brian Warner [Thu, 12 Apr 2007 21:24:11 +0000 (14:24 -0700)]
hashtree.py: reindent from 2-spaces to 4-spaces. No functional changes.
Brian Warner [Thu, 12 Apr 2007 20:13:25 +0000 (13:13 -0700)]
rename chunk.py to hashtree.py
Brian Warner [Thu, 12 Apr 2007 20:07:40 +0000 (13:07 -0700)]
verify hash chains on incoming blocks
Implement enough of chunk.IncompleteHashTree to be usable.
Rearrange download: all block/hash requests now go through
a ValidatedBucket instance, which is responsible for retrieving
and verifying hashes before providing validated data. Download
was changed to use ValidatedBuckets everywhere instead of
unwrapped RIBucketReader references.
Zooko O'Whielacronx [Mon, 16 Apr 2007 22:12:01 +0000 (15:12 -0700)]
oops -- the previous commit of iputil wasn't the right version
Too bad synchronizing pyutil and allmydata.util includes a manual step.
Zooko O'Whielacronx [Mon, 16 Apr 2007 21:59:13 +0000 (14:59 -0700)]
port iputil to Windows (and Irix, and NetBSD, and Solaris 2, ...)
Zooko O'Whielacronx [Sun, 15 Apr 2007 19:08:12 +0000 (12:08 -0700)]
pyfec: bump version number to 1.0.0a1-2-STABLE
Zooko O'Whielacronx [Sun, 15 Apr 2007 19:07:35 +0000 (12:07 -0700)]
pyfec: add mention of sha256sum to README
Zooko O'Whielacronx [Sun, 15 Apr 2007 01:03:54 +0000 (18:03 -0700)]
pyfec: bump the performance measurement bragging up higher in the README
Zooko O'Whielacronx [Sun, 15 Apr 2007 00:50:40 +0000 (17:50 -0700)]
pyfec: bump version number to 1.0.0a1-1-STABLE
Zooko O'Whielacronx [Sun, 15 Apr 2007 00:48:32 +0000 (17:48 -0700)]
pyfec: add -f option to fec, add more user-friendly handling of filesystem errors and user errors
Zooko O'Whielacronx [Sun, 15 Apr 2007 00:47:48 +0000 (17:47 -0700)]
pyfec: add bragging about how fec is way faster than "par2"
Zooko O'Whielacronx [Sat, 14 Apr 2007 23:00:59 +0000 (16:00 -0700)]
pyfec: more progress indicators, handling of already-existent outfile
Zooko O'Whielacronx [Sat, 14 Apr 2007 23:00:33 +0000 (16:00 -0700)]
pyfec: argparse: add a feature of exclusivecreate to the argparse FileType
Zooko O'Whielacronx [Sat, 14 Apr 2007 22:43:41 +0000 (15:43 -0700)]
pyfec: argparse: simplify
Zooko O'Whielacronx [Sat, 14 Apr 2007 19:54:51 +0000 (12:54 -0700)]
pyfec: fix up docs, version numbers, bump version to 1.0.0a1-0-STABLE
Zooko O'Whielacronx [Sat, 14 Apr 2007 19:02:18 +0000 (12:02 -0700)]
pyfec: update README and bump version number to 0.9.9-0-STABLE
Zooko O'Whielacronx [Sat, 14 Apr 2007 19:00:10 +0000 (12:00 -0700)]
pyfec: add bin/fec and bin/unfec, do better handling and reporting of various errors
Zooko O'Whielacronx [Sat, 14 Apr 2007 18:19:24 +0000 (11:19 -0700)]
pyfec: new filefec with compressed metadata, better error handling, much better unit tests
Zooko O'Whielacronx [Sat, 14 Apr 2007 18:05:54 +0000 (11:05 -0700)]
pyfec: import a copy of the argparse module
Zooko O'Whielacronx [Sat, 14 Apr 2007 17:54:43 +0000 (10:54 -0700)]
pyfec: import a copy of mathutil.py from the pyutil library
Zooko O'Whielacronx [Sat, 14 Apr 2007 17:54:02 +0000 (10:54 -0700)]
pyfec: import a copy of fileutil.py from the pyutil library
Zooko O'Whielacronx [Sat, 14 Apr 2007 17:52:45 +0000 (10:52 -0700)]
pyfec: add easyfec.Decoder(), which is easier to use than fec.Decoder()
Zooko O'Whielacronx [Sat, 14 Apr 2007 17:51:48 +0000 (10:51 -0700)]
pyfec: tighten internal datatype from int to short and undo attempt to make it unsigned (Python doesn't do unsigned short)
Zooko O'Whielacronx [Wed, 11 Apr 2007 17:49:33 +0000 (10:49 -0700)]
give it a version number -- v0.9
I intend to bump it to 1.0 after adding the cmdline tools.
Zooko O'Whielacronx [Wed, 11 Apr 2007 17:48:44 +0000 (10:48 -0700)]
import Version from pyutil
Zooko O'Whielacronx [Wed, 11 Apr 2007 17:34:27 +0000 (10:34 -0700)]
merge changes and fix wrong type -- k and m need more than 8 bits (because they are the count rather than the index, i.e. they are 1-indexed)
Zooko O'Whielacronx [Thu, 1 Feb 2007 02:02:57 +0000 (19:02 -0700)]
pyfec: make unit test failures print out a few useful bytes instead of kibibytes
Zooko O'Whielacronx [Thu, 1 Feb 2007 05:55:28 +0000 (22:55 -0700)]
pyfec: add variant of randomized test which happens to match the interface to benchutil
Zooko O'Whielacronx [Thu, 1 Feb 2007 02:02:24 +0000 (19:02 -0700)]
pyfec: make randomized unit tests more comprehensive
Zooko O'Whielacronx [Wed, 31 Jan 2007 20:55:31 +0000 (13:55 -0700)]
pyfec: tighten internal C types
Zooko O'Whielacronx [Thu, 1 Feb 2007 02:01:40 +0000 (19:01 -0700)]
pyfec: fix precondition checks on k and m to actually check the value before coercing it into a smaller type (oops)
Zooko O'Whielacronx [Wed, 31 Jan 2007 23:53:16 +0000 (16:53 -0700)]
pyfec: loosen preconditions -- you can have up to 256 total shares, not up to 255 total shares
Zooko O'Whielacronx [Wed, 11 Apr 2007 15:50:59 +0000 (08:50 -0700)]
silence warning when building
Zooko O'Whielacronx [Tue, 10 Apr 2007 19:24:32 +0000 (12:24 -0700)]
port GNUmakefile to Windows, cygwin, Linux, MacOSX/PPC, MacOSX/Intel
And of course it may well work on lots of other modern unixes, too, especially the more GNUish ones.
Zooko O'Whielacronx [Tue, 10 Apr 2007 17:33:38 +0000 (10:33 -0700)]
merge TODO into roadmap.txt
Zooko O'Whielacronx [Tue, 10 Apr 2007 16:27:58 +0000 (09:27 -0700)]
add TODO: port to Windows
Zooko O'Whielacronx [Mon, 9 Apr 2007 23:20:34 +0000 (16:20 -0700)]
update TODO (add operational TODOs)
Zooko O'Whielacronx [Mon, 9 Apr 2007 23:14:44 +0000 (16:14 -0700)]
tweak README
Brian Warner [Sun, 8 Apr 2007 20:02:13 +0000 (13:02 -0700)]
test_util: add more coverage for assertutil.py
Brian Warner [Sun, 8 Apr 2007 19:43:01 +0000 (12:43 -0700)]
test_util: add full coverage for mathutil.py
Brian Warner [Sat, 7 Apr 2007 05:51:19 +0000 (22:51 -0700)]
download: retrieve share hashes when downloading. We don't really do much validation with them yet, though.
Brian Warner [Sat, 7 Apr 2007 03:56:39 +0000 (20:56 -0700)]
codec: remove more unused debugging code
Brian Warner [Sat, 7 Apr 2007 03:55:59 +0000 (20:55 -0700)]
node: enable Tub.logLocalFailures now that we require foolscap-0.1.2
Brian Warner [Sat, 7 Apr 2007 03:55:37 +0000 (20:55 -0700)]
codec: remove some unused debugging code
Brian Warner [Sat, 7 Apr 2007 03:51:32 +0000 (20:51 -0700)]
debian: remove the lines that install the now-removed .tac files
Brian Warner [Sat, 7 Apr 2007 03:37:38 +0000 (20:37 -0700)]
misc cleanup: remove old .tac files, move old stuff into misc/
Brian Warner [Sat, 7 Apr 2007 03:34:32 +0000 (20:34 -0700)]
test_system: bump up timeouts for the sake of slow slave1, give each test a separate base directory
Brian Warner [Sat, 7 Apr 2007 01:17:23 +0000 (18:17 -0700)]
include pyfec in the trial tests