Brian Warner [Sun, 21 Jan 2007 11:18:54 +0000 (04:18 -0700)]
more filetree, workqueue-boxes now hold serialized Nodes, move NodeMaker out to a separate module
Brian Warner [Sun, 21 Jan 2007 04:14:41 +0000 (21:14 -0700)]
filetree: start testing IVirtualDrive, beginning with list()
Brian Warner [Sun, 21 Jan 2007 00:04:56 +0000 (17:04 -0700)]
filetree: put SubTreeMaker and NodeMaker in separate classes
Brian Warner [Sat, 20 Jan 2007 22:50:21 +0000 (15:50 -0700)]
more filetree work, more tests now pass
Brian Warner [Sat, 20 Jan 2007 21:41:51 +0000 (14:41 -0700)]
more filetree, it's actually starting to make sense now
Brian Warner [Sat, 20 Jan 2007 20:02:26 +0000 (13:02 -0700)]
filetable: oops, os.listdir() does not guarantee sorted results
Brian Warner [Sat, 20 Jan 2007 12:13:15 +0000 (05:13 -0700)]
filetree: more tests, still very early
Brian Warner [Sat, 20 Jan 2007 11:52:53 +0000 (04:52 -0700)]
filetree: refactor INode serialization, start on tests
Brian Warner [Sat, 20 Jan 2007 11:25:20 +0000 (04:25 -0700)]
more filetree hacking, still too early to test
Brian Warner [Sat, 20 Jan 2007 06:22:39 +0000 (23:22 -0700)]
checkpont more filetree stuff
Brian Warner [Fri, 19 Jan 2007 09:35:36 +0000 (02:35 -0700)]
snapshot filetree work: fix pyflakes complaints
Brian Warner [Fri, 19 Jan 2007 09:23:03 +0000 (02:23 -0700)]
snapshot filetree work: it's getting close
Brian Warner [Sat, 20 Jan 2007 03:05:33 +0000 (20:05 -0700)]
rename bin/allmydata to bin/allmydata-tahoe, so the package can co-exist with the regular (amdlib) package
Brian Warner [Fri, 19 Jan 2007 09:17:48 +0000 (02:17 -0700)]
download.py: fix IDownloader to take a URI
Brian Warner [Fri, 19 Jan 2007 09:17:24 +0000 (02:17 -0700)]
update roadmap
Brian Warner [Thu, 18 Jan 2007 00:00:47 +0000 (17:00 -0700)]
filetree.interfaces: remove not-really-code to improve coverage stats
Brian Warner [Thu, 18 Jan 2007 08:00:11 +0000 (01:00 -0700)]
figleaf.el: add code to auto-enable the annotation mode for all source files
Brian Warner [Wed, 17 Jan 2007 22:34:29 +0000 (15:34 -0700)]
improve test coverage a bit
Brian Warner [Wed, 17 Jan 2007 21:46:02 +0000 (14:46 -0700)]
webish: present real URI in directory listing, not an unnecessarily b2a'ed form
Brian Warner [Wed, 17 Jan 2007 20:54:38 +0000 (13:54 -0700)]
split filetree_new.py up into smaller pieces, in a new subpackage
Brian Warner [Wed, 17 Jan 2007 04:34:34 +0000 (21:34 -0700)]
hush pyflakes warnings
Brian Warner [Wed, 17 Jan 2007 04:29:59 +0000 (21:29 -0700)]
update URI format, include codec name
Brian Warner [Wed, 17 Jan 2007 04:02:00 +0000 (21:02 -0700)]
Makefile: use absolute paths when setting PYTHONPATH
Brian Warner [Wed, 17 Jan 2007 04:01:18 +0000 (21:01 -0700)]
webish: add PBURL to the all-peers table on the welcome page
Brian Warner [Wed, 17 Jan 2007 02:56:30 +0000 (19:56 -0700)]
encode_new.py: recent Foolscap accepts 'None' as a constraint
Brian Warner [Wed, 17 Jan 2007 02:55:53 +0000 (19:55 -0700)]
webish: more verifierid-to-uri transition
Brian Warner [Wed, 17 Jan 2007 02:43:13 +0000 (19:43 -0700)]
vdrive/webish: finish verifierid/uri transition
Brian Warner [Wed, 17 Jan 2007 02:42:28 +0000 (19:42 -0700)]
upload: fix typo in debug messages
Brian Warner [Wed, 17 Jan 2007 02:35:12 +0000 (19:35 -0700)]
upload: rearrange peer-selection code to be more readable, and fix a silly bug
Brian Warner [Wed, 17 Jan 2007 02:07:23 +0000 (19:07 -0700)]
oops, fix that NotEnoughPeersError instrumentation
Brian Warner [Wed, 17 Jan 2007 01:47:52 +0000 (18:47 -0700)]
add better error reporting to NotEnoughPeersError
Brian Warner [Tue, 16 Jan 2007 22:15:26 +0000 (15:15 -0700)]
change upload to push 2 shares instead of 3
Now that peers can talk to themselves, the 5-node system test won't fail
just because one of the shares was hosted on the downloader (and thus
inaccessible until recently). The 3-share push was there to avoid this
problem.
Brian Warner [Tue, 16 Jan 2007 22:12:49 +0000 (15:12 -0700)]
now that foolscap-0.0.7 accepts connect-to-self, allow peers to know about themselves. We now require foolscap-0.0.7
Brian Warner [Tue, 16 Jan 2007 04:31:10 +0000 (21:31 -0700)]
hush pyflakes warnings
Brian Warner [Tue, 16 Jan 2007 04:22:22 +0000 (21:22 -0700)]
rearrange encode/upload, add URIs, switch to ReplicatingEncoder
Added metadata to the bucket store, which is used to hold the share number
(but the bucket doesn't know that, it just gets a string).
Modified the codec interfaces a bit.
Try to pass around URIs to/from download/upload instead of verifierids.
URI format is still in flux.
Change the current (primitive) file encoder to use a ReplicatingEncoder
because it provides ICodecEncoder. We will be moving to the (less primitive)
file encoder (currently in allmydata.encode_new) eventually, but for now
this change lets us test out PyRS or zooko's upcoming C-based RS codec in
something larger than a single unit test. This primitive file encoder only
uses a single segment, and has no merkle trees.
Also added allmydata.util.deferredutil for a DeferredList wrapper that
errbacks (but only when all component Deferreds have fired) if there were
any errors, which unfortunately is not a behavior available from the standard
DeferredList.
Brian Warner [Mon, 15 Jan 2007 21:01:22 +0000 (14:01 -0700)]
add simple metadata (a single string) to the storage protocol
Brian Warner [Fri, 12 Jan 2007 03:57:14 +0000 (20:57 -0700)]
change IEncoder to ICodecEncoder, to match the previous change
Brian Warner [Fri, 12 Jan 2007 03:51:27 +0000 (20:51 -0700)]
use the word 'codec' for erasure coding, for now. 'encode' is used for file-level segmentation/hashing
Brian Warner [Wed, 10 Jan 2007 04:13:05 +0000 (21:13 -0700)]
sendOnly: oops, I keep forgetting that you can't really use f.trap in a lambda, because it returns the failure it trapped
Brian Warner [Wed, 10 Jan 2007 02:40:36 +0000 (19:40 -0700)]
test_system.py: exercise queen.Roster._lost_node too
Brian Warner [Tue, 9 Jan 2007 08:58:50 +0000 (01:58 -0700)]
workqueue: more tests
Brian Warner [Tue, 9 Jan 2007 08:36:12 +0000 (01:36 -0700)]
workqueue: more tests, coverage now at 63.4%, yay
Brian Warner [Tue, 9 Jan 2007 05:29:42 +0000 (22:29 -0700)]
workqueue: more improvements, more tests
Brian Warner [Tue, 9 Jan 2007 04:29:33 +0000 (21:29 -0700)]
workqueue: start adding tests
Brian Warner [Tue, 9 Jan 2007 04:29:03 +0000 (21:29 -0700)]
figleaf.el: when converting to the elisp-readable format, use our in-tree version of figleaf rather than one found on the system. Also change the keybinding to toggle annotations to C-cC-a, which is easier to type
Brian Warner [Tue, 9 Jan 2007 04:28:08 +0000 (21:28 -0700)]
figleaf: delete coverage file (.figleaf) before each test run, otherwise I suspect coverage data will accumulate from one test run to the next
Brian Warner [Mon, 8 Jan 2007 23:29:14 +0000 (16:29 -0700)]
rename the new filetable code to 'filetree', since robk astutely pointed out that 'table' is misleading and implies a flat list of files in a single directory
Brian Warner [Sun, 7 Jan 2007 20:10:15 +0000 (13:10 -0700)]
figleaf.el: add some emacs22 comments
Brian Warner [Sun, 7 Jan 2007 20:09:37 +0000 (13:09 -0700)]
Makefile: add a default target to just build the tree
Brian Warner [Sat, 6 Jan 2007 04:15:59 +0000 (21:15 -0700)]
fix pyflakes warning in debugshell, by providing a dummy value for 'app' that will be overwritten when the manhole connection is established
Brian Warner [Sat, 6 Jan 2007 04:12:26 +0000 (21:12 -0700)]
use absolute import of 'allmydata.Crypto' rather than a relative import of just 'Crypto', to make it clear that we're using our own form rather than relying upon the system version
Brian Warner [Sat, 6 Jan 2007 04:11:19 +0000 (21:11 -0700)]
workqueue: fix pyflakes warnings, code is still quite incomplete
Brian Warner [Sat, 6 Jan 2007 03:06:51 +0000 (20:06 -0700)]
debian: give incrementing version numbers to .deb packages, using timestamps
Brian Warner [Sat, 6 Jan 2007 01:53:22 +0000 (18:53 -0700)]
test_encode_share.py: fix some pyflakes warnings
Brian Warner [Sat, 6 Jan 2007 01:46:03 +0000 (18:46 -0700)]
add some (disabled) encoder benchmarking code
Brian Warner [Sat, 6 Jan 2007 01:12:45 +0000 (18:12 -0700)]
change Encoder to use py_ecc, now that it works
Brian Warner [Sat, 6 Jan 2007 01:12:04 +0000 (18:12 -0700)]
disable figleaf tracing during py_ecc, since it takes *forever*, especially on the slow buildslave
Brian Warner [Fri, 5 Jan 2007 07:48:42 +0000 (00:48 -0700)]
encode.py: add some timing comments
Brian Warner [Fri, 5 Jan 2007 07:42:52 +0000 (00:42 -0700)]
only run a single (short) py_ecc test on slave3, since it is so slow the tests timeout
Brian Warner [Fri, 5 Jan 2007 07:06:42 +0000 (00:06 -0700)]
encoding: fix the last py_ecc problem, tests pass now
Brian Warner [Fri, 5 Jan 2007 06:51:35 +0000 (23:51 -0700)]
change encode_new to use IEncoder
Brian Warner [Fri, 5 Jan 2007 06:50:21 +0000 (23:50 -0700)]
fix our use of py_ecc (set log2FieldSize=8 explicitly)
Brian Warner [Fri, 5 Jan 2007 04:52:51 +0000 (21:52 -0700)]
establish IEncoder/IDecoder, create suitable interfaces for both the simple replicating encoder and the py_ecc one, add a (failing) unit test for it
Brian Warner [Fri, 5 Jan 2007 04:52:40 +0000 (21:52 -0700)]
figleaf.el: announce annotation/deannotation
Brian Warner [Fri, 5 Jan 2007 01:06:20 +0000 (18:06 -0700)]
more pyflakes cleanups
Brian Warner [Fri, 5 Jan 2007 01:04:43 +0000 (18:04 -0700)]
fix pyflakes warnings in figleaf
Zooko O'Whielacronx [Fri, 5 Jan 2007 00:58:14 +0000 (17:58 -0700)]
add padding to segments and convert type from string to list of integers before passing to py_ecc
But then I get an error that I don't understand about ECC math...
Brian Warner [Thu, 4 Jan 2007 08:26:43 +0000 (01:26 -0700)]
improve figleaf: fix some line-numbering bugs, add an alphabetically-sorted index HTML page
Brian Warner [Thu, 4 Jan 2007 08:25:36 +0000 (01:25 -0700)]
more figleaf emacs work: enable the minor mode, then type C-cA to toggle annotations
Brian Warner [Thu, 4 Jan 2007 05:23:25 +0000 (22:23 -0700)]
modify figleaf2html to show module names instead of .py filenames, also add a --root argument to restrict coverage to a specific parent directory
Brian Warner [Thu, 4 Jan 2007 05:06:51 +0000 (22:06 -0700)]
move figleaf2html and figleaf_htmlizer.py into our tree, for easier customization
Brian Warner [Thu, 4 Jan 2007 04:38:29 +0000 (21:38 -0700)]
figleaf: move a copy into allmydata.util.figleaf, update Makefile/trial stuff
Brian Warner [Tue, 2 Jan 2007 06:48:42 +0000 (23:48 -0700)]
add some experimental emacs test-coverage-annotation tools, still in development
Brian Warner [Tue, 2 Jan 2007 06:47:16 +0000 (23:47 -0700)]
checkpoint work-in-progress for WorkQueue, a disk-persistent list of work to be done
Brian Warner [Tue, 2 Jan 2007 06:44:12 +0000 (23:44 -0700)]
filetable_new: import py_ecc, make the tests get a little bit further
Brian Warner [Tue, 2 Jan 2007 06:43:24 +0000 (23:43 -0700)]
add __init__ and setup.py glue for py_ecc, so we can import it
Zooko O'Whielacronx [Fri, 29 Dec 2006 20:50:53 +0000 (13:50 -0700)]
make encode_new use py_ecc for real live erasure coding
(This patch is not tested -- I'm working on a Mac which doesn't have gcc
installed... (gcc is necessary for the crypto module.) I will now attempt to
connect to a better development computer over my mom's staticy, failure-prone,
14.4 K modem...)
Zooko O'Whielacronx [Fri, 29 Dec 2006 20:50:42 +0000 (13:50 -0700)]
import mathutil from pyutil
Zooko O'Whielacronx [Fri, 29 Dec 2006 20:46:45 +0000 (13:46 -0700)]
import py_ecc, a pure python fec by Emin Martinian, which is under a permissive licence
It is too slow for a real product, but is a quick way to get a working prototype, and also is freely redistributable by us...
Zooko O'Whielacronx [Fri, 29 Dec 2006 19:40:10 +0000 (12:40 -0700)]
encode_new: fix a bug in subshare hashes and tidy-up a couple of things
Brian Warner [Mon, 25 Dec 2006 07:56:18 +0000 (00:56 -0700)]
more filetable_new tests
Brian Warner [Sun, 24 Dec 2006 19:47:03 +0000 (12:47 -0700)]
filetable_new: fix the test
Brian Warner [Sun, 24 Dec 2006 19:39:24 +0000 (12:39 -0700)]
checkpointing new filetable work.. tests don't pass yet
Brian Warner [Thu, 14 Dec 2006 11:33:57 +0000 (04:33 -0700)]
filetable: shuffle lines a bit to appease figleaf's confusion
Brian Warner [Thu, 14 Dec 2006 11:31:17 +0000 (04:31 -0700)]
encode: start to fix a few problems, still a lot of work left to go
Brian Warner [Thu, 14 Dec 2006 11:17:01 +0000 (04:17 -0700)]
make initial simple encode_new test pass
Brian Warner [Thu, 14 Dec 2006 11:16:43 +0000 (04:16 -0700)]
Makefile: fix location of figleaf.excludes
Brian Warner [Thu, 14 Dec 2006 11:13:29 +0000 (04:13 -0700)]
Makefile: allow 'make test TEST=something.else'
Brian Warner [Thu, 14 Dec 2006 11:08:09 +0000 (04:08 -0700)]
Makefile: improve 'clean' target to remove debian/ symlink too
Brian Warner [Thu, 14 Dec 2006 11:05:22 +0000 (04:05 -0700)]
Makefile: add pyflakes target, only covering allmydata code (not pycrypto)
Brian Warner [Thu, 14 Dec 2006 10:59:51 +0000 (03:59 -0700)]
Makefile: add correct generated build/lib.linux-i686-2.4 directory to PYTHONPATH for tests
Brian Warner [Thu, 14 Dec 2006 10:39:50 +0000 (03:39 -0700)]
move all packages into src/, fix allmydata.Crypto build. Now you must perform a 'setup.py build' before using anything, and you must add the build directory (build/lib.linux-i686-2.4) to your PYTHONPATH before doing anything
Brian Warner [Thu, 14 Dec 2006 10:29:56 +0000 (03:29 -0700)]
allmydata.Crypto: fix all internal imports
Brian Warner [Thu, 14 Dec 2006 10:25:30 +0000 (03:25 -0700)]
first cut at creating allmydata.Crypto, starting with python-amycrypto-2.0.1.allmydata2 (with working CTR mode)
Brian Warner [Thu, 14 Dec 2006 03:37:00 +0000 (20:37 -0700)]
record some WIP structure for filetable
Brian Warner [Thu, 14 Dec 2006 03:35:12 +0000 (20:35 -0700)]
figleaf doesn't like the last line of a file to be a comment
Brian Warner [Thu, 14 Dec 2006 03:32:35 +0000 (20:32 -0700)]
start work on new encoder, with merkle trees and subshares and stuff
Brian Warner [Wed, 13 Dec 2006 22:14:20 +0000 (15:14 -0700)]
move debugshell.py into allmydata/
Brian Warner [Thu, 7 Dec 2006 22:05:20 +0000 (15:05 -0700)]
minor change to test buildbot triggering for new repository
Brian Warner [Thu, 7 Dec 2006 21:58:23 +0000 (14:58 -0700)]
figleaf gets confused when the last line of a file is a comment