Brian Warner [Wed, 28 Mar 2007 03:14:45 +0000 (20:14 -0700)]
ICodecEncoder: resolve some questions about the API, still more to examine
Brian Warner [Wed, 28 Mar 2007 02:05:09 +0000 (19:05 -0700)]
update the docs on ICodecEncoder and ICodecDecoder
Brian Warner [Wed, 28 Mar 2007 00:44:49 +0000 (17:44 -0700)]
fix some python2.5 incompatibilities, and remove an old webish display that suggested we might know about peers but not be connected to them
Zooko O'Whielacronx [Wed, 28 Mar 2007 00:42:59 +0000 (17:42 -0700)]
add verbosity option to bench
Brian Warner [Wed, 28 Mar 2007 00:16:13 +0000 (17:16 -0700)]
test_introducer: flushEventualQueue at the end of the test run
Brian Warner [Tue, 27 Mar 2007 23:12:11 +0000 (16:12 -0700)]
complete the Introducer changes, separate out vdrive access, make everything work again
Zooko O'Whielacronx [Wed, 7 Mar 2007 07:30:17 +0000 (00:30 -0700)]
add link to mailing list
Peter Secor [Wed, 7 Mar 2007 06:29:08 +0000 (23:29 -0700)]
pyfec repository description of how to download the code via darcs
Zooko O'Whielacronx [Fri, 23 Mar 2007 23:20:26 +0000 (16:20 -0700)]
merge incomplete stuff with other patches
Zooko O'Whielacronx [Fri, 23 Mar 2007 23:15:57 +0000 (16:15 -0700)]
incomplete work to be finished elsewhere
Brian Warner [Fri, 23 Mar 2007 05:22:00 +0000 (22:22 -0700)]
hush a pyflakes warning
Zooko O'Whielacronx [Thu, 22 Mar 2007 21:39:30 +0000 (14:39 -0700)]
separate queen's service of introduction-to-the-network from queen's service of providing-access-to-the-vdrive
Brian Warner [Mon, 12 Mar 2007 23:28:37 +0000 (16:28 -0700)]
more work on a memory-footprint test program
Brian Warner [Fri, 9 Mar 2007 01:12:24 +0000 (18:12 -0700)]
start work on a memory-measuring test tool
Brian Warner [Fri, 9 Mar 2007 00:54:38 +0000 (17:54 -0700)]
add test-pyfec target, not added to the main 'test' target because it adds 20 seconds to the test run
Brian Warner [Fri, 9 Mar 2007 00:30:09 +0000 (17:30 -0700)]
test_system.py: remove the lowered (20s) timeouts, since some buildslaves require more like 30 or 40 seconds to complete the test
Brian Warner [Thu, 8 Mar 2007 23:13:14 +0000 (16:13 -0700)]
test_observer: improve test coverage
Brian Warner [Thu, 8 Mar 2007 22:28:13 +0000 (15:28 -0700)]
Makefile: add an upload-figleaf target, rather than having the buildbot create the rsync command itself
Brian Warner [Thu, 8 Mar 2007 22:12:52 +0000 (15:12 -0700)]
node.py: switch to using get_local_addresses_async, which is slightly more portable (most forms of unix)
Brian Warner [Thu, 8 Mar 2007 22:10:36 +0000 (15:10 -0700)]
rearrange service startup a bit, now Node.startService() returns a Deferred that fires when the tub is actually ready, and there is also a Node.when_tub_ready() hook. This allows get_local_addresses() to be slow and not break everything. Changed all necessary test cases to accomodate this slow startup.
Brian Warner [Thu, 8 Mar 2007 22:09:29 +0000 (15:09 -0700)]
test_observer.py: remove the code that allows observers to run out-of-order, our use of foolscap.eventual removes this error case
Brian Warner [Thu, 8 Mar 2007 22:07:38 +0000 (15:07 -0700)]
add OneShotObserverList from the amdlib tree
Brian Warner [Thu, 8 Mar 2007 03:31:49 +0000 (20:31 -0700)]
control.py: fix get_memory_usage, add a sample client tool
Brian Warner [Thu, 8 Mar 2007 03:03:47 +0000 (20:03 -0700)]
iputil: switch to a scapy-inspired SIOCGIFADDR approach, very linux-specific now
Brian Warner [Thu, 8 Mar 2007 02:29:00 +0000 (19:29 -0700)]
iputil: use the subprocess module instead of os.popen
Brian Warner [Thu, 8 Mar 2007 02:20:27 +0000 (19:20 -0700)]
control: add RIControlClient, fix some pyflakes warnings
Brian Warner [Thu, 8 Mar 2007 02:16:06 +0000 (19:16 -0700)]
add a local foolscap control interface, to upload/download files and check memory usage
Brian Warner [Thu, 8 Mar 2007 01:47:40 +0000 (18:47 -0700)]
iputil: use explicit /sbin/ifconfig, to avoid depending upon PATH
Brian Warner [Thu, 8 Mar 2007 01:43:17 +0000 (18:43 -0700)]
change node startup to put all local addresses in the PBURL, including 127.0.0.1. This should facilitate testing on both connected and disconnected systems.
Brian Warner [Thu, 8 Mar 2007 01:22:30 +0000 (18:22 -0700)]
iputil: add get_local_addresses(), an attempt to enumerate all IPv4 addresses on this host. This is pretty unix-specific for right now (it calls ifconfig)
Brian Warner [Thu, 8 Mar 2007 01:21:42 +0000 (18:21 -0700)]
util.iputil: try to survive not having a global network connection at all
Brian Warner [Tue, 6 Mar 2007 03:57:38 +0000 (20:57 -0700)]
add some questions to allmydata.interfaces
Brian Warner [Tue, 6 Mar 2007 02:46:06 +0000 (19:46 -0700)]
Makefile: update figleaf code to use INSTDIR instead of now-obsolete builddir.py
Brian Warner [Fri, 2 Feb 2007 23:45:45 +0000 (16:45 -0700)]
dummy line to show off buildbot
Brian Warner [Fri, 2 Feb 2007 00:13:01 +0000 (17:13 -0700)]
hush pyflakes warnings
Zooko O'Whielacronx [Thu, 1 Feb 2007 23:30:13 +0000 (16:30 -0700)]
update the use of the encoder API in download.py
Zooko O'Whielacronx [Thu, 1 Feb 2007 23:07:07 +0000 (16:07 -0700)]
fix purely syntactic merge conflict
Zooko O'Whielacronx [Thu, 1 Feb 2007 23:07:00 +0000 (16:07 -0700)]
use pyfec instead of py_ecc for erasure coding and update API to codec
Zooko O'Whielacronx [Thu, 1 Feb 2007 22:55:26 +0000 (15:55 -0700)]
add mathutil.next_power_of_k() and mathutil.ave()
Zooko O'Whielacronx [Thu, 1 Feb 2007 22:49:52 +0000 (15:49 -0700)]
"make clean" recursively runs cleanup in subprojects (pyfec and Crypto)
Zooko O'Whielacronx [Thu, 1 Feb 2007 22:52:35 +0000 (15:52 -0700)]
pyfec: version number bump to v0.99
Zooko O'Whielacronx [Thu, 1 Feb 2007 22:49:15 +0000 (15:49 -0700)]
pyfec: add ACK and TODO
Zooko O'Whielacronx [Thu, 1 Feb 2007 17:37:48 +0000 (10:37 -0700)]
pyfec: add TODO
Zooko O'Whielacronx [Thu, 1 Feb 2007 17:28:46 +0000 (10:28 -0700)]
pyfec: fix typo in unit test
Zooko O'Whielacronx [Thu, 1 Feb 2007 07:03:25 +0000 (00:03 -0700)]
pyfec: fix preconditions and typing, remove unused error-checking, tidy-up naming and documentation
Zooko O'Whielacronx [Wed, 31 Jan 2007 20:54:20 +0000 (13:54 -0700)]
pyfec: licensing tweak
Brian Warner [Thu, 1 Feb 2007 10:39:13 +0000 (03:39 -0700)]
Makefile: $(PWD) doesn't always work, in particular the buildslaves don't
update environment variables like $(PWD) when the launch commands.
Brian Warner [Thu, 1 Feb 2007 02:12:33 +0000 (19:12 -0700)]
Makefile: parameterize 'trial' and 'python' to enable a python2.5 builder
Brian Warner [Wed, 31 Jan 2007 01:10:37 +0000 (18:10 -0700)]
Makefile: count-lines: ignore build/* files, also add total number of .py files
Brian Warner [Wed, 31 Jan 2007 01:03:45 +0000 (18:03 -0700)]
add new build/instdir directories to the boringfile
Zooko O'Whielacronx [Tue, 30 Jan 2007 23:33:05 +0000 (16:33 -0700)]
rename Crypto to allmydata.Crypto
This kind of makes me think that there is a general principle that things
shouldn't know their own names. After all, a name is what *other* people use
to refer to you. And in the general case, some other people might refer to you
in ways incompatible with the ways that other people refer to you.
Zooko O'Whielacronx [Tue, 30 Jan 2007 23:32:52 +0000 (16:32 -0700)]
cleanup new "instdir" on "make clean"
Zooko O'Whielacronx [Tue, 30 Jan 2007 22:58:45 +0000 (15:58 -0700)]
pyfec: make it compatible with Python 2.4 as well as Python 2.5
Zooko O'Whielacronx [Tue, 30 Jan 2007 19:08:46 +0000 (12:08 -0700)]
pyfec: silence compiler warnings, add -Wall to debugmode compilation
Zooko O'Whielacronx [Tue, 30 Jan 2007 21:57:59 +0000 (14:57 -0700)]
change build system to use subpackages pristinely and ask them to install themselves into an "instdir"
Zooko O'Whielacronx [Tue, 30 Jan 2007 18:45:28 +0000 (11:45 -0700)]
pyfec: update performance measurements
Zooko O'Whielacronx [Tue, 30 Jan 2007 17:49:00 +0000 (10:49 -0700)]
pyfec: merge conflict
Zooko O'Whielacronx [Tue, 30 Jan 2007 17:46:40 +0000 (10:46 -0700)]
pyfec: fix docstring
Zooko O'Whielacronx [Tue, 30 Jan 2007 17:37:35 +0000 (10:37 -0700)]
pyfec: fix preconditions, tighten internal C types, fix bugs in the file-encoding and benchmarking utility functions
Zooko O'Whielacronx [Tue, 30 Jan 2007 17:36:42 +0000 (10:36 -0700)]
pyfec: documentation and licensing
Zooko O'Whielacronx [Tue, 30 Jan 2007 17:28:26 +0000 (10:28 -0700)]
pyfec: COPYING
Zooko O'Whielacronx [Sat, 27 Jan 2007 03:25:36 +0000 (20:25 -0700)]
pyfec: add fec/__init__.py to import all the right names into the fec module namespace
Zooko O'Whielacronx [Sat, 27 Jan 2007 03:22:13 +0000 (20:22 -0700)]
pyfec: delete m-k of the tempfiles at random in the benchmark of the to/from files
Zooko O'Whielacronx [Sat, 27 Jan 2007 03:17:59 +0000 (20:17 -0700)]
pyfec: execute tests on import of test_pyfec.py only if it is the __main__
Zooko O'Whielacronx [Sat, 27 Jan 2007 03:16:12 +0000 (20:16 -0700)]
pyfec: weaken preconditions -- you can have up to 256 shares, and you don't have to pad the individual shares except to make them all the same length
Zooko O'Whielacronx [Sat, 27 Jan 2007 03:15:49 +0000 (20:15 -0700)]
pyfec: move benchmark code from test_pyfec.py into the new bench_pyfec.py and add more benchmark code
Zooko O'Whielacronx [Sat, 27 Jan 2007 03:15:27 +0000 (20:15 -0700)]
pyfec: add utility functions to encode into a bunch of temp files and to decode from a bunch of tempfiles
Zooko O'Whielacronx [Sat, 27 Jan 2007 03:10:53 +0000 (20:10 -0700)]
pyfec: trivial formatting tweak
Zooko O'Whielacronx [Sat, 27 Jan 2007 03:10:28 +0000 (20:10 -0700)]
pyfec: remove (optimize) unused code
Zooko O'Whielacronx [Sat, 27 Jan 2007 03:08:50 +0000 (20:08 -0700)]
pyfec: make README.txt much more detailed and rename some internal variables and add some docstrings
Zooko O'Whielacronx [Sat, 27 Jan 2007 02:58:04 +0000 (19:58 -0700)]
rename the C extension from "fec" to "_fec"
Brian Warner [Sat, 27 Jan 2007 00:31:24 +0000 (17:31 -0700)]
fix some of the filetree/workqueue interface definitions
Zooko O'Whielacronx [Fri, 26 Jan 2007 01:47:04 +0000 (18:47 -0700)]
pyfec: add precondition checks on the values of k and m to constructors
Zooko O'Whielacronx [Fri, 26 Jan 2007 01:46:57 +0000 (18:46 -0700)]
pyfec: trivial formatting
Zooko O'Whielacronx [Fri, 26 Jan 2007 01:46:37 +0000 (18:46 -0700)]
add README.txt
Zooko O'Whielacronx [Fri, 26 Jan 2007 01:08:29 +0000 (18:08 -0700)]
pyfec: improve package metadata
Zooko O'Whielacronx [Fri, 26 Jan 2007 01:02:16 +0000 (18:02 -0700)]
add copyright notices, license, attributions
Zooko O'Whielacronx [Fri, 26 Jan 2007 00:50:26 +0000 (17:50 -0700)]
add dummy function to see how fast we can read in a file and invoke a Python function on each segment
Zooko O'Whielacronx [Fri, 26 Jan 2007 00:33:48 +0000 (17:33 -0700)]
add benchmark
Zooko O'Whielacronx [Thu, 25 Jan 2007 22:25:34 +0000 (15:25 -0700)]
fix tests to not require pyutil and other tweaks
Zooko O'Whielacronx [Thu, 25 Jan 2007 22:25:19 +0000 (15:25 -0700)]
add utility method file_fec
Zooko O'Whielacronx [Thu, 25 Jan 2007 22:24:01 +0000 (15:24 -0700)]
use buffers as generic "read" buffers instead of char buffers
This is a typing kludge -- if your buffers have elements of size > 1 then we will be processing only a subset of the elements and treating each byte of the element as a separate entry.
Oh well.
Zooko O'Whielacronx [Thu, 25 Jan 2007 22:23:19 +0000 (15:23 -0700)]
allow any sequence as input, not just List
Zooko O'Whielacronx [Thu, 25 Jan 2007 22:22:43 +0000 (15:22 -0700)]
add warning of subtle gotcha that can happen with mutable data buffers
Brian Warner [Thu, 25 Jan 2007 00:23:22 +0000 (17:23 -0700)]
download: update all users to match Zooko's change to ICodecDecoder.decode (as it now returns a list instead of a single string)
Zooko O'Whielacronx [Wed, 24 Jan 2007 22:34:02 +0000 (15:34 -0700)]
update interfaces and docs for codec
It now takes a sequence of buffers instead of a single string for both encode and decode, and it also takes a separate sequence of shareids for decode instead of a sequence of tuples, and it returns a sequence of buffers instead of a single string.
Zooko O'Whielacronx [Wed, 24 Jan 2007 22:21:45 +0000 (15:21 -0700)]
add comment
Zooko O'Whielacronx [Wed, 24 Jan 2007 22:20:53 +0000 (15:20 -0700)]
nicer API -- you don't have to shuffle the shares into place before calling decode_all()
Instead it does that shuffling in C inside fecmodule.
Zooko O'Whielacronx [Wed, 24 Jan 2007 22:17:51 +0000 (15:17 -0700)]
stricter typing -- using unsigned char for indexes into shares
Brian Warner [Wed, 24 Jan 2007 22:29:22 +0000 (15:29 -0700)]
hush pyflakes
Brian Warner [Wed, 24 Jan 2007 22:10:53 +0000 (15:10 -0700)]
filetree: make delete() work
Zooko O'Whielacronx [Tue, 23 Jan 2007 00:17:31 +0000 (17:17 -0700)]
pyfec v0.9
Here is the change history from the first darcs era, in reverse chronological
order:
Mon Jan 22 16:12:56 MST 2007 "Zooko O'Whielacronx <zooko@zooko.com>"
* move everything into a subdirectory so that I can merge this darcs repo with the tahoe darcs repo
./fec -> ./pyfec/fec
./setup.py -> ./pyfec/setup.py
A ./pyfec/
Mon Jan 22 16:10:17 MST 2007 "Zooko O'Whielacronx <zooko@zooko.com>"
* clean up and minimize fec.c
* strip out unused code
* hard-code GF_BITS to 8
* reindent and reformat curly bracket placement
M ./fec/fec.c -655 +324
M ./fec/fec.h -25
Mon Jan 22 14:24:32 MST 2007 "Zooko O'Whielacronx <zooko@zooko.com>"
* change API to allow a subset of the shares to be produced, and to just pass back pointers to primary shares instead of copying them
M ./fec/fec.c -24 +40
M ./fec/fec.h -5 +17
M ./fec/fecmodule.c -63 +144
M ./fec/test/test_pyfec.py -16 +25
M ./setup.py -2 +27
Tue Jan 16 23:01:44 MST 2007 "Zooko O'Whielacronx <zooko@zooko.com>"
* split encoder from decoder
M ./fec/fecmodule.c -48 +161
M ./fec/test/test_pyfec.py -3 +4
Tue Jan 16 14:35:25 MST 2007 "Zooko O'Whielacronx <zooko@zooko.com>"
* it compiles now!
./fec.c -> ./pyfec/fec.c
./fec.h -> ./pyfec/fec.h
./fecmodule.c -> ./pyfec/fecmodule.c
./pyfec -> ./fec
M ./fec/fec.c -109 +85 r13
M ./fec/fec.h -3 +2 r13
M ./fec/fecmodule.c -23 +241 r13
A ./fec/test/
A ./fec/test/test_pyfec.py
A ./pyfec/
A ./setup.py
Tue Jan 9 10:47:58 MST 2007 zooko@zooko.com
* start of new fecmodule.c
A ./fecmodule.c
Mon Jan 1 15:00:04 MST 2007 zooko@zooko.com
* tidy up error handling
M ./fec.c -26 +16
Mon Jan 1 14:06:30 MST 2007 zooko@zooko.com
* remove the on-the-fly encoding option
We don't currently need it.
M ./fec.c -68
M ./fec.h -22
Mon Jan 1 13:53:28 MST 2007 zooko@zooko.com
* original import from Mnet project
A ./fec.c
A ./fec.h
Brian Warner [Mon, 22 Jan 2007 08:17:39 +0000 (01:17 -0700)]
Makefile: add target to count lines of code and TODOs
Brian Warner [Mon, 22 Jan 2007 08:06:09 +0000 (01:06 -0700)]
filetree: change the way addpath works, now we add workqueue steps for all involved subtrees at about the same time, rather than letting one step add the next when it runs. Finally add a (passing) test for uploading files to CHK-based directories
Brian Warner [Sun, 21 Jan 2007 23:03:15 +0000 (16:03 -0700)]
filetree: add vdrive upload/download test, change workqueue relative-filename semantics
Brian Warner [Sun, 21 Jan 2007 23:01:09 +0000 (16:01 -0700)]
queen.sendOnly: ignore DeadReferenceError too
Brian Warner [Sun, 21 Jan 2007 22:15:31 +0000 (15:15 -0700)]
move IWorkQueue into allmydata.interfaces, give VirtualDrive an uploader
Brian Warner [Sun, 21 Jan 2007 22:01:34 +0000 (15:01 -0700)]
move upload/download interfaces to allmydata.interfaces, let SubTreeMaker assert IDownloader-ness of its 'downloader' argument
Brian Warner [Sun, 21 Jan 2007 20:39:40 +0000 (13:39 -0700)]
test_filetree: more vdrive._get_file_uri() coverage
Brian Warner [Sun, 21 Jan 2007 20:31:16 +0000 (13:31 -0700)]
filetree: mark leaf nodes by adding is_leaf_subtree(), stop traversing when we hit them, to make vdrive._get_file_uri() work