tahoe-lafs/tahoe-lafs.git
15 years agosetup: more verbose assertion failure in test_runner
Zooko O'Whielacronx [Thu, 29 Jan 2009 17:49:06 +0000 (10:49 -0700)]
setup: more verbose assertion failure in test_runner

15 years agosetup: subclass setuptools.Command instead of distutils Command
Zooko O'Whielacronx [Thu, 29 Jan 2009 14:00:58 +0000 (07:00 -0700)]
setup: subclass setuptools.Command instead of distutils Command
There's almost no difference between them, but let's be consistent try to use the setuptools API as it was intended.

15 years agosetup: temporarily comment-out the horrible kludge to work-around setuptools #17...
Zooko O'Whielacronx [Thu, 29 Jan 2009 14:00:00 +0000 (07:00 -0700)]
setup: temporarily comment-out the horrible kludge to work-around setuptools #17, while I figure out how to solve it better

15 years agosetup: always create a support dir and populate it with a site-packages and add same...
Zooko O'Whielacronx [Thu, 29 Jan 2009 05:56:08 +0000 (22:56 -0700)]
setup: always create a support dir and populate it with a site-packages and add same to the PYTHONPATH, just in case someone is going to do "build", "develop", or "test" or something else which triggers a build
I think there must be a much better solution for this -- probably to fix setuptools #54 and ship our own fork of setuptools and rely on it.

15 years agosetup: if any of "build", "develop", or "test" appear in the sys.argv then that means...
Zooko O'Whielacronx [Thu, 29 Jan 2009 05:55:34 +0000 (22:55 -0700)]
setup: if any of "build", "develop", or "test" appear in the sys.argv then that means we'll be doing a develop, so add the workarounds for setuptools #17 in any case
I think there must be a much better solution for this -- probably to fix setuptools #17 and ship our own fork of setuptools and rely on it.

15 years agosetup: add metadata indicating compatibility with python 2.6
Zooko O'Whielacronx [Thu, 29 Jan 2009 01:26:28 +0000 (18:26 -0700)]
setup: add metadata indicating compatibility with python 2.6

15 years agosetup: a new improved way to create tahoe executables
Zooko O'Whielacronx [Thu, 29 Jan 2009 01:07:16 +0000 (18:07 -0700)]
setup: a new improved way to create tahoe executables
Create the 'tahoe-script.py' file under the 'bin' directory. The 'tahoe-script.py' file is exactly the same as the 'tahoe-script.template' script except that the shebang line is rewritten to use our sys.executable for the interpreter. On Windows, create a tahoe.exe will execute it.  On non-Windows, make a symlink to it from 'tahoe'.  The tahoe.exe will be copied from the setuptools egg's cli.exe and this will work from a zip-safe and non-zip-safe setuptools egg.

15 years agostorage: enable the test of a share larger than 2 GiB; this will probably be too...
Zooko O'Whielacronx [Wed, 28 Jan 2009 23:33:12 +0000 (16:33 -0700)]
storage: enable the test of a share larger than 2 GiB; this will probably be too expensive on Mac OS X, but I think it won't be on all the other platforms ; I will probably set it to SkipTest if the sys.platform is Mac after seeing the results of this buildbot run

15 years agosetup: bundle darcsver-1.1.8
Zooko O'Whielacronx [Wed, 28 Jan 2009 21:03:26 +0000 (14:03 -0700)]
setup: bundle darcsver-1.1.8

15 years agosetup: remove old bundled dependency darcsver-1.1.5
Zooko O'Whielacronx [Wed, 28 Jan 2009 19:20:12 +0000 (12:20 -0700)]
setup: remove old bundled dependency darcsver-1.1.5

15 years agoCLI: fix examples in tahoe put --help
Brian Warner [Tue, 27 Jan 2009 22:39:09 +0000 (15:39 -0700)]
CLI: fix examples in tahoe put --help

15 years agotrivial: remove unused imports noticed by pyflakes
Zooko O'Whielacronx [Tue, 27 Jan 2009 22:11:48 +0000 (15:11 -0700)]
trivial: remove unused imports noticed by pyflakes

15 years agosetup: fix test_runner to invoke bin/tahoe.exe instead of bin/tahoe if on Windows...
Zooko O'Whielacronx [Tue, 27 Jan 2009 21:37:17 +0000 (14:37 -0700)]
setup: fix test_runner to invoke bin/tahoe.exe instead of bin/tahoe if on Windows (this is what happens when a user invokes bin/tahoe on Windows)

15 years agosetup: fix test_runner to assert that lines which are output to stderr must end with...
Zooko O'Whielacronx [Tue, 27 Jan 2009 21:35:05 +0000 (14:35 -0700)]
setup: fix test_runner to assert that lines which are output to stderr must end with a punctuation mark (apparently re.search("x$", "x\r\n") does not match.  :-()

15 years agosetup: fix test_runner.RunNode.test_baddir -- it was left incomplete and broken by...
Zooko O'Whielacronx [Tue, 27 Jan 2009 21:32:45 +0000 (14:32 -0700)]
setup: fix test_runner.RunNode.test_baddir -- it was left incomplete and broken by a recent patch

15 years agosetup: hack the sys.argv to set poll reactor if "python ./setup.py test" in addition...
Zooko O'Whielacronx [Tue, 27 Jan 2009 05:40:46 +0000 (22:40 -0700)]
setup: hack the sys.argv to set poll reactor if "python ./setup.py test" in addition to if "python ./setup.py trial"; remove another hack which has been moved setup.cfg; remove setup_requires Twisted since now we actually setup_require setuptools_trial and it install_requires Twisted.

15 years agosetup: go ahead and check for noise in test_client_no_noise
Zooko O'Whielacronx [Tue, 27 Jan 2009 00:46:16 +0000 (17:46 -0700)]
setup: go ahead and check for noise in test_client_no_noise

15 years agosetup: always run "build" before running "test"
Zooko O'Whielacronx [Tue, 27 Jan 2009 00:32:40 +0000 (17:32 -0700)]
setup: always run "build" before running "test"

15 years agosetup: add a test for a warning message from importing nevow, marked as TODO, comment...
Zooko O'Whielacronx [Tue, 27 Jan 2009 00:30:46 +0000 (17:30 -0700)]
setup: add a test for a warning message from importing nevow, marked as TODO, comment-out the assertion of no-noise inside other test_runner tests

15 years agosetup: always run build_tahoe before running tests
Zooko O'Whielacronx [Tue, 27 Jan 2009 00:30:24 +0000 (17:30 -0700)]
setup: always run build_tahoe before running tests

15 years agodiskwatcher: cache growth results (which take 30s to generate) for 3 minutes, to...
Brian Warner [Mon, 26 Jan 2009 00:06:39 +0000 (17:06 -0700)]
diskwatcher: cache growth results (which take 30s to generate) for 3 minutes, to help munin, which invokes it 6 times in a row every 5 minutes

15 years agomake streaming-manifest stop doing work after the HTTP connection is dropped
Brian Warner [Sat, 24 Jan 2009 02:39:08 +0000 (19:39 -0700)]
make streaming-manifest stop doing work after the HTTP connection is dropped

15 years agosetup: sys.exit() raises a SystemExit exception on Python 2.4 on Solaris -- fix it...
Zooko O'Whielacronx [Sat, 24 Jan 2009 01:49:11 +0000 (18:49 -0700)]
setup: sys.exit() raises a SystemExit exception on Python 2.4 on Solaris -- fix it so that bin/tahoe doesn't interpret this as its brother having been non-executable

15 years agosetup: find a "bin/tahoe" executable to test based on allmydata.__file__ instead...
Zooko O'Whielacronx [Sat, 24 Jan 2009 01:34:37 +0000 (18:34 -0700)]
setup: find a "bin/tahoe" executable to test based on allmydata.__file__ instead of based on the CWD
This means that the tests still work if you are executing them from a CWD other than the src dir -- *if* the "bin/tahoe" is found at os.path.dirname(os.path.dirname(os.path.dirname(allmydata.__file__))).
If no file is found at that location, then just skip the tests of executing the "tahoe" executable, because we don't want to accidentally run those tests against an executable from a different version of tahoe.

15 years agorollback the #591 fix, since it breaks quicktest entirely
Brian Warner [Sat, 24 Jan 2009 00:28:12 +0000 (17:28 -0700)]
rollback the #591 fix, since it breaks quicktest entirely

15 years ago#509: remove non-streaming 'tahoe manifest' CLI form
Brian Warner [Sat, 24 Jan 2009 00:00:02 +0000 (17:00 -0700)]
#509: remove non-streaming 'tahoe manifest' CLI form

15 years agoCLI.txt: improve docs on 'tahoe manifest' to cover --verify-cap, --repair-cap, and...
Brian Warner [Fri, 23 Jan 2009 23:59:39 +0000 (16:59 -0700)]
CLI.txt: improve docs on 'tahoe manifest' to cover --verify-cap, --repair-cap, and streaming JSON

15 years ago#509 CLI: add 'tahoe manifest --stream'
Brian Warner [Fri, 23 Jan 2009 23:33:21 +0000 (16:33 -0700)]
#509 CLI: add 'tahoe manifest --stream'

15 years ago#509: test_system.py: add test for streamed-manifest
Brian Warner [Fri, 23 Jan 2009 23:32:47 +0000 (16:32 -0700)]
#509: test_system.py: add test for streamed-manifest

15 years agotest_system: rearrange DeepCheckWebGood to make it easier to add CLI tests
Brian Warner [Fri, 23 Jan 2009 23:13:06 +0000 (16:13 -0700)]
test_system: rearrange DeepCheckWebGood to make it easier to add CLI tests

15 years agosetup: use "trial" executable instead of the setuptools_trial plugin for "make quicktest"
Zooko O'Whielacronx [Fri, 23 Jan 2009 23:58:30 +0000 (16:58 -0700)]
setup: use "trial" executable instead of the setuptools_trial plugin for "make quicktest"
This is to fix #591 ("make quicktest" could be quicker and less noisy).  This means that "make quicktest" won't work unless the user has manually installed Twisted already such that the "trial" executable is on their PATH and the Twisted package is on their PYTHONPATH.  This bypasses the behavior of setuptools_trial which builds and checks package dependencies before executing the tests.

15 years agoCLI: remove the '-v' alias for --version-and-path, to free up '-v' for more tradition...
Brian Warner [Fri, 23 Jan 2009 22:21:50 +0000 (15:21 -0700)]
CLI: remove the '-v' alias for --version-and-path, to free up '-v' for more traditional uses (like --verbose)

15 years ago#590: add webish t=stream-manifest
Brian Warner [Fri, 23 Jan 2009 05:01:36 +0000 (22:01 -0700)]
#590: add webish t=stream-manifest

15 years agodirnode: add get_repair_cap()
Brian Warner [Fri, 23 Jan 2009 04:44:49 +0000 (21:44 -0700)]
dirnode: add get_repair_cap()

15 years agodirnode.deep_traverse: fix docstring to describe the correct return value
Brian Warner [Fri, 23 Jan 2009 04:39:50 +0000 (21:39 -0700)]
dirnode.deep_traverse: fix docstring to describe the correct return value

15 years agofilenode: add get_repair_cap(), which uses the read-write filecap for immutable files...
Brian Warner [Fri, 23 Jan 2009 04:38:36 +0000 (21:38 -0700)]
filenode: add get_repair_cap(), which uses the read-write filecap for immutable files, and the verifycap for immutable files

15 years agosetup: make the bin/tahoe executable exit with the appropriate exit code
Zooko O'Whielacronx [Thu, 22 Jan 2009 22:54:05 +0000 (15:54 -0700)]
setup: make the bin/tahoe executable exit with the appropriate exit code

15 years agosetup: add test that the tests are testing the right source code
Zooko O'Whielacronx [Thu, 22 Jan 2009 22:52:40 +0000 (15:52 -0700)]
setup: add test that the tests are testing the right source code
This is a test of #145, and I think that now the code passes this test.

15 years agotrivial: removed unused imports noticed by pyflakes
Zooko O'Whielacronx [Thu, 22 Jan 2009 22:52:13 +0000 (15:52 -0700)]
trivial: removed unused imports noticed by pyflakes

15 years agosetup: change test_runner to invoke "bin/tahoe" in a subprocess instead of executing...
Zooko O'Whielacronx [Thu, 22 Jan 2009 22:38:18 +0000 (15:38 -0700)]
setup: change test_runner to invoke "bin/tahoe" in a subprocess instead of executing runner.runner()
This is necessary because loading allmydata code now depends on PYTHONPATH manipulation which is done in the "bin/tahoe" script.  Unfortunately it makes test_runner slower since it launches and waits for many subprocesses.

15 years agosetup: fix "tahoe start" to work on Windows even when a Tahoe base dir hasn't been...
Zooko O'Whielacronx [Wed, 21 Jan 2009 19:47:20 +0000 (12:47 -0700)]
setup: fix "tahoe start" to work on Windows even when a Tahoe base dir hasn't been configured in the Windows registry

15 years agodocs: trivial naming change
Zooko O'Whielacronx [Wed, 21 Jan 2009 03:50:42 +0000 (20:50 -0700)]
docs: trivial naming change

15 years agorollback the feature of making "ambient upload authority" configurable
Zooko O'Whielacronx [Wed, 21 Jan 2009 03:47:35 +0000 (20:47 -0700)]
rollback the feature of making "ambient upload authority" configurable

This reverses some, but not all, of the changes that were committed in the following set of patches.

rolling back:

Sun Jan 18 09:54:30 MST 2009  toby.murray
  * add 'web.ambient_upload_authority' as a paramater to tahoe.cfg

    M ./src/allmydata/client.py -1 +3
    M ./src/allmydata/test/common.py -7 +9
    A ./src/allmydata/test/test_ambient_upload_authority.py
    M ./src/allmydata/web/root.py +12
    M ./src/allmydata/webish.py -1 +4
Sun Jan 18 09:56:08 MST 2009  zooko@zooko.com
  * trivial: whitespace
  I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.

    M ./src/allmydata/test/test_ambient_upload_authority.py -9 +8
    M ./src/allmydata/web/root.py -2 +1
    M ./src/allmydata/webish.py -2 +1
Mon Jan 19 14:16:19 MST 2009  zooko@zooko.com
  * trivial: remove unused import noticed by pyflakes

    M ./src/allmydata/test/test_ambient_upload_authority.py -1
Mon Jan 19 21:38:35 MST 2009  toby.murray
  * doc: describe web.ambient_upload_authority

    M ./docs/configuration.txt +14
    M ./docs/frontends/webapi.txt +11
Mon Jan 19 21:38:57 MST 2009  zooko@zooko.com
  * doc: add Toby Murray to the CREDITS

    M ./CREDITS +4

15 years agosetup: add new bundled setuptools_trial-0.4
Zooko O'Whielacronx [Wed, 21 Jan 2009 00:40:12 +0000 (17:40 -0700)]
setup: add new bundled setuptools_trial-0.4

15 years agosetup: remove old bundled setuptools_trial-0.2
Zooko O'Whielacronx [Wed, 21 Jan 2009 00:39:18 +0000 (17:39 -0700)]
setup: remove old bundled setuptools_trial-0.2

15 years agosetup: require darcsver always, and not just when we see the string "darcsver" in...
Zooko O'Whielacronx [Tue, 20 Jan 2009 19:42:29 +0000 (12:42 -0700)]
setup: require darcsver always, and not just when we see the string "darcsver" in sys.argv, because the new aliases hack means that the string might not appear in sys.argv

15 years agosetup: fix test_system to require tahoe under its package==distribution name "allmyda...
Zooko O'Whielacronx [Tue, 20 Jan 2009 19:38:09 +0000 (12:38 -0700)]
setup: fix test_system to require tahoe under its package==distribution name "allmydata-tahoe" instead of its module name "allmydata"

15 years agosetup: use setup.cfg aliases to map "setup.py test" to "setup.py trial" and "setup...
Zooko O'Whielacronx [Tue, 20 Jan 2009 19:37:23 +0000 (12:37 -0700)]
setup: use setup.cfg aliases to map "setup.py test" to "setup.py trial" and "setup.py build" to "setup.py darcsver --count-all-patches build_tahoe"
Thanks to dpeterson for the suggestion.

15 years agodoc: add Toby Murray to the CREDITS
Zooko O'Whielacronx [Tue, 20 Jan 2009 05:38:57 +0000 (22:38 -0700)]
doc: add Toby Murray to the CREDITS

15 years agodoc: describe web.ambient_upload_authority
toby.murray [Tue, 20 Jan 2009 05:38:35 +0000 (22:38 -0700)]
doc: describe web.ambient_upload_authority

15 years agotrivial: remove unused import noticed by pyflakes
Zooko O'Whielacronx [Mon, 19 Jan 2009 22:16:19 +0000 (15:16 -0700)]
trivial: remove unused import noticed by pyflakes

15 years agosetup: simplify install.html a tad
Zooko O'Whielacronx [Mon, 19 Jan 2009 22:04:47 +0000 (15:04 -0700)]
setup: simplify install.html a tad

15 years agosetup: refactor versions-and-paths and use pkg_resources to find them
Zooko O'Whielacronx [Mon, 19 Jan 2009 22:04:35 +0000 (15:04 -0700)]
setup: refactor versions-and-paths and use pkg_resources to find them
Using pkg_resources is probably better if it works -- zope.interface doesn't have a __version__ attribute that we can query, but pkg_resources knows zope.interface's version number, for one thing.
This code falls back to the old way -- looking at the __version__ attributes and __file__ attributes -- if the pkg_resources way doesn't answer.
Note that this patch also changes the capitalization of "Nevow", "Twisted", and "pyOpenSSL", and the spelling of "allmydata-tahoe".  These changes are not frivolous: they are reflecting the fact that we are naming Python packages (technically called Python "distributions") instead of Python modules (technically and confusingly called Python "packages") here.  The package ("distribution") is named "allmydata-tahoe".  The module ("package") is named "allmydata".

15 years agosetup: undo (for the second time) the use of the --multi-version feature
Zooko O'Whielacronx [Mon, 19 Jan 2009 21:53:52 +0000 (14:53 -0700)]
setup: undo (for the second time) the use of the --multi-version feature
When this feature is turned on, then setuptools doesn't create easy-install.pth, setuptools.pth, or site.py in the target site-packages dir.  I don't know why not and we should probably open a ticket on the setuptools tracker and/or hack setuptools to create those files anyway.  But for now (for the Tahoe-1.3.0 release), we're going to leave --multi-version mode off and require users to manually uninstall any packages which are too old and thus conflict with our newer dependencies.

15 years agotrivial: whitespace
Zooko O'Whielacronx [Sun, 18 Jan 2009 17:58:15 +0000 (10:58 -0700)]
trivial: whitespace
Ran "M-x whitespace-cleanup" on files that Toby's recent patch touched, even though they didn't have trailing whitespace.

15 years agotrivial: whitespace
Zooko O'Whielacronx [Sun, 18 Jan 2009 17:56:08 +0000 (10:56 -0700)]
trivial: whitespace
I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.

15 years agotrivial: whitespace
Zooko O'Whielacronx [Sun, 18 Jan 2009 17:54:58 +0000 (10:54 -0700)]
trivial: whitespace

15 years agoadd 'web.ambient_upload_authority' as a paramater to tahoe.cfg
toby.murray [Sun, 18 Jan 2009 17:54:30 +0000 (10:54 -0700)]
add 'web.ambient_upload_authority' as a paramater to tahoe.cfg

15 years agodoc: add Larry Hosken to CREDITS
Zooko O'Whielacronx [Sat, 17 Jan 2009 17:49:43 +0000 (10:49 -0700)]
doc: add Larry Hosken to CREDITS

15 years agorun build_tahoe command with trial commmand
cgalvan [Sat, 17 Jan 2009 01:00:47 +0000 (18:00 -0700)]
run build_tahoe command with trial commmand

15 years agoadding multi-version support
cgalvan [Sat, 17 Jan 2009 00:03:26 +0000 (17:03 -0700)]
adding multi-version support

15 years agosetup: addition of setuptools_trial egg to the base dir is boring from the point...
Zooko O'Whielacronx [Fri, 16 Jan 2009 21:05:54 +0000 (14:05 -0700)]
setup: addition of setuptools_trial egg to the base dir is boring from the point of view of revision control

15 years agoprevent --site-dirs from being passed to the 'install' command
cgalvan [Fri, 16 Jan 2009 20:57:32 +0000 (13:57 -0700)]
prevent --site-dirs from being passed to the 'install' command

15 years agoadd option to show version and path to the tahoe executable
cgalvan [Fri, 16 Jan 2009 19:47:51 +0000 (12:47 -0700)]
add option to show version and path to the tahoe executable

15 years agosetup: put back configuration of the PYTHONPATH which is necessary to build the Windo...
Zooko O'Whielacronx [Thu, 15 Jan 2009 03:37:51 +0000 (20:37 -0700)]
setup: put back configuration of the PYTHONPATH which is necessary to build the Windows packages

15 years agonode.py: use NODEDIR/tmp for the 'tempfile' module's temporary directory, so webapi...
Brian Warner [Thu, 15 Jan 2009 03:00:15 +0000 (20:00 -0700)]
node.py: use NODEDIR/tmp for the 'tempfile' module's temporary directory, so webapi upload tempfiles are put there instead of /tmp . You can set it to something else by setting [node]tempdir in tahoe.cfg

15 years agoweb/operations: undo the disable-ophandle-expiration change that inadvertently got...
Brian Warner [Thu, 15 Jan 2009 02:14:59 +0000 (19:14 -0700)]
web/operations: undo the disable-ophandle-expiration change that inadvertently got included in zooko's recent 'rename wapi.txt to webapi.txt' patch, causing test failures

15 years agocli: tests: skip symlink test if there is no os.symlink
Zooko O'Whielacronx [Thu, 15 Jan 2009 01:10:10 +0000 (18:10 -0700)]
cli: tests: skip symlink test if there is no os.symlink

15 years agowebapi.txt: explain POST /uri/?t=upload, thanks to Toby Murray for the catch
Brian Warner [Thu, 15 Jan 2009 01:08:03 +0000 (18:08 -0700)]
webapi.txt: explain POST /uri/?t=upload, thanks to Toby Murray for the catch

15 years agomutable: move recent operation history management code (MutableWatcher) into history...
Brian Warner [Thu, 15 Jan 2009 00:36:20 +0000 (17:36 -0700)]
mutable: move recent operation history management code (MutableWatcher) into history.py, have History provide stats

15 years agodownload: tiny cleanup of history code
Brian Warner [Wed, 14 Jan 2009 23:41:51 +0000 (16:41 -0700)]
download: tiny cleanup of history code

15 years agoupload: move upload history into History object
Brian Warner [Wed, 14 Jan 2009 23:41:06 +0000 (16:41 -0700)]
upload: move upload history into History object

15 years agoimmutable/download.py move recent-downloads history out of Downloader and into a...
Brian Warner [Wed, 14 Jan 2009 23:14:24 +0000 (16:14 -0700)]
immutable/download.py move recent-downloads history out of Downloader and into a separate class. upload/etc will follow soon.

15 years agodocs: rename wapi.txt to webapi.txt
Zooko O'Whielacronx [Wed, 14 Jan 2009 20:53:48 +0000 (13:53 -0700)]
docs: rename wapi.txt to webapi.txt
Because Brian argues that the file contains a description of the wui as well as of the wapi, and because the name "webapi.txt" might be more obvious to the untrained eye.

15 years agosetup: configure setup.cfg to point setuptools at the index.html page instead of...
Zooko O'Whielacronx [Wed, 14 Jan 2009 19:50:23 +0000 (12:50 -0700)]
setup: configure setup.cfg to point setuptools at the index.html page instead of straight at the tahoe directory listing
This is necessary, because if setuptools looks at the directory listing, then it follows the link named "More Info" in attempt to download the file, and this fails.

15 years agosetup: fix previous patch to set reactor to poll reactor on linux or cygwin
Zooko O'Whielacronx [Wed, 14 Jan 2009 17:40:22 +0000 (10:40 -0700)]
setup: fix previous patch to set reactor to poll reactor on linux or cygwin

15 years agosetup: use poll reactor for trial if on linux2 or cygwin
Zooko O'Whielacronx [Wed, 14 Jan 2009 16:15:46 +0000 (09:15 -0700)]
setup: use poll reactor for trial if on linux2 or cygwin

15 years agodocs: rename frontends/webapi.txt to frontends/wapi.txt
Zooko O'Whielacronx [Wed, 14 Jan 2009 03:51:43 +0000 (20:51 -0700)]
docs: rename frontends/webapi.txt to frontends/wapi.txt
rename CLI.txt to frontends/CLI.txt
change a few mentions of "webapi" to "wapi"
fixes #582

15 years agoupload: use WriteBucketProxy_v2 when uploading a large file (with shares larger than...
Brian Warner [Tue, 13 Jan 2009 03:14:42 +0000 (20:14 -0700)]
upload: use WriteBucketProxy_v2 when uploading a large file (with shares larger than 4GiB). This finally closes #346. I think we can now handle immutable files up to 48EiB.

15 years agodeep-check-and-repair: improve results and their HTML representation
Brian Warner [Tue, 13 Jan 2009 01:56:19 +0000 (18:56 -0700)]
deep-check-and-repair: improve results and their HTML representation

15 years agotest_repairer.py: hush pyflakes: remove duplicate/shadowed function name, by using...
Brian Warner [Mon, 12 Jan 2009 22:45:09 +0000 (15:45 -0700)]
test_repairer.py: hush pyflakes: remove duplicate/shadowed function name, by using the earlier definition (which is identical)

15 years agohush pyflakes by removing unused imports
Brian Warner [Mon, 12 Jan 2009 22:41:20 +0000 (15:41 -0700)]
hush pyflakes by removing unused imports

15 years agoimmutable repairer
Zooko O'Whielacronx [Mon, 12 Jan 2009 18:00:22 +0000 (11:00 -0700)]
immutable repairer
This implements an immutable repairer by marrying a CiphertextDownloader to a CHKUploader.  It extends the IDownloadTarget interface so that the downloader can provide some metadata that the uploader requires.
The processing is incremental -- it uploads the first segments before it finishes downloading the whole file.  This is necessary so that you can repair large files without running out of RAM or using a temporary file on the repairer.
It requires only a verifycap, not a readcap.  That is: it doesn't need or use the decryption key, only the integrity check codes.
There are several tests marked TODO and several instances of XXX in the source code.  I intend to open tickets to document further improvements to functionality and testing, but the current version is probably good enough for Tahoe-1.3.0.

15 years agoutil: dictutil: add DictOfSets.union(key, values) and DictOfSets.update(otherdictofsets)
Zooko O'Whielacronx [Mon, 12 Jan 2009 17:55:39 +0000 (10:55 -0700)]
util: dictutil: add DictOfSets.union(key, values) and DictOfSets.update(otherdictofsets)

15 years agosetup: update doc in setup.cfg
Zooko O'Whielacronx [Sun, 11 Jan 2009 16:13:19 +0000 (09:13 -0700)]
setup: update doc in setup.cfg

15 years agosetup: Point setuptools at a directory on the allmydata.org test grid to find depende...
Zooko O'Whielacronx [Sun, 11 Jan 2009 16:11:26 +0000 (09:11 -0700)]
setup: Point setuptools at a directory on the allmydata.org test grid to find dependencies.
Don't include an unrouteable IP address in find_links (fixes #574).

15 years agoimmutable: separate tests of immutable upload/download from tests of immutable checki...
Zooko O'Whielacronx [Sat, 10 Jan 2009 22:07:39 +0000 (15:07 -0700)]
immutable: separate tests of immutable upload/download from tests of immutable checking/repair

15 years agotrivial: minor changes to in-line comments -- mark plaintext-hash-tree as obsolete
Zooko O'Whielacronx [Sat, 10 Jan 2009 21:56:01 +0000 (14:56 -0700)]
trivial: minor changes to in-line comments -- mark plaintext-hash-tree as obsolete

15 years agoimmutable: make the web display of upload results more human-friendly, like they...
Zooko O'Whielacronx [Sat, 10 Jan 2009 21:02:09 +0000 (14:02 -0700)]
immutable: make the web display of upload results more human-friendly, like they were before my recent change to the meaning of the "sharemap"

15 years agoimmutable: fix edit-o in interfaces.py documentation introduced in recent patch
Zooko O'Whielacronx [Sat, 10 Jan 2009 19:54:08 +0000 (12:54 -0700)]
immutable: fix edit-o in interfaces.py documentation introduced in recent patch

15 years agoimmutable: redefine the "sharemap" member of the upload results to be a map from...
Zooko O'Whielacronx [Sat, 10 Jan 2009 18:46:23 +0000 (11:46 -0700)]
immutable: redefine the "sharemap" member of the upload results to be a map from shnum to set of serverids
It used to be a map from shnum to a string saying "placed this share on XYZ server".  The new definition is more in keeping with the "sharemap" object that results from immutable file checking and repair, and it is more useful to the repairer, which is a consumer of immutable upload results.

15 years agonaming: finish renaming "CheckerResults" to "CheckResults"
Zooko O'Whielacronx [Sat, 10 Jan 2009 01:00:52 +0000 (18:00 -0700)]
naming: finish renaming "CheckerResults" to "CheckResults"

15 years agostorage.py : replace 4294967295 with 2**32-1: python does constant folding, I measure...
Brian Warner [Sat, 10 Jan 2009 02:52:22 +0000 (19:52 -0700)]
storage.py : replace 4294967295 with 2**32-1: python does constant folding, I measured this statement as taking 50ns, versus the 400ns for the call to min(), or the 9us required for the 'assert not os.path.exists' syscall

15 years agostorage.py: announce a maximum-immutable-share-size based upon a 'df' of the disk...
Brian Warner [Sat, 10 Jan 2009 02:37:36 +0000 (19:37 -0700)]
storage.py: announce a maximum-immutable-share-size based upon a 'df' of the disk. Fixes #569, and this should be the last requirement for #346 (remove 12GiB filesize limit)

15 years agoset bin/tahoe executable permissions and leave build_tahoe in sys.argv
cgalvan [Fri, 9 Jan 2009 22:06:40 +0000 (15:06 -0700)]
set bin/tahoe executable permissions and leave build_tahoe in sys.argv

15 years agosetup: merge relaxation of the version of setuptools that we require at runtime with...
Zooko O'Whielacronx [Fri, 9 Jan 2009 20:09:49 +0000 (13:09 -0700)]
setup: merge relaxation of the version of setuptools that we require at runtime with an indentation change

16 years agosetup: remove custom Trial class inside our setup.py and use the setuptools_trial...
Zooko O'Whielacronx [Sat, 6 Dec 2008 00:22:07 +0000 (17:22 -0700)]
setup: remove custom Trial class inside our setup.py and use the setuptools_trial plugin

16 years agosetup: we require pywin32 if building on Windows (plus some formatting and comment...
Zooko O'Whielacronx [Sat, 6 Dec 2008 00:19:11 +0000 (17:19 -0700)]
setup: we require pywin32 if building on Windows (plus some formatting and comment fixes)

15 years agofix bin/tahoe executable for Windows
cgalvan [Fri, 9 Jan 2009 19:42:22 +0000 (12:42 -0700)]
fix bin/tahoe executable for Windows

15 years agouse subprocess.call instead of os.execve in bin/tahoe
cgalvan [Fri, 9 Jan 2009 19:03:00 +0000 (12:03 -0700)]
use subprocess.call instead of os.execve in bin/tahoe