Brian Warner [Fri, 6 Feb 2009 05:07:01 +0000 (22:07 -0700)]
#598: add cli+backupdb tests, improve user display, update docs, move docs out of proposed/
Brian Warner [Fri, 6 Feb 2009 02:56:40 +0000 (19:56 -0700)]
#598: add backupdb to 'tahoe backup' command, enable it by default
Brian Warner [Fri, 6 Feb 2009 01:17:56 +0000 (18:17 -0700)]
add sqlite-based backupdb, for #598 and others (including 'tahoe cp'). Not enabled yet.
Zooko O'Whielacronx [Thu, 5 Feb 2009 16:29:23 +0000 (09:29 -0700)]
setup: require new bundled setuptools-0.6c12dev
Zooko O'Whielacronx [Thu, 5 Feb 2009 16:28:18 +0000 (09:28 -0700)]
setup: bundle setuptools-0.6c12dev (our own toothpick of setuptools) this version completes my patch to fix http://bugs.python.org/setuptools/issue54 , which is necessary for tahoe to build with --prefix=support without doing a lot of PYTHONPATH gymnastics around the call to setup.py
Zooko O'Whielacronx [Thu, 5 Feb 2009 16:27:58 +0000 (09:27 -0700)]
setup: remove old bundled setuptools-0.6c11dev (our own toothpick of setuptools)
Zooko O'Whielacronx [Thu, 5 Feb 2009 01:06:20 +0000 (18:06 -0700)]
setup: add a case to execute "python .../twistd.py" if "twistd" is not found
Zooko O'Whielacronx [Wed, 4 Feb 2009 22:38:40 +0000 (15:38 -0700)]
doc: specify Python >= 2.4.2
Zooko O'Whielacronx [Wed, 4 Feb 2009 20:32:22 +0000 (13:32 -0700)]
setup: merge recent patches that change the set of bundled tools in misc/dependencies/
Zooko O'Whielacronx [Wed, 4 Feb 2009 20:24:54 +0000 (13:24 -0700)]
setup: bundle new setuptools_trial
Zooko O'Whielacronx [Wed, 4 Feb 2009 20:24:42 +0000 (13:24 -0700)]
setup: remove old bundled setuptools_trial
Zooko O'Whielacronx [Wed, 4 Feb 2009 20:24:29 +0000 (13:24 -0700)]
setup: bundle new setuptools_darcs
Zooko O'Whielacronx [Wed, 4 Feb 2009 20:24:18 +0000 (13:24 -0700)]
setup: remove old bundled setuptools_darcs
Zooko O'Whielacronx [Wed, 4 Feb 2009 08:38:36 +0000 (01:38 -0700)]
roll back the upgrade of the bundled setuptools_trial from 0.5 to 0.5.1
Upgrading setuptools trial in that way leads to a very tricky problem in setuptools's handling of recursive installation of the build-time dependencies of build-time dependencies...
rolling back:
Tue Feb 3 22:17:18 MST 2009 zooko@zooko.com
* setup: bundle new setuptools_trial-0.5.1
A ./misc/dependencies/setuptools_trial-0.5.1.tar
Tue Feb 3 22:17:32 MST 2009 zooko@zooko.com
* setup: remove old bundled setuptools_trial
R ./misc/dependencies/setuptools_trial-0.5.tar
Zooko O'Whielacronx [Wed, 4 Feb 2009 06:24:05 +0000 (23:24 -0700)]
setup: remove a couple of horrible work-arounds in setup.py now that we rely on our own toothpick of setuptools which fixes those issues
also specify that we need our latest revision (11dev) of our toothpick of setuptools
also *always* setup_require setuptools_darcs at module import time. Formerly we added setup_require setuptools_darcs only if the PKG-INFO file were not already created. There is some weird, irreproducible bug to do with setuptool_darcs, and I guess that the fact that whether it is required or not depends on that autogenerated file might have something to do with it. Anyway, this is simpler.
Zooko O'Whielacronx [Wed, 4 Feb 2009 06:17:32 +0000 (23:17 -0700)]
setup: remove old bundled setuptools_trial
Zooko O'Whielacronx [Wed, 4 Feb 2009 06:17:18 +0000 (23:17 -0700)]
setup: bundle new setuptools_trial-0.5.1
Zooko O'Whielacronx [Wed, 4 Feb 2009 06:17:04 +0000 (23:17 -0700)]
setup: bundle new setuptools_darcs-1.2.5
Zooko O'Whielacronx [Wed, 4 Feb 2009 06:16:40 +0000 (23:16 -0700)]
setup: remove old bundled setuptools_darcs
Zooko O'Whielacronx [Wed, 4 Feb 2009 06:16:28 +0000 (23:16 -0700)]
setup: remove bundled darcsver in gzipped format
Zooko O'Whielacronx [Wed, 4 Feb 2009 06:16:17 +0000 (23:16 -0700)]
setup: bundle darcsver in ungzipped format
Zooko O'Whielacronx [Wed, 4 Feb 2009 04:34:10 +0000 (21:34 -0700)]
docs: mention issues using flogtool on Windows
Brian Warner [Wed, 4 Feb 2009 02:22:48 +0000 (19:22 -0700)]
webapi: add verifycap (spelled 'verify_url') to the t=json output on files and directories. Closes #559.
Zooko O'Whielacronx [Tue, 3 Feb 2009 05:23:52 +0000 (22:23 -0700)]
setup: stop trying to add specific dirs to site-dirs to work-around setuptools #17
Zooko O'Whielacronx [Tue, 3 Feb 2009 05:23:23 +0000 (22:23 -0700)]
setup: require and automatically use setuptools-0.6c11dev (our own toothpick of setuptools) which is bundled
Zooko O'Whielacronx [Tue, 3 Feb 2009 05:23:04 +0000 (22:23 -0700)]
setup: remove old bundled setuptools-0.6c10dev (our own toothpick of setuptools)
Zooko O'Whielacronx [Tue, 3 Feb 2009 05:19:50 +0000 (22:19 -0700)]
setup: bundle setuptools-0.6c11dev (our own toothpick of setuptools)
Hopefully this one fixes the issue with easy_install not searching the sys.path for packages that were requested to be installed, (setuptools #17), thus allowing us to do away with the "--site-dirs=" kludges, which are currently breaking some of our buildbots.
Brian Warner [Tue, 3 Feb 2009 04:12:32 +0000 (21:12 -0700)]
CLI: tahoe_check: stop escaping the JSON output when using --raw
Brian Warner [Tue, 3 Feb 2009 04:09:02 +0000 (21:09 -0700)]
#598: first cut of 'tahoe backup' command: no backupdb, but yes shared-unchanged-directories and Archives/TIMESTAMP and Latest/
Brian Warner [Tue, 3 Feb 2009 04:08:56 +0000 (21:08 -0700)]
CLI: move node-url -should-end-in-slash code into a common place, so other tools can rely upon it later
Brian Warner [Tue, 3 Feb 2009 04:07:20 +0000 (21:07 -0700)]
tahoe_ls: CLI command should return rc=0, not None
Zooko O'Whielacronx [Tue, 3 Feb 2009 01:09:55 +0000 (18:09 -0700)]
setup: add site-dirs for Debian and for some (?) Mac OS X
Brian Warner [Tue, 3 Feb 2009 00:36:57 +0000 (17:36 -0700)]
misc/dependencies: update setuptools_trial from 0.4 to 0.5
Brian Warner [Tue, 3 Feb 2009 00:03:33 +0000 (17:03 -0700)]
.darcs-boringfile: ignore bin/tahoe and bin/tahoe-script.py, since both are now generated
Brian Warner [Tue, 3 Feb 2009 00:03:07 +0000 (17:03 -0700)]
Makefile: fix 'clean' target to remove bin/tahoe and bin/tahoe-script.py, since both are now generated
Zooko O'Whielacronx [Sat, 31 Jan 2009 05:16:49 +0000 (22:16 -0700)]
storage: disable test_large_share again: my linux laptop has less than 4 GiB free
Brian Warner [Sat, 31 Jan 2009 02:32:05 +0000 (19:32 -0700)]
web/directory: add a link from readwrite directories to a read-only version, and fix the 'SI=xxx' header to actually use the storage index, not the writekey
Brian Warner [Sat, 31 Jan 2009 02:31:10 +0000 (19:31 -0700)]
uri: add abbrev_si() method, which returns the abbreviated storage index
Zooko O'Whielacronx [Fri, 30 Jan 2009 21:38:19 +0000 (14:38 -0700)]
setup: make sure you use darcsver whenever you are going to run trial
This fixes the bug Brian had where he ran "python ./setup.py trial" and the allmydata-tahoe version number came out as 0.0.0.
Zooko O'Whielacronx [Fri, 30 Jan 2009 05:31:33 +0000 (22:31 -0700)]
setup: require setuptools_trial >= 0.5, and delegate to it the job of deciding which Twisted reactor to use for the current platform
Brian Warner [Fri, 30 Jan 2009 11:25:36 +0000 (04:25 -0700)]
Makefile: use 'setup.py test' for test/quicktest targets (instead of
'setup.py trial'). 'setup.py trial' clobbers the tahoe .egg's PKG-INFO
"Version:" field (resetting it to 0.0.0), possibly because it isn't invoking
the darcsver subcommand that 'setup.py test' does before it runs the 'trial'
subcommand.
This slows down quicktest by another couple of seconds (see #591) and adds
more noise to its output, but without this change, 'make test' and 'make
quicktest' fail on test_runner (which spawns bin/tahoe as a subprocess, and
with a mangled Version:, the setuptools-based entry point script refuses to
recognize our source tree as a usable version of Tahoe).
Brian Warner [Fri, 30 Jan 2009 02:01:31 +0000 (19:01 -0700)]
Makefile: remove the obsolete stats-gatherer-run target
Zooko O'Whielacronx [Thu, 29 Jan 2009 20:59:52 +0000 (13:59 -0700)]
setup: remove the "build three times in a row" kludge now that #229 is fixed, and spell build "build" instead of "build_tahoe"
Zooko O'Whielacronx [Thu, 29 Jan 2009 19:56:40 +0000 (12:56 -0700)]
setup: require darcsver >= 1.2.0 and rely exclusively on darcsver to set the version string
Zooko O'Whielacronx [Thu, 29 Jan 2009 19:51:15 +0000 (12:51 -0700)]
setup: add new bundled darcsver-1.2.0
Zooko O'Whielacronx [Thu, 29 Jan 2009 19:51:02 +0000 (12:51 -0700)]
setup: remove old bundled darcsver-1.1.8
Zooko O'Whielacronx [Thu, 29 Jan 2009 18:48:02 +0000 (11:48 -0700)]
setup: add doc explaining why we set zip_safe=False
Zooko O'Whielacronx [Thu, 29 Jan 2009 17:58:15 +0000 (10:58 -0700)]
setup: setup.cfg aliases get expanded only once, so put everything you want to happen during the "test" step in the alias
Zooko O'Whielacronx [Thu, 29 Jan 2009 17:51:25 +0000 (10:51 -0700)]
setup: invoke darcsver whenever doing an sdist
Zooko O'Whielacronx [Thu, 29 Jan 2009 17:49:06 +0000 (10:49 -0700)]
setup: more verbose assertion failure in test_runner
Zooko O'Whielacronx [Thu, 29 Jan 2009 14:00:58 +0000 (07:00 -0700)]
setup: subclass setuptools.Command instead of distutils Command
There's almost no difference between them, but let's be consistent try to use the setuptools API as it was intended.
Zooko O'Whielacronx [Thu, 29 Jan 2009 14:00:00 +0000 (07:00 -0700)]
setup: temporarily comment-out the horrible kludge to work-around setuptools #17, while I figure out how to solve it better
Zooko O'Whielacronx [Thu, 29 Jan 2009 05:56:08 +0000 (22:56 -0700)]
setup: always create a support dir and populate it with a site-packages and add same to the PYTHONPATH, just in case someone is going to do "build", "develop", or "test" or something else which triggers a build
I think there must be a much better solution for this -- probably to fix setuptools #54 and ship our own fork of setuptools and rely on it.
Zooko O'Whielacronx [Thu, 29 Jan 2009 05:55:34 +0000 (22:55 -0700)]
setup: if any of "build", "develop", or "test" appear in the sys.argv then that means we'll be doing a develop, so add the workarounds for setuptools #17 in any case
I think there must be a much better solution for this -- probably to fix setuptools #17 and ship our own fork of setuptools and rely on it.
Zooko O'Whielacronx [Thu, 29 Jan 2009 01:26:28 +0000 (18:26 -0700)]
setup: add metadata indicating compatibility with python 2.6
Zooko O'Whielacronx [Thu, 29 Jan 2009 01:07:16 +0000 (18:07 -0700)]
setup: a new improved way to create tahoe executables
Create the 'tahoe-script.py' file under the 'bin' directory. The 'tahoe-script.py' file is exactly the same as the 'tahoe-script.template' script except that the shebang line is rewritten to use our sys.executable for the interpreter. On Windows, create a tahoe.exe will execute it. On non-Windows, make a symlink to it from 'tahoe'. The tahoe.exe will be copied from the setuptools egg's cli.exe and this will work from a zip-safe and non-zip-safe setuptools egg.
Zooko O'Whielacronx [Wed, 28 Jan 2009 23:33:12 +0000 (16:33 -0700)]
storage: enable the test of a share larger than 2 GiB; this will probably be too expensive on Mac OS X, but I think it won't be on all the other platforms ; I will probably set it to SkipTest if the sys.platform is Mac after seeing the results of this buildbot run
Zooko O'Whielacronx [Wed, 28 Jan 2009 21:03:26 +0000 (14:03 -0700)]
setup: bundle darcsver-1.1.8
Zooko O'Whielacronx [Wed, 28 Jan 2009 19:20:12 +0000 (12:20 -0700)]
setup: remove old bundled dependency darcsver-1.1.5
Brian Warner [Tue, 27 Jan 2009 22:39:09 +0000 (15:39 -0700)]
CLI: fix examples in tahoe put --help
Zooko O'Whielacronx [Tue, 27 Jan 2009 22:11:48 +0000 (15:11 -0700)]
trivial: remove unused imports noticed by pyflakes
Zooko O'Whielacronx [Tue, 27 Jan 2009 21:37:17 +0000 (14:37 -0700)]
setup: fix test_runner to invoke bin/tahoe.exe instead of bin/tahoe if on Windows (this is what happens when a user invokes bin/tahoe on Windows)
Zooko O'Whielacronx [Tue, 27 Jan 2009 21:35:05 +0000 (14:35 -0700)]
setup: fix test_runner to assert that lines which are output to stderr must end with a punctuation mark (apparently re.search("x$", "x\r\n") does not match. :-()
Zooko O'Whielacronx [Tue, 27 Jan 2009 21:32:45 +0000 (14:32 -0700)]
setup: fix test_runner.RunNode.test_baddir -- it was left incomplete and broken by a recent patch
Zooko O'Whielacronx [Tue, 27 Jan 2009 05:40:46 +0000 (22:40 -0700)]
setup: hack the sys.argv to set poll reactor if "python ./setup.py test" in addition to if "python ./setup.py trial"; remove another hack which has been moved setup.cfg; remove setup_requires Twisted since now we actually setup_require setuptools_trial and it install_requires Twisted.
Zooko O'Whielacronx [Tue, 27 Jan 2009 00:46:16 +0000 (17:46 -0700)]
setup: go ahead and check for noise in test_client_no_noise
Zooko O'Whielacronx [Tue, 27 Jan 2009 00:32:40 +0000 (17:32 -0700)]
setup: always run "build" before running "test"
Zooko O'Whielacronx [Tue, 27 Jan 2009 00:30:46 +0000 (17:30 -0700)]
setup: add a test for a warning message from importing nevow, marked as TODO, comment-out the assertion of no-noise inside other test_runner tests
Zooko O'Whielacronx [Tue, 27 Jan 2009 00:30:24 +0000 (17:30 -0700)]
setup: always run build_tahoe before running tests
Brian Warner [Mon, 26 Jan 2009 00:06:39 +0000 (17:06 -0700)]
diskwatcher: cache growth results (which take 30s to generate) for 3 minutes, to help munin, which invokes it 6 times in a row every 5 minutes
Brian Warner [Sat, 24 Jan 2009 02:39:08 +0000 (19:39 -0700)]
make streaming-manifest stop doing work after the HTTP connection is dropped
Zooko O'Whielacronx [Sat, 24 Jan 2009 01:49:11 +0000 (18:49 -0700)]
setup: sys.exit() raises a SystemExit exception on Python 2.4 on Solaris -- fix it so that bin/tahoe doesn't interpret this as its brother having been non-executable
Zooko O'Whielacronx [Sat, 24 Jan 2009 01:34:37 +0000 (18:34 -0700)]
setup: find a "bin/tahoe" executable to test based on allmydata.__file__ instead of based on the CWD
This means that the tests still work if you are executing them from a CWD other than the src dir -- *if* the "bin/tahoe" is found at os.path.dirname(os.path.dirname(os.path.dirname(allmydata.__file__))).
If no file is found at that location, then just skip the tests of executing the "tahoe" executable, because we don't want to accidentally run those tests against an executable from a different version of tahoe.
Brian Warner [Sat, 24 Jan 2009 00:28:12 +0000 (17:28 -0700)]
rollback the #591 fix, since it breaks quicktest entirely
Brian Warner [Sat, 24 Jan 2009 00:00:02 +0000 (17:00 -0700)]
#509: remove non-streaming 'tahoe manifest' CLI form
Brian Warner [Fri, 23 Jan 2009 23:59:39 +0000 (16:59 -0700)]
CLI.txt: improve docs on 'tahoe manifest' to cover --verify-cap, --repair-cap, and streaming JSON
Brian Warner [Fri, 23 Jan 2009 23:33:21 +0000 (16:33 -0700)]
#509 CLI: add 'tahoe manifest --stream'
Brian Warner [Fri, 23 Jan 2009 23:32:47 +0000 (16:32 -0700)]
#509: test_system.py: add test for streamed-manifest
Brian Warner [Fri, 23 Jan 2009 23:13:06 +0000 (16:13 -0700)]
test_system: rearrange DeepCheckWebGood to make it easier to add CLI tests
Zooko O'Whielacronx [Fri, 23 Jan 2009 23:58:30 +0000 (16:58 -0700)]
setup: use "trial" executable instead of the setuptools_trial plugin for "make quicktest"
This is to fix #591 ("make quicktest" could be quicker and less noisy). This means that "make quicktest" won't work unless the user has manually installed Twisted already such that the "trial" executable is on their PATH and the Twisted package is on their PYTHONPATH. This bypasses the behavior of setuptools_trial which builds and checks package dependencies before executing the tests.
Brian Warner [Fri, 23 Jan 2009 22:21:50 +0000 (15:21 -0700)]
CLI: remove the '-v' alias for --version-and-path, to free up '-v' for more traditional uses (like --verbose)
Brian Warner [Fri, 23 Jan 2009 05:01:36 +0000 (22:01 -0700)]
#590: add webish t=stream-manifest
Brian Warner [Fri, 23 Jan 2009 04:44:49 +0000 (21:44 -0700)]
dirnode: add get_repair_cap()
Brian Warner [Fri, 23 Jan 2009 04:39:50 +0000 (21:39 -0700)]
dirnode.deep_traverse: fix docstring to describe the correct return value
Brian Warner [Fri, 23 Jan 2009 04:38:36 +0000 (21:38 -0700)]
filenode: add get_repair_cap(), which uses the read-write filecap for immutable files, and the verifycap for immutable files
Zooko O'Whielacronx [Thu, 22 Jan 2009 22:54:05 +0000 (15:54 -0700)]
setup: make the bin/tahoe executable exit with the appropriate exit code
Zooko O'Whielacronx [Thu, 22 Jan 2009 22:52:40 +0000 (15:52 -0700)]
setup: add test that the tests are testing the right source code
This is a test of #145, and I think that now the code passes this test.
Zooko O'Whielacronx [Thu, 22 Jan 2009 22:52:13 +0000 (15:52 -0700)]
trivial: removed unused imports noticed by pyflakes
Zooko O'Whielacronx [Thu, 22 Jan 2009 22:38:18 +0000 (15:38 -0700)]
setup: change test_runner to invoke "bin/tahoe" in a subprocess instead of executing runner.runner()
This is necessary because loading allmydata code now depends on PYTHONPATH manipulation which is done in the "bin/tahoe" script. Unfortunately it makes test_runner slower since it launches and waits for many subprocesses.
Zooko O'Whielacronx [Wed, 21 Jan 2009 19:47:20 +0000 (12:47 -0700)]
setup: fix "tahoe start" to work on Windows even when a Tahoe base dir hasn't been configured in the Windows registry
Zooko O'Whielacronx [Wed, 21 Jan 2009 03:50:42 +0000 (20:50 -0700)]
docs: trivial naming change
Zooko O'Whielacronx [Wed, 21 Jan 2009 03:47:35 +0000 (20:47 -0700)]
rollback the feature of making "ambient upload authority" configurable
This reverses some, but not all, of the changes that were committed in the following set of patches.
rolling back:
Sun Jan 18 09:54:30 MST 2009 toby.murray
* add 'web.ambient_upload_authority' as a paramater to tahoe.cfg
M ./src/allmydata/client.py -1 +3
M ./src/allmydata/test/common.py -7 +9
A ./src/allmydata/test/test_ambient_upload_authority.py
M ./src/allmydata/web/root.py +12
M ./src/allmydata/webish.py -1 +4
Sun Jan 18 09:56:08 MST 2009 zooko@zooko.com
* trivial: whitespace
I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.
M ./src/allmydata/test/test_ambient_upload_authority.py -9 +8
M ./src/allmydata/web/root.py -2 +1
M ./src/allmydata/webish.py -2 +1
Mon Jan 19 14:16:19 MST 2009 zooko@zooko.com
* trivial: remove unused import noticed by pyflakes
M ./src/allmydata/test/test_ambient_upload_authority.py -1
Mon Jan 19 21:38:35 MST 2009 toby.murray
* doc: describe web.ambient_upload_authority
M ./docs/configuration.txt +14
M ./docs/frontends/webapi.txt +11
Mon Jan 19 21:38:57 MST 2009 zooko@zooko.com
* doc: add Toby Murray to the CREDITS
M ./CREDITS +4
Zooko O'Whielacronx [Wed, 21 Jan 2009 00:40:12 +0000 (17:40 -0700)]
setup: add new bundled setuptools_trial-0.4
Zooko O'Whielacronx [Wed, 21 Jan 2009 00:39:18 +0000 (17:39 -0700)]
setup: remove old bundled setuptools_trial-0.2
Zooko O'Whielacronx [Tue, 20 Jan 2009 19:42:29 +0000 (12:42 -0700)]
setup: require darcsver always, and not just when we see the string "darcsver" in sys.argv, because the new aliases hack means that the string might not appear in sys.argv
Zooko O'Whielacronx [Tue, 20 Jan 2009 19:38:09 +0000 (12:38 -0700)]
setup: fix test_system to require tahoe under its package==distribution name "allmydata-tahoe" instead of its module name "allmydata"
Zooko O'Whielacronx [Tue, 20 Jan 2009 19:37:23 +0000 (12:37 -0700)]
setup: use setup.cfg aliases to map "setup.py test" to "setup.py trial" and "setup.py build" to "setup.py darcsver --count-all-patches build_tahoe"
Thanks to dpeterson for the suggestion.
Zooko O'Whielacronx [Tue, 20 Jan 2009 05:38:57 +0000 (22:38 -0700)]
doc: add Toby Murray to the CREDITS
toby.murray [Tue, 20 Jan 2009 05:38:35 +0000 (22:38 -0700)]
doc: describe web.ambient_upload_authority
Zooko O'Whielacronx [Mon, 19 Jan 2009 22:16:19 +0000 (15:16 -0700)]
trivial: remove unused import noticed by pyflakes