Zooko O'Whielacronx [Tue, 2 Nov 2010 05:33:57 +0000 (22:33 -0700)]
tests: make test-with-fake-pkg's unacceptably-old fake pycryptopp be 0.5.13 instead of 0.5.17, since 0.5.17 is acceptably new on non-x86 archs
Also move the fakepkgs directory to be in misc/build_helpers/ instead of in the base of the source tree.
david-sarah [Tue, 2 Nov 2010 03:59:05 +0000 (20:59 -0700)]
test_runner.py: fix test failure in test_the_right_code after applying zooko's change to test-with-fake-pkg.py
Zooko O'Whielacronx [Mon, 1 Nov 2010 05:21:42 +0000 (22:21 -0700)]
setup: FreeStorm's WinXP-x86-py2.6 buildslave has informed us that there is yet a fourth way to spell "i386" in Python
Zooko O'Whielacronx [Mon, 1 Nov 2010 04:27:21 +0000 (21:27 -0700)]
setup: add platform.machine() to show-tool-versions, because FreeStorm WinXP builder apparently thinks that its platform.machine() is not one of ['i386', 'x86_64', 'amd64', ''], since it is requiring pycryptopp >= 0.5.14 instead of pycryptopp >= 0.5.20
Zooko O'Whielacronx [Mon, 1 Nov 2010 04:26:02 +0000 (21:26 -0700)]
edit docs for English usage, rename "Tahoe" to "Tahoe-LAFS" in docs/configuration.rst, rename a variable in bin/tahoe-script.template for clarity, update François's CREDITS entry
david-sarah [Mon, 1 Nov 2010 00:33:16 +0000 (17:33 -0700)]
allmydata/__init__.py: move the call to require_auto_deps() to the top again, since the [4784] patch turned out not to be the cause of the failure on the 'mm netbsd5' buildslave.
david-sarah [Sun, 31 Oct 2010 17:46:49 +0000 (10:46 -0700)]
allmydata/__init__.py: put the _auto_deps.require_auto_deps() call back in exactly the same place it was prior to [4784].
david-sarah [Sun, 31 Oct 2010 16:06:36 +0000 (09:06 -0700)]
allmydata/__init__.py: suppress a DeprecationWarning for the sha module on importing pycrypto.
david-sarah [Sun, 31 Oct 2010 16:03:32 +0000 (09:03 -0700)]
misc/build_helpers/run_trial.py: fix another typo in last patch.
david-sarah [Sun, 31 Oct 2010 15:52:15 +0000 (08:52 -0700)]
misc/build_helpers/run_trial.py: fix typo in last patch.
david-sarah [Sun, 31 Oct 2010 15:38:28 +0000 (08:38 -0700)]
allmydata/__init__.py: call require_auto_deps() after importing nevow and twisted, reverting change in [4784]. Also fix a missing 'warnings.filters.pop()'.
david-sarah [Sun, 31 Oct 2010 15:36:13 +0000 (08:36 -0700)]
misc/build_helpers/run_trial.py: fix false positive on directory check that can occur when running run_trial from test-with-fake-pkg manually.
david-sarah [Sun, 31 Oct 2010 05:50:03 +0000 (22:50 -0700)]
allmydata.__init__.py: temporary hack to debug failure on midnightmagic's buildslave
david-sarah [Sun, 31 Oct 2010 05:14:26 +0000 (22:14 -0700)]
NEWS: entries for #1190 and #1212, and minor cleanups. refs #1190, #1212
david-sarah [Sun, 31 Oct 2010 03:52:31 +0000 (20:52 -0700)]
tahoe backup: perform tilde expansion in exclude-from filename (version 2). fixes #1241
Francois Deppierraz [Sat, 30 Oct 2010 11:11:30 +0000 (04:11 -0700)]
NEWS: add news entry for #1223
Francois Deppierraz [Sat, 30 Oct 2010 10:13:51 +0000 (03:13 -0700)]
NEWS: add a NEWS entry about bug #1045
Zooko O'Whielacronx [Sat, 30 Oct 2010 08:10:35 +0000 (01:10 -0700)]
setup: run require_auto_deps() before attempting to import any deps in __init__.py
For one thing, this makes missing-dependency failures into DistributionNotFound errors instead of ImportErrors, which might be more useful to the user. For another thing, if someone is using distributions that were installed with --multi-version, then they might be not importable until after require_auto_deps() has been run. (The docs claim that this would be the case, but we don't have an example of this happening at this time.)
Zooko O'Whielacronx [Sat, 30 Oct 2010 07:02:33 +0000 (00:02 -0700)]
setup: show-tool-versions: emit module and __version__ information even when module name != distribution (package) name, and add TwistedCore, TwistedWeb, and TwistedConch
david-sarah [Sat, 30 Oct 2010 03:43:03 +0000 (20:43 -0700)]
misc/build_helpers/test-with-fake-pkg.py: look for eggs in the parent of the src directory. refs #1190
david-sarah [Sat, 30 Oct 2010 00:31:49 +0000 (17:31 -0700)]
scripts/runner.py: fix unused import of allmydata. refs #1190
david-sarah [Fri, 29 Oct 2010 23:53:28 +0000 (16:53 -0700)]
scripts/runner.py: remove pkg_resources.require() calls. These are at best redundant because we have already called _auto_deps.require_auto_deps() (from allmydata.__init__) at that point, and they are causing failure of the test-from-prefixdir step on some buildslaves. refs #1190
david-sarah [Fri, 29 Oct 2010 23:03:29 +0000 (16:03 -0700)]
misc/build_helpers/run_trial.py: look for zetuptoolz egg in the parent directory, not the cwd of run_trial. refs #1190
david-sarah [Fri, 29 Oct 2010 22:31:11 +0000 (15:31 -0700)]
bundled zetuptoolz: if __main__.__requires__ exists then do not add packages to the working set if they provide an incompatible version of a package. Also put a complete __requires__ listing the transitive closure of dependencies at the beginning of generated scripts, rather than a shallow __requires__ specifying only the application version. refs #1190
david-sarah [Fri, 29 Oct 2010 22:28:25 +0000 (15:28 -0700)]
setup.py, misc/build_helpers/run_trial.py: use undocumented __requires__ variable to cause setuptools/zetuptoolz to put the correct versions of dependencies on sys.path. Also ensure that run_trial adds the bundled zetuptoolz egg at the start of sys.path if present. Make the source directory comparison work correctly for the test-with-fake-pkg build step. refs #1190
david-sarah [Fri, 29 Oct 2010 22:11:23 +0000 (15:11 -0700)]
test_runner.py: fix error in BinTahoe.test_version_no_noise introduced by last patch. refs #1235
david-sarah [Fri, 29 Oct 2010 20:42:46 +0000 (13:42 -0700)]
test_runner.py: also allow 'from pkg_resources import load_entry_point' as noise. refs #1235.
david-sarah [Fri, 29 Oct 2010 19:18:04 +0000 (12:18 -0700)]
test_runner.py: if the only noise is 'UserWarning: Unbuilt egg for setuptools', skip instead of failing the no_noise tests. This version avoids 'any' to be compatible with Python < 2.5. refs #1235.
david-sarah [Thu, 28 Oct 2010 18:00:46 +0000 (11:00 -0700)]
NEWS: clarify (strengthen) description of what backdoors.rst declares, and add bugfix entries for 'tahoe cp' and Windows console bugs. refs #1216, #1224, #1232
david-sarah [Wed, 27 Oct 2010 04:33:02 +0000 (21:33 -0700)]
make ResponseCache smarter to avoid memory leaks: don't record timestamps, use DataSpans to merge entries, and clear the cache when we see a new seqnum. refs #1045, #1229
david-sarah [Wed, 27 Oct 2010 02:16:36 +0000 (19:16 -0700)]
windows/fixups.py: limit length of string passed in a single call to WriteConsoleW. fixes #1232.
Brian Warner [Fri, 29 Oct 2010 09:42:23 +0000 (02:42 -0700)]
scripts/tahoe_backup.py: oops, fix missing import, thanks pyflakes
test_cli.py: hush minor pyflakes complaint
Brian Warner [Fri, 29 Oct 2010 09:14:24 +0000 (02:14 -0700)]
mutable/servermap.py: update comment. Closes #1231.
Francois Deppierraz [Sat, 16 Oct 2010 09:09:22 +0000 (11:09 +0200)]
tahoe_cp.py: Don't call urllib.quote with an Unicode argument, fix #1224
tahoe_backup.py: Fix another (potential) occurrence of calling urllib.quote()
with an Unicode parameter
Brian Warner [Fri, 29 Oct 2010 08:20:36 +0000 (01:20 -0700)]
fix #1223, crash+inefficiency during repair due to read overrun
* repairer (really the uploader) reads beyond end of input file (Uploadable)
* new-downloader does not tolerate overreads
* uploader does lots of tiny reads (inefficient)
This fixes the last two. The uploader still does a single overread at the end
of the input file, but now that's ok so we can leave it in place. The
uploader now expects the Uploadable to behave like a normal disk
file (reading beyond EOF will return less data than was asked for), and now
the new-downloadable behaves that way.
david-sarah [Fri, 29 Oct 2010 02:51:50 +0000 (19:51 -0700)]
add misc/build_helpers/test-with-fake-pkg.py. refs #1190
david-sarah [Thu, 28 Oct 2010 01:48:05 +0000 (18:48 -0700)]
startstop_node.py: pyflakes import fix. refs #1190
Brian Warner [Wed, 27 Oct 2010 06:13:42 +0000 (23:13 -0700)]
'tahoe start': use import+call rather than find+spawn
This removes the need to use a locally-built (dependency) bin/twistd, and
removes a big chunk of behavior differences between unix and windows. It
also happens to resolve the "client node probably started" uncertainty.
Might help with #1190, #602, and #71.
david-sarah [Sun, 24 Oct 2010 23:42:59 +0000 (16:42 -0700)]
docs/known_issues.rst: Add section on traffic analysis. Fix URL for current version of file.
david-sarah [Sun, 24 Oct 2010 19:34:09 +0000 (12:34 -0700)]
test_mutable.py: add test for ResponseCache memory leak. refs #1045, #1129
david-sarah [Sat, 23 Oct 2010 03:58:10 +0000 (20:58 -0700)]
test_encodingutil.py: test_argv_to_unicode modified the wrong encoding variable. fixes #1214
david-sarah [Sun, 24 Oct 2010 17:15:00 +0000 (10:15 -0700)]
docs/running.html: fix missing end-quote, and change frontends/ doc references to .rst.
david-sarah [Sun, 24 Oct 2010 17:04:31 +0000 (10:04 -0700)]
docs/running.html: 'tahoe create-client' now creates a node with storage disabled. Also change configuration.txt references to configuration.rst.
Zooko O'Whielacronx [Fri, 15 Oct 2010 06:02:02 +0000 (23:02 -0700)]
doc: add explanation of the motivation for the surprising and awkward API to erasure coding
Zooko O'Whielacronx [Fri, 15 Oct 2010 05:44:40 +0000 (22:44 -0700)]
setup: catch and log ValueError from locale.getdefaultlocale() in show-tool-versions.py
I got a bug report from Mathias Baert showing that locale.getdefaultlocale() raises an exception on his Mac OS X system. Heh.
Zooko O'Whielacronx [Fri, 15 Oct 2010 05:44:13 +0000 (22:44 -0700)]
docs: update how-to-make-a-release doc with a few tweaks from the 1.8.0 process
Zooko O'Whielacronx [Fri, 15 Oct 2010 05:37:19 +0000 (22:37 -0700)]
docs: update NEWS ref: #1216
Brian Warner [Fri, 15 Oct 2010 06:04:18 +0000 (23:04 -0700)]
docs: fix tab-vs-spaces, make some CLI examples <tt>/"literal", wrap some to
80-cols, remove spurious whitespace. Add rst2html.py rule to Makefile.
Zooko O'Whielacronx [Fri, 15 Oct 2010 05:32:42 +0000 (22:32 -0700)]
docs: add Peter Secor, Shawn Willden, and Terrell Russell as signatories to docs/backdoors.rst
Zooko O'Whielacronx [Fri, 15 Oct 2010 05:29:13 +0000 (22:29 -0700)]
docs: convert all .txt docs to .rst thanks to Ravi Pinjala
fixes #1225
Zooko O'Whielacronx [Wed, 6 Oct 2010 05:11:47 +0000 (22:11 -0700)]
docs: add statement on our refusal to insert backdoors
Zooko O'Whielacronx [Tue, 5 Oct 2010 18:23:50 +0000 (11:23 -0700)]
setup: add --multi-version to the "setup.py develop" command-line
fixes #530. I earlier tried this twice (see #530 for history) and then twice rolled it back due to some problems that arose. However, I didn't write down what the problems were in enough detail on the ticket that I can tell today whether those problems are still issues, so here goes the third attempt. (I did write down on the ticket that it would not create site.py or .pth files in the target directory with --multi-version mode, but I didn't explain why *that* was a problem.)
Zooko O'Whielacronx [Mon, 6 Sep 2010 05:57:14 +0000 (22:57 -0700)]
setup: use execfile to access _auto_deps.py in its proper location of src/allmydata/ instead of copying it into place when setup.py is executed
Zooko O'Whielacronx [Fri, 3 Sep 2010 14:47:12 +0000 (07:47 -0700)]
trivial: M-x whitespace-cleanup
Zooko O'Whielacronx [Thu, 30 Sep 2010 15:37:08 +0000 (08:37 -0700)]
minor: remove unused interface declaration, change allmydata.org to tahoe-lafs.org in email address, fix wording in relnotes.txt
Kevan Carstensen [Mon, 27 Sep 2010 20:01:02 +0000 (13:01 -0700)]
immutable/repairer.py: don't use the default happiness setting when repairing
david-sarah [Fri, 24 Sep 2010 19:12:07 +0000 (12:12 -0700)]
NEWS: note dependency updates to pycryptopp and pycrypto.
Zooko O'Whielacronx [Fri, 24 Sep 2010 02:16:31 +0000 (19:16 -0700)]
TAG allmydata-tahoe-1.8.0
Zooko O'Whielacronx [Fri, 24 Sep 2010 02:15:52 +0000 (19:15 -0700)]
docs: timestamp the 1.8.0 release
Zooko O'Whielacronx [Fri, 24 Sep 2010 02:08:51 +0000 (19:08 -0700)]
docs: update quickstart.html to include "In Case Of Trouble" and to mention Python 2.7
Zooko O'Whielacronx [Fri, 24 Sep 2010 02:02:45 +0000 (19:02 -0700)]
docs: mention default values of K, H, and M
Zooko O'Whielacronx [Thu, 23 Sep 2010 07:23:25 +0000 (00:23 -0700)]
setup: add flappclient to the set of tools reported on by show-tool-versions
Zooko O'Whielacronx [Wed, 22 Sep 2010 07:08:08 +0000 (00:08 -0700)]
setup: raise minimum required version of pycryptopp, even on non-x86/amd64 platforms, to v0.5.14 of pycryptopp
Zooko O'Whielacronx [Tue, 21 Sep 2010 14:03:44 +0000 (07:03 -0700)]
setup: allow an older version of pycryptopp if we are not on x86 or amd64 architecture (which are the architectures on which older versions of Crypto++ had bugs in asm implementations of AES and SHA-256)
Zooko O'Whielacronx [Tue, 21 Sep 2010 07:32:58 +0000 (00:32 -0700)]
setup: force the bundled version of darcsver to take precedence over any other version of darcsver
Zooko O'Whielacronx [Tue, 21 Sep 2010 07:30:46 +0000 (00:30 -0700)]
setup: upgrade bundled copy of darcsver to v1.6.3
Zooko O'Whielacronx [Mon, 20 Sep 2010 22:54:15 +0000 (15:54 -0700)]
setup: format the output of show-tool-versions for better human-readability
david-sarah [Sun, 19 Sep 2010 05:03:35 +0000 (22:03 -0700)]
relnotes.txt and docs/quickstart.html for 1.8.0 release
david-sarah [Sun, 19 Sep 2010 04:57:43 +0000 (21:57 -0700)]
_auto_deps.py: require pycrypto >= 0.5.20 (fixes bugs in SHA-256 and AES)
david-sarah [Sun, 19 Sep 2010 04:44:12 +0000 (21:44 -0700)]
NEWS, docs/known_issues.txt for 1.8.0 release
Zooko O'Whielacronx [Sun, 19 Sep 2010 01:16:36 +0000 (18:16 -0700)]
docs: a few simple updates to links and naming, and also recommend torsocks instead of the old, unmaintained tsocks, for use with Tor
Zooko O'Whielacronx [Sun, 19 Sep 2010 01:15:54 +0000 (18:15 -0700)]
setup: make show-tool-versions report entire verbose exception stack traces when it fails to find a Python package
Zooko O'Whielacronx [Sat, 18 Sep 2010 22:04:10 +0000 (15:04 -0700)]
setup: add information about setuptools_darcs and darcsver to the show-tool-versions output
david-sarah [Tue, 14 Sep 2010 04:04:56 +0000 (21:04 -0700)]
docs/quickstart.html: note dependency on python development files, e.g. on Ubuntu
Zooko O'Whielacronx [Sun, 12 Sep 2010 06:22:25 +0000 (23:22 -0700)]
TAG allmydata-tahoe-1.8.0c4
Zooko O'Whielacronx [Sun, 12 Sep 2010 06:14:23 +0000 (23:14 -0700)]
docs: update relnotes.txt, NEWS, and quickstart.html for the 1.8.0c4 release
Zooko O'Whielacronx [Sat, 11 Sep 2010 04:14:42 +0000 (21:14 -0700)]
misc: add docstring to bench_spans.py telling how to run it
ref #1182
david-sarah [Sun, 12 Sep 2010 00:32:51 +0000 (17:32 -0700)]
test_download.py: simplify and possibly fix test_lost_servers; refs #1197
david-sarah [Sat, 11 Sep 2010 22:19:13 +0000 (15:19 -0700)]
test_runner.py: fix spurious failures of test_path on case-insensitive filesystems, and restore the check that the version is not 'unknown'.
david-sarah [Sat, 11 Sep 2010 00:31:47 +0000 (17:31 -0700)]
docs/frontends/webapi.txt: document that the meaning of the 'healthy' field may change in future to reflect servers-of-happiness; refs #614
david-sarah [Sat, 11 Sep 2010 00:25:48 +0000 (17:25 -0700)]
docs/frontends/webapi.txt: note that 'count-good-share-hosts' is computed incorrectly; refs #1115
david-sarah [Fri, 10 Sep 2010 19:54:22 +0000 (12:54 -0700)]
docs/frontends/FTP-and-SFTP.txt: docs/performance.txt, architecture.txt: updates taking into account new downloader (revised). refs #798
david-sarah [Fri, 10 Sep 2010 19:32:34 +0000 (12:32 -0700)]
docs/frontends/FTP-and-SFTP.txt: warn users about connecting to the FTP and SFTP servers remotely. Fixes #1192
Zooko O'Whielacronx [Fri, 10 Sep 2010 15:41:35 +0000 (08:41 -0700)]
test: make tests stop relying on pyutil version class accepting the string 'unknown' for its version, and make them forward-compatible with the future Python Rational Version Numbering standard
Zooko O'Whielacronx [Fri, 10 Sep 2010 06:14:11 +0000 (23:14 -0700)]
setup: copy in this fix from zetuptoolz and the accompanying new version number of zetuptoolz: http://tahoe-lafs.org/trac/zetuptoolz/ticket/1
Brian Warner [Fri, 10 Sep 2010 04:30:38 +0000 (21:30 -0700)]
immutable download: have the finder inform its share consumer "no more shares" in a subsequent tick, thus avoiding accidentally telling it "no more shares" now and then telling it "here's another share" in a subsequent tick
fixes #1191
Patch by Brian. This patch description was actually written by Zooko, but I forged Brian's name on the "author" field so that he would get credit for this patch in revision control history.
Zooko O'Whielacronx [Thu, 9 Sep 2010 04:16:54 +0000 (21:16 -0700)]
immutable downloader: add a test specifically of whether the finder sometimes announces "no more shares ever" and then announces a newly found share
(The current code fails this test, ref #1191.)
marc.doudiet [Fri, 13 Aug 2010 14:08:53 +0000 (07:08 -0700)]
docs/frontends/FTP-and-SFTP.txt : ftpd and sftpd doesn't listen on loopback interface only
Zooko O'Whielacronx [Sat, 4 Sep 2010 04:15:15 +0000 (21:15 -0700)]
tests: assign the storage servers to a fixed order which triggers a bug in new downloader every time this test is run (formerly this test would detect the bug in new-downloader only sporadically)
If you are investigating the bug in new-downloader, one way to investigate might be to change this ordering to a different fixed order (e.g. rotate by 4 instead of rotate by 5) and observe how the behavior of new-downloader differs in that case.
Zooko O'Whielacronx [Thu, 2 Sep 2010 21:21:40 +0000 (14:21 -0700)]
TAG allmydata-tahoe-1.8.0c3
Zooko O'Whielacronx [Thu, 2 Sep 2010 21:21:11 +0000 (14:21 -0700)]
docs: update relnotes.txt for v1.8.0c3
Zooko O'Whielacronx [Thu, 2 Sep 2010 16:15:41 +0000 (09:15 -0700)]
download status: fix bug from me committing the wrong one of François's #1172 patches, fixes #1172
francois [Thu, 2 Sep 2010 10:17:28 +0000 (03:17 -0700)]
DownloadStatus: show active immutable downloads in Active Operations, Fix #1172
Zooko O'Whielacronx [Thu, 2 Sep 2010 05:38:01 +0000 (22:38 -0700)]
tests: bump up the allowed number of reads
Kyle's OpenBSD buildslave used 41 reads when doing this test. The fact that I'm blindly bumping this number up to match the observed behavior probably means this isn't a good criterion to be testing for anyway. But perhaps someone else (Brian) could investigate why that run on Kyle's OpenBSD box took four more reads than we expected, and whether the fact that it took 41 reads to do this operation is indicative of an actual problem.
Brian Warner [Wed, 1 Sep 2010 01:37:02 +0000 (18:37 -0700)]
SegmentFetcher: use new diversity-seeking share-selection algorithm, and
deliver all shares at once instead of feeding them out one-at-a-time.
Also fix distribution of real-number-of-segments information: now all
CommonShares (not just the ones used for the first segment) get a
correctly-sized hashtree. Previously, the late ones might not, which would
make them crash and get dropped (causing the download to fail if the initial
set were insufficient, perhaps because one of their servers went away).
Update tests, add some TODO notes, improve variable names and comments.
Improve logging: add logparents, set more appropriate levels.
Brian Warner [Wed, 1 Sep 2010 01:35:58 +0000 (18:35 -0700)]
Share: drop received data after each block finishes. Quick fix for the #1170 spans.py complexity bug.
Zooko O'Whielacronx [Sun, 29 Aug 2010 22:27:58 +0000 (15:27 -0700)]
docs: a couple of small edits to CREDITS and how_to_make_a_tahoe-lafs_release.txt
Zooko O'Whielacronx [Thu, 19 Aug 2010 03:06:30 +0000 (20:06 -0700)]
add simulator to explore the trade-offs for hashed-based digital signatures
david-sarah [Sun, 29 Aug 2010 23:00:38 +0000 (16:00 -0700)]
_auto_deps.py: change pycrypto version requirement to avoid https://bugs.launchpad.net/pycrypto/+bug/620253
francois [Sun, 15 Aug 2010 14:19:33 +0000 (07:19 -0700)]
web: refactor rate computation, fixes #1166