Brian Warner [Wed, 4 Aug 2010 07:28:08 +0000 (00:28 -0700)]
lazily create DownloadNode upon first read()/get_segment()
Brian Warner [Wed, 4 Aug 2010 07:28:00 +0000 (00:28 -0700)]
test_hung_server: update comments, remove dead "stage_4_d" code
Brian Warner [Wed, 4 Aug 2010 07:27:52 +0000 (00:27 -0700)]
copy the rest of David-Sarah's changes to make my tree match 1.8.0beta
Brian Warner [Wed, 4 Aug 2010 07:27:41 +0000 (00:27 -0700)]
ShareFinder: add 10s OVERDUE timer, send new requests to replace overdue ones
The fixed 10-second timer will eventually be replaced with a per-server
value, calculated based on observed response times.
test_hung_server.py: enhance to exercise DYHB=OVERDUE state. Split existing
mutable+immutable tests into two pieces for clarity. Reenabled several tests.
Deleted the now-obsolete "test_failover_during_stage_4".
Brian Warner [Wed, 4 Aug 2010 07:27:10 +0000 (00:27 -0700)]
Rewrite immutable downloader (#798). This patch adds and updates unit tests.
Brian Warner [Wed, 4 Aug 2010 07:27:02 +0000 (00:27 -0700)]
Rewrite immutable downloader (#798). This patch includes higher-level
integration into the NodeMaker, and updates the web-status display to handle
the new download events.
Brian Warner [Wed, 4 Aug 2010 07:26:39 +0000 (00:26 -0700)]
Rewrite immutable downloader (#798). This patch rearranges the rest of src/allmydata/immutable/ .
Brian Warner [Wed, 4 Aug 2010 07:26:29 +0000 (00:26 -0700)]
Rewrite immutable downloader (#798). This patch adds the new downloader itself.
Brian Warner [Wed, 4 Aug 2010 07:26:12 +0000 (00:26 -0700)]
util/observer.py: add EventStreamObserver
Brian Warner [Wed, 4 Aug 2010 07:26:00 +0000 (00:26 -0700)]
Add a byte-spans utility class, like perl's Set::IntSpan for .newsrc files.
Also a data-spans class, which records a byte (instead of a bit) for each
index.
Brian Warner [Wed, 4 Aug 2010 07:17:13 +0000 (00:17 -0700)]
check-umids: oops, forgot to add the tool
Brian Warner [Wed, 4 Aug 2010 07:11:31 +0000 (00:11 -0700)]
coverage tools: ignore errors, display lines-uncovered in elisp mode. Fix Makefile paths.
Brian Warner [Wed, 4 Aug 2010 07:10:42 +0000 (00:10 -0700)]
check-umids: new tool to check uniqueness of umids
Brian Warner [Wed, 4 Aug 2010 07:09:42 +0000 (00:09 -0700)]
misc/simulators/sizes.py: update, we now use SHA256 (not SHA1), so large-file overhead grows to 0.5%
Brian Warner [Wed, 4 Aug 2010 07:08:15 +0000 (00:08 -0700)]
storage-overhead: try to fix, probably still broken
david-sarah [Tue, 3 Aug 2010 23:32:54 +0000 (16:32 -0700)]
docs/quickstart.html: link to 1.8.0beta zip, and note 'bin\tahoe' on Windows.
Zooko O'Whielacronx [Tue, 3 Aug 2010 15:49:13 +0000 (08:49 -0700)]
docs: relnotes.txt for 1.8.0β
david-sarah [Tue, 3 Aug 2010 10:20:58 +0000 (03:20 -0700)]
test_storage.py: avoid spurious test failure by accepting either 'Next crawl in 59 minutes' or 'Next crawl in 60 minutes'. fixes #1140
david-sarah [Tue, 3 Aug 2010 10:11:28 +0000 (03:11 -0700)]
misc/build_helpers/show-tool-versions.py: get sys.std{out,err}.encoding and 'as' version correctly, and improve formatting.
david-sarah [Tue, 3 Aug 2010 09:48:12 +0000 (02:48 -0700)]
misc/build_helpers/show-tool-versions.py: avoid error message when 'as -version' does not create a.out.
david-sarah [Tue, 3 Aug 2010 08:54:16 +0000 (01:54 -0700)]
CLI: further improve consistency of basedir options and add tests. addresses #118
david-sarah [Tue, 3 Aug 2010 08:53:59 +0000 (01:53 -0700)]
CLI: make the synopsis for 'tahoe unlink' say unlink instead of rm.
david-sarah [Tue, 3 Aug 2010 08:48:01 +0000 (01:48 -0700)]
CLI: make all of the option descriptions imperative sentences.
david-sarah [Tue, 3 Aug 2010 08:47:20 +0000 (01:47 -0700)]
test_cli.py: make 'tahoe mkdir' tests slightly less dumb (check for 'URI:' in the output).
david-sarah [Tue, 3 Aug 2010 08:35:38 +0000 (01:35 -0700)]
test_cli.py: use u-escapes instead of UTF-8.
david-sarah [Tue, 3 Aug 2010 05:08:27 +0000 (22:08 -0700)]
NEWS: remove XXX comment and separate description of #890.
Zooko O'Whielacronx [Tue, 3 Aug 2010 04:46:18 +0000 (21:46 -0700)]
docs: more updates to NEWS for 1.8.0β
Zooko O'Whielacronx [Mon, 2 Aug 2010 07:28:40 +0000 (00:28 -0700)]
docs: incomplete beginnings of a NEWS update for v1.8β
david-sarah [Tue, 3 Aug 2010 00:49:38 +0000 (17:49 -0700)]
docs/quickstart.html: extra step to open a new Command Prompt or log out/in on Windows.
david-sarah [Tue, 3 Aug 2010 00:38:15 +0000 (17:38 -0700)]
update bundled zetuptools with doc changes, change to script setup for Windows XP, and to have the 'develop' command run script setup.
david-sarah [Mon, 2 Aug 2010 22:45:05 +0000 (15:45 -0700)]
bundled setuptools/command/scriptsetup.py: use SendMessageTimeoutW, to test whether that broadcasts environment changes any better.
david-sarah [Mon, 2 Aug 2010 07:21:29 +0000 (00:21 -0700)]
bundled zetuptoolz: add missing setuptools/command/scriptsetup.py
david-sarah [Mon, 2 Aug 2010 06:25:58 +0000 (23:25 -0700)]
test_runner.py: add test_run_with_python_options, which checks that the Windows script changes haven't broken 'python <options> bin/tahoe'.
david-sarah [Mon, 2 Aug 2010 06:09:02 +0000 (23:09 -0700)]
test_runner.py: fix missing import of get_filesystem_encoding
david-sarah [Mon, 2 Aug 2010 06:06:02 +0000 (23:06 -0700)]
Bundle setuptools-0.6c16dev (with Windows script changes, and the change to only warn if site.py wasn't generated by setuptools) instead of 0.6c15dev. addresses #565, #1073, #1074
david-sarah [Mon, 2 Aug 2010 05:03:13 +0000 (22:03 -0700)]
.darcs-boringfile: changes needed to take account of egg directories being bundled. Also, make _trial_temp a prefix rather than exact match.
david-sarah [Mon, 2 Aug 2010 05:01:28 +0000 (22:01 -0700)]
.darcs-boringfile: changes needed to take account of pyscript wrappers on Windows.
david-sarah [Mon, 2 Aug 2010 04:55:35 +0000 (21:55 -0700)]
misc/build_helpers/run_trial.py: check that the root from which the module we are testing was loaded is the current directory. This version of the patch folds in later fixes to the logic for caculating the directories to compare, and improvements to error messages. addresses #1137
david-sarah [Wed, 28 Jul 2010 06:27:31 +0000 (23:27 -0700)]
Skip option arguments to the python interpreter when reconstructing Unicode argv on Windows.
david-sarah [Tue, 27 Jul 2010 18:19:21 +0000 (11:19 -0700)]
windows/fixups.py: improve comments and reference some relevant Python bugs.
david-sarah [Mon, 26 Jul 2010 22:19:04 +0000 (15:19 -0700)]
windows/fixups.py: make errors reported to original_stderr have enough information to debug even if we can't see the traceback.
david-sarah [Mon, 26 Jul 2010 21:47:36 +0000 (14:47 -0700)]
windows/fixups.py: fix paste-o in name of Unicode stderr wrapper.
david-sarah [Mon, 26 Jul 2010 04:50:19 +0000 (21:50 -0700)]
windows/fixups.py: Don't rely on buggy MSVCRT library for Unicode output, use the Win32 API instead. This should make it work on XP. Also, change how we handle the case where sys.stdout and sys.stderr are redirected, since the .encoding attribute isn't necessarily writeable.
david-sarah [Sun, 25 Jul 2010 18:20:08 +0000 (11:20 -0700)]
test_runner.py: change to code for locating the bin/tahoe script that was missed when rebasing the patch for #1074.
david-sarah [Sun, 25 Jul 2010 09:28:49 +0000 (02:28 -0700)]
Add missing windows/fixups.py (for setting up Unicode args and output on Windows).
david-sarah [Sun, 25 Jul 2010 08:32:16 +0000 (01:32 -0700)]
Changes to Tahoe needed to work with new zetuptoolz (that does not use .exe wrappers on Windows), and to support Unicode arguments and stdout/stderr -- v5
david-sarah [Mon, 2 Aug 2010 06:49:29 +0000 (23:49 -0700)]
scripts/common.py: fix an error introduced when rebasing to the ticket798 branch, which caused base directories to be duplicated in self.basedirs.
david-sarah [Mon, 2 Aug 2010 04:30:04 +0000 (21:30 -0700)]
Basedir/node directory option improvements for ticket798 branch. addresses #188, #706, #715, #772, #890
david-sarah [Mon, 2 Aug 2010 00:02:12 +0000 (17:02 -0700)]
scripts/create_node.py: allow nickname to be Unicode. Also ensure webport is validly encoded in config file.
david-sarah [Mon, 2 Aug 2010 01:35:26 +0000 (18:35 -0700)]
test_runner.py: Fix error in message arguments to 'fail' calls.
david-sarah [Mon, 2 Aug 2010 01:05:52 +0000 (18:05 -0700)]
Additional Unicode basedir changes for ticket798 branch.
david-sarah [Sun, 1 Aug 2010 23:53:10 +0000 (16:53 -0700)]
Unicode basedir changes for ticket798 branch.
david-sarah [Sun, 25 Jul 2010 22:26:03 +0000 (15:26 -0700)]
fileutil: change WindowsError to OSError in abspath_expanduser_unicode, because WindowsError might not exist.
david-sarah [Sat, 24 Jul 2010 03:21:23 +0000 (20:21 -0700)]
test_system: correct a failure in _test_runner caused by Unicode basedir patch on non-Unicode platforms.
david-sarah [Sun, 25 Jul 2010 01:03:18 +0000 (18:03 -0700)]
Fix test failures due to Unicode basedir patches.
david-sarah [Fri, 23 Jul 2010 07:53:14 +0000 (00:53 -0700)]
util.encodingutil: change quote_output to do less unnecessary escaping, and to use double-quotes more consistently when needed. This version avoids u-escaping for characters that are representable in the output encoding, when double quotes are used, and includes tests. fixes #1135
david-sarah [Thu, 22 Jul 2010 00:14:18 +0000 (17:14 -0700)]
Replace uses of os.path.abspath with abspath_expanduser_unicode where necessary. This makes basedir paths consistently represented as Unicode.
david-sarah [Wed, 21 Jul 2010 23:15:07 +0000 (16:15 -0700)]
util.fileutil, test.test_util: add abspath_expanduser_unicode function, to work around <http://bugs.python.org/issue3426>. util.encodingutil: add a convenience function argv_to_abspath.
Zooko O'Whielacronx [Mon, 2 Aug 2010 07:17:48 +0000 (00:17 -0700)]
setup: increase requirement on foolscap from >= 0.4.1 to >= 0.5.1 to avoid the foolscap performance bug with transferring large mutable files
Zooko O'Whielacronx [Mon, 2 Aug 2010 07:02:12 +0000 (00:02 -0700)]
upload: tidy up logging messages
reformat code to be less than 100 chars wide, refactor formatting of logging messages, add log levels to some logging messages, M-x whitespace-cleanup
jacob.lyles [Thu, 29 Jul 2010 21:06:38 +0000 (14:06 -0700)]
abbreviate time edge case python2.5 unit test
david-sarah [Thu, 29 Jul 2010 15:29:27 +0000 (08:29 -0700)]
test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 -- fix .todo reference.
david-sarah [Thu, 29 Jul 2010 14:22:50 +0000 (07:22 -0700)]
test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 for consistency.
Zooko O'Whielacronx [Mon, 2 Aug 2010 06:33:39 +0000 (23:33 -0700)]
tests: remove debug print
Zooko O'Whielacronx [Mon, 2 Aug 2010 06:33:14 +0000 (23:33 -0700)]
docs: update the list of forums to announce Tahoe-LAFS too, add empty checkboxes
Zooko O'Whielacronx [Mon, 2 Aug 2010 06:20:04 +0000 (23:20 -0700)]
immutable: tidy-up some code by using a set instead of list to hold homeless_shares
Zooko O'Whielacronx [Sun, 1 Aug 2010 16:42:07 +0000 (09:42 -0700)]
setup: fix a couple instances of hard-coded 'allmydata-tahoe' in the scripts, tighten the tests (as suggested by David-Sarah)
Zooko O'Whielacronx [Sun, 1 Aug 2010 16:05:17 +0000 (09:05 -0700)]
setup: replace hardcoded 'allmydata-tahoe' with allmydata.__appname__
Zooko O'Whielacronx [Sun, 1 Aug 2010 03:43:38 +0000 (20:43 -0700)]
NEWS: describe #1055
Zooko O'Whielacronx [Mon, 19 Jul 2010 08:20:00 +0000 (01:20 -0700)]
immutable: use PrefixingLogMixin to organize logging in Tahoe2PeerSelector and add more detailed messages about peer
Zooko O'Whielacronx [Mon, 19 Jul 2010 04:49:48 +0000 (21:49 -0700)]
benchmarking: update bench_dirnode to be correct and use the shiniest new pyutil.benchutil features concerning what units you measure in
Zooko O'Whielacronx [Mon, 19 Jul 2010 04:47:44 +0000 (21:47 -0700)]
trivial: rename and add in-line doc to clarify "used_peers" => "upload_servers"
Zooko O'Whielacronx [Fri, 30 Jul 2010 23:05:00 +0000 (16:05 -0700)]
docs: add Jacob Lyles to CREDITS
jacob.lyles [Fri, 30 Jul 2010 22:05:50 +0000 (15:05 -0700)]
web: don't use %d formatting on a potentially large negative float -- there is a bug in Python 2.5 in that case
fixes #1055
Zooko O'Whielacronx [Thu, 29 Jul 2010 05:29:23 +0000 (22:29 -0700)]
docs: fix licensing typo that was earlier fixed in [
20090921164651-92b7f-
7f97b58101d93dc588445c52a9aaa56a2c7ae336]
I wish we didn't copies of this licensing text in several different files so that changes can be accidentally omitted from some of them.
david-sarah [Mon, 26 Jul 2010 22:57:29 +0000 (15:57 -0700)]
misc/build_helpers/run-with-pythonpath.py: fix stale comment, and remove 'trial' example that is not the right way to run trial.
david-sarah [Fri, 23 Jul 2010 06:16:16 +0000 (23:16 -0700)]
docs/specifications/dirnodes.txt: 'mesh'->'grid'.
david-sarah [Fri, 23 Jul 2010 05:47:03 +0000 (22:47 -0700)]
docs/specifications/dirnodes.txt: bring layer terminology up-to-date with architecture.txt, and a few other updates (e.g. note that the MAC is no longer verified, and that URIs can be unknown). Also 'Tahoe'->'Tahoe-LAFS'.
Zooko O'Whielacronx [Wed, 21 Jul 2010 01:05:43 +0000 (18:05 -0700)]
docs: use current cap to Zooko's wiki page in example text
fixes #1134
david-sarah [Tue, 20 Jul 2010 01:19:39 +0000 (18:19 -0700)]
__init__.py: silence DeprecationWarning about BaseException.message globally. fixes #1129
david-sarah [Tue, 20 Jul 2010 01:13:45 +0000 (18:13 -0700)]
test_runner: test that 'tahoe --version' outputs no noise (e.g. DeprecationWarnings).
Zooko O'Whielacronx [Mon, 19 Jul 2010 13:13:52 +0000 (06:13 -0700)]
TAG allmydata-tahoe-1.7.1
Zooko O'Whielacronx [Mon, 19 Jul 2010 08:30:59 +0000 (01:30 -0700)]
relnotes.txt: updated for v1.7.1 release!
Zooko O'Whielacronx [Mon, 19 Jul 2010 08:16:12 +0000 (01:16 -0700)]
immutable: add test case of #1128, fix test case of #1118
Zooko O'Whielacronx [Mon, 19 Jul 2010 08:12:48 +0000 (01:12 -0700)]
NEWS: add #1118 and reflow
Zooko O'Whielacronx [Mon, 19 Jul 2010 07:54:26 +0000 (00:54 -0700)]
immutable: fix bug in which preexisting_shares and merged were shallowly referencing the same sets
This bug had the effect of making uploads sometimes (rarely) appear to succeed when they had actually not distributed the shares well enough to achieve the desired servers-of-happiness level.
david-sarah [Mon, 19 Jul 2010 04:46:55 +0000 (21:46 -0700)]
upload.py: fix #1118 by aborting newly-homeless buckets when reassignment runs. This makes a previously failing assert correct. This version refactors 'abort' into two methods, rather than using a default argument.
david-sarah [Sun, 18 Jul 2010 23:59:40 +0000 (16:59 -0700)]
docs/known_issues.txt: update release version and date.
david-sarah [Sun, 18 Jul 2010 23:54:37 +0000 (16:54 -0700)]
relnotes.txt, docs/quickstart.html: prepare for 1.7.1 release. Don't claim to work on Cygwin (this might work but is untested).
Zooko O'Whielacronx [Mon, 19 Jul 2010 04:50:47 +0000 (21:50 -0700)]
immutable: extend the tests to check that the shares that got uploaded really do make a sufficiently Happy distribution
This patch also renames some instances of "find_shares()" to "find_all_shares()" and other instances to "find_uri_shares()" as appropriate -- the conflation between those names confused me at first when writing these tests.
Zooko O'Whielacronx [Sun, 18 Jul 2010 22:15:37 +0000 (15:15 -0700)]
immutable: test for #1118
Zooko O'Whielacronx [Sun, 18 Jul 2010 22:29:07 +0000 (15:29 -0700)]
immutable: test for #1124
david-sarah [Sun, 18 Jul 2010 23:04:20 +0000 (16:04 -0700)]
docs/logging.txt: document that _trial_temp/test.log does not receive messages below level=OPERATIONAL, due to <http://foolscap.lothar.com/trac/ticket/154>.
Zooko O'Whielacronx [Sun, 18 Jul 2010 21:51:33 +0000 (14:51 -0700)]
trivial: fix unused import (sorry about that, pyflakes)
Zooko O'Whielacronx [Sun, 18 Jul 2010 20:32:25 +0000 (13:32 -0700)]
tests, NEWS, CREDITS re: #1117
Give Brian and Kevan promotions, move release date in NEWS to the 18th, commit Brian's test for #1117.
fixes #1117
Kevan Carstensen [Fri, 16 Jul 2010 00:10:46 +0000 (17:10 -0700)]
test/test_upload.py: test to see that aborted buckets are ignored by the storage server
Kevan Carstensen [Thu, 15 Jul 2010 23:21:48 +0000 (16:21 -0700)]
test/test_storage.py: test for the new remote_abort semantics.
Kevan Carstensen [Thu, 15 Jul 2010 23:21:05 +0000 (16:21 -0700)]
storage/immutable.py: make remote_abort btell the storage server about aborted buckets.
Kevan Carstensen [Thu, 15 Jul 2010 23:18:20 +0000 (16:18 -0700)]
test/test_upload.py: changes to test plumbing for #1117 tests
- Add a callRemoteOnly method to FakeBucketWriter.
- Change the abort method in FakeBucketWriter to not return a
RuntimeError.
Kevan Carstensen [Thu, 15 Jul 2010 23:17:14 +0000 (16:17 -0700)]
immutable/upload.py: abort buckets if peer selection fails