Zooko O'Whielacronx [Thu, 2 Sep 2010 21:21:11 +0000 (14:21 -0700)]
docs: update relnotes.txt for v1.8.0c3
Zooko O'Whielacronx [Thu, 2 Sep 2010 16:15:41 +0000 (09:15 -0700)]
download status: fix bug from me committing the wrong one of François's #1172 patches, fixes #1172
francois [Thu, 2 Sep 2010 10:17:28 +0000 (03:17 -0700)]
DownloadStatus: show active immutable downloads in Active Operations, Fix #1172
Zooko O'Whielacronx [Thu, 2 Sep 2010 05:38:01 +0000 (22:38 -0700)]
tests: bump up the allowed number of reads
Kyle's OpenBSD buildslave used 41 reads when doing this test. The fact that I'm blindly bumping this number up to match the observed behavior probably means this isn't a good criterion to be testing for anyway. But perhaps someone else (Brian) could investigate why that run on Kyle's OpenBSD box took four more reads than we expected, and whether the fact that it took 41 reads to do this operation is indicative of an actual problem.
Brian Warner [Wed, 1 Sep 2010 01:37:02 +0000 (18:37 -0700)]
SegmentFetcher: use new diversity-seeking share-selection algorithm, and
deliver all shares at once instead of feeding them out one-at-a-time.
Also fix distribution of real-number-of-segments information: now all
CommonShares (not just the ones used for the first segment) get a
correctly-sized hashtree. Previously, the late ones might not, which would
make them crash and get dropped (causing the download to fail if the initial
set were insufficient, perhaps because one of their servers went away).
Update tests, add some TODO notes, improve variable names and comments.
Improve logging: add logparents, set more appropriate levels.
Brian Warner [Wed, 1 Sep 2010 01:35:58 +0000 (18:35 -0700)]
Share: drop received data after each block finishes. Quick fix for the #1170 spans.py complexity bug.
Zooko O'Whielacronx [Sun, 29 Aug 2010 22:27:58 +0000 (15:27 -0700)]
docs: a couple of small edits to CREDITS and how_to_make_a_tahoe-lafs_release.txt
Zooko O'Whielacronx [Thu, 19 Aug 2010 03:06:30 +0000 (20:06 -0700)]
add simulator to explore the trade-offs for hashed-based digital signatures
david-sarah [Sun, 29 Aug 2010 23:00:38 +0000 (16:00 -0700)]
_auto_deps.py: change pycrypto version requirement to avoid https://bugs.launchpad.net/pycrypto/+bug/620253
francois [Sun, 15 Aug 2010 14:19:33 +0000 (07:19 -0700)]
web: refactor rate computation, fixes #1166
Zooko O'Whielacronx [Thu, 19 Aug 2010 02:14:46 +0000 (19:14 -0700)]
docs: update NEWS a bit about New-Downloader
Zooko O'Whielacronx [Thu, 19 Aug 2010 02:14:20 +0000 (19:14 -0700)]
misc: add benchmarking tool for spans
Zooko O'Whielacronx [Sat, 14 Aug 2010 05:41:17 +0000 (22:41 -0700)]
docs: doc of the download status page
ref: http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1169#comment:1
Zooko O'Whielacronx [Wed, 11 Aug 2010 07:17:58 +0000 (00:17 -0700)]
docs: NEWS: edit English usage, remove ticket numbers for regressions vs. 1.7.1 that were fixed again before 1.8.0c2
Zooko O'Whielacronx [Wed, 11 Aug 2010 07:13:03 +0000 (00:13 -0700)]
docs: NEWS: more detail about new-downloader
david-sarah [Tue, 10 Aug 2010 07:38:47 +0000 (00:38 -0700)]
TAG allmydata-tahoe-1.8.0c2
david-sarah [Tue, 10 Aug 2010 07:38:32 +0000 (00:38 -0700)]
quickstart.html: update tarball link.
david-sarah [Tue, 10 Aug 2010 06:41:27 +0000 (23:41 -0700)]
webapi.txt: fix grammatical error.
david-sarah [Tue, 10 Aug 2010 06:32:43 +0000 (23:32 -0700)]
relnotes.txt: update revision of NEWS.
david-sarah [Tue, 10 Aug 2010 06:28:51 +0000 (23:28 -0700)]
NEWS, relnotes and known-issues for 1.8.0c2.
Brian Warner [Tue, 10 Aug 2010 06:06:03 +0000 (23:06 -0700)]
DownloadStatus: put real numbers in progress/status rows, not placeholders.
Improve tests.
Brian Warner [Mon, 9 Aug 2010 22:03:42 +0000 (15:03 -0700)]
web download-status: tolerate DYHBs that haven't retired yet. Fixes #1160.
Also add a better unit test for it.
Brian Warner [Mon, 9 Aug 2010 22:49:09 +0000 (15:49 -0700)]
immutable/filenode.py: put off DownloadStatus creation until first read() call
This avoids spamming the "recent uploads and downloads" /status page from
FileNode instances that were created for a directory read but which nobody is
ever going to read from. I also cleaned up the way DownloadStatus instances
are made to only ever do it in the CiphertextFileNode, not in the
higher-level plaintext FileNode. Also fixed DownloadStatus handling of read
size, thanks to David-Sarah for the catch.
Brian Warner [Fri, 6 Aug 2010 07:07:14 +0000 (00:07 -0700)]
Share: hush log entries in the main loop() after the fetch has been completed.
david-sarah [Sun, 8 Aug 2010 18:50:05 +0000 (11:50 -0700)]
test_runner.py: correct and simplify normalization of package directory for case-insensitive filesystems.
david-sarah [Sun, 8 Aug 2010 17:13:40 +0000 (10:13 -0700)]
test_runner.py: make test_path work for test-from-installdir.
david-sarah [Sun, 8 Aug 2010 17:12:35 +0000 (10:12 -0700)]
src/allmydata/__init__.py: make the package paths more accurate when we fail to get them from setuptools.
david-sarah [Sun, 8 Aug 2010 15:43:07 +0000 (08:43 -0700)]
test_runner.py: another try at calculating the rootdir correctly for test-from-egg and test-from-prefixdir.
david-sarah [Sun, 8 Aug 2010 04:28:17 +0000 (21:28 -0700)]
test_runner.py: calculate the location of bin/tahoe correctly for test-from-prefixdir (by copying code from misc/build_helpers/run_trial.py). Also fix the false-positive check for Unicode paths in test_the_right_code, which was causing skips that should have been failures.
david-sarah [Sat, 7 Aug 2010 00:45:46 +0000 (17:45 -0700)]
TAG allmydata-tahoe-1.8.0c1
david-sarah [Sat, 7 Aug 2010 00:42:54 +0000 (17:42 -0700)]
how_to_make_a_tahoe-lafs_release.txt: add step to check that release will report itself as the intended version.
david-sarah [Sat, 7 Aug 2010 00:36:46 +0000 (17:36 -0700)]
relnotes.txt: 1.8.0c1 release
david-sarah [Fri, 6 Aug 2010 23:51:11 +0000 (16:51 -0700)]
NEWS, quickstart.html and known_issues.txt for 1.8.0c1 release.
Brian Warner [Fri, 6 Aug 2010 08:04:50 +0000 (01:04 -0700)]
TAG allmydata-tahoe-1.8.0rc1
Brian Warner [Fri, 6 Aug 2010 08:02:28 +0000 (01:02 -0700)]
update NEWS and other docs in preparation for 1.8.0rc1
in particular, merge the various 1.8.0b1/b2 sections, and remove the
datestamp. NEWS gets updated just before a release, doesn't need to precisely
describe pre-release candidates, and the datestamp gets updated just before
the final release is tagged
Also, I removed the BOM from some files. My toolchain made it hard to retain,
and BOMs in UTF-8 don't make a whole lot of sense anyway. Sorry if that
messes anything up.
Brian Warner [Fri, 6 Aug 2010 07:07:05 +0000 (00:07 -0700)]
downloader.Segmentation: unregisterProducer when asked to stopProducing, this
seems to avoid the #1155 log message which reveals the URI (and filecap).
Also add an [ERROR] marker to the flog entry, since unregisterProducer also
makes interrupted downloads appear "200 OK"; this makes it more obvious that
the download did not complete.
david-sarah [Fri, 6 Aug 2010 05:24:15 +0000 (22:24 -0700)]
TAG allmydata-tahoe-1.8.0b2
david-sarah [Fri, 6 Aug 2010 04:08:23 +0000 (21:08 -0700)]
relnotes.txt and docs/known_issues.txt for 1.8.0beta2.
david-sarah [Fri, 6 Aug 2010 05:00:51 +0000 (22:00 -0700)]
test_util.py: use SHA-256 from pycryptopp instead of MD5 from hashlib (for uses in which any hash will do), since hashlib was only added to the stdlib in Python 2.5.
david-sarah [Fri, 6 Aug 2010 04:26:01 +0000 (21:26 -0700)]
test_runner.py: increase timeout to cater for Francois' ARM buildslave.
david-sarah [Fri, 6 Aug 2010 04:16:16 +0000 (21:16 -0700)]
test_util.py: remove use of 'a if p else b' syntax that requires Python 2.5.
david-sarah [Fri, 6 Aug 2010 03:51:12 +0000 (20:51 -0700)]
NEWS and docs/quickstart.html for 1.8.0beta2.
david-sarah [Fri, 6 Aug 2010 00:24:35 +0000 (17:24 -0700)]
docs/quickstart.html: remove link to tahoe-lafs-ticket798-1.8.0b.zip, due to appname regression. refs #1159
Brian Warner [Thu, 5 Aug 2010 18:41:08 +0000 (11:41 -0700)]
test_download.DownloadTest.test_simultaneous_goodguess: enable some disabled
checks that used to fail but work now.
Brian Warner [Thu, 5 Aug 2010 18:45:49 +0000 (11:45 -0700)]
DownloadNode: fix lost-progress in fetch_failed, tolerate cancel when no segment-fetch is active. Fixes #1154.
The lost-progress bug occurred when two simultanous read() calls fetched
different segments, and the first one failed (due to corruption, or the other
bugs in #1154): the second read() would never complete. While in this state,
cancelling the second read by having its consumer call stopProducing) would
trigger the cancel-intolerance bug. Finally, in downloader.node.Cancel,
prevent late cancels by adding an 'active' flag
Brian Warner [Thu, 5 Aug 2010 17:46:18 +0000 (10:46 -0700)]
util/spans.py: __nonzero__ cannot return a long either. for #1154
david-sarah [Thu, 5 Aug 2010 02:26:12 +0000 (19:26 -0700)]
test_storage.py: change skip note for test_large_share to say that Windows doesn't support sparse files. refs #569
Brian Warner [Wed, 4 Aug 2010 18:45:49 +0000 (11:45 -0700)]
One fix for bug #1154: webapi GETs with a 'Range' header broke new-downloader.
The Range header causes n.read() to be called with an offset= of type 'long',
which eventually got used in a Spans/DataSpans object's __len__ method.
Apparently python doesn't permit __len__() to return longs, only ints.
Rewrote Spans/DataSpans to use s.len() instead of len(s) aka s.__len__() .
Added a test in test_download. Note that test_web didn't catch this because
it uses mock FileNodes for speed: it's probably time to rewrite that.
There is still an unresolved error-recovery problem in #1154, so I'm not
closing the ticket quite yet.
Brian Warner [Wed, 4 Aug 2010 17:55:55 +0000 (10:55 -0700)]
test_download: minor cleanup
Brian Warner [Wed, 4 Aug 2010 07:28:14 +0000 (00:28 -0700)]
fetcher.py: improve comments
Brian Warner [Wed, 4 Aug 2010 07:28:08 +0000 (00:28 -0700)]
lazily create DownloadNode upon first read()/get_segment()
Brian Warner [Wed, 4 Aug 2010 07:28:00 +0000 (00:28 -0700)]
test_hung_server: update comments, remove dead "stage_4_d" code
Brian Warner [Wed, 4 Aug 2010 07:27:52 +0000 (00:27 -0700)]
copy the rest of David-Sarah's changes to make my tree match 1.8.0beta
Brian Warner [Wed, 4 Aug 2010 07:27:41 +0000 (00:27 -0700)]
ShareFinder: add 10s OVERDUE timer, send new requests to replace overdue ones
The fixed 10-second timer will eventually be replaced with a per-server
value, calculated based on observed response times.
test_hung_server.py: enhance to exercise DYHB=OVERDUE state. Split existing
mutable+immutable tests into two pieces for clarity. Reenabled several tests.
Deleted the now-obsolete "test_failover_during_stage_4".
Brian Warner [Wed, 4 Aug 2010 07:27:10 +0000 (00:27 -0700)]
Rewrite immutable downloader (#798). This patch adds and updates unit tests.
Brian Warner [Wed, 4 Aug 2010 07:27:02 +0000 (00:27 -0700)]
Rewrite immutable downloader (#798). This patch includes higher-level
integration into the NodeMaker, and updates the web-status display to handle
the new download events.
Brian Warner [Wed, 4 Aug 2010 07:26:39 +0000 (00:26 -0700)]
Rewrite immutable downloader (#798). This patch rearranges the rest of src/allmydata/immutable/ .
Brian Warner [Wed, 4 Aug 2010 07:26:29 +0000 (00:26 -0700)]
Rewrite immutable downloader (#798). This patch adds the new downloader itself.
Brian Warner [Wed, 4 Aug 2010 07:26:12 +0000 (00:26 -0700)]
util/observer.py: add EventStreamObserver
Brian Warner [Wed, 4 Aug 2010 07:26:00 +0000 (00:26 -0700)]
Add a byte-spans utility class, like perl's Set::IntSpan for .newsrc files.
Also a data-spans class, which records a byte (instead of a bit) for each
index.
Brian Warner [Wed, 4 Aug 2010 07:17:13 +0000 (00:17 -0700)]
check-umids: oops, forgot to add the tool
Brian Warner [Wed, 4 Aug 2010 07:11:31 +0000 (00:11 -0700)]
coverage tools: ignore errors, display lines-uncovered in elisp mode. Fix Makefile paths.
Brian Warner [Wed, 4 Aug 2010 07:10:42 +0000 (00:10 -0700)]
check-umids: new tool to check uniqueness of umids
Brian Warner [Wed, 4 Aug 2010 07:09:42 +0000 (00:09 -0700)]
misc/simulators/sizes.py: update, we now use SHA256 (not SHA1), so large-file overhead grows to 0.5%
Brian Warner [Wed, 4 Aug 2010 07:08:15 +0000 (00:08 -0700)]
storage-overhead: try to fix, probably still broken
david-sarah [Tue, 3 Aug 2010 23:32:54 +0000 (16:32 -0700)]
docs/quickstart.html: link to 1.8.0beta zip, and note 'bin\tahoe' on Windows.
Zooko O'Whielacronx [Tue, 3 Aug 2010 15:49:13 +0000 (08:49 -0700)]
docs: relnotes.txt for 1.8.0β
david-sarah [Tue, 3 Aug 2010 10:20:58 +0000 (03:20 -0700)]
test_storage.py: avoid spurious test failure by accepting either 'Next crawl in 59 minutes' or 'Next crawl in 60 minutes'. fixes #1140
david-sarah [Tue, 3 Aug 2010 10:11:28 +0000 (03:11 -0700)]
misc/build_helpers/show-tool-versions.py: get sys.std{out,err}.encoding and 'as' version correctly, and improve formatting.
david-sarah [Tue, 3 Aug 2010 09:48:12 +0000 (02:48 -0700)]
misc/build_helpers/show-tool-versions.py: avoid error message when 'as -version' does not create a.out.
david-sarah [Tue, 3 Aug 2010 08:54:16 +0000 (01:54 -0700)]
CLI: further improve consistency of basedir options and add tests. addresses #118
david-sarah [Tue, 3 Aug 2010 08:53:59 +0000 (01:53 -0700)]
CLI: make the synopsis for 'tahoe unlink' say unlink instead of rm.
david-sarah [Tue, 3 Aug 2010 08:48:01 +0000 (01:48 -0700)]
CLI: make all of the option descriptions imperative sentences.
david-sarah [Tue, 3 Aug 2010 08:47:20 +0000 (01:47 -0700)]
test_cli.py: make 'tahoe mkdir' tests slightly less dumb (check for 'URI:' in the output).
david-sarah [Tue, 3 Aug 2010 08:35:38 +0000 (01:35 -0700)]
test_cli.py: use u-escapes instead of UTF-8.
david-sarah [Tue, 3 Aug 2010 05:08:27 +0000 (22:08 -0700)]
NEWS: remove XXX comment and separate description of #890.
Zooko O'Whielacronx [Tue, 3 Aug 2010 04:46:18 +0000 (21:46 -0700)]
docs: more updates to NEWS for 1.8.0β
Zooko O'Whielacronx [Mon, 2 Aug 2010 07:28:40 +0000 (00:28 -0700)]
docs: incomplete beginnings of a NEWS update for v1.8β
david-sarah [Tue, 3 Aug 2010 00:49:38 +0000 (17:49 -0700)]
docs/quickstart.html: extra step to open a new Command Prompt or log out/in on Windows.
david-sarah [Tue, 3 Aug 2010 00:38:15 +0000 (17:38 -0700)]
update bundled zetuptools with doc changes, change to script setup for Windows XP, and to have the 'develop' command run script setup.
david-sarah [Mon, 2 Aug 2010 22:45:05 +0000 (15:45 -0700)]
bundled setuptools/command/scriptsetup.py: use SendMessageTimeoutW, to test whether that broadcasts environment changes any better.
david-sarah [Mon, 2 Aug 2010 07:21:29 +0000 (00:21 -0700)]
bundled zetuptoolz: add missing setuptools/command/scriptsetup.py
david-sarah [Mon, 2 Aug 2010 06:25:58 +0000 (23:25 -0700)]
test_runner.py: add test_run_with_python_options, which checks that the Windows script changes haven't broken 'python <options> bin/tahoe'.
david-sarah [Mon, 2 Aug 2010 06:09:02 +0000 (23:09 -0700)]
test_runner.py: fix missing import of get_filesystem_encoding
david-sarah [Mon, 2 Aug 2010 06:06:02 +0000 (23:06 -0700)]
Bundle setuptools-0.6c16dev (with Windows script changes, and the change to only warn if site.py wasn't generated by setuptools) instead of 0.6c15dev. addresses #565, #1073, #1074
david-sarah [Mon, 2 Aug 2010 05:03:13 +0000 (22:03 -0700)]
.darcs-boringfile: changes needed to take account of egg directories being bundled. Also, make _trial_temp a prefix rather than exact match.
david-sarah [Mon, 2 Aug 2010 05:01:28 +0000 (22:01 -0700)]
.darcs-boringfile: changes needed to take account of pyscript wrappers on Windows.
david-sarah [Mon, 2 Aug 2010 04:55:35 +0000 (21:55 -0700)]
misc/build_helpers/run_trial.py: check that the root from which the module we are testing was loaded is the current directory. This version of the patch folds in later fixes to the logic for caculating the directories to compare, and improvements to error messages. addresses #1137
david-sarah [Wed, 28 Jul 2010 06:27:31 +0000 (23:27 -0700)]
Skip option arguments to the python interpreter when reconstructing Unicode argv on Windows.
david-sarah [Tue, 27 Jul 2010 18:19:21 +0000 (11:19 -0700)]
windows/fixups.py: improve comments and reference some relevant Python bugs.
david-sarah [Mon, 26 Jul 2010 22:19:04 +0000 (15:19 -0700)]
windows/fixups.py: make errors reported to original_stderr have enough information to debug even if we can't see the traceback.
david-sarah [Mon, 26 Jul 2010 21:47:36 +0000 (14:47 -0700)]
windows/fixups.py: fix paste-o in name of Unicode stderr wrapper.
david-sarah [Mon, 26 Jul 2010 04:50:19 +0000 (21:50 -0700)]
windows/fixups.py: Don't rely on buggy MSVCRT library for Unicode output, use the Win32 API instead. This should make it work on XP. Also, change how we handle the case where sys.stdout and sys.stderr are redirected, since the .encoding attribute isn't necessarily writeable.
david-sarah [Sun, 25 Jul 2010 18:20:08 +0000 (11:20 -0700)]
test_runner.py: change to code for locating the bin/tahoe script that was missed when rebasing the patch for #1074.
david-sarah [Sun, 25 Jul 2010 09:28:49 +0000 (02:28 -0700)]
Add missing windows/fixups.py (for setting up Unicode args and output on Windows).
david-sarah [Sun, 25 Jul 2010 08:32:16 +0000 (01:32 -0700)]
Changes to Tahoe needed to work with new zetuptoolz (that does not use .exe wrappers on Windows), and to support Unicode arguments and stdout/stderr -- v5
david-sarah [Mon, 2 Aug 2010 06:49:29 +0000 (23:49 -0700)]
scripts/common.py: fix an error introduced when rebasing to the ticket798 branch, which caused base directories to be duplicated in self.basedirs.
david-sarah [Mon, 2 Aug 2010 04:30:04 +0000 (21:30 -0700)]
Basedir/node directory option improvements for ticket798 branch. addresses #188, #706, #715, #772, #890
david-sarah [Mon, 2 Aug 2010 00:02:12 +0000 (17:02 -0700)]
scripts/create_node.py: allow nickname to be Unicode. Also ensure webport is validly encoded in config file.
david-sarah [Mon, 2 Aug 2010 01:35:26 +0000 (18:35 -0700)]
test_runner.py: Fix error in message arguments to 'fail' calls.