tahoe_cp.py: Don't call urllib.quote with an Unicode argument, fix #1224
tahoe_backup.py: Fix another (potential) occurrence of calling urllib.quote()
with an Unicode parameter
Brian Warner [Fri, 29 Oct 2010 08:20:36 +0000 (01:20 -0700)]
fix #1223, crash+inefficiency during repair due to read overrun
* repairer (really the uploader) reads beyond end of input file (Uploadable)
* new-downloader does not tolerate overreads
* uploader does lots of tiny reads (inefficient)
This fixes the last two. The uploader still does a single overread at the end
of the input file, but now that's ok so we can leave it in place. The
uploader now expects the Uploadable to behave like a normal disk
file (reading beyond EOF will return less data than was asked for), and now
the new-downloadable behaves that way.
Brian Warner [Wed, 27 Oct 2010 06:13:42 +0000 (23:13 -0700)]
'tahoe start': use import+call rather than find+spawn
This removes the need to use a locally-built (dependency) bin/twistd, and
removes a big chunk of behavior differences between unix and windows. It
also happens to resolve the "client node probably started" uncertainty.
Might help with #1190, #602, and #71.
setup: catch and log ValueError from locale.getdefaultlocale() in show-tool-versions.py
I got a bug report from Mathias Baert showing that locale.getdefaultlocale() raises an exception on his Mac OS X system. Heh.
setup: add --multi-version to the "setup.py develop" command-line
fixes #530. I earlier tried this twice (see #530 for history) and then twice rolled it back due to some problems that arose. However, I didn't write down what the problems were in enough detail on the ticket that I can tell today whether those problems are still issues, so here goes the third attempt. (I did write down on the ticket that it would not create site.py or .pth files in the target directory with --multi-version mode, but I didn't explain why *that* was a problem.)
setup: allow an older version of pycryptopp if we are not on x86 or amd64 architecture (which are the architectures on which older versions of Crypto++ had bugs in asm implementations of AES and SHA-256)
test: make tests stop relying on pyutil version class accepting the string 'unknown' for its version, and make them forward-compatible with the future Python Rational Version Numbering standard
Brian Warner [Fri, 10 Sep 2010 04:30:38 +0000 (21:30 -0700)]
immutable download: have the finder inform its share consumer "no more shares" in a subsequent tick, thus avoiding accidentally telling it "no more shares" now and then telling it "here's another share" in a subsequent tick
fixes #1191
Patch by Brian. This patch description was actually written by Zooko, but I forged Brian's name on the "author" field so that he would get credit for this patch in revision control history.
immutable downloader: add a test specifically of whether the finder sometimes announces "no more shares ever" and then announces a newly found share
(The current code fails this test, ref #1191.)
tests: assign the storage servers to a fixed order which triggers a bug in new downloader every time this test is run (formerly this test would detect the bug in new-downloader only sporadically)
If you are investigating the bug in new-downloader, one way to investigate might be to change this ordering to a different fixed order (e.g. rotate by 4 instead of rotate by 5) and observe how the behavior of new-downloader differs in that case.
tests: bump up the allowed number of reads
Kyle's OpenBSD buildslave used 41 reads when doing this test. The fact that I'm blindly bumping this number up to match the observed behavior probably means this isn't a good criterion to be testing for anyway. But perhaps someone else (Brian) could investigate why that run on Kyle's OpenBSD box took four more reads than we expected, and whether the fact that it took 41 reads to do this operation is indicative of an actual problem.
Brian Warner [Wed, 1 Sep 2010 01:37:02 +0000 (18:37 -0700)]
SegmentFetcher: use new diversity-seeking share-selection algorithm, and
deliver all shares at once instead of feeding them out one-at-a-time.
Also fix distribution of real-number-of-segments information: now all
CommonShares (not just the ones used for the first segment) get a
correctly-sized hashtree. Previously, the late ones might not, which would
make them crash and get dropped (causing the download to fail if the initial
set were insufficient, perhaps because one of their servers went away).
Update tests, add some TODO notes, improve variable names and comments.
Improve logging: add logparents, set more appropriate levels.
Brian Warner [Mon, 9 Aug 2010 22:49:09 +0000 (15:49 -0700)]
immutable/filenode.py: put off DownloadStatus creation until first read() call
This avoids spamming the "recent uploads and downloads" /status page from
FileNode instances that were created for a directory read but which nobody is
ever going to read from. I also cleaned up the way DownloadStatus instances
are made to only ever do it in the CiphertextFileNode, not in the
higher-level plaintext FileNode. Also fixed DownloadStatus handling of read
size, thanks to David-Sarah for the catch.
david-sarah [Sun, 8 Aug 2010 04:28:17 +0000 (21:28 -0700)]
test_runner.py: calculate the location of bin/tahoe correctly for test-from-prefixdir (by copying code from misc/build_helpers/run_trial.py). Also fix the false-positive check for Unicode paths in test_the_right_code, which was causing skips that should have been failures.
Brian Warner [Fri, 6 Aug 2010 08:02:28 +0000 (01:02 -0700)]
update NEWS and other docs in preparation for 1.8.0rc1
in particular, merge the various 1.8.0b1/b2 sections, and remove the
datestamp. NEWS gets updated just before a release, doesn't need to precisely
describe pre-release candidates, and the datestamp gets updated just before
the final release is tagged
Also, I removed the BOM from some files. My toolchain made it hard to retain,
and BOMs in UTF-8 don't make a whole lot of sense anyway. Sorry if that
messes anything up.
Brian Warner [Fri, 6 Aug 2010 07:07:05 +0000 (00:07 -0700)]
downloader.Segmentation: unregisterProducer when asked to stopProducing, this
seems to avoid the #1155 log message which reveals the URI (and filecap).
Also add an [ERROR] marker to the flog entry, since unregisterProducer also
makes interrupted downloads appear "200 OK"; this makes it more obvious that
the download did not complete.
david-sarah [Fri, 6 Aug 2010 05:00:51 +0000 (22:00 -0700)]
test_util.py: use SHA-256 from pycryptopp instead of MD5 from hashlib (for uses in which any hash will do), since hashlib was only added to the stdlib in Python 2.5.
projects / tahoe-lafs / tahoe-lafs.git / log