Brian Warner [Fri, 1 Aug 2008 22:05:14 +0000 (15:05 -0700)]
tests: simplify CLI tests that use stdin, now that runner supports it
Brian Warner [Fri, 1 Aug 2008 18:46:24 +0000 (11:46 -0700)]
CLI: simplify argument-passing, use options= for everthing, including stdout
Zooko O'Whielacronx [Thu, 31 Jul 2008 00:20:15 +0000 (17:20 -0700)]
tests: add test that verifier notices any (randomly chosen) bit flipped in the verifiable part of any (randomly chosen) share
The currently verifier doesn't (usually) pass this randomized test, hence the TODO.
Zooko O'Whielacronx [Wed, 30 Jul 2008 23:54:20 +0000 (16:54 -0700)]
tests: test that checker doesn't cause reads on the storage servers
It would still pass the test if it noticed a corrupted share. (It won't
notice, of course.) But it is required to do its work without causing storage
servers to read blocks from the filesystem.
Zooko O'Whielacronx [Wed, 30 Jul 2008 22:51:07 +0000 (15:51 -0700)]
storage: make storage servers declare oldest supported version == 1.0, and storage clients declare oldest supported version == 1.0
See comments in patch for intended semantics.
Zooko O'Whielacronx [Wed, 30 Jul 2008 22:49:20 +0000 (15:49 -0700)]
tests: use the handy dandy TestCase.mktemp() function from trial to give unique and nicely named directories for each testcase
Zooko O'Whielacronx [Wed, 30 Jul 2008 22:35:36 +0000 (15:35 -0700)]
tests: don't use SignalMixin
It seems like we no longer need it, and it screws up something internal in
trial which causes trial's TestCase.mktemp() method to exhibit wrong behavior
(always using a certain test method name instead of using the current test
method name), and I wish to use TestCase.mktemp().
Of course, it is possible that the buildbot is about to tell me that we do
still require SignalMixin on some of our platforms...
Zooko O'Whielacronx [Wed, 30 Jul 2008 20:58:06 +0000 (13:58 -0700)]
setup: if the user passes a TRIALOPT env var then pass that on to trial
This is useful for --reporter=bwverbose, for example.
Zooko O'Whielacronx [Wed, 30 Jul 2008 18:12:17 +0000 (11:12 -0700)]
setup: turn back on reactor=poll for cygwin trial (else it runs out of fds)
Zooko O'Whielacronx [Wed, 30 Jul 2008 16:04:29 +0000 (09:04 -0700)]
setup: fix bug in Makefile -- ifeq, not ifneq -- so that now it sets poll reactor only if the user hasn't specified a REACTOR variable, instead of setting poll reactor only if the user has specified a REACTOR variable
Zooko O'Whielacronx [Wed, 30 Jul 2008 03:23:58 +0000 (20:23 -0700)]
setup: whoops, really remove the default reactor=poll this time
Zooko O'Whielacronx [Wed, 30 Jul 2008 03:16:56 +0000 (20:16 -0700)]
setup: instead of setting --reactor=poll for trial in all cases (which fails on platforms that don't have poll reactor, such as Windows and some Mac OS X), just set --reactor=poll for linux2.
Zooko O'Whielacronx [Wed, 30 Jul 2008 02:39:06 +0000 (19:39 -0700)]
setup: pass --reactor=poll to trial unless REACTOR variable is set, in which case pass --reactor=$(REACTOR)
This hopefully works around the problem that Twisted v8.1.0 has a bug when used
with pyOpenSSL v0.7 which bug causes some unit tests to spuriously fail -- see
known_issues.txt r2788:
http://allmydata.org/trac/tahoe/browser/docs/known_issues.txt?rev=2788#L122
Also it matches with the fact that --reactor=poll is required on cygwin.
Zooko O'Whielacronx [Wed, 30 Jul 2008 02:10:41 +0000 (19:10 -0700)]
setup: require secure_connections from foolscap
This causes a problem on debian sid, since the pyOpenSSL v0.6 .deb doesn't come
with .egg-info, so setuptools will not know that it is already installed and
will try to install pyOpenSSL, and if it installs pyOpenSSL v0.7, then this
will trigger the bug in Twisted v8.1.0 when used with pyOpenSSL v0.7.
http://twistedmatrix.com/trac/ticket/3218
Now the comments in twisted #3218 suggest that it happens only with the select
reactor, so maybe using --reactor=poll will avoid it.
Zooko O'Whielacronx [Mon, 28 Jul 2008 23:43:17 +0000 (16:43 -0700)]
tests: add test_system.Checker which tests basic checking (without verification) functionality
Zooko O'Whielacronx [Mon, 28 Jul 2008 23:42:17 +0000 (16:42 -0700)]
test: add testutil.flip_one_bit which flips a randomly chosen bit of the input string
Zooko O'Whielacronx [Mon, 28 Jul 2008 23:40:29 +0000 (16:40 -0700)]
tests: make it so that you can use common.py's SystemTestMixin.set_up_nodes() more than once with the same introducer
Brian Warner [Mon, 28 Jul 2008 21:57:31 +0000 (14:57 -0700)]
download.py: set up self._paused before registering the producer, since they might call pauseProducing right away
Brian Warner [Mon, 28 Jul 2008 19:44:21 +0000 (12:44 -0700)]
test/common.py: use pre-computed Tub certificates for the system-test mixin, to speed such tests up by maybe 15%. The goal is to encourage more full-grid tests.
Brian Warner [Mon, 28 Jul 2008 19:42:33 +0000 (12:42 -0700)]
munin/tahoe_spacetime: show 2wk data even if 4wk data is unavailable
Brian Warner [Sat, 26 Jul 2008 00:41:10 +0000 (17:41 -0700)]
web: add /status/?t=json, with active upload/download ops. Addresses #493.
Brian Warner [Sat, 26 Jul 2008 00:24:27 +0000 (17:24 -0700)]
web: make t=json stats pages use text/plain, instead of leaving it at text/html
Brian Warner [Fri, 25 Jul 2008 22:33:49 +0000 (15:33 -0700)]
test_system.py: factor SystemTestMixin out of SystemTest
Brian Warner [Fri, 25 Jul 2008 22:29:31 +0000 (15:29 -0700)]
test_system.py: modify system-test setup code in preparation for merge with common.SystemTestMixin
Brian Warner [Fri, 25 Jul 2008 22:17:58 +0000 (15:17 -0700)]
test_system.py: move SystemTestMixin out into common.py, where further improvements will occur
Brian Warner [Fri, 25 Jul 2008 22:13:00 +0000 (15:13 -0700)]
test_system.py: create SystemTestMixin, with less cruft, for faster system-like tests
Zooko O'Whielacronx [Tue, 22 Jul 2008 01:46:08 +0000 (18:46 -0700)]
TAG allmydata-tahoe-1.2.0
Zooko O'Whielacronx [Tue, 22 Jul 2008 01:04:03 +0000 (18:04 -0700)]
relnotes.txt: update and edit for the 1.2.0 release!
Zooko O'Whielacronx [Tue, 22 Jul 2008 01:02:49 +0000 (18:02 -0700)]
known_issues.txt: fix up the argv leakage issue -- it applies to Tahoe 1.2.0. Other editing corrections.
Zooko O'Whielacronx [Tue, 22 Jul 2008 01:02:29 +0000 (18:02 -0700)]
setup: update README to point to known_issues.txt
Zooko O'Whielacronx [Tue, 22 Jul 2008 00:43:34 +0000 (17:43 -0700)]
known_issues.txt: command-line arguments are leaked to other processes
Brian Warner [Tue, 22 Jul 2008 00:28:28 +0000 (17:28 -0700)]
storage: remove update_write_enabler method, it won't serve the desired purpose, and I have a better scheme in mind. See #489 for details
Brian Warner [Mon, 21 Jul 2008 23:29:30 +0000 (16:29 -0700)]
NEWS: add user-visible changes since the previous release
Brian Warner [Mon, 21 Jul 2008 22:33:09 +0000 (15:33 -0700)]
docs/CLI.txt: add a warning about leaking dircaps through argv in add-alias
Zooko O'Whielacronx [Mon, 21 Jul 2008 21:11:06 +0000 (14:11 -0700)]
simulate_load: put it in friendnet mode and print out a sum of doubled shares
Zooko O'Whielacronx [Mon, 21 Jul 2008 17:44:06 +0000 (10:44 -0700)]
known_issues.txt: edits suggested by Brian
Zooko O'Whielacronx [Tue, 1 Jul 2008 20:19:36 +0000 (13:19 -0700)]
docs: start updating the relnotes.txt in preparation for the next release
Zooko O'Whielacronx [Mon, 21 Jul 2008 17:21:01 +0000 (10:21 -0700)]
known_issues.txt: add issue #491 and renumber issues
Brian Warner [Mon, 21 Jul 2008 16:49:36 +0000 (09:49 -0700)]
add note to mutable-DSA.txt about the need for a ciphertext hash, for #492
Zooko O'Whielacronx [Mon, 21 Jul 2008 16:31:02 +0000 (09:31 -0700)]
immutable file download: make the ciphertext hash tree mandatory
This fixes #491 (URIs do not refer to unique files in Allmydata Tahoe).
Fortunately all of the versions of Tahoe currently in use are already producing
this ciphertext hash tree when uploading, so there is no
backwards-compatibility problem with having the downloader require it to be
present.
Zooko O'Whielacronx [Mon, 21 Jul 2008 16:28:49 +0000 (09:28 -0700)]
setup: don't use "python" in Makefile, use $(PYTHON) everywhere
Brian Warner [Sat, 19 Jul 2008 01:58:57 +0000 (18:58 -0700)]
test_web: test that save=true filename=unicode doesn't crash
Brian Warner [Sat, 19 Jul 2008 01:06:50 +0000 (18:06 -0700)]
web: for GET save=true, don't interpret the filename= arg with any character set, just copy the bytes back into the Content-Disposition header. This seems to make it maximally compatible with Firefox and IE7
Brian Warner [Fri, 18 Jul 2008 04:09:23 +0000 (21:09 -0700)]
first pass at a mutable repairer. not tested at all yet, but of course all existing tests pass
Brian Warner [Fri, 18 Jul 2008 00:32:17 +0000 (17:32 -0700)]
interfaces: add IRepairable
Brian Warner [Fri, 18 Jul 2008 00:06:20 +0000 (17:06 -0700)]
oops, fix import/pyflakes problems
Brian Warner [Thu, 17 Jul 2008 23:47:09 +0000 (16:47 -0700)]
deep-check: add webapi, add 'DEEP-CHECK' button to wui, add tests, rearrange checker API a bit
Brian Warner [Thu, 17 Jul 2008 21:37:04 +0000 (14:37 -0700)]
dirnode deep-check: add tests of cycles, fix failures
Brian Warner [Thu, 17 Jul 2008 21:25:04 +0000 (14:25 -0700)]
dirnode deep-check: rearrange traversal approach, simplify code a bit
Brian Warner [Thu, 17 Jul 2008 01:20:57 +0000 (18:20 -0700)]
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
Brian Warner [Wed, 16 Jul 2008 23:31:47 +0000 (16:31 -0700)]
immutable download: remove dead LiteralDownloader, now that we use filenodes for download
Brian Warner [Wed, 16 Jul 2008 23:08:17 +0000 (16:08 -0700)]
munin/tahoe_server_latency: add percentile to graph title
Brian Warner [Wed, 16 Jul 2008 22:42:56 +0000 (15:42 -0700)]
checker: re-enable checker web results (although they just say 'Healthy' right now)
Brian Warner [Wed, 16 Jul 2008 20:14:39 +0000 (13:14 -0700)]
move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected.
Brian Warner [Wed, 16 Jul 2008 19:36:42 +0000 (12:36 -0700)]
munin/serverops: improve graph title
Brian Warner [Wed, 16 Jul 2008 01:47:41 +0000 (18:47 -0700)]
debian/rules: put munin plugins in /usr/share/PACKAGENAME/munin/, make them +x, remove packagename from rules to make branches easier to manage
Brian Warner [Wed, 16 Jul 2008 01:42:38 +0000 (18:42 -0700)]
sid/control: set python versions to 2.4,2.5 , to match feisty/control, since sid has had 2.5 available forever now
Brian Warner [Wed, 16 Jul 2008 00:23:25 +0000 (17:23 -0700)]
overhaul checker invocation
Removed the Checker service, removed checker results storage (both in-memory
and the tiny stub of sqlite-based storage). Added ICheckable, all
check/verify is now done by calling the check() method on filenodes and
dirnodes (immutable files, literal files, mutable files, and directory
instances).
Checker results are returned in a Results instance, with an html() method for
display. Checker results have been temporarily removed from the wui directory
listing until we make some other fixes.
Also fixed client.create_node_from_uri() to create LiteralFileNodes properly,
since they have different checking behavior. Previously we were creating full
FileNodes with LIT uris inside, which were downloadable but not checkable.
Brian Warner [Tue, 15 Jul 2008 22:23:41 +0000 (15:23 -0700)]
debian/changelog: remove all versions but 0.0.1, so debchange can always override it. The 'deb-X' Makefile targets should not be used; run 'make deb-X-head' instead
Brian Warner [Mon, 14 Jul 2008 22:25:21 +0000 (15:25 -0700)]
download: fix stopProducing failure ('self._paused_at not defined'), add tests
Brian Warner [Mon, 14 Jul 2008 21:53:25 +0000 (14:53 -0700)]
setup.py: cosmetic, remove trailing whitespace
Brian Warner [Mon, 14 Jul 2008 20:25:35 +0000 (13:25 -0700)]
munin/tahoe_stats: add server operations per second (allocate/get/writev/readv
Brian Warner [Mon, 14 Jul 2008 20:25:01 +0000 (13:25 -0700)]
munin/tahoe_stats: rearrange config a bit, no functional changes
Brian Warner [Mon, 14 Jul 2008 20:18:47 +0000 (13:18 -0700)]
munin/tahoe_server_operations: fix comments
Brian Warner [Mon, 14 Jul 2008 20:18:11 +0000 (13:18 -0700)]
munin: add plugin for storage-server operations per second
Brian Warner [Mon, 14 Jul 2008 19:56:38 +0000 (12:56 -0700)]
copy debian/sid changes to debian/feisty
Brian Warner [Mon, 14 Jul 2008 19:55:34 +0000 (12:55 -0700)]
debian/sid: add more docs to package, including munin plugins
Brian Warner [Mon, 14 Jul 2008 19:29:19 +0000 (12:29 -0700)]
misc/munin: add server_latency plugin
Zooko O'Whielacronx [Sat, 12 Jul 2008 23:54:29 +0000 (16:54 -0700)]
misc/simulate_load.py: fix bug in which only K shares were uploaded instead of N, and fix the x axis in place so that successive runs can be compared
Zooko O'Whielacronx [Sat, 12 Jul 2008 22:37:27 +0000 (15:37 -0700)]
misc/simulate_load.py: use a more realistic distribution of file sizes, and automate the formatting of the ascii art so that it fits in 70 columns and the number of files uploaded is properly marked
Zooko O'Whielacronx [Sat, 12 Jul 2008 21:26:22 +0000 (14:26 -0700)]
add misc/simulate_load.py in an attempt to understand why permuting the peerlist per each storage index matters
Brian Warner [Sat, 12 Jul 2008 04:51:02 +0000 (21:51 -0700)]
storage: rename the latency key names so they sort properly
Brian Warner [Sat, 12 Jul 2008 04:34:36 +0000 (21:34 -0700)]
add 1%,10% percentiles to the storage server latency output
Brian Warner [Fri, 11 Jul 2008 22:21:13 +0000 (15:21 -0700)]
misc/munin: rename plugins to make munin happy: use underscores rather than hyphens, remove .py suffix
Brian Warner [Fri, 11 Jul 2008 22:17:33 +0000 (15:17 -0700)]
remove obsolete tahoe-sharesperfile munin plugin
Brian Warner [Fri, 11 Jul 2008 22:14:28 +0000 (15:14 -0700)]
remove tahoe-dirnodes.py munin plugin, it is way obsolete
Brian Warner [Thu, 10 Jul 2008 01:06:55 +0000 (18:06 -0700)]
storage: add add_lease/update_write_enabler to remote API, revamp lease handling
Brian Warner [Thu, 10 Jul 2008 00:37:22 +0000 (17:37 -0700)]
welcome page: add link to statistics page
Brian Warner [Tue, 8 Jul 2008 00:36:00 +0000 (17:36 -0700)]
implement a mutable checker+verifier. No repair yet. Part of #205.
Brian Warner [Mon, 7 Jul 2008 22:06:15 +0000 (15:06 -0700)]
servermap.shares_available: report N too, not just k
Brian Warner [Mon, 7 Jul 2008 21:37:36 +0000 (14:37 -0700)]
interfaces: add verify= and repair= args to check()
Brian Warner [Mon, 7 Jul 2008 21:11:02 +0000 (14:11 -0700)]
'tahoe dump-share': show verify-cap too
Brian Warner [Mon, 7 Jul 2008 19:18:10 +0000 (12:18 -0700)]
test_mutable.py: add tests for no-servers conditions, closes #463.
Brian Warner [Mon, 7 Jul 2008 07:18:16 +0000 (00:18 -0700)]
web/root.py: fix minor typo
Brian Warner [Mon, 7 Jul 2008 07:17:41 +0000 (00:17 -0700)]
mutable: if there are no servers available, return an error, rather than hanging forever
Brian Warner [Mon, 7 Jul 2008 06:49:08 +0000 (23:49 -0700)]
re-enable incident-reporting, but disable it in unit tests, because they take 150% longer, and the leftover trailing timers cause errors
Brian Warner [Thu, 3 Jul 2008 01:24:00 +0000 (18:24 -0700)]
disable incident reporting: unit tests are failing because of the leftover post-incident timer
Brian Warner [Thu, 3 Jul 2008 00:40:29 +0000 (17:40 -0700)]
start using Foolscap's 'incident-logging' feature, which requires foolscap-0.2.9
Brian Warner [Thu, 26 Jun 2008 18:36:17 +0000 (11:36 -0700)]
test_storage: oops, update the test to match the leave-incoming/ change
Brian Warner [Thu, 26 Jun 2008 18:23:44 +0000 (11:23 -0700)]
catalog-shares: ignore incoming/
Brian Warner [Thu, 26 Jun 2008 18:07:57 +0000 (11:07 -0700)]
storage.py: leave the storage/shares/incoming/ directory in place when the bucket is closed
Brian Warner [Wed, 25 Jun 2008 00:21:18 +0000 (17:21 -0700)]
storage: record latency stats in a flat dict, not nested, to conform to RIStatsProvider
Brian Warner [Tue, 24 Jun 2008 18:08:10 +0000 (11:08 -0700)]
test_mutable.py: add more tests of post-mapupdate corruption, to support #474 testing
Zooko O'Whielacronx [Sat, 21 Jun 2008 21:04:26 +0000 (14:04 -0700)]
setup: remove bundled darcsver-1.1.2.tar
Zooko O'Whielacronx [Sat, 21 Jun 2008 21:01:09 +0000 (14:01 -0700)]
setup: setup_requires darcsver >= 1.1.5
Zooko O'Whielacronx [Sat, 21 Jun 2008 20:56:24 +0000 (13:56 -0700)]
setup: bundle darcsver-1.1.5 into misc/dependencies
Once we implement #249 (move bundled dependencies out of revision control history and make them optional) then we can stop doing this. Until then we try not to do it unless we really need to...
Brian Warner [Sat, 21 Jun 2008 04:55:25 +0000 (21:55 -0700)]
docs/accounts-pubkey.txt: when sending FURLs (signed or otherwise), also send
the object that it references, to bypass the Foolscap getReferenceByName
roundtrip that would otherwise be required. Recipients must still use the
FURL for security, but pre-populating the foolscap table is a good speed-up.
Brian Warner [Thu, 19 Jun 2008 00:04:41 +0000 (17:04 -0700)]
introducer: move the relevant interfaces out to introducer/interfaces.py
Brian Warner [Wed, 18 Jun 2008 23:58:34 +0000 (16:58 -0700)]
introducer: add old (V1) introducer code, add test framework for compatibility testing
Brian Warner [Thu, 19 Jun 2008 19:39:52 +0000 (12:39 -0700)]
check_grid.py: make it work, move node start/stop responsibility from Makefile to script