Brian Warner [Wed, 27 Aug 2008 00:37:29 +0000 (17:37 -0700)]
servermap: don't log late arrivals, and don't log DeadReferenceError at log.WEIRD
Brian Warner [Tue, 26 Aug 2008 23:34:54 +0000 (16:34 -0700)]
mutable: make mutable-repair work for non-verifier runs, add tests
Zooko O'Whielacronx [Tue, 26 Aug 2008 15:50:55 +0000 (08:50 -0700)]
mutable: remove work-around for a flaw in an older version of foolscap
We now require "foolscap[secure_connections] >= 0.3.0", per [source:_auto_deps.py].
Zooko O'Whielacronx [Tue, 26 Aug 2008 15:49:29 +0000 (08:49 -0700)]
docs: edit install.html a tad
Brian Warner [Tue, 26 Aug 2008 01:59:18 +0000 (18:59 -0700)]
misc/make_umid: little script and elisp fragment to insert umid= arguments
Brian Warner [Tue, 26 Aug 2008 01:57:59 +0000 (18:57 -0700)]
logging: add 'unique-message-ids' (or 'umids') to each WEIRD-or-higher log.msg call, to make it easier to correlate log message with source code
Brian Warner [Tue, 26 Aug 2008 00:51:55 +0000 (17:51 -0700)]
logging cleanups: lower DeadReferenceError from WEIRD (which provokes Incidents) to merely UNUSUAL, don't pre-format Failures in others
Zooko O'Whielacronx [Mon, 25 Aug 2008 21:44:07 +0000 (14:44 -0700)]
checker: make the log() function of SimpleCHKFileVerifier compatible with the log() function of its superclasses and subclasses
Zooko O'Whielacronx [Fri, 22 Aug 2008 15:46:05 +0000 (08:46 -0700)]
docs: warn that the "garbage-collection and accounting" section of architecture.txt is out of date, and clarify that "deleted" therein means ciphertext getting garbage-collected
Brian Warner [Thu, 21 Aug 2008 20:59:01 +0000 (13:59 -0700)]
docs/filesystem-notes.txt: add notes about enabling the 'directory index' feature on ext3 filesystems for storage server lookup speed
Zooko O'Whielacronx [Fri, 15 Aug 2008 17:22:34 +0000 (10:22 -0700)]
setup: doc string describing what the require_auto_deps() function is for
Brian Warner [Wed, 13 Aug 2008 03:50:20 +0000 (20:50 -0700)]
mutable/checker: log a WEIRD-level event when we see a hash failure, to trigger an Incident
Brian Warner [Wed, 13 Aug 2008 03:35:30 +0000 (20:35 -0700)]
immutable checker: add a status_report field
Brian Warner [Wed, 13 Aug 2008 03:35:06 +0000 (20:35 -0700)]
mutable/servermap: lower the priority of many log messages
Brian Warner [Wed, 13 Aug 2008 03:34:26 +0000 (20:34 -0700)]
web/deep-check: show the webapi runtime at the bottom of the page
Brian Warner [Wed, 13 Aug 2008 02:50:50 +0000 (19:50 -0700)]
CLI: tolerate blank lines in the aliases file
Brian Warner [Wed, 13 Aug 2008 02:45:20 +0000 (19:45 -0700)]
test_web: workaround broken HEAD behavior in twisted-2.5.0 and earlier
Brian Warner [Wed, 13 Aug 2008 02:04:51 +0000 (19:04 -0700)]
test_web: oops, actually use HEAD (instead of GET) in the HEAD test
Brian Warner [Wed, 13 Aug 2008 02:04:10 +0000 (19:04 -0700)]
web: use get_size_of_best_version for HEAD requests, provide correct content-type
Brian Warner [Wed, 13 Aug 2008 02:02:52 +0000 (19:02 -0700)]
mutable: add get_size_of_best_version to the interface, to simplify the web HEAD code, and tests
Brian Warner [Wed, 13 Aug 2008 00:05:01 +0000 (17:05 -0700)]
CLI: add 'tahoe debug corrupt-share', and use it for deep-verify tests, and fix non-deep web checker API to pass verify=true into node
Brian Warner [Tue, 12 Aug 2008 23:14:07 +0000 (16:14 -0700)]
IFilesystemNode: add get_storage_index(), it makes tests easier
Brian Warner [Tue, 12 Aug 2008 22:59:32 +0000 (15:59 -0700)]
test_system: rename Checker to ImmutableChecker, to make room for a mutable one
Brian Warner [Tue, 12 Aug 2008 21:46:56 +0000 (14:46 -0700)]
'tahoe debug dump-share': add --offsets, to show section offsets
Brian Warner [Tue, 12 Aug 2008 21:46:34 +0000 (14:46 -0700)]
test_cli: oops, fix tests after recent stdout/stderr cleanup
Brian Warner [Tue, 12 Aug 2008 20:55:17 +0000 (13:55 -0700)]
scripts/debug: split out dump_immutable_share
Brian Warner [Tue, 12 Aug 2008 20:52:42 +0000 (13:52 -0700)]
scripts/debug: clean up use of stdout/stderr
Brian Warner [Tue, 12 Aug 2008 20:40:17 +0000 (13:40 -0700)]
CLI: move the 'repl' command to 'tahoe debug repl'
Brian Warner [Tue, 12 Aug 2008 20:37:32 +0000 (13:37 -0700)]
CLI: move all debug commands (dump-share, dump-cap, find-shares, catalog-shares) into a 'debug' subcommand, and improve --help output
Brian Warner [Tue, 12 Aug 2008 04:24:23 +0000 (21:24 -0700)]
hush a pyflakes warning
Brian Warner [Tue, 12 Aug 2008 04:24:09 +0000 (21:24 -0700)]
web/directory: enable verify=true in t=deep-check
Brian Warner [Tue, 12 Aug 2008 04:23:38 +0000 (21:23 -0700)]
dirnode: add some deep-check logging
Brian Warner [Tue, 12 Aug 2008 04:23:06 +0000 (21:23 -0700)]
checker_results.problems: don't str the whole Failure, just extract the reason string
Brian Warner [Tue, 12 Aug 2008 04:03:26 +0000 (21:03 -0700)]
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
Brian Warner [Tue, 12 Aug 2008 03:20:33 +0000 (20:20 -0700)]
mutable/checker: rearrange a bit, change checker-results to have a status_report string
Brian Warner [Tue, 12 Aug 2008 03:19:30 +0000 (20:19 -0700)]
mutable/servermap: add summarize_version
Brian Warner [Tue, 12 Aug 2008 01:20:23 +0000 (18:20 -0700)]
CLI: make 'tahoe webopen' command accept aliases like 'tahoe ls'
Brian Warner [Mon, 11 Aug 2008 20:34:31 +0000 (13:34 -0700)]
munin diskusage/doomsday: oops, fix labels, everything was reported in the 1hr column
Brian Warner [Thu, 7 Aug 2008 21:40:08 +0000 (14:40 -0700)]
munin/tahoe_overhead: don't emit non-sensicial numbers
Brian Warner [Thu, 7 Aug 2008 20:39:25 +0000 (13:39 -0700)]
munin: add tahoe_overhead plugin, to measure effectiveness of GC and deleting data from inactive accounts
Brian Warner [Thu, 7 Aug 2008 20:12:14 +0000 (13:12 -0700)]
diskwatcher.tac: include total-bytes-used
Zooko O'Whielacronx [Thu, 7 Aug 2008 19:30:29 +0000 (12:30 -0700)]
setup: remove accidentally duplicated lines from Makefile
Brian Warner [Thu, 7 Aug 2008 18:45:46 +0000 (11:45 -0700)]
misc/dependencies: remove the no-longer-useful foolscap-0.2.5 tarball
Brian Warner [Thu, 7 Aug 2008 18:30:01 +0000 (11:30 -0700)]
Makefile: avoid bare quotes, since the emacs syntax-highlighter gets confused by them
Brian Warner [Thu, 7 Aug 2008 17:34:33 +0000 (10:34 -0700)]
diskwatcher.tac: don't report negative timeleft
Brian Warner [Thu, 7 Aug 2008 06:20:21 +0000 (23:20 -0700)]
diskwatcher.tac: reduce the polling rate to once per hour
Brian Warner [Thu, 7 Aug 2008 06:00:03 +0000 (23:00 -0700)]
misc/spacetime: add munin plugins, add everything to .deb
Brian Warner [Thu, 7 Aug 2008 05:04:27 +0000 (22:04 -0700)]
diskwatcher.tac: hush pyflakes
Brian Warner [Thu, 7 Aug 2008 05:03:27 +0000 (22:03 -0700)]
diskwatcher.tac: add async-GET code, but leave it commented out: urlopen() seems to work better for now
Brian Warner [Thu, 7 Aug 2008 04:38:01 +0000 (21:38 -0700)]
cpu-watcher.tac: improve error message
Brian Warner [Thu, 7 Aug 2008 04:22:22 +0000 (21:22 -0700)]
disk-watcher: first draft of a daemon to use the HTTP stats interface and its new storage_server.disk_avail feature, to track changes in disk space over time
Brian Warner [Thu, 7 Aug 2008 04:17:05 +0000 (21:17 -0700)]
misc/cpu-watcher.tac: tolerate missing pidfiles, just skip over that sample
Zooko O'Whielacronx [Wed, 6 Aug 2008 23:27:42 +0000 (16:27 -0700)]
setup: don't attempt to escape quote marks, just delete them. Ugly, but it works okay.
Zooko O'Whielacronx [Wed, 6 Aug 2008 23:11:43 +0000 (16:11 -0700)]
setup: escape any double-quote chars in the PATH before using the PATH to find and invoke trial
Brian Warner [Wed, 6 Aug 2008 21:06:02 +0000 (14:06 -0700)]
storage: include disk-free information in the stats-gatherer output
Brian Warner [Wed, 6 Aug 2008 19:06:07 +0000 (12:06 -0700)]
mutable: more repair tests, one with force=True to check out merging
Brian Warner [Wed, 6 Aug 2008 19:05:52 +0000 (12:05 -0700)]
test/common: add ShouldFailMixin
Brian Warner [Wed, 6 Aug 2008 17:38:50 +0000 (10:38 -0700)]
test_mutable: add comment about minimal-bandwidth repairer, comma lack of
Brian Warner [Wed, 6 Aug 2008 17:38:04 +0000 (10:38 -0700)]
test_mutable: factor out common setup code
Brian Warner [Wed, 6 Aug 2008 06:12:39 +0000 (23:12 -0700)]
mutable: start adding Repair tests, fix a simple bug
Brian Warner [Wed, 6 Aug 2008 06:12:19 +0000 (23:12 -0700)]
mutable.txt: add warning about out-of-date section
Brian Warner [Wed, 6 Aug 2008 01:49:58 +0000 (18:49 -0700)]
test_system: factor out find_shares/replace_shares to a common class, so they can be used by other tests
Brian Warner [Wed, 6 Aug 2008 01:32:22 +0000 (18:32 -0700)]
debian/control: update dependencies to match _auto_deps: foolscap-0.3.0, pycryptopp-0.5
Brian Warner [Tue, 5 Aug 2008 23:58:28 +0000 (16:58 -0700)]
bump foolscap dependency to 0.3.0, for the new incident-gathering interfaces
Brian Warner [Tue, 5 Aug 2008 19:09:21 +0000 (12:09 -0700)]
web: add 'report incident' button at the bottom of the welcome page
Brian Warner [Mon, 4 Aug 2008 20:26:43 +0000 (13:26 -0700)]
test_cli: more coverage for 'tahoe put' modifying a mutable file in-place, by filename, closes #441
Brian Warner [Sat, 2 Aug 2008 02:48:56 +0000 (19:48 -0700)]
check_grid.py: update to match new CLI: 'put - TARGET' instead of 'put TARGET'
Brian Warner [Sat, 2 Aug 2008 02:47:34 +0000 (19:47 -0700)]
test_cli: remove windows-worrying newlines from test data
Brian Warner [Sat, 2 Aug 2008 02:29:38 +0000 (19:29 -0700)]
test_cli.py: factor out CLITestMixin
Brian Warner [Sat, 2 Aug 2008 02:27:29 +0000 (19:27 -0700)]
CLI: change one-arg forms of 'tahoe put' to make an unlinked file, fix replace-mutable #441
Brian Warner [Sat, 2 Aug 2008 02:10:41 +0000 (19:10 -0700)]
CLI: add create-alias command, to merge mkdir and add-alias into a single (secure-from-argv-snooping) step
Brian Warner [Fri, 1 Aug 2008 22:10:09 +0000 (15:10 -0700)]
test_cli: add system-based tests for PUT, including a mutable put that fails/todo (#441)
Brian Warner [Fri, 1 Aug 2008 22:05:14 +0000 (15:05 -0700)]
tests: simplify CLI tests that use stdin, now that runner supports it
Brian Warner [Fri, 1 Aug 2008 18:46:24 +0000 (11:46 -0700)]
CLI: simplify argument-passing, use options= for everthing, including stdout
Zooko O'Whielacronx [Thu, 31 Jul 2008 00:20:15 +0000 (17:20 -0700)]
tests: add test that verifier notices any (randomly chosen) bit flipped in the verifiable part of any (randomly chosen) share
The currently verifier doesn't (usually) pass this randomized test, hence the TODO.
Zooko O'Whielacronx [Wed, 30 Jul 2008 23:54:20 +0000 (16:54 -0700)]
tests: test that checker doesn't cause reads on the storage servers
It would still pass the test if it noticed a corrupted share. (It won't
notice, of course.) But it is required to do its work without causing storage
servers to read blocks from the filesystem.
Zooko O'Whielacronx [Wed, 30 Jul 2008 22:51:07 +0000 (15:51 -0700)]
storage: make storage servers declare oldest supported version == 1.0, and storage clients declare oldest supported version == 1.0
See comments in patch for intended semantics.
Zooko O'Whielacronx [Wed, 30 Jul 2008 22:49:20 +0000 (15:49 -0700)]
tests: use the handy dandy TestCase.mktemp() function from trial to give unique and nicely named directories for each testcase
Zooko O'Whielacronx [Wed, 30 Jul 2008 22:35:36 +0000 (15:35 -0700)]
tests: don't use SignalMixin
It seems like we no longer need it, and it screws up something internal in
trial which causes trial's TestCase.mktemp() method to exhibit wrong behavior
(always using a certain test method name instead of using the current test
method name), and I wish to use TestCase.mktemp().
Of course, it is possible that the buildbot is about to tell me that we do
still require SignalMixin on some of our platforms...
Zooko O'Whielacronx [Wed, 30 Jul 2008 20:58:06 +0000 (13:58 -0700)]
setup: if the user passes a TRIALOPT env var then pass that on to trial
This is useful for --reporter=bwverbose, for example.
Zooko O'Whielacronx [Wed, 30 Jul 2008 18:12:17 +0000 (11:12 -0700)]
setup: turn back on reactor=poll for cygwin trial (else it runs out of fds)
Zooko O'Whielacronx [Wed, 30 Jul 2008 16:04:29 +0000 (09:04 -0700)]
setup: fix bug in Makefile -- ifeq, not ifneq -- so that now it sets poll reactor only if the user hasn't specified a REACTOR variable, instead of setting poll reactor only if the user has specified a REACTOR variable
Zooko O'Whielacronx [Wed, 30 Jul 2008 03:23:58 +0000 (20:23 -0700)]
setup: whoops, really remove the default reactor=poll this time
Zooko O'Whielacronx [Wed, 30 Jul 2008 03:16:56 +0000 (20:16 -0700)]
setup: instead of setting --reactor=poll for trial in all cases (which fails on platforms that don't have poll reactor, such as Windows and some Mac OS X), just set --reactor=poll for linux2.
Zooko O'Whielacronx [Wed, 30 Jul 2008 02:39:06 +0000 (19:39 -0700)]
setup: pass --reactor=poll to trial unless REACTOR variable is set, in which case pass --reactor=$(REACTOR)
This hopefully works around the problem that Twisted v8.1.0 has a bug when used
with pyOpenSSL v0.7 which bug causes some unit tests to spuriously fail -- see
known_issues.txt r2788:
http://allmydata.org/trac/tahoe/browser/docs/known_issues.txt?rev=2788#L122
Also it matches with the fact that --reactor=poll is required on cygwin.
Zooko O'Whielacronx [Wed, 30 Jul 2008 02:10:41 +0000 (19:10 -0700)]
setup: require secure_connections from foolscap
This causes a problem on debian sid, since the pyOpenSSL v0.6 .deb doesn't come
with .egg-info, so setuptools will not know that it is already installed and
will try to install pyOpenSSL, and if it installs pyOpenSSL v0.7, then this
will trigger the bug in Twisted v8.1.0 when used with pyOpenSSL v0.7.
http://twistedmatrix.com/trac/ticket/3218
Now the comments in twisted #3218 suggest that it happens only with the select
reactor, so maybe using --reactor=poll will avoid it.
Zooko O'Whielacronx [Mon, 28 Jul 2008 23:43:17 +0000 (16:43 -0700)]
tests: add test_system.Checker which tests basic checking (without verification) functionality
Zooko O'Whielacronx [Mon, 28 Jul 2008 23:42:17 +0000 (16:42 -0700)]
test: add testutil.flip_one_bit which flips a randomly chosen bit of the input string
Zooko O'Whielacronx [Mon, 28 Jul 2008 23:40:29 +0000 (16:40 -0700)]
tests: make it so that you can use common.py's SystemTestMixin.set_up_nodes() more than once with the same introducer
Brian Warner [Mon, 28 Jul 2008 21:57:31 +0000 (14:57 -0700)]
download.py: set up self._paused before registering the producer, since they might call pauseProducing right away
Brian Warner [Mon, 28 Jul 2008 19:44:21 +0000 (12:44 -0700)]
test/common.py: use pre-computed Tub certificates for the system-test mixin, to speed such tests up by maybe 15%. The goal is to encourage more full-grid tests.
Brian Warner [Mon, 28 Jul 2008 19:42:33 +0000 (12:42 -0700)]
munin/tahoe_spacetime: show 2wk data even if 4wk data is unavailable
Brian Warner [Sat, 26 Jul 2008 00:41:10 +0000 (17:41 -0700)]
web: add /status/?t=json, with active upload/download ops. Addresses #493.
Brian Warner [Sat, 26 Jul 2008 00:24:27 +0000 (17:24 -0700)]
web: make t=json stats pages use text/plain, instead of leaving it at text/html
Brian Warner [Fri, 25 Jul 2008 22:33:49 +0000 (15:33 -0700)]
test_system.py: factor SystemTestMixin out of SystemTest
Brian Warner [Fri, 25 Jul 2008 22:29:31 +0000 (15:29 -0700)]
test_system.py: modify system-test setup code in preparation for merge with common.SystemTestMixin
Brian Warner [Fri, 25 Jul 2008 22:17:58 +0000 (15:17 -0700)]
test_system.py: move SystemTestMixin out into common.py, where further improvements will occur
Brian Warner [Fri, 25 Jul 2008 22:13:00 +0000 (15:13 -0700)]
test_system.py: create SystemTestMixin, with less cruft, for faster system-like tests
Zooko O'Whielacronx [Tue, 22 Jul 2008 01:46:08 +0000 (18:46 -0700)]
TAG allmydata-tahoe-1.2.0
Zooko O'Whielacronx [Tue, 22 Jul 2008 01:04:03 +0000 (18:04 -0700)]
relnotes.txt: update and edit for the 1.2.0 release!