Brian Warner [Mon, 18 May 2009 23:43:26 +0000 (16:43 -0700)]
util/pipeline.py: new utility class to manage size-limited work pipelines, for #392
Zooko O'Whielacronx [Thu, 14 May 2009 23:20:59 +0000 (16:20 -0700)]
docs: add a diagram that I'm about to show to the Boulder Linux Users Group: network-and-reliance-topology.svg
Zooko O'Whielacronx [Thu, 7 May 2009 22:31:51 +0000 (15:31 -0700)]
tests: mark test_runner as coded in utf-8 instead of ascii
Zooko O'Whielacronx [Thu, 7 May 2009 21:50:12 +0000 (14:50 -0700)]
tests: raise timeout on test_runner.RunNode.test_introducer from 120s to 240s, since it hit the 120s time-out on François Lenny-armv5tel
Zooko O'Whielacronx [Thu, 7 May 2009 21:50:03 +0000 (14:50 -0700)]
setup: fix comment in setup.py
Zooko O'Whielacronx [Thu, 7 May 2009 21:49:32 +0000 (14:49 -0700)]
docs: how_to_make_a_tahoe_release.txt: a couple of small edits
Brian Warner [Wed, 15 Apr 2009 21:05:50 +0000 (14:05 -0700)]
.darcs-boringfile: also ignore .gitignore
Brian Warner [Wed, 15 Apr 2009 20:59:29 +0000 (13:59 -0700)]
.darcs-boringfile: ignore .git, I'm starting to play around with it
Brian Warner [Tue, 14 Apr 2009 20:14:00 +0000 (13:14 -0700)]
fix quicktest: stop using setuptools, add misc/run-with-pythonpath.py, to make it run faster
Zooko O'Whielacronx [Tue, 14 Apr 2009 02:56:36 +0000 (19:56 -0700)]
TAG allmydata-tahoe-1.4.1
Zooko O'Whielacronx [Tue, 14 Apr 2009 02:54:30 +0000 (19:54 -0700)]
docs: inaugurate Tahoe-1.4.1, since I left out a handful of patches from the Tahoe-1.4.0 release
Zooko O'Whielacronx [Tue, 14 Apr 2009 02:43:42 +0000 (19:43 -0700)]
docs: add "darcs pull" to how_to_make_a_tahoe_release.txt, and renumber
Zooko O'Whielacronx [Tue, 14 Apr 2009 02:20:15 +0000 (19:20 -0700)]
TAG allmydata-tahoe-1.4.0
Zooko O'Whielacronx [Tue, 14 Apr 2009 02:19:13 +0000 (19:19 -0700)]
docs: edit relnotes.txt and promote Tahoe from filesystem to cloud storage thingie
Zooko O'Whielacronx [Mon, 13 Apr 2009 18:52:10 +0000 (11:52 -0700)]
docs: add note about pywin32 to install.html
Zooko O'Whielacronx [Mon, 13 Apr 2009 17:41:38 +0000 (10:41 -0700)]
trivial: fix comment
Zooko O'Whielacronx [Mon, 13 Apr 2009 16:08:37 +0000 (09:08 -0700)]
docs: a few edits/updates about dirnodes
Zooko O'Whielacronx [Mon, 13 Apr 2009 16:06:49 +0000 (09:06 -0700)]
docs: change install.html to point to the 1.4.0 release zip file instead of 1.3.0
Zooko O'Whielacronx [Mon, 13 Apr 2009 16:06:12 +0000 (09:06 -0700)]
docs: edit install.html regarding versions of Python
Zooko O'Whielacronx [Mon, 13 Apr 2009 16:04:14 +0000 (09:04 -0700)]
docs: some small edits to install.html
Brian Warner [Mon, 13 Apr 2009 11:04:46 +0000 (04:04 -0700)]
NEWS: mention that the new 'reliability' page requires numpy
Zooko O'Whielacronx [Mon, 13 Apr 2009 04:14:05 +0000 (21:14 -0700)]
docs: update relnotes.txt, NEWS for Tahoe-1.4.0 release!
Zooko O'Whielacronx [Mon, 13 Apr 2009 02:36:21 +0000 (19:36 -0700)]
tests: remove obsolete test (it tests functionality that is long gone) which occasionally ERRORs now that we have more metadata (since [
20090411225205-92b7f-
7adfb89cb4db4ac7d28427934dea3d2c108f6476])
Zooko O'Whielacronx [Sun, 12 Apr 2009 02:17:42 +0000 (19:17 -0700)]
trivial: remove trailing whitespace and unused import
Zooko O'Whielacronx [Sat, 11 Apr 2009 22:52:05 +0000 (15:52 -0700)]
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API
Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.
Zooko O'Whielacronx [Sat, 11 Apr 2009 22:48:28 +0000 (15:48 -0700)]
docs: webapi.txt edits to explain a few things better, adjust indentation, editing
Zooko O'Whielacronx [Sat, 11 Apr 2009 18:19:06 +0000 (11:19 -0700)]
Zooko O'Whielacronx [Fri, 10 Apr 2009 17:43:45 +0000 (10:43 -0700)]
setup: update comments about what versions we require of our dependencies
Brian Warner [Fri, 10 Apr 2009 01:56:55 +0000 (18:56 -0700)]
NEWS: finish formatting new items. Still needs proofreading and updating the release name+date
Brian Warner [Thu, 9 Apr 2009 23:59:59 +0000 (16:59 -0700)]
#622: disable 'Repair' button on check-results page until we make it work correctly
Brian Warner [Wed, 8 Apr 2009 02:13:40 +0000 (19:13 -0700)]
web: make sure that PUT /uri?mutable=false really means immutable, fixes #675
Brian Warner [Tue, 7 Apr 2009 21:15:14 +0000 (14:15 -0700)]
NEWS: format some (but not all) items
Zooko O'Whielacronx [Tue, 7 Apr 2009 18:54:59 +0000 (11:54 -0700)]
wui: edit some of the human-readable parts of the wui such as button labels
(The word "parent" suggests that you can go up a directory hierarchy -- perhaps that word is vestigial.)
Zooko O'Whielacronx [Tue, 7 Apr 2009 18:28:34 +0000 (11:28 -0700)]
wui: fix bug in which empty directory is marked as "unreadable", add test, remove exclamation point
Brian Warner [Tue, 7 Apr 2009 02:11:55 +0000 (19:11 -0700)]
NEWS: add unformatted list of code changes since previous release
Brian Warner [Tue, 7 Apr 2009 02:11:35 +0000 (19:11 -0700)]
docs: modify how-to-relase notes a tiny bit
Brian Warner [Sat, 4 Apr 2009 04:35:27 +0000 (21:35 -0700)]
hashtree.py: another micro-optimization, expand on a comment
Zooko O'Whielacronx [Fri, 3 Apr 2009 23:56:14 +0000 (16:56 -0700)]
hashtree: fix tests of depth_of
Zooko O'Whielacronx [Fri, 3 Apr 2009 23:33:15 +0000 (16:33 -0700)]
trivial: source code metadata
Zooko O'Whielacronx [Fri, 3 Apr 2009 23:33:02 +0000 (16:33 -0700)]
trivial: whitespace
Zooko O'Whielacronx [Fri, 3 Apr 2009 23:30:37 +0000 (16:30 -0700)]
hashtree: a couple of tiny improvements
* calculate depth-first with math instead of traversing the actual tree
* don't mark a node with a red dot if you instead compare it with an extant hash value (tiny optimization)
* edit a comment about checking the root node
Zooko O'Whielacronx [Fri, 3 Apr 2009 22:59:04 +0000 (15:59 -0700)]
leases, time_format: modify time stamping in lease description
* emit lease expiry date in ISO-8601'ish format as well as Brian's format
* rename iso_utc_time_to_localseconds() to iso_utc_time_to_seconds()
* add iso_utc_date()
* simplify the body of iso_utc_time_to_seconds()
Brian Warner [Tue, 31 Mar 2009 20:21:27 +0000 (13:21 -0700)]
hashtree: fix O(N**2) behavior, to improve fatal alacrity problems in a 10GB file (#670). Also improve docstring.
Zooko O'Whielacronx [Wed, 25 Mar 2009 03:54:59 +0000 (20:54 -0700)]
docs: setup: Norm Hardy suggested that it would be easier if users realized that they already had Python (especially true for Mac users)
Brian Warner [Tue, 24 Mar 2009 20:51:37 +0000 (13:51 -0700)]
expirer: include crawler progress in the JSON status output
Brian Warner [Tue, 24 Mar 2009 01:57:52 +0000 (18:57 -0700)]
docs/proposed: new Accounting overview, discuss in #666
Brian Warner [Tue, 24 Mar 2009 01:54:57 +0000 (18:54 -0700)]
docs/proposed: move old accounting docs out of the way
Brian Warner [Mon, 23 Mar 2009 23:08:20 +0000 (16:08 -0700)]
move GC docs out of proposed/, since it's all implemented now. Add reference to configuration.txt . Add expire.*= suggestions to tahoe.cfg .
Brian Warner [Mon, 23 Mar 2009 03:21:28 +0000 (20:21 -0700)]
storage: use constant-time comparison for write-enablers and lease-secrets
Brian Warner [Mon, 23 Mar 2009 03:20:55 +0000 (20:20 -0700)]
hashutil: add constant-time comparison function, to avoid timing attacks when python's short-circuiting data-dependent == operator is used to, say, check a write-enabler
Brian Warner [Fri, 20 Mar 2009 23:58:09 +0000 (16:58 -0700)]
WUI: fix display of empty directories, it threw an exception before
Brian Warner [Fri, 20 Mar 2009 22:44:50 +0000 (15:44 -0700)]
storage webstatus: insert spaces when we're configured to expire multiple sharetypes
Brian Warner [Fri, 20 Mar 2009 19:18:16 +0000 (12:18 -0700)]
expirer: tolerate empty buckets, refactor bucketsize-counting code a bit, don't increment -mutable/-immutable counters unless we actually know the sharetype
Brian Warner [Thu, 19 Mar 2009 18:48:37 +0000 (11:48 -0700)]
storage: improve wording of status message
Brian Warner [Thu, 19 Mar 2009 18:44:43 +0000 (11:44 -0700)]
parse_date: insist that it returns an int
Brian Warner [Thu, 19 Mar 2009 18:07:56 +0000 (11:07 -0700)]
storage status: report expiration-cutoff-date like 19-Mar-2009 (as opposed to the tahoe.cfg input format of 2009-03-19), for redundancy: someone who gets the month and day switched will have a better chance to spot the problem in the storage-status output if it's in a different format
Brian Warner [Thu, 19 Mar 2009 01:00:09 +0000 (18:00 -0700)]
expirer: clean up constructor args, add tahoe.cfg controls, use cutoff_date instead of date_cutoff
Brian Warner [Thu, 19 Mar 2009 00:58:14 +0000 (17:58 -0700)]
util/time_format: new routine to parse dates like 2009-03-18, switch expirer to use it. I'd prefer to use 18-Mar-2009, but it is surprisingly non-trivial to build a parser that will take UTC dates instead of local dates
Brian Warner [Thu, 19 Mar 2009 00:21:38 +0000 (17:21 -0700)]
expirer: change setup, config options, in preparation for adding tahoe.cfg controls
Brian Warner [Wed, 18 Mar 2009 20:25:04 +0000 (13:25 -0700)]
expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison
Brian Warner [Tue, 17 Mar 2009 07:01:17 +0000 (00:01 -0700)]
add utility function to parse durations, for lease-expiration config
Brian Warner [Tue, 17 Mar 2009 06:51:18 +0000 (23:51 -0700)]
expirer: add mode to expire only-mutable or only-immutable shares
Brian Warner [Tue, 17 Mar 2009 05:12:06 +0000 (22:12 -0700)]
tahoe_ls.py: add comment about error cases to improve
Brian Warner [Tue, 17 Mar 2009 05:10:41 +0000 (22:10 -0700)]
GC: add date-cutoff -based expiration, add proposed docs
Brian Warner [Mon, 16 Mar 2009 00:51:34 +0000 (17:51 -0700)]
test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly
Brian Warner [Sun, 15 Mar 2009 23:19:58 +0000 (16:19 -0700)]
tahoe cp -r: add --caps-only flag, to write filecaps into local files instead of actual file contents. Used only for debugging and as a quick tree-comparison tool.
Brian Warner [Fri, 13 Mar 2009 23:31:35 +0000 (16:31 -0700)]
dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237.
Brian Warner [Fri, 13 Mar 2009 08:27:51 +0000 (01:27 -0700)]
consolidate: remove pointless 'else' after for loop
Brian Warner [Fri, 13 Mar 2009 08:24:51 +0000 (01:24 -0700)]
consolidate: add eta, flush stdout
Brian Warner [Fri, 13 Mar 2009 06:54:02 +0000 (23:54 -0700)]
consolidate: tolerate unicode dirnames
Brian Warner [Fri, 13 Mar 2009 06:50:46 +0000 (23:50 -0700)]
dirnode.py: when doing deep-traverse, walk each directory in alphabetical order, to make things like 'manifest' more predictable
Brian Warner [Fri, 13 Mar 2009 05:47:28 +0000 (22:47 -0700)]
consolidator: add progress to scan-old-directory passes
Brian Warner [Fri, 13 Mar 2009 04:22:29 +0000 (21:22 -0700)]
consolidator: fix cycle detection to not trigger on merely shared directories, add snapshot counter to progress
Brian Warner [Fri, 13 Mar 2009 03:48:01 +0000 (20:48 -0700)]
consolidator: re-use more directories, add total directories seen-vs-used counts
Brian Warner [Fri, 13 Mar 2009 01:16:00 +0000 (18:16 -0700)]
tahoe_backup.py: tolerate more time formats
Brian Warner [Thu, 12 Mar 2009 23:29:00 +0000 (16:29 -0700)]
consolidator: add more verbose traversal of directories
Brian Warner [Thu, 12 Mar 2009 23:04:27 +0000 (16:04 -0700)]
consolidate: create multiple numbered backups of the original Archives directory, not just the first time
Brian Warner [Thu, 12 Mar 2009 20:56:06 +0000 (13:56 -0700)]
add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'.
Brian Warner [Thu, 12 Mar 2009 20:53:45 +0000 (13:53 -0700)]
tahoe_cp.py: return 0 for success, instead of None
Zooko O'Whielacronx [Wed, 11 Mar 2009 22:59:02 +0000 (15:59 -0700)]
setup: specify in the debian/control files that tahoe is compatible with Python 2.6
Zooko O'Whielacronx [Wed, 11 Mar 2009 15:29:52 +0000 (08:29 -0700)]
setup: update the debian/copyright text to reflect the current licences
Brian Warner [Mon, 9 Mar 2009 20:42:17 +0000 (13:42 -0700)]
expirer: fix prediction math, thanks to Zandr for the catch
Brian Warner [Mon, 9 Mar 2009 03:55:16 +0000 (20:55 -0700)]
storage.expirer: oops, fix upgrade-handler code
Brian Warner [Mon, 9 Mar 2009 03:42:20 +0000 (20:42 -0700)]
storage.expirer: handle upgrades better
Brian Warner [Mon, 9 Mar 2009 03:38:28 +0000 (20:38 -0700)]
storage.expirer: exercise the last missing line of webstatus code
Brian Warner [Mon, 9 Mar 2009 03:08:40 +0000 (20:08 -0700)]
expirer: tolerate corrupt shares, add them to the state and history for future examination
Brian Warner [Mon, 9 Mar 2009 03:07:32 +0000 (20:07 -0700)]
storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols.
Brian Warner [Mon, 9 Mar 2009 03:05:27 +0000 (20:05 -0700)]
storage: add test coverage for BucketReader.__repr__
Brian Warner [Mon, 9 Mar 2009 02:02:01 +0000 (19:02 -0700)]
storage/mutable: raise a specific error upon seeing bad magic, instead of using assert
Zooko O'Whielacronx [Sun, 8 Mar 2009 03:50:39 +0000 (20:50 -0700)]
nodeadmin: node stops itself if a hotline file hasn't been touched in 120 seconds now, instead of in 60 seconds
A test failed on draco (MacPPC) because it took 67.1 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 60 seconds old.
Brian Warner [Sat, 7 Mar 2009 23:14:42 +0000 (16:14 -0700)]
expirer: make web display a bit more consistent
Brian Warner [Sat, 7 Mar 2009 23:02:43 +0000 (16:02 -0700)]
web/storage.py: tolerate unknown-future displays, I'm not sure why LeaseCrawler.test_unpredictable_future didn't catch this
Brian Warner [Sat, 7 Mar 2009 12:40:51 +0000 (05:40 -0700)]
tahoe_cp.py: improve error reporting slightly: don't json-interpret HTTP errors, pass through tahoe webapi error messages
Brian Warner [Sat, 7 Mar 2009 12:08:15 +0000 (05:08 -0700)]
test_cli.py: assert that 'ls' on an unrecoverable file now gives a better error message
Brian Warner [Sat, 7 Mar 2009 11:57:07 +0000 (04:57 -0700)]
test_web: test behavior of broken-dirnode GET, both html and json
Brian Warner [Sat, 7 Mar 2009 11:56:01 +0000 (04:56 -0700)]
web: when a dirnode can't be read, emit a regular HTML page but with the child-table and upload-forms replaced with an apologetic message. Make sure to include the 'get info' links so the user can do a filecheck
Brian Warner [Sat, 7 Mar 2009 11:54:08 +0000 (04:54 -0700)]
web/common: split out exception-to-explanation+code mapping to a separate humanize_failure() function, so it can be used by other code. Add explanation for mutable UnrecoverableFileError.
Brian Warner [Sat, 7 Mar 2009 10:04:28 +0000 (03:04 -0700)]
test_cli: use explicit (and stable) testdirs, instead of using self.mktemp
Brian Warner [Sat, 7 Mar 2009 09:45:18 +0000 (02:45 -0700)]
test_storage: solaris doesn't appear to give the same block count as other platforms, so don't assert as much about 'diskbytes' recovered
Brian Warner [Sat, 7 Mar 2009 09:44:21 +0000 (02:44 -0700)]
tahoe_cp.py: don't be verbose unless --verbose is passed: default is one line of success