Brian Warner [Thu, 22 May 2008 00:35:21 +0000 (17:35 -0700)]
tahoe_cp: rewrite, make --recursive work
Brian Warner [Thu, 22 May 2008 00:34:52 +0000 (17:34 -0700)]
tahoe_ls: improve error message when the target is missing
Brian Warner [Thu, 22 May 2008 00:31:19 +0000 (17:31 -0700)]
scripts/common_http.py: add comment about why we need .seek
Brian Warner [Wed, 21 May 2008 19:09:13 +0000 (12:09 -0700)]
tahoe_cp.py: fix pyflakes complaint
Brian Warner [Wed, 21 May 2008 18:49:22 +0000 (11:49 -0700)]
cli: initial implementation of 'cp -r', probably doesn't work yet
Brian Warner [Wed, 21 May 2008 16:43:49 +0000 (09:43 -0700)]
util.dictutil: move DictOfSets out to a separate utility module
Brian Warner [Tue, 20 May 2008 23:56:03 +0000 (16:56 -0700)]
CLI: implement the easy part of cp (no -r, only two arguments)
Brian Warner [Tue, 20 May 2008 22:40:49 +0000 (15:40 -0700)]
web t=json: add 'mutable' key to the information dict
Brian Warner [Tue, 20 May 2008 22:21:46 +0000 (15:21 -0700)]
test_web: improve test coverage of web.common utility code
Brian Warner [Tue, 20 May 2008 22:14:19 +0000 (15:14 -0700)]
web: fix JSON output for mutable files
Brian Warner [Tue, 20 May 2008 21:36:04 +0000 (14:36 -0700)]
CLI: add 'list-aliases', factor out get_aliases
Brian Warner [Tue, 20 May 2008 20:30:31 +0000 (13:30 -0700)]
CLI: add 'ln', just like move but without the delete
Brian Warner [Tue, 20 May 2008 20:20:48 +0000 (13:20 -0700)]
test_system.py: remove dead code
Brian Warner [Tue, 20 May 2008 20:07:50 +0000 (13:07 -0700)]
tahoe_ls: improve output formatting
Brian Warner [Tue, 20 May 2008 19:49:47 +0000 (12:49 -0700)]
CLI mv: if we can't created the new child, don't delete the old one
Brian Warner [Tue, 20 May 2008 19:49:11 +0000 (12:49 -0700)]
CLI ls: add --readonly-uri to display readonly URIs for all children
Brian Warner [Tue, 20 May 2008 19:36:55 +0000 (12:36 -0700)]
CLI: add put --mutable, enhance ls to show mutable vs immutable as rw/r-
Brian Warner [Tue, 20 May 2008 19:36:02 +0000 (12:36 -0700)]
web: handle PUT mutable=true properly
Brian Warner [Tue, 20 May 2008 18:47:43 +0000 (11:47 -0700)]
test_web: add HEAD coverage
Brian Warner [Tue, 20 May 2008 18:35:47 +0000 (11:35 -0700)]
web: more mutable-file coverage
Brian Warner [Tue, 20 May 2008 18:33:14 +0000 (11:33 -0700)]
web: even more test coverage
Brian Warner [Tue, 20 May 2008 18:13:12 +0000 (11:13 -0700)]
web: improve test coverage, remove some dead code
Brian Warner [Tue, 20 May 2008 06:28:52 +0000 (23:28 -0700)]
test_web/test_system: improve test coverage
Brian Warner [Tue, 20 May 2008 03:24:12 +0000 (20:24 -0700)]
test_system: fix windows test failure, open files with binary mode
Brian Warner [Tue, 20 May 2008 02:28:50 +0000 (19:28 -0700)]
overhaul CLI: not quite complete but it works a lot better than it used to. The new scheme uses 'tahoe add-alias' and rsync/scp-style 'alias:foo/bar.txt' arguments
Brian Warner [Tue, 20 May 2008 01:52:19 +0000 (18:52 -0700)]
test/common: oops, forgot the FakeMutableFileNode.get_readonly fix
Brian Warner [Tue, 20 May 2008 01:38:39 +0000 (18:38 -0700)]
web: improve test coverage
Brian Warner [Tue, 20 May 2008 01:38:28 +0000 (18:38 -0700)]
web/filenode: remove unreachable code
Brian Warner [Tue, 20 May 2008 01:37:28 +0000 (18:37 -0700)]
web/directory: fix rw_uri output in t=JSON to reflect mutable files properly
Brian Warner [Mon, 19 May 2008 22:19:25 +0000 (15:19 -0700)]
web: factor out identical renderHTTP methods
Brian Warner [Mon, 19 May 2008 21:34:13 +0000 (14:34 -0700)]
webish: censor URLs that start with /file and /named too
Brian Warner [Mon, 19 May 2008 21:28:39 +0000 (14:28 -0700)]
test_web: remove leftover import to hush pyflakes
Brian Warner [Mon, 19 May 2008 21:09:31 +0000 (14:09 -0700)]
test_system: match webapi change, making new files now returns 201 Created, not 200 OK
Brian Warner [Mon, 19 May 2008 20:03:00 +0000 (13:03 -0700)]
filenode: add is_mutable to non-MutableFileNode classes
Brian Warner [Mon, 19 May 2008 19:57:04 +0000 (12:57 -0700)]
webish: complete rewrite, break into smaller pieces, auto-create directories, improve error handling
Brian Warner [Mon, 19 May 2008 19:56:02 +0000 (12:56 -0700)]
test_web: implement API changes from the recent webapi.txt overhaul
Brian Warner [Mon, 19 May 2008 19:47:46 +0000 (12:47 -0700)]
webapi.txt: overhaul documentation. API changes are as follows:
* download/upload localdir=/localfile= has been removed. This sort of ambient
authority was unsafe to expose over the web (CSRF), and at some point
soon we'll have 'cp -r' in the CLI to replace it.
* GET save=filename -> GET filename=filename&save=true
* GET t=download removed
* side-effect causing operations now use POST where appropriate, not PUT
* to create multiple directories, either use
* POST /uri/DIRCAP/parent?t=mkdir&name=child (more form/browser oriented)
* POST /uri/DIRCAP/parent/child?t=mkdir (more machine oriented)
The t=mkdir-p form is still accepted, but not preferred (since it leaks
the child name queryarg into the logs)
* use PUT /uri/MUTABLEFILECAP or PUT /uri/DIRCAP/child (on a mutable file) to
replace its contents, or POST /same?t=upload from forms
* response bodies and codes are better specified than before
Brian Warner [Mon, 19 May 2008 19:33:39 +0000 (12:33 -0700)]
test_web.py: minor cleanups, improved error reporting
Brian Warner [Mon, 19 May 2008 19:32:09 +0000 (12:32 -0700)]
test_web.py: localdir=/localfile= is going away, so remove the tests that exercise it
Brian Warner [Mon, 19 May 2008 21:24:41 +0000 (14:24 -0700)]
debug dump-share: print size of encprivkey and signature too
Brian Warner [Mon, 19 May 2008 21:16:02 +0000 (14:16 -0700)]
servermap.py: add size_of_version(), to retrieve the size of a mutable file version
Peter Secor [Mon, 19 May 2008 20:00:29 +0000 (13:00 -0700)]
native client - Mike updated build to allow for AllmydataTray to stop/start the Manager process
Peter Secor [Mon, 19 May 2008 19:47:31 +0000 (12:47 -0700)]
windows installer - fix quoting on start/stop of Allmydata Manager and update version #
Peter Secor [Mon, 19 May 2008 19:08:05 +0000 (12:08 -0700)]
windows installer - added startup line for AllmydataManager, removal of the service at uninstall
Peter Secor [Sat, 17 May 2008 01:57:31 +0000 (18:57 -0700)]
windows installer - trying to get the new Allmydata Manager service to start automatically
Peter Secor [Sat, 17 May 2008 01:12:57 +0000 (18:12 -0700)]
windows installer - changed to update for Mikes recent changes to add AllmydataManager which uses a named pipe to avoid some permissions issues on Vista and fixes some netowkr problems
Peter Secor [Sat, 17 May 2008 01:04:05 +0000 (18:04 -0700)]
native client - Mikes changes to have a manager to start/stop services with a named pipe, limit the size of the logfile to 5mb, install the loopback adapter, still need to make changes to the installer to automatically start them up
Brian Warner [Fri, 16 May 2008 23:09:47 +0000 (16:09 -0700)]
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
Brian Warner [Fri, 16 May 2008 23:08:38 +0000 (16:08 -0700)]
testutil.shouldFail: mention the 'which' string in substring failures too
Brian Warner [Wed, 14 May 2008 21:32:21 +0000 (14:32 -0700)]
webish: add /file links, change directory page to use them. This fixes filenames in wget. Closes #221.
Brian Warner [Wed, 14 May 2008 20:55:47 +0000 (13:55 -0700)]
introweb.py: tolerate foolscap>=0.2.6, which changed the internals of .locationHints
Brian Warner [Wed, 14 May 2008 20:10:41 +0000 (13:10 -0700)]
test_mutable.py: test replacing a file that has one new outlier share present: closes #272
Brian Warner [Sat, 10 May 2008 01:06:29 +0000 (18:06 -0700)]
CLI.txt: document proposed scp:-based CLI syntax
Brian Warner [Fri, 9 May 2008 19:36:19 +0000 (12:36 -0700)]
docs/CLI.txt: provide an overview of bin/tahoe subcommands
Brian Warner [Fri, 9 May 2008 01:02:55 +0000 (18:02 -0700)]
use a weakref cache in the client to manage singleton filenodes/dirnodes, to avoid autocollision. Should close #391.
Brian Warner [Fri, 9 May 2008 01:01:31 +0000 (18:01 -0700)]
check_memory: oops, fix HTTP-based modes: trailing slash in node.url vs no slash
Brian Warner [Thu, 8 May 2008 23:53:35 +0000 (16:53 -0700)]
test_dirnode: improve coverage of not-mutable-error a bit
Brian Warner [Thu, 8 May 2008 23:49:27 +0000 (16:49 -0700)]
check_memory: don't use fixed webports, they cause occasional test failures
Brian Warner [Thu, 8 May 2008 23:19:42 +0000 (16:19 -0700)]
deep-stats: add file-size histogram
Brian Warner [Thu, 8 May 2008 20:42:42 +0000 (13:42 -0700)]
test_dirnode: the size of the directories varies, tolerate it as low as 500 bytes
Brian Warner [Thu, 8 May 2008 20:33:07 +0000 (13:33 -0700)]
dirnode: refactor deep-stats a bit
Brian Warner [Thu, 8 May 2008 20:21:14 +0000 (13:21 -0700)]
dirnode: add a deep_stats(), like deep-size but with more information. webish adds t=deeps-size too.
Brian Warner [Thu, 8 May 2008 18:37:30 +0000 (11:37 -0700)]
run a stats provider even if there's no gatherer, since the HTTP /statistics page is then useful. Only run the once-per-second load-monitor if there is a gatherer configured
Brian Warner [Thu, 8 May 2008 01:36:37 +0000 (18:36 -0700)]
dirnode: use the concurrency limiter in t=manifest and t=deep-size, allow 10 retrievals in parallel
Brian Warner [Thu, 8 May 2008 01:35:53 +0000 (18:35 -0700)]
limiter: fix bug that ignores the limit= argument
robk-tahoe [Wed, 7 May 2008 23:42:20 +0000 (16:42 -0700)]
tahoefuse: fix typo which could cause data corruption
a typo in the 'flags2mode' code would wind up passing the O_APPEND
flag into the os open() call, which would cause the file to be opened
in 'strict append' mode, i.e. all writes extend the file, regardless of
calls to seek.
this causes a problem for tahoefuse in that the seek() calls made to
filehandles open through fuse would be ignored when write()s occurred.
this was evidenced by corruption seen when using rsync. it turns out
that rsync actually makes overlapping writes in some cases, i.e. even
when writing a new fresh file out, it still doesn't write a simple
contiguous span of data, but will make writes overlapping data already
written. this is probably related to the way it manages data blocks
internally for rolling checksums etc. at any rate, this bug would
thus cause rsync in those cases to write a chunk of duplicate data
into the file - leading to tahoe securely and reliably storing the
wrong data.
fixing this, so that non-append file opens do not pass O_APPEND seems
to eliminate this problem.
robk-tahoe [Wed, 7 May 2008 23:40:09 +0000 (16:40 -0700)]
tahoefuse: return bogus but useful data to statfs call
previously tahoefuse returned the fs stat for the filesystem the fuse plugin
was running upon (e.g. '/'). this works ok until you need to copy more to
tahoe than the local machine has free disk space, at which point Finder will
refuse to copy 'too much' data.
this changes it so that tahoe always reports 2TiB used of an 8TiB filesystem
this is entirely bogus, but allows copies of up to 2TiB to be initiated.
Brian Warner [Thu, 8 May 2008 00:00:47 +0000 (17:00 -0700)]
test_util: improve ConcurrencyLimiter tests
Brian Warner [Wed, 7 May 2008 23:53:30 +0000 (16:53 -0700)]
add a basic concurrency limiter utility
Brian Warner [Wed, 7 May 2008 23:52:48 +0000 (16:52 -0700)]
boringfile: add Twisted .egg files
Brian Warner [Wed, 7 May 2008 20:53:58 +0000 (13:53 -0700)]
web status: abbreviate file size in the upload+download summary page
Brian Warner [Wed, 7 May 2008 20:19:08 +0000 (13:19 -0700)]
munin: add tahoe_cpu_watcher.py, to track the data from misc/cpu-watcher.tac
Brian Warner [Wed, 7 May 2008 19:34:29 +0000 (12:34 -0700)]
misc/cpu-watcher*: add some tools to monitor CPU usage of arbitrary processes, like tahoe nodes
Zooko O'Whielacronx [Wed, 7 May 2008 15:39:03 +0000 (08:39 -0700)]
docs: fix a few stale comments in code
Zooko O'Whielacronx [Wed, 7 May 2008 12:37:11 +0000 (05:37 -0700)]
setup: tiny fix to syntax in makefile
Brian Warner [Tue, 6 May 2008 22:59:06 +0000 (15:59 -0700)]
configuration.txt: describe helper config
Zooko O'Whielacronx [Tue, 6 May 2008 22:29:04 +0000 (15:29 -0700)]
docs: shorter running.html
Brian Warner [Tue, 6 May 2008 20:49:01 +0000 (13:49 -0700)]
docs/helper.txt: explain more about the helper
Zooko O'Whielacronx [Tue, 6 May 2008 20:39:35 +0000 (13:39 -0700)]
docs: mention configuration, suggested by ben hyde's question about storage servers
Zooko O'Whielacronx [Tue, 6 May 2008 19:31:15 +0000 (12:31 -0700)]
docs: edit to install.html suggested by Brian
Zooko O'Whielacronx [Tue, 6 May 2008 19:30:56 +0000 (12:30 -0700)]
setup: trivial formatting change in _auto_deps.py
Zooko O'Whielacronx [Tue, 6 May 2008 19:09:00 +0000 (12:09 -0700)]
setup: remove specific checks for twisted dependency in makefile
Now that the twisted dependency is handled by the automatic dependency mechanism.
Zooko O'Whielacronx [Tue, 6 May 2008 18:17:47 +0000 (11:17 -0700)]
setup: require pycryptopp >= v0.5
Zooko O'Whielacronx [Tue, 6 May 2008 15:24:46 +0000 (08:24 -0700)]
setup: remove bundled pycryptopp-0.3.0.tar
Zooko O'Whielacronx [Tue, 6 May 2008 15:23:36 +0000 (08:23 -0700)]
setup: bundle pycryptopp-0.5.1.tar
pycryptopp-0.3.0 incorrectly crypts AES (depending on compiler and version of Crypto++ used)
Very soon now we're going to set up an "ext" repository to hold all tarballs and no longer check them into our trunk source tree.
Brian Warner [Wed, 30 Apr 2008 18:52:31 +0000 (11:52 -0700)]
test_stats.py: improve test coverage
Brian Warner [Wed, 30 Apr 2008 18:39:13 +0000 (11:39 -0700)]
stats: add tests for CPUUsageMonitor, modify it a bit to facilitate testing
Zooko O'Whielacronx [Wed, 30 Apr 2008 20:22:04 +0000 (13:22 -0700)]
setup: don't catch ImportError when importing _auto_deps in allmydata/__init__.py
Nowadays pkg_resources is a runtime requirement, and if there is something screwed up in the installation, we want an explicit ImportError exception as early as possible.
Brian Warner [Wed, 30 Apr 2008 01:24:57 +0000 (18:24 -0700)]
test_mutable: update notify_publish() to match new signature
Brian Warner [Wed, 30 Apr 2008 01:20:05 +0000 (18:20 -0700)]
mutable stats: track mutable bytes published too
Brian Warner [Wed, 30 Apr 2008 01:12:53 +0000 (18:12 -0700)]
stats: add CPU-percentage monitor, with 1min/5min/15min moving-window averages, using time.clock()
Zooko O'Whielacronx [Tue, 29 Apr 2008 22:51:58 +0000 (15:51 -0700)]
docs: remove the redundant (and therefore bit-rotting) parts of mutable-DSA.txt and instead refer to mutable.txt
Zooko O'Whielacronx [Tue, 29 Apr 2008 22:10:14 +0000 (15:10 -0700)]
wui: reorganize the welcome.xhtml page
Jake Edge tried Tahoe out and didn't notice the /status page. Hopefully with this new organization people like he will see that link more easily. This also addresses drewp's suggestion that the controls appear above the list of servers instead of below. (I think that was his suggestion.) I also reordered the controls.
Brian Warner [Mon, 28 Apr 2008 20:27:55 +0000 (13:27 -0700)]
offloaded: add 'resumes' counter to stats: how many upload requests are resuming an interrupted upload?
Zooko O'Whielacronx [Mon, 28 Apr 2008 14:05:44 +0000 (07:05 -0700)]
contrib: add a note about Armin Rigo's fuse implementation
ben [Mon, 28 Apr 2008 07:41:40 +0000 (00:41 -0700)]
support freebsd 6
Brian Warner [Thu, 24 Apr 2008 23:01:04 +0000 (16:01 -0700)]
debian: add python-setuptools to the debian install-time dependencies. Should close #382.
Brian Warner [Thu, 24 Apr 2008 18:30:38 +0000 (11:30 -0700)]
Makefile: add quicktest-figleaf: this is in my edit-test-repeat loop, and I need it to be fast
Brian Warner [Thu, 24 Apr 2008 18:28:35 +0000 (11:28 -0700)]
munin/tahoe_estimate_files.py: tool to estimate the total number of slots (mutable and immutable combined) in the grid, from a small sample
Zooko O'Whielacronx [Thu, 24 Apr 2008 17:29:17 +0000 (10:29 -0700)]
setup: pyOpenSSL is now easy_installable, and pycryptopp now includes Crypto++, so we can remove those two from the Manual Dependencies