Kevan Carstensen [Sun, 7 Aug 2011 00:44:14 +0000 (17:44 -0700)]
test/test_mutable: tests for MDMF
These are their own patch because they cut across a lot of the changes
I've made in implementing MDMF in such a way as to make it difficult to
split them up into the other patches.
Kevan Carstensen [Sun, 7 Aug 2011 00:43:48 +0000 (17:43 -0700)]
webapi changes for MDMF
- Learn how to create MDMF files and directories through the
mutable-type argument.
- Operate with the interface changes associated with MDMF and #993.
- Learn how to do partial updates of mutable files.
Kevan Carstensen [Sun, 7 Aug 2011 00:42:59 +0000 (17:42 -0700)]
mutable/servermap: Rework the servermap to work with MDMF mutable files
Kevan Carstensen [Sun, 7 Aug 2011 00:42:24 +0000 (17:42 -0700)]
dirnode: teach dirnode to make MDMF directories
david-sarah [Tue, 2 Aug 2011 03:28:10 +0000 (20:28 -0700)]
Fix some test failures caused by #393 patch.
Kevan Carstensen [Tue, 2 Aug 2011 02:20:56 +0000 (19:20 -0700)]
docs: amend configuration, webapi documentation to talk about MDMF
Kevan Carstensen [Tue, 2 Aug 2011 02:16:13 +0000 (19:16 -0700)]
cli: teach CLI how to create MDMF mutable files
Specifically, 'tahoe mkdir' and 'tahoe put' now take a --mutable-type
argument.
Kevan Carstensen [Tue, 2 Aug 2011 02:12:07 +0000 (19:12 -0700)]
frontends/sftpd: Resolve incompatibilities between SFTP frontend and MDMF changes
Kevan Carstensen [Tue, 2 Aug 2011 02:11:20 +0000 (19:11 -0700)]
mutable/layout: Define MDMF share format, write tools for working with MDMF share format
The changes in layout.py are mostly concerned with the MDMF share
format. In particular, we define read and write proxy objects used by
retrieval, publishing, and other code to write and read the MDMF share
format. We create equivalent proxies for SDMF objects so that these
objects can be suitably general.
Kevan Carstensen [Tue, 2 Aug 2011 02:09:05 +0000 (19:09 -0700)]
immutable/filenode: implement unified filenode interface
Kevan Carstensen [Tue, 2 Aug 2011 02:08:14 +0000 (19:08 -0700)]
immutable/literal.py: Implement interface changes in literal nodes.
Kevan Carstensen [Tue, 2 Aug 2011 01:56:43 +0000 (18:56 -0700)]
test/common: Alter common test code to work with MDMF.
This mostly has to do with making the test code implement the new
unified filenode interfaces.
Kevan Carstensen [Tue, 2 Aug 2011 01:51:40 +0000 (18:51 -0700)]
mutable: train checker and repairer to work with MDMF mutable files
Kevan Carstensen [Tue, 2 Aug 2011 01:49:26 +0000 (18:49 -0700)]
nodemaker: teach nodemaker about MDMF caps
Kevan Carstensen [Tue, 2 Aug 2011 01:48:11 +0000 (18:48 -0700)]
client: teach client how to create and work with MDMF files
Kevan Carstensen [Tue, 2 Aug 2011 01:45:01 +0000 (18:45 -0700)]
mutable/filenode: Modify mutable filenodes for use with MDMF
In particular:
- Break MutableFileNode and MutableFileVersion into distinct classes.
- Implement the interface modifications made for MDMF.
- Be aware of MDMF caps.
- Learn how to create and work with MDMF files.
Kevan Carstensen [Tue, 2 Aug 2011 01:42:58 +0000 (18:42 -0700)]
nodemaker: teach nodemaker how to create MDMF mutable files
Kevan Carstensen [Tue, 2 Aug 2011 01:41:19 +0000 (18:41 -0700)]
interfaces: change interfaces to work with MDMF
A lot of this work concerns #993, in that it unifies (to an extent) the
interfaces of mutable and immutable files.
Kevan Carstensen [Tue, 2 Aug 2011 01:39:31 +0000 (18:39 -0700)]
mutable/publish: teach the publisher how to publish MDMF mutable files
Like the downloader, the publisher needs some substantial changes to handle multiple segment mutable files.
Kevan Carstensen [Tue, 2 Aug 2011 01:35:24 +0000 (18:35 -0700)]
mutable/retrieve: rework the mutable downloader to handle multiple-segment files
The downloader needs substantial reworking to handle multiple segment
mutable files, which it needs to handle for MDMF.
david-sarah [Fri, 19 Aug 2011 20:48:36 +0000 (13:48 -0700)]
Fix repeated 'the' in license text.
david-sarah [Fri, 19 Aug 2011 20:47:42 +0000 (13:47 -0700)]
Remove Non-Profit Open Software License from the set of 'added permission' licenses. Although it actually does qualify as an Open Source license (because it allows relicensing under plain OSL), its wording is unclear and could easily be misunderstood, and it contributes to incompatible license proliferation.
Zooko O'Whielacronx [Fri, 19 Aug 2011 06:01:42 +0000 (23:01 -0700)]
docs: change links that pointed to COPYING.TGPPL.html to point to COPYING.TGPPL.rst instead
Zooko O'Whielacronx [Fri, 19 Aug 2011 06:01:10 +0000 (23:01 -0700)]
docs: formatting: reflow to fill-column 77
Zooko O'Whielacronx [Fri, 19 Aug 2011 06:00:41 +0000 (23:00 -0700)]
docs: formatting: M-x whitespace-cleanup
Zooko O'Whielacronx [Fri, 19 Aug 2011 05:46:56 +0000 (22:46 -0700)]
licensing: add to the list of licenses that we grant the added permission for
Added: most of the ones listed on the FSF's "List of Free Software, GPL Incompatible Licenses", plus the Non-Profit Open Software License.
Zooko O'Whielacronx [Fri, 19 Aug 2011 05:30:59 +0000 (22:30 -0700)]
docs: reflow the added text at the top of COPYING.GPL to fill-column 77
Zooko O'Whielacronx [Fri, 19 Aug 2011 05:27:53 +0000 (22:27 -0700)]
docs: reformat COPYING.TGPPL.html to COPYING.TGPPL.rst
Zooko O'Whielacronx [Fri, 19 Aug 2011 04:41:03 +0000 (21:41 -0700)]
docs: reflow docs/logging.rst to fill-column 77
Zooko O'Whielacronx [Fri, 19 Aug 2011 04:39:46 +0000 (21:39 -0700)]
doc: fix formatting error in docs/logging.rst
david-sarah [Thu, 18 Aug 2011 04:07:49 +0000 (21:07 -0700)]
Cleanups for suppression of UserWarnings. refs #1435
Zooko O'Whielacronx [Wed, 17 Aug 2011 20:31:34 +0000 (13:31 -0700)]
suppress warning emitted by newer zope.interface with Nevow 0.10
refs #1435
Zooko O'Whielacronx [Tue, 9 Aug 2011 13:25:10 +0000 (06:25 -0700)]
doc: formatting: reflow to fill-column=77
david-sarah [Mon, 15 Aug 2011 02:53:47 +0000 (19:53 -0700)]
_auto_deps.py: change the requirement for zope.interface to <= 3.6.2, >= 3.6.6. fixes #1435
david-sarah [Mon, 15 Aug 2011 03:51:53 +0000 (20:51 -0700)]
allmydata/__init__.py, test_version.py: make version parsing understand '<=', with test. refs #1435
david-sarah [Mon, 1 Aug 2011 03:19:52 +0000 (20:19 -0700)]
Makefile and setup.py: remove setup.py commands that we no longer need, and their uses in the Makefile. Delete a stale and incorrect comment about updating _version.py. Also fix some coding style checks in the Makefile to operate on all source files.
Brian Warner [Thu, 11 Aug 2011 18:27:05 +0000 (11:27 -0700)]
remove misc/debian[_helpers], rely upon official packaging instead. fixes #1454
david-sarah [Mon, 1 Aug 2011 03:18:57 +0000 (20:18 -0700)]
Makefile: remove targets that used misc/debian[_helpers] which no longer exist. Also change docs/debian.rst to reflect the fact that we no longer support building .debs using those targets. refs #1454
Brian Warner [Thu, 11 Aug 2011 17:37:04 +0000 (10:37 -0700)]
replace tabs with spaces in the #1441 'tahoe debug' synopsis
david-sarah [Sun, 24 Jul 2011 16:25:30 +0000 (09:25 -0700)]
Correct the information printed by '/usr/bin/tahoe debug --help' on Debian/Ubuntu. fixes #1441
Zooko O'Whielacronx [Thu, 4 Aug 2011 19:34:09 +0000 (12:34 -0700)]
doc: edit the explanation of K-of-N tradeoffs
Zooko O'Whielacronx [Thu, 4 Aug 2011 19:27:22 +0000 (12:27 -0700)]
doc: clean up formatting of doc/configuration.rst
reflow to 77 chars line width, M-x white-space cleanup, blank link between name and definition
david-sarah [Sun, 24 Jul 2011 21:16:59 +0000 (14:16 -0700)]
Add test for webopen. fixes #1149
david-sarah [Wed, 10 Aug 2011 05:25:04 +0000 (22:25 -0700)]
test_client.py: relax a check in test_create_drop_uploader so that it should pass on Python 2.4.x. refs #1429
david-sarah [Wed, 10 Aug 2011 05:19:06 +0000 (22:19 -0700)]
test/common_util.py: correct fix to mkdir_nonascii. refs #1472
david-sarah [Wed, 10 Aug 2011 04:42:35 +0000 (21:42 -0700)]
test/common_util.py: fix a typo. refs #1472
david-sarah [Wed, 10 Aug 2011 03:45:05 +0000 (20:45 -0700)]
test_client.py, test_drop_upload.py: fix pyflakes warnings.
david-sarah [Wed, 10 Aug 2011 03:15:58 +0000 (20:15 -0700)]
Factor out methods dealing with non-ASCII directories and filenames from test_drop_upload.py into common_util.py. refs #1429, #1472
david-sarah [Wed, 10 Aug 2011 03:05:38 +0000 (20:05 -0700)]
test_client.py: add a test that the drop-uploader is initialized correctly by client.py. Also give the DropUploader service a name, which is necessary for the test. refs #1429
david-sarah [Wed, 10 Aug 2011 03:03:45 +0000 (20:03 -0700)]
drop-upload: rename 'start' method to 'startService', which is what you're supposed to use to start a Service. refs #1429
david-sarah [Wed, 10 Aug 2011 02:59:42 +0000 (19:59 -0700)]
test_drop_upload.py: add comment explaining why we don't use FilePath.setContent. refs #1429
david-sarah [Tue, 9 Aug 2011 22:12:31 +0000 (15:12 -0700)]
test_drop_upload.py: fix some grammatical and spelling nits. refs #1429
Zooko O'Whielacronx [Tue, 9 Aug 2011 22:09:30 +0000 (15:09 -0700)]
drop-upload: report the configured local directory being absent differently from it being a file
refs #1429
Zooko O'Whielacronx [Tue, 9 Aug 2011 22:05:08 +0000 (15:05 -0700)]
drop-upload: rename the 'upload.uri' parameter to 'upload.dircap', and a couple of cleanups to error messages. refs #1429
I rerecorded this patch, originally by David-Sarah, to use "darcs replace" instead of editing to do the renames. This uncovered one missed rename in Client.init_drop_uploader. (Which also means that code isn't exercised by the current unit tests.)
refs #1429
Zooko O'Whielacronx [Tue, 9 Aug 2011 22:01:15 +0000 (15:01 -0700)]
drop-upload test for non-existent local dir separately from test for non-directory local dir
A candidate patch for #1429 has a bug when it is using FilePath.is_dir() to detect whether the configured local dir exists and is a directory. FilePath.is_dir() raises exception, instead of returning False, if the thing doesn't exist. This test is to make sure that DropUploader.__init__ raise different exceptions for those two cases.
refs #1429
Zooko O'Whielacronx [Tue, 9 Aug 2011 21:59:13 +0000 (14:59 -0700)]
drop-upload: unit tests for the configuration options being named "cap" instead of "uri"
This is a subset of a patch that David-Sarah attached to #1429. This is just the unit-tests part of that patch, and uses darcs record instead of hunks to change the names.
refs #1429
david-sarah [Tue, 19 Jul 2011 02:27:52 +0000 (19:27 -0700)]
src/allmydata/storage/server.py: use the filesystem of storage/shares/, rather than storage/, to calculate remaining space. fixes #1384
david-sarah [Tue, 9 Aug 2011 19:07:22 +0000 (12:07 -0700)]
test_storage.py: test that we are using the filesystem of storage/shares/, rather than storage/, to calculate remaining space, and that the HTML status output reflects the values returned by fileutil.get_disk_stats. This version works with older versions of the mock library. refs #1384
david-sarah [Tue, 9 Aug 2011 01:23:34 +0000 (18:23 -0700)]
Work around ref #1472 by having test_drop_upload delete the non-ASCII directories it creates.
david-sarah [Tue, 9 Aug 2011 00:11:17 +0000 (17:11 -0700)]
Remove all trailing whitespace from .py files.
david-sarah [Mon, 8 Aug 2011 23:54:22 +0000 (16:54 -0700)]
test_drop_upload.py: fix unused imports. refs #1429
david-sarah [Mon, 8 Aug 2011 18:21:46 +0000 (11:21 -0700)]
Documentation for drop-upload frontend. refs #1429
david-sarah [Mon, 8 Aug 2011 23:40:49 +0000 (16:40 -0700)]
Drop-upload frontend, rerecorded for 1.9 beta (and correcting a minor mistake). Includes some fixes for Windows but not the Windows inotify implementation. fixes #1429
david-sarah [Sat, 6 Aug 2011 22:19:34 +0000 (15:19 -0700)]
node.py: ensure that client and introducer nodes record their port number and use that port on the next restart, fixing a regression caused by #1385. fixes #1469.
david-sarah [Sat, 6 Aug 2011 23:18:42 +0000 (16:18 -0700)]
test_runner.py: fix a race condition in the test when NODE_URL_FILE is written before PORTNUM_FILE. refs #1469
david-sarah [Sat, 6 Aug 2011 23:16:52 +0000 (16:16 -0700)]
test_runner.py: cleanups of HOTLINE_FILE writing and removal.
david-sarah [Sat, 6 Aug 2011 22:14:16 +0000 (15:14 -0700)]
test_runner.py: remove an unused constant.
david-sarah [Sat, 6 Aug 2011 22:10:07 +0000 (15:10 -0700)]
node.py: fix the error path for a missing config option so that it works for a Unicode base directory.
david-sarah [Sat, 6 Aug 2011 22:06:35 +0000 (15:06 -0700)]
test_runner.py: test that client and introducer nodes record their port number and use that port on the next restart. This tests for a regression caused by ref #1385.
david-sarah [Thu, 4 Aug 2011 00:30:32 +0000 (17:30 -0700)]
test_runner.py: fix a bug in CreateNode.do_create introduced in changeset [5114] when the tahoe.cfg file has been written with CRLF line endings. refs #1385
david-sarah [Wed, 3 Aug 2011 23:50:36 +0000 (16:50 -0700)]
test_client.py: repair Basic.test_error_on_old_config_files. refs #1385
david-sarah [Wed, 3 Aug 2011 21:40:42 +0000 (14:40 -0700)]
test_checker.py: increase timeout for TooParallel.test_immutable again. The ARM buildslave took 38 seconds, so 40 seconds is too close to the edge; make it 80.
david-sarah [Wed, 3 Aug 2011 18:09:17 +0000 (11:09 -0700)]
test_runner.py: fix RunNode.test_introducer to not rely on the mtime of introducer.furl to detect when the node has restarted. Instead we detect when node.url has been written. refs #1385
david-sarah [Wed, 3 Aug 2011 17:45:46 +0000 (10:45 -0700)]
Further improve error message about old config files. refs #1385
david-sarah [Wed, 3 Aug 2011 16:38:48 +0000 (09:38 -0700)]
Slightly improve error message about old config files (avoid unnecessary Unicode escaping). refs #1385
david-sarah [Wed, 3 Aug 2011 16:32:13 +0000 (09:32 -0700)]
test_checker.py: increase timeout for TooParallel.test_immutable (was consistently failing on ARM buildslave).
david-sarah [Wed, 3 Aug 2011 01:32:12 +0000 (18:32 -0700)]
Fix the bug that prevents an introducer from starting when introducer.furl already exists. Also remove some dead code that used to read old config files, and rename 'warn_about_old_config_files' to reflect that it's not a warning. refs #1385
david-sarah [Wed, 3 Aug 2011 01:27:04 +0000 (18:27 -0700)]
test_runner.py: modify RunNode.test_introducer to test that starting an introducer works when the introducer.furl file already exists. refs #1385
david-sarah [Tue, 2 Aug 2011 17:24:37 +0000 (10:24 -0700)]
verifier: correct a bug introduced in changeset [5106] that caused us to only verify the first block of a file. refs #1395
david-sarah [Tue, 2 Aug 2011 17:58:41 +0000 (10:58 -0700)]
test_repairer: add a deterministic test of share data corruption that always flips the bits of the last byte of the share data. refs #1395
Zooko O'Whielacronx [Tue, 2 Aug 2011 06:37:03 +0000 (23:37 -0700)]
verifier: serialize the fetching of blocks within a share so that we don't use too much RAM
Shares are still verified in parallel, but within a share, don't request a
block until the previous block has been verified and the memory we used to hold
it has been freed up.
Patch originally due to Brian. This version has a mockery-patchery-style test
which is "low tech" (it implements the patching inline in the test code instead
of using an extension of the mock.patch() function from the mock library) and
which unpatches in case of exception.
fixes #1395
Brian Warner [Tue, 2 Aug 2011 04:45:41 +0000 (21:45 -0700)]
add docs about timing-channel attacks
Brian Warner [Mon, 7 Feb 2011 17:25:34 +0000 (09:25 -0800)]
'test-coverage' now needs PYTHONPATH=. to find TOP/twisted/plugins/
Brian Warner [Mon, 1 Aug 2011 22:43:17 +0000 (15:43 -0700)]
remove nodeid from WriteBucketProxy classes and customers
refs #1363
Brian Warner [Mon, 1 Aug 2011 22:43:07 +0000 (15:43 -0700)]
remove get_serverid() from ReadBucketProxy and customers, including Checker
and debug.py dump-share commands
refs #1363
Zooko O'Whielacronx [Mon, 1 Aug 2011 23:24:23 +0000 (16:24 -0700)]
reject old-style (pre-Tahoe-LAFS-v1.3) configuration files
Check for the existence of any of them and if any are found raise exception which will abort the startup of the node.
This is a backwards-incompatible change for anyone who is still using old-style configuration files.
fixes #1385
Zooko O'Whielacronx [Mon, 25 Jul 2011 01:55:46 +0000 (18:55 -0700)]
whitespace-cleanup
Zooko O'Whielacronx [Thu, 31 Mar 2011 14:54:27 +0000 (07:54 -0700)]
tests: use fileutil.write() instead of open() to ensure timely close even without CPython-style reference counting
Some of these already had an explicit close() but I went ahead and replaced them with fileutil.write() as well for the sake of uniformity.
david-sarah [Mon, 1 Aug 2011 22:13:17 +0000 (15:13 -0700)]
Address Kevan's comment in #776 about Options classes missed when adding 'self.command_name'. refs #776, #1359
david-sarah [Mon, 1 Aug 2011 22:09:19 +0000 (15:09 -0700)]
docs/frontends/webapi.rst: change some more instances of 'delete' or 'remove' to 'unlink', change some section titles, and use two blank lines between all sections. refs #776, #1104
david-sarah [Mon, 1 Aug 2011 22:01:08 +0000 (15:01 -0700)]
cleanup: implement rm as a synonym for unlink rather than vice-versa. refs #776
david-sarah [Mon, 1 Aug 2011 20:53:56 +0000 (13:53 -0700)]
docs/webapi.rst: address Kevan's comments about use of 'delete' on ref #1104
david-sarah [Wed, 13 Jul 2011 00:27:22 +0000 (17:27 -0700)]
docs: some changes of 'delete' or 'rm' to 'unlink'. refs #1104
david-sarah [Wed, 13 Jul 2011 00:12:18 +0000 (17:12 -0700)]
WUI: change the label of the button to unlink a file from 'del' to 'unlink'. Also change some internal names to 'unlink', and allow 't=unlink' as a synonym for 't=delete' in the web-API interface. Incidentally, improve a test to check for the rename button as well as the unlink button. fixes #1104
david-sarah [Mon, 1 Aug 2011 20:30:09 +0000 (13:30 -0700)]
src/allmydata/web/filenode.py: delete a stale comment that was made incorrect by changeset [3133].
Brian Warner [Mon, 1 Aug 2011 20:03:41 +0000 (13:03 -0700)]
fix typo introduced during rebasing of 'remove get_serverid from
DownloadStatus.add_dyhb_request and customers' patch, to fix test failure.
Zooko O'Whielacronx [Mon, 1 Aug 2011 18:54:01 +0000 (11:54 -0700)]
remove get_serverid from DownloadStatus.add_dyhb_request and customers
This patch is a rebase of a patch originally written by Brian. I didn't change any of the intent of Brian's patch, just ported it to current trunk.
refs #1363
Zooko O'Whielacronx [Mon, 1 Aug 2011 18:53:44 +0000 (11:53 -0700)]
remove get_serverid from DownloadStatus.add_block_request and customers
This is a rebase of a patch Brian originally wrote. I haven't changed the intent of that patch, just ported it to trunk.
refs #1363
Brian Warner [Mon, 1 Aug 2011 17:44:52 +0000 (10:44 -0700)]
apply zooko's advice: storage_client get_known_servers() returns a frozenset, caller sorts
refs #1363
Brian Warner [Mon, 1 Aug 2011 17:44:44 +0000 (10:44 -0700)]
test_immutable.Test: rewrite to use NoNetworkGrid, now takes 2.7s not 97s
remove now-unused ShareManglingMixin
refs #1363