David-Sarah Hopwood [Mon, 18 Feb 2013 22:33:30 +0000 (22:33 +0000)]
openstack_container.py: factor out HTTP response code checking.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Fri, 15 Feb 2013 23:05:20 +0000 (23:05 +0000)]
docs/backends/cloud.rst: add documentation for OpenStack config parameters.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Fri, 15 Feb 2013 20:59:06 +0000 (20:59 +0000)]
openstack_container.py: improve name of _auth_lock; simplify by using DeferredLock.run.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Fri, 15 Feb 2013 03:13:13 +0000 (03:13 +0000)]
OpenStack: change provider names to rackspace.com and rackspace.co.uk.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 14 Feb 2013 22:30:51 +0000 (22:30 +0000)]
test_storage.py: add tests for OpenStackContainer.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 14 Feb 2013 22:30:04 +0000 (22:30 +0000)]
openstack_container.py: disable or remove debug prints.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 14 Feb 2013 22:29:17 +0000 (22:29 +0000)]
openstack_container.py: fix a bug in type of ContainerListing.is_truncated.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 14 Feb 2013 22:24:08 +0000 (22:24 +0000)]
Move classes common to mock and OpenStack cloud services, to cloud_common.py.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 14 Feb 2013 22:19:34 +0000 (22:19 +0000)]
test_storage.py: refactor OpenStackCloudBackend to make it easier to add new tests.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 13 Feb 2013 22:08:46 +0000 (22:08 +0000)]
OpenStack: mostly complete implementation of OpenStackContainer.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 13 Feb 2013 22:07:55 +0000 (22:07 +0000)]
OpenStack: improve logging in openstack_container.py.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 12 Feb 2013 01:58:51 +0000 (01:58 +0000)]
OpenStack: add openstack.container config parameter.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 12 Feb 2013 01:44:43 +0000 (01:44 +0000)]
Add test for OpenStack authentication client.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Mon, 11 Feb 2013 02:43:36 +0000 (02:43 +0000)]
openstack_container.py: add shutdown() to avoid unclean reactor errors in tests.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Mon, 11 Feb 2013 02:42:23 +0000 (02:42 +0000)]
openstack_container.py: add _ prefix to private attributes.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 5 Feb 2013 19:58:32 +0000 (19:58 +0000)]
setup.py: add allmydata.storage.backends.cloud.openstack module.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 5 Feb 2013 19:42:17 +0000 (19:42 +0000)]
test_client.py: add OpenStack config tests.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 5 Feb 2013 19:42:01 +0000 (19:42 +0000)]
test_client.py: cleanups to S3 config tests.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 5 Feb 2013 19:41:31 +0000 (19:41 +0000)]
OpenStack service: add AuthenticationClient.
Configure using properties relevant to OpenStack.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 29 Jan 2013 23:17:31 +0000 (23:17 +0000)]
Add stub OpenStack container impl. as a copy of S3 container impl.
Generalize the container instantiation to work for either.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 20 Feb 2013 06:25:32 +0000 (06:25 +0000)]
Fix interface violations introduced in cloud merge.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Mon, 15 Apr 2013 19:50:40 +0000 (20:50 +0100)]
Use "PRAGMA synchronous = OFF" for leasedb.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 07:19:47 +0000 (07:19 +0000)]
Makefile: have 'make tmpfstest' unmount and remove stale temp directories.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 07:18:55 +0000 (07:18 +0000)]
Makefile: the timing for 'make tmpfstest' should exclude filesystem
mounting/unmounting and entering the password if needed.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 20:27:09 +0000 (21:27 +0100)]
Makefile: allow tmpfs size to be more easily overridden, and use 500 MiB by default.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 07:12:05 +0000 (07:12 +0000)]
test_runner.py: add test for 'tahoe debug trial'.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 20:10:06 +0000 (21:10 +0100)]
Fixes to tests. Some tests are applied to multiple backends.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 19:39:20 +0000 (20:39 +0100)]
Fixes to test infrastructure.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 19:37:35 +0000 (20:37 +0100)]
Rename 'delete_all_shares' to 'empty_sharedir' to avoid name clash.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 07:04:31 +0000 (07:04 +0000)]
Miscellaneous corrections and additions.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 07:01:59 +0000 (07:01 +0000)]
Changes to crawler classes (ShareCrawler and AccountingCrawler).
Pass in a Clock to allow (in theory) deterministic testing, although this isn't used yet by tests.
Simplify the generic ShareCrawler code by not attempting to track state during processing
of a single prefix.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:40:37 +0000 (19:40 +0100)]
Changes to fileutil.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 06:54:02 +0000 (06:54 +0000)]
Fix bugs in Accountant.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 06:53:24 +0000 (06:53 +0000)]
Simplify Account.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:38:16 +0000 (19:38 +0100)]
Changes to debug.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:26:45 +0000 (19:26 +0100)]
Changes to node classes (Node, Client and StorageServer).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 8 Apr 2014 23:56:58 +0000 (00:56 +0100)]
Changes to Bucket{Reader,Writer} and disk backend (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 06:23:16 +0000 (06:23 +0000)]
Changes and additions to interface documentation.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Wed, 9 Apr 2014 00:33:33 +0000 (01:33 +0100)]
Add new files for cloud merge (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Thu, 22 Nov 2012 06:43:49 +0000 (06:43 +0000)]
Move BucketWriter and BucketReader to storage/bucket.py.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 22 Nov 2012 05:33:35 +0000 (05:33 +0000)]
Move code around and add new directories for cloud backend merge.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:22:15 +0000 (19:22 +0100)]
Add dependency on our fork of txAWS (0.2.1.post5).
This also makes our dependency on six unconditional, rather than only
depending on six when pyOpenSSL >= 0.14 is used.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 8 Apr 2014 23:46:57 +0000 (00:46 +0100)]
.gitignore: changes to facilitate cloud backend merge (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 8 Apr 2014 23:43:30 +0000 (00:43 +0100)]
Add documentation for each storage backend (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Sat, 15 Dec 2012 06:27:21 +0000 (06:27 +0000)]
Add test_leasedb.py.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sat, 15 Dec 2012 06:24:41 +0000 (06:24 +0000)]
leasedb: use a semantic primary key (storage_index, shnum, account_id), rather than an integer, for the leases table.
Take advantage of this to simplify add_or_renew_leases.
Fix a bug in add_starter_lease (which is not used yet) when a starter lease already exists.
Clean up leftover accesses to self._dirty.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 06:22:18 +0000 (06:22 +0000)]
Rename 'buckets' to 'sharesets' on storage status page.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:08:22 +0000 (19:08 +0100)]
Remove unused files storage/lease.py and storage/expirer.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:04:40 +0000 (19:04 +0100)]
Main leasedb changes (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 05:00:13 +0000 (05:00 +0000)]
Remove the 'original-*' and 'configured-*' lease crawler output that won't be supported by the accounting crawler.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 05:04:48 +0000 (05:04 +0000)]
Cosmetics.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 04:13:19 +0000 (04:13 +0000)]
test_storage.py: ss -> server for cases that will remain a server after the server/account split.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 02:52:40 +0000 (02:52 +0000)]
Asyncify crawlers. Note that this breaks tests for the LeaseCrawler
(which is going away, to be replaced by the AccountingCrawler).
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 02:19:14 +0000 (02:19 +0000)]
scripts/debug.py: remove display of lease information and secrets.
This version replaces the expiration field with '-' instead of '0', per Zooko's comments.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:00:57 +0000 (19:00 +0100)]
Remove the [storage]expire.{mutable,immutable} options.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 17:59:20 +0000 (18:59 +0100)]
Remove support for [storage]debug_discard option.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 05:17:11 +0000 (05:17 +0000)]
Add new files for leasedb.
Authors: Brian Warner <warner@lothar.com> and David-Sarah Hopwood
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sat, 17 Nov 2012 00:32:24 +0000 (00:32 +0000)]
Changes to specification of add_lease and renew_lease in RIStorageServer.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 30 Oct 2012 20:34:30 +0000 (20:34 +0000)]
docs/garbage-collection.rst: update text for leasedb changes.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Fri, 16 Nov 2012 23:26:41 +0000 (23:26 +0000)]
util/fileutil.py: add get_used_space. This version does not use FilePath.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 16:59:58 +0000 (17:59 +0100)]
backupdb.py: cosmetics
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 16:59:34 +0000 (17:59 +0100)]
Make backupdb use dbutil.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 16:57:49 +0000 (17:57 +0100)]
Add util/dbutil.py: open/create/update sqlite databases given some schema.
Author: Brian Warner <warner@lothar.com>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Fri, 23 Nov 2012 00:23:54 +0000 (00:23 +0000)]
Some useful Deferred utilities, originally from the cloud backend branch.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Mon, 13 Apr 2015 23:08:47 +0000 (00:08 +0100)]
Merge pull request #152 from vu3rdd/182-osx-packaging-7
strip the single quotes around $VERSION
Ramakrishnan Muthukrishnan [Sat, 11 Apr 2015 09:05:18 +0000 (14:35 +0530)]
strip the single quotes around $VERSION
Without stripping the single quotes, the generated pagkage filename
also has the undesired single quotes around the version string.
(eg: tahoe-lafs-'1.10.0.post309'-osx.pkg).
Daira Hopwood [Fri, 10 Apr 2015 22:42:57 +0000 (23:42 +0100)]
README.rst: change description to the new wording on tahoe-lafs.org (with "file store" instead of "filesystem").
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 2 Apr 2015 15:16:46 +0000 (16:16 +0100)]
Add discussion and rejection of Change Journals.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 2 Apr 2015 15:16:17 +0000 (16:16 +0100)]
Add link to otf-magic-folder-objective ticket query.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 2 Apr 2015 15:15:39 +0000 (16:15 +0100)]
Copyediting.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 2 Apr 2015 14:26:25 +0000 (15:26 +0100)]
Collaborative design from Daira and David.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Brian Warner [Tue, 31 Mar 2015 18:07:48 +0000 (11:07 -0700)]
Merge branch 'ftp-2394'
closes ticket:2394
Brian Warner [Tue, 31 Mar 2015 18:07:21 +0000 (11:07 -0700)]
typo
Brian Warner [Tue, 31 Mar 2015 16:41:55 +0000 (09:41 -0700)]
bump Twisted dep to 11.1.0, thus simplify IntishPermissions
Brian Warner [Tue, 31 Mar 2015 16:30:17 +0000 (09:30 -0700)]
test_ftp: fix comparison against integer
Brian Warner [Thu, 26 Mar 2015 01:23:33 +0000 (18:23 -0700)]
Fix ftp 'ls' to work with current Twisted-15.0.0
refs ticket:2394
It's kind of a hack, but Twisted changed the API and I couldn't find a
cleaner way to detect which form of "permissions" value the Twisted FTP
server wants.
I've manually tested it against 14.0.2 and 15.0.0.
Daira Hopwood [Tue, 31 Mar 2015 16:47:06 +0000 (17:47 +0100)]
Merge pull request #149 from warner/mutable-2034
use more servers for test_mutable.Update, to test #2034 properly
Brian Warner [Thu, 26 Mar 2015 01:49:37 +0000 (18:49 -0700)]
use more servers for test_mutable.Update, to test #2034 properly
Daira Hopwood [Tue, 24 Mar 2015 17:10:00 +0000 (17:10 +0000)]
Add a test, add missing imports. refs #2388
This tests ftpd, but not sftpd. Doing this sort of test on sftpd
requires the creation of a valid pubkey/privkey file pair, which is more
work than I want to do right now.
init_ftp/init_sftp were changed to interpret the configured
accounts.file as relative to the node's basedir, with
abspath_expanduser_unicode(accountfile, base=self.basedir).
This would happen naturally in a real node, since it os.chdir()s
to the basedir before doing anything. But tests don't do that.
Author: Brian Warner <warner@lothar.com>
Author: Daira Hopwood <daira@jacaranda.org>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 3 Mar 2015 20:06:35 +0000 (20:06 +0000)]
Ensure that path parameters to SFTPServer and FTPServer constructors are unicode. refs #2388
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 3 Mar 2015 20:04:57 +0000 (20:04 +0000)]
Add from_utf8_or_none and tests.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 24 Mar 2015 16:44:54 +0000 (16:44 +0000)]
Fix a missing import in test-osx-pkg.py. refs #2393
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 24 Mar 2015 16:42:27 +0000 (16:42 +0000)]
More careful quoting in build-osx-pkg.sh. refs #2393
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 24 Mar 2015 16:41:43 +0000 (16:41 +0000)]
Move computation of Tahoe version from Makefile to build-osx-pkg.sh. refs #2393
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 24 Mar 2015 16:11:30 +0000 (16:11 +0000)]
Merge pull request #146 from warner/versioned-osx-pkg
put a version string in the name of the OS-X .pkg file
Brian Warner [Sat, 21 Mar 2015 21:20:00 +0000 (14:20 -0700)]
create/test/upload tahoe-lafs-VERSION-osx.pkg, not tahoe-lafs-osx.pkg
Closes ticket:2393
Brian Warner [Sat, 21 Mar 2015 21:09:33 +0000 (14:09 -0700)]
move build-osx-pkg out to a separate shell script
Also remove the duplicate test-osx-pkg (i.e. don't run the test as part
of the build, and have the caller do it themselves later).
Brian Warner [Fri, 20 Mar 2015 17:52:48 +0000 (10:52 -0700)]
Makefile: add upload-osx-pkg rule
Brian Warner [Fri, 20 Mar 2015 01:59:16 +0000 (18:59 -0700)]
check_memory.py: announce 32-bit vs 64-bit
Brian Warner [Tue, 17 Mar 2015 18:53:30 +0000 (11:53 -0700)]
Merge branch '2329-2': fix "tahoe cp -r" exception
Brian Warner [Tue, 17 Mar 2015 18:16:06 +0000 (11:16 -0700)]
test_cli_cp.py: hush pyflakes
Brian Warner [Tue, 17 Mar 2015 18:13:57 +0000 (11:13 -0700)]
tahoe_cp: be consistent with "source.basename() is None" testing
Brian Warner [Tue, 17 Mar 2015 18:13:33 +0000 (11:13 -0700)]
tahoe_cp: minor cleanups, no behavior changes
Brian Warner [Tue, 17 Mar 2015 18:11:11 +0000 (11:11 -0700)]
tahoe_cp: make populate(recurse=) more explicit
Brian Warner [Tue, 17 Mar 2015 17:41:01 +0000 (10:41 -0700)]
test_cli_cp: test cleanups
* remove redundant test cases
* add more internal asserts
* clean up set usage
Brian Warner [Tue, 17 Mar 2015 16:59:34 +0000 (09:59 -0700)]
test_cli_cp: improve test cases
Brian Warner [Tue, 10 Mar 2015 16:17:17 +0000 (09:17 -0700)]
improve error message for slash-terminated non-directories
Brian Warner [Wed, 4 Mar 2015 02:20:03 +0000 (18:20 -0800)]
test_cli_cp.py: turn off debug mode, tests now assert for real
Brian Warner [Wed, 4 Mar 2015 02:19:58 +0000 (18:19 -0800)]
tahoe cp: overhaul target assignment, update tests
This substantially changes the internals of "tahoe cp", to behave in
accordance with the scheme developed in ticket:2329. test_cli_cp.py got
a large new test to exercise all the various combinations. This also
changes the set of error messages that "tahoe cp" can produce.
This modifies try_copy(), inserts a new implementation of
copy_things_to_directory() (and supporting methods), and fixes a few
bugs elsewhere.
fixes ticket:2329
Brian Warner [Wed, 4 Mar 2015 02:18:17 +0000 (18:18 -0800)]
tahoe_cp: delete copy_to_directory() code
This code will be replaced in the next commit with an entirely different
approach, and modifying it in a single commit would yield a completely
unreadable diff.