Itamar Turner-Trauring [Mon, 4 Mar 2013 15:53:45 +0000 (10:53 -0500)]
Tests for googlestorage_container.AuthenticationClient.
Author: Itamar Turner-Trauring <itamar@futurefoundries.com>
Itamar Turner-Trauring [Fri, 1 Mar 2013 18:58:10 +0000 (13:58 -0500)]
Sketch of working Google Cloud Storage authentication, with some demo code.
David-Sarah Hopwood [Tue, 26 Feb 2013 03:36:08 +0000 (03:36 +0000)]
docs/backends/cloud.rst: clarify how to get to API Access in the Rackspace console.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 26 Feb 2013 03:35:35 +0000 (03:35 +0000)]
OpenStack: fix a type error introduced by the fix to #1921.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 24 Feb 2013 04:08:23 +0000 (04:08 +0000)]
OpenStack: support HP Cloud Object Storage.
Also make the choice of auth protocol for Rackspace configurable via
openstack.provider, and change the reauth period to 11 hours.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Fri, 22 Feb 2013 03:59:48 +0000 (03:59 +0000)]
leasedb/accounting crawler: only treat stable shares as disappeared or unleased.
fixes #1921
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 21 Feb 2013 22:04:16 +0000 (22:04 +0000)]
Cloud backend: move potentially reusable HTTP request utilities to cloud_common.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 21 Feb 2013 22:17:06 +0000 (22:17 +0000)]
OpenStack: if we get a 401 Unauthorized response, reauthenticate immediately.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 21 Feb 2013 22:14:27 +0000 (22:14 +0000)]
cloud_common.py: generalize ContainerRetryMixin to allow the container class to specify what to retry.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 21 Feb 2013 21:37:54 +0000 (21:37 +0000)]
openstack_container.py: remove a superfluous argument to get_auth_info_locked.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 21 Feb 2013 01:59:48 +0000 (01:59 +0000)]
accounting_crawler.py: disable removing leasedb entries for disappeared shares.
This works around ticket #1921 for now.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 21 Feb 2013 01:58:56 +0000 (01:58 +0000)]
openstack_container.py: avoid logging secrets in request headers.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 20 Feb 2013 23:26:15 +0000 (23:26 +0000)]
docs: add references to OpenStack/cloud backend in configuration.rst and running.rst.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 20 Feb 2013 06:21:25 +0000 (06:21 +0000)]
OpenStack: generalize to support multiple auth protocols, and add V2 protocol.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 19 Feb 2013 18:47:14 +0000 (18:47 +0000)]
OpenStack: add _http_request helper.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Mon, 18 Feb 2013 22:33:30 +0000 (22:33 +0000)]
openstack_container.py: factor out HTTP response code checking.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Fri, 15 Feb 2013 23:05:20 +0000 (23:05 +0000)]
docs/backends/cloud.rst: add documentation for OpenStack config parameters.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Fri, 15 Feb 2013 20:59:06 +0000 (20:59 +0000)]
openstack_container.py: improve name of _auth_lock; simplify by using DeferredLock.run.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Fri, 15 Feb 2013 03:13:13 +0000 (03:13 +0000)]
OpenStack: change provider names to rackspace.com and rackspace.co.uk.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 14 Feb 2013 22:30:51 +0000 (22:30 +0000)]
test_storage.py: add tests for OpenStackContainer.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 14 Feb 2013 22:30:04 +0000 (22:30 +0000)]
openstack_container.py: disable or remove debug prints.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 14 Feb 2013 22:29:17 +0000 (22:29 +0000)]
openstack_container.py: fix a bug in type of ContainerListing.is_truncated.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 14 Feb 2013 22:24:08 +0000 (22:24 +0000)]
Move classes common to mock and OpenStack cloud services, to cloud_common.py.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 14 Feb 2013 22:19:34 +0000 (22:19 +0000)]
test_storage.py: refactor OpenStackCloudBackend to make it easier to add new tests.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 13 Feb 2013 22:08:46 +0000 (22:08 +0000)]
OpenStack: mostly complete implementation of OpenStackContainer.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 13 Feb 2013 22:07:55 +0000 (22:07 +0000)]
OpenStack: improve logging in openstack_container.py.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 12 Feb 2013 01:58:51 +0000 (01:58 +0000)]
OpenStack: add openstack.container config parameter.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 12 Feb 2013 01:44:43 +0000 (01:44 +0000)]
Add test for OpenStack authentication client.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Mon, 11 Feb 2013 02:43:36 +0000 (02:43 +0000)]
openstack_container.py: add shutdown() to avoid unclean reactor errors in tests.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Mon, 11 Feb 2013 02:42:23 +0000 (02:42 +0000)]
openstack_container.py: add _ prefix to private attributes.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 5 Feb 2013 19:58:32 +0000 (19:58 +0000)]
setup.py: add allmydata.storage.backends.cloud.openstack module.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 5 Feb 2013 19:42:17 +0000 (19:42 +0000)]
test_client.py: add OpenStack config tests.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 5 Feb 2013 19:42:01 +0000 (19:42 +0000)]
test_client.py: cleanups to S3 config tests.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 5 Feb 2013 19:41:31 +0000 (19:41 +0000)]
OpenStack service: add AuthenticationClient.
Configure using properties relevant to OpenStack.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 29 Jan 2013 23:17:31 +0000 (23:17 +0000)]
Add stub OpenStack container impl. as a copy of S3 container impl.
Generalize the container instantiation to work for either.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 20 Feb 2013 06:25:32 +0000 (06:25 +0000)]
Fix interface violations introduced in cloud merge.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Mon, 15 Apr 2013 19:50:40 +0000 (20:50 +0100)]
Make backupdb use dbutil.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 07:19:47 +0000 (07:19 +0000)]
Makefile: have 'make tmpfstest' unmount and remove stale temp directories.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 07:18:55 +0000 (07:18 +0000)]
Makefile: the timing for 'make tmpfstest' should exclude filesystem
mounting/unmounting and entering the password if needed.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 04:23:01 +0000 (05:23 +0100)]
Makefile: allow tmpfs size to be more easily overridden, and use 500 MiB by default (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 07:12:05 +0000 (07:12 +0000)]
test_runner.py: add test for 'tahoe debug trial'.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 04:21:17 +0000 (05:21 +0100)]
Fixes to tests. Some tests are applied to multiple backends.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 03:43:07 +0000 (04:43 +0100)]
Fixes to test infrastructure.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 07:04:31 +0000 (07:04 +0000)]
Miscellaneous corrections and additions.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 07:01:59 +0000 (07:01 +0000)]
Changes to crawler classes (ShareCrawler and AccountingCrawler).
Pass in a Clock to allow (in theory) deterministic testing, although this isn't used yet by tests.
Simplify the generic ShareCrawler code by not attempting to track state during processing
of a single prefix.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 03:35:27 +0000 (04:35 +0100)]
Changes to fileutil.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 06:54:02 +0000 (06:54 +0000)]
Fix bugs in Accountant.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 06:53:24 +0000 (06:53 +0000)]
Simplify Account.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 03:34:00 +0000 (04:34 +0100)]
Changes to debug.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 03:23:15 +0000 (04:23 +0100)]
Changes to node classes (Node, Client and StorageServer).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 8 Apr 2014 23:56:58 +0000 (00:56 +0100)]
Changes to Bucket{Reader,Writer} and disk backend (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 06:23:16 +0000 (06:23 +0000)]
Changes and additions to interface documentation.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Wed, 9 Apr 2014 00:33:33 +0000 (01:33 +0100)]
Add new files for cloud merge (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Thu, 22 Nov 2012 06:43:49 +0000 (06:43 +0000)]
Move BucketWriter and BucketReader to storage/bucket.py.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Thu, 22 Nov 2012 05:33:35 +0000 (05:33 +0000)]
Move code around and add new directories for cloud backend merge.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 03:20:08 +0000 (04:20 +0100)]
Add dependency on our fork of txAWS (0.2.1.post5).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 8 Apr 2014 23:46:57 +0000 (00:46 +0100)]
.gitignore: changes to facilitate cloud backend merge (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 8 Apr 2014 23:43:30 +0000 (00:43 +0100)]
Add documentation for each storage backend (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Sat, 15 Dec 2012 06:27:21 +0000 (06:27 +0000)]
Add test_leasedb.py.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sat, 15 Dec 2012 06:24:41 +0000 (06:24 +0000)]
leasedb: use a semantic primary key (storage_index, shnum, account_id), rather than an integer, for the leases table.
Take advantage of this to simplify add_or_renew_leases.
Fix a bug in add_starter_lease (which is not used yet) when a starter lease already exists.
Clean up leftover accesses to self._dirty.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 06:22:18 +0000 (06:22 +0000)]
Rename 'buckets' to 'sharesets' on storage status page.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 03:10:53 +0000 (04:10 +0100)]
Remove unused files storage/lease.py and storage/expirer.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 03:09:46 +0000 (04:09 +0100)]
Main leasedb changes (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 05:00:13 +0000 (05:00 +0000)]
Remove the 'original-*' and 'configured-*' lease crawler output that won't be supported by the accounting crawler.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 05:04:48 +0000 (05:04 +0000)]
Cosmetics.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 04:13:19 +0000 (04:13 +0000)]
test_storage.py: ss -> server for cases that will remain a server after the server/account split.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 02:52:40 +0000 (02:52 +0000)]
Asyncify crawlers. Note that this breaks tests for the LeaseCrawler
(which is going away, to be replaced by the AccountingCrawler).
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sun, 18 Nov 2012 02:19:14 +0000 (02:19 +0000)]
scripts/debug.py: remove display of lease information and secrets.
This version replaces the expiration field with '-' instead of '0', per Zooko's comments.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 03:06:08 +0000 (04:06 +0100)]
Remove the [storage]expire.{mutable,immutable} options.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 10 Jul 2015 03:04:58 +0000 (04:04 +0100)]
Remove support for [storage]debug_discard option.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
David-Sarah Hopwood [Wed, 12 Dec 2012 05:17:11 +0000 (05:17 +0000)]
Add new files for leasedb.
Authors: Brian Warner <warner@lothar.com> and David-Sarah Hopwood
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Sat, 17 Nov 2012 00:32:24 +0000 (00:32 +0000)]
Changes to specification of add_lease and renew_lease in RIStorageServer.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Tue, 30 Oct 2012 20:34:30 +0000 (20:34 +0000)]
docs/garbage-collection.rst: update text for leasedb changes.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
David-Sarah Hopwood [Fri, 16 Nov 2012 23:26:41 +0000 (23:26 +0000)]
util/fileutil.py: add get_used_space. This version does not use FilePath.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Brian Warner [Sat, 27 Jun 2015 01:39:29 +0000 (18:39 -0700)]
docs/how-to-release: add more wiki pages to update
Brian Warner [Tue, 16 Jun 2015 20:41:21 +0000 (13:41 -0700)]
add test-pip-install
Brian Warner [Tue, 16 Jun 2015 01:35:55 +0000 (18:35 -0700)]
release checklist: use signed git tag next time
Brian Warner [Tue, 16 Jun 2015 00:38:01 +0000 (17:38 -0700)]
README: update copyright date
Brian Warner [Mon, 15 Jun 2015 23:29:37 +0000 (16:29 -0700)]
update docs for 1.10.1 release
Brian Warner [Mon, 15 Jun 2015 21:40:19 +0000 (14:40 -0700)]
Dockerfile: git-pull more history, to get a version string
The Docker build process seems to use 'git pull --depth=1', which
doesn't fetch enough history to see a tag, which means tahoe's setup.py
can't compute a version (then 'tahoe --version' reports "unknown").
Pulling a hundred commits should be enough to see a tag without making
the resulting image unnecessarily large. If we go more than 100 commits
between tags, we'll need to increase this, or come up with something
smarter.
Daira Hopwood [Thu, 11 Jun 2015 20:33:09 +0000 (21:33 +0100)]
Finish user-interface.rst. fixes ticket:2443
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 4 Jun 2015 16:42:29 +0000 (17:42 +0100)]
Make a start on the Magic Folder Objective 5 (user interface design) doc. refs ticket:2443
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 9 Jun 2015 17:48:07 +0000 (18:48 +0100)]
NEWS.rst: fix missing link.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 9 Jun 2015 17:45:14 +0000 (18:45 +0100)]
NEWS.rst for 1.10.1: links, formatting and nitpicks.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Brian Warner [Tue, 9 Jun 2015 08:48:25 +0000 (01:48 -0700)]
NEWS: finish 1.10.1 edits. Good enough for beta1.
Brian Warner [Mon, 8 Jun 2015 22:59:35 +0000 (15:59 -0700)]
NEWS: rearrange
Brian Warner [Mon, 8 Jun 2015 22:58:48 +0000 (15:58 -0700)]
NEWS: remove many non-user-visible items
Brian Warner [Mon, 8 Jun 2015 22:55:23 +0000 (15:55 -0700)]
NEWS: first editing pass
Brian Warner [Mon, 8 Jun 2015 06:54:28 +0000 (23:54 -0700)]
NEWS: classify changes as user-visible or not
Brian Warner [Mon, 8 Jun 2015 05:03:19 +0000 (22:03 -0700)]
NEWS.rst: update to include latest changes
Brian Warner [Sun, 7 Jun 2015 21:30:58 +0000 (14:30 -0700)]
move Dockerfile back to root. DockerHub can't handle it elsewhere.
It looks like DockerHub's automated builds only have access to the
subtree of the source checkout at+below the Dockerfile. Putting the
Dockerfile in misc/build_helpers/ meant that the build process only had
access to misc/build_helpers/, not the full source tree.
Brian Warner [Sun, 7 Jun 2015 21:24:04 +0000 (14:24 -0700)]
Dockerfile: fix ADD path to deal with non-root location
Brian Warner [Sun, 7 Jun 2015 20:40:26 +0000 (13:40 -0700)]
move Dockerfile down ito misc/build_helpers/
Daira Hopwood [Sat, 6 Jun 2015 21:45:42 +0000 (22:45 +0100)]
Magic Folder design doc: rename "parent DMD" to "collective directory", and don't assume it is mutable.
(An immutable collective directory could be interesting for some use cases, and is no more difficult to support.)
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 2 Jun 2015 17:12:23 +0000 (18:12 +0100)]
Remove packages from not_import_versionable that we now can always version.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 2 Jun 2015 17:07:20 +0000 (18:07 +0100)]
Add OpenSSL version check and tests. refs ticket:2215
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 29 May 2015 21:10:33 +0000 (22:10 +0100)]
remote-to-local-sync.rst: fix a minor error.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 29 May 2015 02:03:46 +0000 (03:03 +0100)]
Magic Folder: add remote-to-local sync design doc.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Brian Warner [Tue, 26 May 2015 20:23:31 +0000 (13:23 -0700)]
Merge branch 'globalopts2'
Improves CLI rendering of --help for global options. Closes ticket:2233.
Brian Warner [Tue, 26 May 2015 18:32:14 +0000 (11:32 -0700)]
wrap long lines, and tolerate various-width wrappings of the --help output
test_cli.Help was too sensitive to the way that the --help output was
wrapped, which caused failures on travis when COLUMNS= was set low and
the expected strings were split across separate lines.