Daira Hopwood [Thu, 22 Nov 2012 05:33:35 +0000 (05:33 +0000)]
Move code around and add new directories for cloud backend merge.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:22:15 +0000 (19:22 +0100)]
Add dependency on our fork of txAWS (0.2.1.post5).
This also makes our dependency on six unconditional, rather than only
depending on six when pyOpenSSL >= 0.14 is used.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 8 Apr 2014 23:46:57 +0000 (00:46 +0100)]
.gitignore: changes to facilitate cloud backend merge (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 8 Apr 2014 23:43:30 +0000 (00:43 +0100)]
Add documentation for each storage backend (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Sat, 15 Dec 2012 06:27:21 +0000 (06:27 +0000)]
Add test_leasedb.py.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Sat, 15 Dec 2012 06:24:41 +0000 (06:24 +0000)]
leasedb: use a semantic primary key (storage_index, shnum, account_id), rather than an integer, for the leases table.
Take advantage of this to simplify add_or_renew_leases.
Fix a bug in add_starter_lease (which is not used yet) when a starter lease already exists.
Clean up leftover accesses to self._dirty.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Sun, 18 Nov 2012 06:22:18 +0000 (06:22 +0000)]
Rename 'buckets' to 'sharesets' on storage status page.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:08:22 +0000 (19:08 +0100)]
Remove unused files storage/lease.py and storage/expirer.py.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:04:40 +0000 (19:04 +0100)]
Main leasedb changes (rebased).
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Sun, 18 Nov 2012 05:00:13 +0000 (05:00 +0000)]
Remove the 'original-*' and 'configured-*' lease crawler output that won't be supported by the accounting crawler.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Sun, 18 Nov 2012 05:04:48 +0000 (05:04 +0000)]
Cosmetics.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Sun, 18 Nov 2012 04:13:19 +0000 (04:13 +0000)]
test_storage.py: ss -> server for cases that will remain a server after the server/account split.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Sun, 18 Nov 2012 02:52:40 +0000 (02:52 +0000)]
Asyncify crawlers. Note that this breaks tests for the LeaseCrawler
(which is going away, to be replaced by the AccountingCrawler).
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Sun, 18 Nov 2012 02:19:14 +0000 (02:19 +0000)]
scripts/debug.py: remove display of lease information and secrets.
This version replaces the expiration field with '-' instead of '0', per Zooko's comments.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 18:00:57 +0000 (19:00 +0100)]
Remove the [storage]expire.{mutable,immutable} options.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 17:59:20 +0000 (18:59 +0100)]
Remove support for [storage]debug_discard option.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Wed, 12 Dec 2012 05:17:11 +0000 (05:17 +0000)]
Add new files for leasedb.
Authors: Brian Warner <warner@lothar.com> and David-Sarah Hopwood
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Sat, 17 Nov 2012 00:32:24 +0000 (00:32 +0000)]
Changes to specification of add_lease and renew_lease in RIStorageServer.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Tue, 30 Oct 2012 20:34:30 +0000 (20:34 +0000)]
docs/garbage-collection.rst: update text for leasedb changes.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 16 Nov 2012 23:26:41 +0000 (23:26 +0000)]
util/fileutil.py: add get_used_space. This version does not use FilePath.
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 16:59:58 +0000 (17:59 +0100)]
backupdb.py: cosmetics
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 16:59:34 +0000 (17:59 +0100)]
Make backupdb use dbutil.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 17 Apr 2015 16:57:49 +0000 (17:57 +0100)]
Add util/dbutil.py: open/create/update sqlite databases given some schema.
Author: Brian Warner <warner@lothar.com>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 23 Nov 2012 00:23:54 +0000 (00:23 +0000)]
Some useful Deferred utilities, originally from the cloud backend branch.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Mon, 13 Apr 2015 23:08:47 +0000 (00:08 +0100)]
Merge pull request #152 from vu3rdd/182-osx-packaging-7
strip the single quotes around $VERSION
Ramakrishnan Muthukrishnan [Sat, 11 Apr 2015 09:05:18 +0000 (14:35 +0530)]
strip the single quotes around $VERSION
Without stripping the single quotes, the generated pagkage filename
also has the undesired single quotes around the version string.
(eg: tahoe-lafs-'1.10.0.post309'-osx.pkg).
Daira Hopwood [Fri, 10 Apr 2015 22:42:57 +0000 (23:42 +0100)]
README.rst: change description to the new wording on tahoe-lafs.org (with "file store" instead of "filesystem").
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 2 Apr 2015 15:16:46 +0000 (16:16 +0100)]
Add discussion and rejection of Change Journals.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 2 Apr 2015 15:16:17 +0000 (16:16 +0100)]
Add link to otf-magic-folder-objective ticket query.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 2 Apr 2015 15:15:39 +0000 (16:15 +0100)]
Copyediting.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 2 Apr 2015 14:26:25 +0000 (15:26 +0100)]
Collaborative design from Daira and David.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Brian Warner [Tue, 31 Mar 2015 18:07:48 +0000 (11:07 -0700)]
Merge branch 'ftp-2394'
closes ticket:2394
Brian Warner [Tue, 31 Mar 2015 18:07:21 +0000 (11:07 -0700)]
typo
Brian Warner [Tue, 31 Mar 2015 16:41:55 +0000 (09:41 -0700)]
bump Twisted dep to 11.1.0, thus simplify IntishPermissions
Brian Warner [Tue, 31 Mar 2015 16:30:17 +0000 (09:30 -0700)]
test_ftp: fix comparison against integer
Brian Warner [Thu, 26 Mar 2015 01:23:33 +0000 (18:23 -0700)]
Fix ftp 'ls' to work with current Twisted-15.0.0
refs ticket:2394
It's kind of a hack, but Twisted changed the API and I couldn't find a
cleaner way to detect which form of "permissions" value the Twisted FTP
server wants.
I've manually tested it against 14.0.2 and 15.0.0.
Daira Hopwood [Tue, 31 Mar 2015 16:47:06 +0000 (17:47 +0100)]
Merge pull request #149 from warner/mutable-2034
use more servers for test_mutable.Update, to test #2034 properly
Brian Warner [Thu, 26 Mar 2015 01:49:37 +0000 (18:49 -0700)]
use more servers for test_mutable.Update, to test #2034 properly
Daira Hopwood [Tue, 24 Mar 2015 17:10:00 +0000 (17:10 +0000)]
Add a test, add missing imports. refs #2388
This tests ftpd, but not sftpd. Doing this sort of test on sftpd
requires the creation of a valid pubkey/privkey file pair, which is more
work than I want to do right now.
init_ftp/init_sftp were changed to interpret the configured
accounts.file as relative to the node's basedir, with
abspath_expanduser_unicode(accountfile, base=self.basedir).
This would happen naturally in a real node, since it os.chdir()s
to the basedir before doing anything. But tests don't do that.
Author: Brian Warner <warner@lothar.com>
Author: Daira Hopwood <daira@jacaranda.org>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 3 Mar 2015 20:06:35 +0000 (20:06 +0000)]
Ensure that path parameters to SFTPServer and FTPServer constructors are unicode. refs #2388
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 3 Mar 2015 20:04:57 +0000 (20:04 +0000)]
Add from_utf8_or_none and tests.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 24 Mar 2015 16:44:54 +0000 (16:44 +0000)]
Fix a missing import in test-osx-pkg.py. refs #2393
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 24 Mar 2015 16:42:27 +0000 (16:42 +0000)]
More careful quoting in build-osx-pkg.sh. refs #2393
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 24 Mar 2015 16:41:43 +0000 (16:41 +0000)]
Move computation of Tahoe version from Makefile to build-osx-pkg.sh. refs #2393
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 24 Mar 2015 16:11:30 +0000 (16:11 +0000)]
Merge pull request #146 from warner/versioned-osx-pkg
put a version string in the name of the OS-X .pkg file
Brian Warner [Sat, 21 Mar 2015 21:20:00 +0000 (14:20 -0700)]
create/test/upload tahoe-lafs-VERSION-osx.pkg, not tahoe-lafs-osx.pkg
Closes ticket:2393
Brian Warner [Sat, 21 Mar 2015 21:09:33 +0000 (14:09 -0700)]
move build-osx-pkg out to a separate shell script
Also remove the duplicate test-osx-pkg (i.e. don't run the test as part
of the build, and have the caller do it themselves later).
Brian Warner [Fri, 20 Mar 2015 17:52:48 +0000 (10:52 -0700)]
Makefile: add upload-osx-pkg rule
Brian Warner [Fri, 20 Mar 2015 01:59:16 +0000 (18:59 -0700)]
check_memory.py: announce 32-bit vs 64-bit
Brian Warner [Tue, 17 Mar 2015 18:53:30 +0000 (11:53 -0700)]
Merge branch '2329-2': fix "tahoe cp -r" exception
Brian Warner [Tue, 17 Mar 2015 18:16:06 +0000 (11:16 -0700)]
test_cli_cp.py: hush pyflakes
Brian Warner [Tue, 17 Mar 2015 18:13:57 +0000 (11:13 -0700)]
tahoe_cp: be consistent with "source.basename() is None" testing
Brian Warner [Tue, 17 Mar 2015 18:13:33 +0000 (11:13 -0700)]
tahoe_cp: minor cleanups, no behavior changes
Brian Warner [Tue, 17 Mar 2015 18:11:11 +0000 (11:11 -0700)]
tahoe_cp: make populate(recurse=) more explicit
Brian Warner [Tue, 17 Mar 2015 17:41:01 +0000 (10:41 -0700)]
test_cli_cp: test cleanups
* remove redundant test cases
* add more internal asserts
* clean up set usage
Brian Warner [Tue, 17 Mar 2015 16:59:34 +0000 (09:59 -0700)]
test_cli_cp: improve test cases
Brian Warner [Tue, 10 Mar 2015 16:17:17 +0000 (09:17 -0700)]
improve error message for slash-terminated non-directories
Brian Warner [Wed, 4 Mar 2015 02:20:03 +0000 (18:20 -0800)]
test_cli_cp.py: turn off debug mode, tests now assert for real
Brian Warner [Wed, 4 Mar 2015 02:19:58 +0000 (18:19 -0800)]
tahoe cp: overhaul target assignment, update tests
This substantially changes the internals of "tahoe cp", to behave in
accordance with the scheme developed in ticket:2329. test_cli_cp.py got
a large new test to exercise all the various combinations. This also
changes the set of error messages that "tahoe cp" can produce.
This modifies try_copy(), inserts a new implementation of
copy_things_to_directory() (and supporting methods), and fixes a few
bugs elsewhere.
fixes ticket:2329
Brian Warner [Wed, 4 Mar 2015 02:18:17 +0000 (18:18 -0800)]
tahoe_cp: delete copy_to_directory() code
This code will be replaced in the next commit with an entirely different
approach, and modifying it in a single commit would yield a completely
unreadable diff.
Brian Warner [Wed, 4 Mar 2015 01:51:28 +0000 (17:51 -0800)]
tahoe_cp: simplify some checks, rename some functions
no behavioral changes
Daira Hopwood [Fri, 27 Feb 2015 23:59:18 +0000 (23:59 +0000)]
.travis.yml: emove 'sudo: false' as requested by https://github.com/travis-ci/travis-ci/issues/2389#issuecomment-
76202191
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Brian Warner [Thu, 26 Feb 2015 06:04:25 +0000 (22:04 -0800)]
NEWS.rst: add partial list of changes
Zooko [Tue, 24 Feb 2015 18:11:30 +0000 (18:11 +0000)]
zetuptoolz: print out more context in case of parse_requirements error
Brian Warner [Tue, 24 Feb 2015 10:07:57 +0000 (02:07 -0800)]
tahoe_cp.py: use s.basename() instead of tracking it separately
Brian Warner [Tue, 24 Feb 2015 09:55:53 +0000 (01:55 -0800)]
tahoe_cp.py: store basename in the Source instance
Brian Warner [Mon, 23 Feb 2015 06:31:14 +0000 (22:31 -0800)]
test_cli.py: split the largest test classes out into separate files
That file was getting unruly. No behavioral changes, apart from tests
now having different names.
Daira Hopwood [Thu, 19 Feb 2015 17:41:09 +0000 (17:41 +0000)]
Discourage future readers of iputil.py from using Win32 APIs in preference to route.exe.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 17 Feb 2015 19:11:57 +0000 (19:11 +0000)]
It is Twisted 12.2.0 that starts depending on pywin32 for endpoints, not Twisted 12.3.0.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 17 Feb 2015 19:10:54 +0000 (19:10 +0000)]
Explicitly using "git.cmd" on Windows doesn't always work; use shell=True on Windows instead.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 10 Feb 2015 18:28:09 +0000 (18:28 +0000)]
Merge pull request #142 from tahoe-lafs/754.location-AUTO.2
Allow combining autodetected and statically configured locations
Daira Hopwood [Tue, 10 Feb 2015 18:23:59 +0000 (18:23 +0000)]
Documentation change for AUTO in tub.location. refs #754
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 10 Feb 2015 17:31:10 +0000 (17:31 +0000)]
Allow multiple occurrences of AUTO. refs #754
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Mon, 9 Feb 2015 19:48:06 +0000 (19:48 +0000)]
Add tests for combining autodetected and statically configured locations. refs #754
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Mon, 9 Feb 2015 19:45:31 +0000 (19:45 +0000)]
test_node.py: refactor test_location* to reduce code duplication. refs #754
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Mon, 9 Feb 2015 19:20:42 +0000 (19:20 +0000)]
Only get the local addresses if they are actually used. refs #754
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Mon, 9 Feb 2015 19:18:58 +0000 (19:18 +0000)]
Allow combining autodetected and statically configured locations. fixes #754
Replaces the location 'AUTO' with the autodetected IP/port combination.
Author: Chris Kerr <debdepba@dasganma.tk>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Sat, 7 Feb 2015 02:45:10 +0000 (02:45 +0000)]
.travis.yml: add "sudo: false" option, to attempt to resolve "Connection reset by peer" problems during build.
See https://github.com/travis-ci/travis-ci/issues/2389 .
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 6 Feb 2015 23:59:12 +0000 (23:59 +0000)]
Fix warnings due to bitrot in MANIFEST.in. closes #2380
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 5 Feb 2015 02:22:27 +0000 (02:22 +0000)]
Test the quotemarks=True and quotemarks=False options of quote_local_unicode_path.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 5 Feb 2015 02:13:23 +0000 (02:13 +0000)]
Test the quotemarks=False option of encodingutil.quote_path.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Wed, 4 Feb 2015 17:02:02 +0000 (17:02 +0000)]
Further refine error handling in windows_getenv to correctly handle zero-length environment values. refs #1674
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Brian Warner [Wed, 4 Feb 2015 09:35:44 +0000 (01:35 -0800)]
remove stray trailing whitespace
Brian Warner [Wed, 4 Feb 2015 09:31:46 +0000 (01:31 -0800)]
Merge branch 'pr138': fix long paths on windows
Brian Warner [Wed, 4 Feb 2015 06:09:40 +0000 (22:09 -0800)]
test that web.static= is really treated as a relative path
Also ssh.authorized_keys_file .
Daira Hopwood [Wed, 4 Feb 2015 00:10:21 +0000 (00:10 +0000)]
Quote the default node-directory correctly in help output. refs #2235
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Tue, 3 Feb 2015 23:47:31 +0000 (23:47 +0000)]
More robust error handling in windows_getenv. refs #1674
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Brian Warner [Tue, 3 Feb 2015 19:10:36 +0000 (11:10 -0800)]
tahoe_cp.py: clean up unicode handling
Daira Hopwood [Sat, 11 Oct 2014 22:45:13 +0000 (23:45 +0100)]
Changes filename to unicode before placing the file. refs #2027
Author: Mark Berger <mark.berger.j@gmail.com>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Sat, 11 Oct 2014 22:43:23 +0000 (23:43 +0100)]
Adds test_ticket_2027 to test_cli.Cp. refs #2027
Author: Mark Berger <mark.berger.j@gmail.com>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Thu, 29 Jan 2015 18:32:05 +0000 (18:32 +0000)]
Fix user-path-expansion on Windows for non-ASCII home directories. refs #1674
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 30 Jan 2015 00:50:18 +0000 (00:50 +0000)]
Change uses of os.path.expanduser and os.path.abspath. refs #2235
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Mon, 25 Aug 2014 18:09:40 +0000 (19:09 +0100)]
Use absolute paths in tahoe cp and tahoe backup. refs #2235
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 30 Jan 2015 00:47:09 +0000 (00:47 +0000)]
Add support in abspath_expanduser_unicode for expanding relative to a base path. refs #2235
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 30 Jan 2015 00:04:11 +0000 (00:04 +0000)]
Quote local paths correctly. refs #2235
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Fri, 30 Jan 2015 00:05:14 +0000 (00:05 +0000)]
Use "long" paths prefixed with \\?\ on Windows. refs #2235
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Daira Hopwood [Mon, 25 Aug 2014 17:24:59 +0000 (18:24 +0100)]
fileutil cleanup: delete the unused open_or_create function.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Mark Berger [Tue, 23 Jul 2013 16:53:22 +0000 (12:53 -0400)]
Adds _servers_with_shares to ServermapUpdater
When calculating the query boundary for updates to mutable files,
instead of using servers that used to have shares, use servers we
have added to the servermap. This way the querying process won't finish
until we have finished interacting with the servers that have shares.
This fixes the race condition which sometimes caused the querying process
to finish before the updater was done talking to servers with shares.
Brian Warner [Tue, 27 Jan 2015 06:05:38 +0000 (22:05 -0800)]
Revert "Enable DeprecationWarnings for Travis-CI even on Python 2.7. refs #2366"
This reverts commit
cec7727bf980f62ba5e98d11776edb658860e350.
It caused travis (for py2.7) to fail on these tests:
* allmydata.test.test_runner.BinTahoe.test_version_no_noise
* allmydata.test.test_runner.RunNode.test_client_no_noise
* allmydata.test.test_system.SystemTest.test_filesystem_with_cli_in_subprocess
Daira Hopwood [Thu, 22 Jan 2015 02:49:16 +0000 (02:49 +0000)]
Enable DeprecationWarnings for Travis-CI even on Python 2.7. refs #2366
Signed-off-by: Daira Hopwood <daira@jacaranda.org>