leasedb: use a semantic primary key (storage_index, shnum, account_id), rather than an integer, for the leases table.
Take advantage of this to simplify add_or_renew_leases.
Fix a bug in add_starter_lease (which is not used yet) when a starter lease already exists.
Clean up leftover accesses to self._dirty.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

Rename 'buckets' to 'sharesets' on storage status page.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

Remove unused files storage/lease.py and storage/expirer.py.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Main leasedb changes (rebased).

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Remove the 'original-*' and 'configured-*' lease crawler output that won't be supported by the accounting crawler.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

Cosmetics.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

test_storage.py: ss -> server for cases that will remain a server after the server/account split.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

Asyncify crawlers. Note that this breaks tests for the LeaseCrawler
(which is going away, to be replaced by the AccountingCrawler).

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

scripts/debug.py: remove display of lease information and secrets.
This version replaces the expiration field with '-' instead of '0', per Zooko's comments.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

Remove the [storage]expire.{mutable,immutable} options.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Remove support for [storage]debug_discard option.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Add new files for leasedb.

Authors: Brian Warner <warner@lothar.com> and David-Sarah Hopwood
Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

Changes to specification of add_lease and renew_lease in RIStorageServer.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

docs/garbage-collection.rst: update text for leasedb changes.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

util/fileutil.py: add get_used_space. This version does not use FilePath.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>

docs/how-to-release: add more wiki pages to update

add test-pip-install

release checklist: use signed git tag next time

README: update copyright date

update docs for 1.10.1 release

Dockerfile: git-pull more history, to get a version string

The Docker build process seems to use 'git pull --depth=1', which
doesn't fetch enough history to see a tag, which means tahoe's setup.py
can't compute a version (then 'tahoe --version' reports "unknown").
Pulling a hundred commits should be enough to see a tag without making
the resulting image unnecessarily large. If we go more than 100 commits
between tags, we'll need to increase this, or come up with something
smarter.

Finish user-interface.rst. fixes ticket:2443

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Make a start on the Magic Folder Objective 5 (user interface design) doc. refs ticket:2443

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

NEWS.rst: fix missing link.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

NEWS.rst for 1.10.1: links, formatting and nitpicks.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

NEWS: finish 1.10.1 edits. Good enough for beta1.

NEWS: rearrange

NEWS: remove many non-user-visible items

NEWS: first editing pass

NEWS: classify changes as user-visible or not

NEWS.rst: update to include latest changes

move Dockerfile back to root. DockerHub can't handle it elsewhere.

It looks like DockerHub's automated builds only have access to the
subtree of the source checkout at+below the Dockerfile. Putting the
Dockerfile in misc/build_helpers/ meant that the build process only had
access to misc/build_helpers/, not the full source tree.

Dockerfile: fix ADD path to deal with non-root location

move Dockerfile down ito misc/build_helpers/

Magic Folder design doc: rename "parent DMD" to "collective directory", and don't assume it is mutable.
(An immutable collective directory could be interesting for some use cases, and is no more difficult to support.)

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Remove packages from not_import_versionable that we now can always version.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Add OpenSSL version check and tests. refs ticket:2215

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

remote-to-local-sync.rst: fix a minor error.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Magic Folder: add remote-to-local sync design doc.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Merge branch 'globalopts2'

Improves CLI rendering of --help for global options. Closes ticket:2233.

wrap long lines, and tolerate various-width wrappings of the --help output

test_cli.Help was too sensitive to the way that the --help output was
wrapped, which caused failures on travis when COLUMNS= was set low and
the expected strings were split across separate lines.

cli: improve formatting of all commands

Also:

* do some light refactoring of create-client/node
* make it clear that these commands' --basedir options do the same as
  the global --node-directory option
* use "global-options" instead of "global-opts"

scripts: improve rendering of synopsis/usage

Subcommands "--help" is now rendered as:

```
 tahoe [global-options] COMMAND [options] ARGS
 (use 'tahoe --help' to view global options)
 USAGE (flags/options)
 DESCRIPTION
 DESCRIPTION_UNWRAPPED
```

The new .description and .description_unwrapped fields allow
commands (subclasses of twisted.python.usage.Usage) better control over
how their explanations are rendered: the old .longdesc field was wrapped
unpleasantly.

scripts/runner.py: put command group descriptions in parens

refactor scripts/common.py: move _default_nodedir out to its own file

Merge pull request #169 from zooko/2433-dead-code-NamedTemporaryDirectory

remove dead code: NamedTemporaryDirectory

remove dead code: NamedTemporaryDirectory

fixes #2433

NEWS: update lists of tickets closed/referenced-as-not-closed

test-desert-island wasn't catching https downloads

I think the desert-island build is broken, but the automated test wasn't
catching it because of this bug.

NEWS: update with all recent changes. Still unedited.

MANIFEST.in: add some missing files so tarballs are complete

fileutil.py: use ctypes.get_last_error() instead of GetLastError(). refs #1531

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

On Windows, the user's home directory may be either %USERPROFILE% or %HOMEDRIVE%%HOMEPATH% depending on the Windows version. fixes ticket:2417

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

interfaces.py: fix arguments to create_subdirectory.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

drop-upload.py: fix error messages.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Fix a pyflakes warning.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Update copy of gen-package-table.py in source tree.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Retire the setup_requires hack in cases where it isn't needed (and can cause build problems). refs #2286

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

NEWS.rst: add dependency updates. fixes #2415

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

_auto_deps.py: update comments. refs #2416

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Allow working with Twisted >= 13 on Windows if pywin32 is manually installed. fixes #2416

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Merge pull request #165 from pataquets/add-dockerfile

Base Dockerfile initial version.

Remove .bashrc alias uncommenting.

docs/frontends/CLI.rst: clarification.
The implementation (in [source:src/allmydata/scripts/common.py]) actually tests whether there is a slash anywhere before the first colon.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

docs/frontends/CLI.rst: remove a superfluous TODO.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Base Dockerfile initial version.

Merge branch '2399-cp-mkdir'

closes ticket:2399

apply review feedback

cp: trailing slash on source filename is an error, just like on targets

tahoe cp: ignore trailing slash on source arguments

This avoids an error case where an empty child name resulted in a
duplicate mkdir. It adds a precondition check to guard against empty
child names, and some test cases. It also cleans up a funny redundancy
noticed earlier (refs ticket:2329).

Improve docs on 'cp -r', noting the recent 2329 changes

refs ticket:2329

Update title of OpenSSL and pyOpenSSL build instructions.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Add OpenSSL and pyOpenSSL build instructions for 32-bit Windows.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Add doc for building pyOpenSSL.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

OS X icons and dialog box help text

fix OS X logo generation

add osx bundle meta data and applescript script to show an alert.

Build Tahoe-LAFS icon for OS X and Windows. refs ticket:2323

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

zetuptoolz: don't add ';' to an empty path when appending a new item.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

control.py: minor improvements

refs ticket:1737

Bump zetuptoolz version (move directory).

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Bump zetuptoolz version (file changes).

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

zetuptoolz: tolerate a single string for __requires__, and improve error reporting. fixes ticket:2242

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Change foolscap dependency to >= 0.8.0. refs #2400

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

CREDITS: add my GPG key

Fix a test that was dependent on Windows path separators.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Reduce differences between Windows and non-Windows dependencies.
This also fixes some warnings on Windows when using Twisted 12.1.0.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

fix check-memory test, with new new (safe) control-port methods

Ensure the 'base' argument to abspath_expanduser_unicode takes effect on Windows. fixes #2398

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

finish removing control-port upload/download functions

remove control-port upload/download file interfaces

FURLs are unguessable, but an attacker who somehow learned this FURL
could overwrite files and read sensitive data.

This will break the memory tests. I will add a new interface to support
the memory tests soon.

refs ticket:1737

backupdb.py: cosmetics

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Make backupdb use dbutil.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Add util/dbutil.py: open/create/update sqlite databases given some schema.

Author: Brian Warner <warner@lothar.com>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Some useful Deferred utilities, originally from the cloud backend branch.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Merge pull request #152 from vu3rdd/182-osx-packaging-7

strip the single quotes around $VERSION

strip the single quotes around $VERSION

Without stripping the single quotes, the generated pagkage filename
also has the undesired single quotes around the version string.
(eg: tahoe-lafs-'1.10.0.post309'-osx.pkg).

README.rst: change description to the new wording on tahoe-lafs.org (with "file store" instead of "filesystem").

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Add discussion and rejection of Change Journals.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Add link to otf-magic-folder-objective ticket query.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>