tahoe-lafs/tahoe-lafs.git
11 years agoAllow T (terabytes), P (petabytes), and E (exabytes), case-insensitive, in parse_abbr...
David-Sarah Hopwood [Thu, 27 Dec 2012 20:08:00 +0000 (20:08 +0000)]
Allow T (terabytes), P (petabytes), and E (exabytes), case-insensitive, in parse_abbreviated_size.
This also simplifies how case-insensitivity is handled, and fixes a corner case
where the wrong exception was raised when the size ends in "BB".
fixes #1812

Signed-off-by: David-Sarah Hopwood <davidsarah@mint>
11 years agoMake the system test nodes have non-ASCII nicknames. 25/head
David-Sarah Hopwood [Thu, 27 Dec 2012 19:00:22 +0000 (19:00 +0000)]
Make the system test nodes have non-ASCII nicknames.
This ensures that non-ASCII nicknames are exercised with the real server code.
refs #1298

Signed-off-by: David-Sarah Hopwood <davidsarah@mint>
11 years agoAllow welcome and storage status pages to display a non-ASCII node nickname. fixes...
David-Sarah Hopwood [Thu, 27 Dec 2012 05:08:53 +0000 (05:08 +0000)]
Allow welcome and storage status pages to display a non-ASCII node nickname. fixes #1298

Signed-off-by: David-Sarah Hopwood <davidsarah@jacaranda.org>
11 years agoRemove ResponseCache in favor of MDMFSlotReadProxy's cache. closes #1240.
David-Sarah Hopwood [Thu, 27 Dec 2012 00:00:17 +0000 (00:00 +0000)]
Remove ResponseCache in favor of MDMFSlotReadProxy's cache. closes #1240.

This contains several merged patches. Individual messages follow, latest first:

* Fix a warning from check-miscaptures.
* In retrieve.py, explicitly test whether a key is in self.servermap.proxies
  rather than catching KeyError.
* Added a new comment to the MDMF version of the test I removed, explaining
  the removal of the SDMF version.
* Removed test_corrupt_all_block_hash_tree_late, since the entire block_hash_tree
  is cached in the servermap for an SDMF file.
* Fixed several tests that require files larger than the servermap cache.
* Remove unused test_response_cache_memory_leak().
* Exercise the cache.
* Test infrastructure for counting cache misses on MDMF files.
* Removed the ResponseCache. Instead, the MDMFSlotReadProxy initialized
  by ServerMap is kept around so Retrieve can access it. The ReadProxy
  has a cache of the first 1000 bytes initially read from each share by
  the ServerMap. We're able to satisfy a number of requests out of this
  cache, so roundtrips are reduced from 84 to 60 in test_deepcheck_mdmf.
  There is still some mystery about under what conditions the cache has
  fewer than 1000 bytes. Also this breaks some existing unit tests that
  depend on the inner behavior of ResponseCache.
* The servermap.proxies (a cache of SlotReadProxies) is now keyed
  by (verinfo,serverid,shnum) rather than just (serverid,shnum)
* Minor cosmetic changes
* Added a test failure if the number of cache misses is too high.

Author: Andrew Miller <amiller@dappervision.com>
Signed-off-by: David-Sarah Hopwood <davidsarah@jacaranda.org>
11 years agodocs/logging.rst: fix rst2html warnings.
David-Sarah Hopwood [Sat, 22 Dec 2012 23:47:47 +0000 (23:47 +0000)]
docs/logging.rst: fix rst2html warnings.

Signed-off-by: David-Sarah Hopwood <davidsarah@mint>
11 years agodocs/logging.rst: remove note about a bug that was fixed in foolscap 0.6.0.
David-Sarah Hopwood [Sat, 22 Dec 2012 23:39:11 +0000 (23:39 +0000)]
docs/logging.rst: remove note about a bug that was fixed in foolscap 0.6.0.

Signed-off-by: David-Sarah Hopwood <davidsarah@mint>
12 years agotest/common_util.py: fix a bug in NonASCIIPathMixin that would cause spurious log...
David-Sarah Hopwood [Sun, 2 Dec 2012 01:54:47 +0000 (01:54 +0000)]
test/common_util.py: fix a bug in NonASCIIPathMixin that would cause spurious log entries on Windows.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agotests: add more diagnostics when code-under-test fails 23/head
Zooko O'Whielacronx [Thu, 29 Nov 2012 21:11:47 +0000 (14:11 -0700)]
tests: add more diagnostics when code-under-test fails

This prints out which things are different when two sets are expected to be the
same. This was useful to me when debugging the code under test. Hm, this
pattern might be more generally useful...

12 years agoremove unnecessary request from upload-helper
Zooko O'Whielacronx [Thu, 29 Nov 2012 20:44:32 +0000 (13:44 -0700)]
remove unnecessary request from upload-helper

The unnecessary request was from the upload helper to the sender, and it was
there in order to trigger the sender to deliver cleartext hashes. But we've
long since removed cleartext hashes.

Unit tests pass both before and after this change, and code-coverage shows that
the block I changed is exercised in unit tests.

12 years agopkg_resources: better error message on bad spec
Zooko O'Whielacronx [Thu, 22 Nov 2012 16:27:28 +0000 (09:27 -0700)]
pkg_resources: better error message on bad spec

Include the text of the offending distribution spec when it is ill-formed.

I got an error message from pkg_resources.py saying that a distribution spec
was ill-formed, because it ended with an '=' and nothing came after the '='.
However, the error message and stack trace didn't tell which distribution spec
it was.

With this patch, it includes the distribution spec itself in the error message.

There is no unit test of this patch, but I did test it manually by inserting a
"raise ValueError('WHATEVER')" into the code and re-running it and observing
that the new output correctly included the distribution

12 years agoNEWS.rst: add a missing dependency update for 1.9.2.
David-Sarah Hopwood [Tue, 20 Nov 2012 05:59:00 +0000 (05:59 +0000)]
NEWS.rst: add a missing dependency update for 1.9.2.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agoMakefile: increase filesystem size for 'make tmpfstest'.
David-Sarah Hopwood [Tue, 13 Nov 2012 16:24:36 +0000 (16:24 +0000)]
Makefile: increase filesystem size for 'make tmpfstest'.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agoweb/unlinked.py: don't use % operator to expand %(uri)s. fixes #1860.
David-Sarah Hopwood [Thu, 15 Nov 2012 04:16:19 +0000 (04:16 +0000)]
web/unlinked.py: don't use % operator to expand %(uri)s. fixes #1860.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agodocs/frontends/CLI.rst: fix markup of * and ?.
David-Sarah Hopwood [Sun, 11 Nov 2012 16:04:23 +0000 (16:04 +0000)]
docs/frontends/CLI.rst: fix markup of * and ?.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agodocs/frontends/CLI.rst: document ./* trick, and that aliases should not be a single...
David-Sarah Hopwood [Sun, 11 Nov 2012 15:50:46 +0000 (15:50 +0000)]
docs/frontends/CLI.rst: document ./* trick, and that aliases should not be a single char on Windows.
Also fix some rst2html warnings. refs #1853, #1840

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agoUpdate docs/network-and-reliance-topology.svg for RAIC, and to be more accurate about...
David-Sarah Hopwood [Sat, 3 Nov 2012 00:15:32 +0000 (00:15 +0000)]
Update docs/network-and-reliance-topology.svg for RAIC, and to be more accurate about frontend protocols.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agoCLI: fix synopsis for 'tahoe ls'. fixes #1839
David-Sarah Hopwood [Thu, 1 Nov 2012 23:40:06 +0000 (23:40 +0000)]
CLI: fix synopsis for 'tahoe ls'. fixes #1839

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agodocs/about.rst: fix broken link for network-and-reliance-topology image.
David-Sarah Hopwood [Thu, 1 Nov 2012 23:09:07 +0000 (23:09 +0000)]
docs/about.rst: fix broken link for network-and-reliance-topology image.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agoMakefile: add 'make tmpfstest', which uses a tmpfs for _trial_temp.
David-Sarah Hopwood [Thu, 25 Oct 2012 01:16:21 +0000 (02:16 +0100)]
Makefile: add 'make tmpfstest', which uses a tmpfs for _trial_temp.

This probably only works on Linux. It uses sudo to mount and unmount the tmpfs,
which may prompt for a password. refs #20

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agotest_runner: we no longer need to skip tests on Twisted < 9.0.
David-Sarah Hopwood [Wed, 24 Oct 2012 22:47:44 +0000 (23:47 +0100)]
test_runner: we no longer need to skip tests on Twisted < 9.0.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agosrc/allmydata/web/directory.py: fix HTML double-encoding issue for filenames.
David-Sarah Hopwood [Thu, 25 Oct 2012 00:01:25 +0000 (01:01 +0100)]
src/allmydata/web/directory.py: fix HTML double-encoding issue for filenames.

Nevow automatically HTML-escapes strings passed in stan without a raw marker.
Written by MK_FG. fixes #1143

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agodocs/frontends/webapi.rst: remove (ignored) "size" keyword from mkdir POST example
Mike Kazantsev [Tue, 9 Oct 2012 17:19:55 +0000 (23:19 +0600)]
docs/frontends/webapi.rst: remove (ignored) "size" keyword from mkdir POST example

12 years agoadd alternating grey-background to make it easier to scan across the table left to...
Zooko O'Whielacronx [Mon, 8 Oct 2012 18:13:33 +0000 (12:13 -0600)]
add alternating grey-background to make it easier to scan across the table left to right

12 years agolink from the tahoe-lafs-dep-eggs/README.html to the relevant wiki pages
Zooko O'Whielacronx [Mon, 8 Oct 2012 17:43:50 +0000 (11:43 -0600)]
link from the tahoe-lafs-dep-eggs/README.html to the relevant wiki pages

12 years agofixed wrong index in tahoe cp --verbose, as explained in #1805
frederik b [Fri, 28 Sep 2012 17:22:53 +0000 (19:22 +0200)]
fixed wrong index in tahoe cp --verbose, as explained in #1805

this bug shows wrong counters for copied files in verbose mode

12 years agoIgnore *~, /miscaptures.txt and /violations.txt.
David-Sarah Hopwood [Mon, 17 Sep 2012 20:28:49 +0000 (21:28 +0100)]
Ignore *~, /miscaptures.txt and /violations.txt.

Signed-off-by: David-Sarah Hopwood <david-sarah@jacaranda.org>
12 years agosrc/allmydata/__init__.py: ensure that if a dependency cannot be imported, we will...
david-sarah [Fri, 14 Sep 2012 02:14:48 +0000 (02:14 +0000)]
src/allmydata/__init__.py: ensure that if a dependency cannot be imported, we will report that rather than reporting a confusing message about the version and location being None. fixes #1804

12 years agodocs/quickstart.rst: swap two paragraphs to make it more likely a user will report...
david-sarah [Fri, 14 Sep 2012 02:10:12 +0000 (02:10 +0000)]
docs/quickstart.rst: swap two paragraphs to make it more likely a user will report import and versioning errors from 'bin/tahoe --version' if there are any, before trying to run tests. Also, change 'python setup.py test' to 'python setup.py trial' to avoid a rebuild that might change the dependencies (if setuptools decides to throw a spanner in the works).

12 years agoApply David-Sarah's recommended changes. Closes #974
Brian Warner [Tue, 4 Sep 2012 22:44:51 +0000 (15:44 -0700)]
Apply David-Sarah's recommended changes. Closes #974

12 years agoWhen the CLI cannot connect to the gateway, it prints an error message rather than...
Andrew Miller [Sat, 31 Mar 2012 22:53:45 +0000 (18:53 -0400)]
When the CLI cannot connect to the gateway, it prints an error message rather than raising a python exception. Includes a unit test

Signed-off-by: Andrew Miller <amiller@dappervision.com>
12 years agoCosmetic changes to match cloud backend branch.
david-sarah [Tue, 24 Jul 2012 03:40:49 +0000 (03:40 +0000)]
Cosmetic changes to match cloud backend branch.

12 years agointerfaces.py: corrections to take into account that lease cancel secrets are no...
david-sarah [Tue, 24 Jul 2012 03:40:06 +0000 (03:40 +0000)]
interfaces.py: corrections to take into account that lease cancel secrets are no longer honoured.

12 years agointerfaces.py: corrections to interfaces to make more of them checkable.
david-sarah [Tue, 24 Jul 2012 03:39:24 +0000 (03:39 +0000)]
interfaces.py: corrections to interfaces to make more of them checkable.

12 years agointerfaces.py: which -> that.
david-sarah [Tue, 24 Jul 2012 03:37:07 +0000 (03:37 +0000)]
interfaces.py: which -> that.

12 years agoDefault arguments in interface declarations should only be used to specify a default...
david-sarah [Tue, 24 Jul 2012 03:34:46 +0000 (03:34 +0000)]
Default arguments in interface declarations should only be used to specify a default, not to specify the type, in order to be checkable.

12 years agoRemove spurious 'self' arguments that should not be included in interface declarations.
david-sarah [Tue, 24 Jul 2012 03:32:56 +0000 (03:32 +0000)]
Remove spurious 'self' arguments that should not be included in interface declarations.

12 years agointerfaces.py: whitespace
david-sarah [Tue, 24 Jul 2012 03:29:14 +0000 (03:29 +0000)]
interfaces.py: whitespace

12 years agodocs: fix link to mutable.rst. Thanks to TimothyA for noticing the broken link
Zooko O'Whielacronx [Fri, 20 Jul 2012 19:24:02 +0000 (19:24 +0000)]
docs: fix link to mutable.rst. Thanks to TimothyA for noticing the broken link

12 years agodocs/configuration.rst: clarify statement about lack of redundancy when shares.happy...
david-sarah [Thu, 12 Jul 2012 20:30:26 +0000 (20:30 +0000)]
docs/configuration.rst: clarify statement about lack of redundancy when shares.happy <= k. fixes #1793

12 years agoUpdate release date for 1.9.2.
david-sarah [Tue, 3 Jul 2012 16:30:10 +0000 (16:30 +0000)]
Update release date for 1.9.2.

12 years agoSlightly improve the error message when a stats.pickle file cannot be read.
david-sarah [Tue, 3 Jul 2012 14:45:22 +0000 (14:45 +0000)]
Slightly improve the error message when a stats.pickle file cannot be read.

12 years agoRelnotes, NEWS and quickstart updates for 1.9.2, with updated release date. refs...
david-sarah [Sun, 1 Jul 2012 23:19:21 +0000 (23:19 +0000)]
Relnotes, NEWS and quickstart updates for 1.9.2, with updated release date. refs #1776

12 years agoFailing to load a crawler state pickle uses default values, but the exception clause...
david-sarah [Mon, 2 Jul 2012 18:18:47 +0000 (18:18 +0000)]
Failing to load a crawler state pickle uses default values, but the exception clause to detect this failure is too narrow; it can also fail with EOFError or KeyError for example.

12 years agoIf a stats.pickle file cannot be read, print a better error message.
david-sarah [Mon, 2 Jul 2012 18:15:55 +0000 (18:15 +0000)]
If a stats.pickle file cannot be read, print a better error message.

12 years agoMakefile: in check-umids, exclude 'old.py' to avoid false positives.
david-sarah [Sun, 1 Jul 2012 20:17:11 +0000 (20:17 +0000)]
Makefile: in check-umids, exclude 'old.py' to avoid false positives.

13 years agosrc/allmydata/__init__.py: head off a possible warning on platforms where twisted...
david-sarah [Mon, 8 Aug 2011 18:20:13 +0000 (18:20 +0000)]
src/allmydata/__init__.py: head off a possible warning on platforms where twisted-conch is a separate package.

12 years ago'tahoe --version': remove inconsistent trailing comma, easier to parse
Brian Warner [Tue, 26 Jun 2012 16:18:45 +0000 (09:18 -0700)]
'tahoe --version': remove inconsistent trailing comma, easier to parse

12 years ago_auto_deps: bump foolscap to >=0.6.3 to tolerate twisted-12.0.0
Brian Warner [Sun, 24 Jun 2012 19:10:24 +0000 (12:10 -0700)]
_auto_deps: bump foolscap to >=0.6.3 to tolerate twisted-12.0.0

The current Twisted release is 12.1.0, which (like 12.0.0 before it)
isn't compatible with foolscap-0.6.2 and earlier. We previously required
foolscap>=0.6.1, since that's all we actually need from foolscap itself.
_auto_deps specifies twisted>=11.0.0, so any system that can't meet that
will install the current Twisted (12.1.0), which will give them
something incompatible with foolscap-0.6.1 and 0.6.2 .

If we're limited to setuptools's declarative constraint language (and
can't have a function which evaluates the available dependency versions
and gives recommendations on which to change), then the only safe
approach is to make sure that any acceptable Foolscap version will be
compatible with all acceptable Twisted versions. So, bump the foolscap
dependency to >=0.6.3, which covers all currently-known
incompatibilities.

12 years agoMakefile: Add 'check-rst' target to check .rst files for errors. Also add 'doc-checks...
david-sarah [Sat, 23 Jun 2012 23:23:19 +0000 (23:23 +0000)]
Makefile: Add 'check-rst' target to check .rst files for errors. Also add 'doc-checks' which at the moment just runs check-rst, but might do more in future. fixes #1780

12 years agoUpdate docs, notably performance.rst, to include MDMF. fixes #1772
david-sarah [Sat, 23 Jun 2012 23:13:38 +0000 (23:13 +0000)]
Update docs, notably performance.rst, to include MDMF. fixes #1772

12 years agoNEWS.rst: hyperlink reference fixes. refs #1776
david-sarah [Sat, 23 Jun 2012 22:02:46 +0000 (22:02 +0000)]
NEWS.rst: hyperlink reference fixes. refs #1776

12 years agoRelease notes, NEWS, and quickstart updates for 1.9.2a1. refs #1776
david-sarah [Sat, 23 Jun 2012 00:17:52 +0000 (00:17 +0000)]
Release notes, NEWS, and quickstart updates for 1.9.2a1. refs #1776

12 years agoChange the maximum mutable share size to 69105 TB, and add a maximum-mutable-share...
david-sarah [Fri, 22 Jun 2012 15:43:54 +0000 (15:43 +0000)]
Change the maximum mutable share size to 69105 TB, and add a maximum-mutable-share-size field to the version announcement. Includes a test. refs #1778

12 years agoRelease notes and NEWS for 1.9.1. refs #1776
david-sarah [Thu, 21 Jun 2012 23:45:56 +0000 (23:45 +0000)]
Release notes and NEWS for 1.9.1. refs #1776

12 years agodocs/frontends/FTP-and-SFTP.rst: remove reference to rekeying bug since it does not...
david-sarah [Mon, 18 Jun 2012 17:03:15 +0000 (17:03 +0000)]
docs/frontends/FTP-and-SFTP.rst: remove reference to rekeying bug since it does not apply now that we depend on Twisted 11.0. refs #1297 refs #1771

12 years ago_auto_deps.py: bump the Twisted dependency to >= 11.0.0 to avoid Twisted bugs #411...
david-sarah [Sun, 17 Jun 2012 00:21:25 +0000 (00:21 +0000)]
_auto_deps.py: bump the Twisted dependency to >= 11.0.0 to avoid Twisted bugs #411 (Deferred recursion limit) and #4395 (SSH server rekeying). refs #1297 fixes #1771

12 years agomutable: fix shape of 'verinfo' tuple returned from MDMFSlotWriteProxy.get_verinfo...
david-sarah [Thu, 21 Jun 2012 20:52:21 +0000 (20:52 +0000)]
mutable: fix shape of 'verinfo' tuple returned from MDMFSlotWriteProxy.get_verinfo(). fixes #1669

12 years agoTests for ref #1669. Also refactor the existing tests to reduce duplicated code and...
david-sarah [Thu, 21 Jun 2012 20:46:27 +0000 (20:46 +0000)]
Tests for ref #1669. Also refactor the existing tests to reduce duplicated code and to fix a cut-and-paste error that caused one case (successful SDMF repair) to go untested.

12 years agoAdd assertions to make sure that set_default_encoding_parameters is always called...
david-sarah [Tue, 19 Jun 2012 02:05:04 +0000 (02:05 +0000)]
Add assertions to make sure that set_default_encoding_parameters is always called, rather than using hardcoded 3/7/10 defaults. Also update affected tests. Note that this by itself cannot fix the bug mentioned in ticket:1212#comment:41, but it might make it easier to reproduce. refs #1212

12 years agodocs/frontends/FTP-and-SFTP.rst: listing of directories containing mutable files...
david-sarah [Mon, 18 Jun 2012 00:24:54 +0000 (00:24 +0000)]
docs/frontends/FTP-and-SFTP.rst: listing of directories containing mutable files (but not reading/writing mutable files) is fixed for FTP. Also remove the orphaned trac link to #1688. refs #680

12 years agodocs: remove mention of #1588 from docs/frontends/FTP-and-SFTP.rst, since it is fixed
Zooko O'Whielacronx [Sun, 17 Jun 2012 03:14:04 +0000 (03:14 +0000)]
docs: remove mention of #1588 from docs/frontends/FTP-and-SFTP.rst, since it is fixed

12 years agoMake the intent of the loop over servers in test_system.Connections.test_rref clearer...
david-sarah [Sat, 16 Jun 2012 18:18:44 +0000 (18:18 +0000)]
Make the intent of the loop over servers in test_system.Connections.test_rref clearer, and able to be the same in 1.9.2 and trunk. Remove the now-unused node_key_s attribute of Client. refs #1636

12 years agoFix a bug in mutable publish that could cause an IndexError when a writer is removed...
david-sarah [Fri, 15 Jun 2012 03:44:37 +0000 (03:44 +0000)]
Fix a bug in mutable publish that could cause an IndexError when a writer is removed in Publish._connection_problem. This version uses DictOfSets as suggested by warner. fixes #1749

12 years agoSkip allmydata.test.test_system.Connections.test_rref unless we have foolscap >=...
david-sarah [Fri, 15 Jun 2012 23:20:28 +0000 (23:20 +0000)]
Skip allmydata.test.test_system.Connections.test_rref unless we have foolscap >= 0.6.4, because of http://foolscap.lothar.com/trac/ticket/196 . refs #1636

12 years agoAfter a server disconnects, make the IServer retain the dead RemoteReference, and...
david-sarah [Fri, 15 Jun 2012 01:48:55 +0000 (01:48 +0000)]
After a server disconnects, make the IServer retain the dead RemoteReference, and continue to return it to anyone who calls get_rref(). This removes the need for callers to guard against receiving a None (as long as the server was connected at least once, which is always the case for servers returned by get_servers_for_psi(), which is how all upload/download code gets servers). Includes test. fixes #1636

12 years agoSince DeprecationWarning about twisted.internet.interfaces.IFinishableConsumer is...
david-sarah [Thu, 14 Jun 2012 21:33:15 +0000 (21:33 +0000)]
Since DeprecationWarning about twisted.internet.interfaces.IFinishableConsumer is suppressed globally, it doesn't need to be suppressed during import. refs #1295

12 years agomisc/coding_tools/check-interfaces.py: clean-ups (warnings about Windows-specific...
david-sarah [Thu, 14 Jun 2012 21:28:29 +0000 (21:28 +0000)]
misc/coding_tools/check-interfaces.py: clean-ups (warnings about Windows-specific modules and error stream handling).

12 years agoSuppress DeprecationWarning about twisted.internet.interfaces.IFinishableConsumer...
david-sarah [Thu, 14 Jun 2012 21:23:08 +0000 (21:23 +0000)]
Suppress DeprecationWarning about twisted.internet.interfaces.IFinishableConsumer. This also unifies the handling of DeprecationWarnings that need to be suppressed globally. refs #1295

12 years agotest/common.py: fix race condition waiting for the helper connection
Brian Warner [Thu, 14 Jun 2012 19:17:55 +0000 (12:17 -0700)]
test/common.py: fix race condition waiting for the helper connection

The wait_for_connections() method, which is used at the start of
test_system to make sure that all the clients are connected to all the
servers, did not also wait for clients to be connected to their Helpers.
Every once in a while, the helper connection would take a bit longer,
and then
test_system.SystemTest.test_filesystem._test_web._got_welcome_helper
would fail, because we'd check for a helper connection before it was
ready.

The fix is to modify wait_for_connections's polling predicate to look
for helper connections (if configured) as well as the regular
introducer- and server- connections.

Tested by temporarily adding a large (30s) delay to the connectTo() call
in Uploader.startService, simulating a long helper
connection-establishment delay. This makes the test fail consistently.
Then I fixed wait_for_connections(), and the test passed (slowly). Then
I removed the delay.

Closes #1467

12 years agoClarify documentation of RIStorageServer.slot_testv_and_readv_and_writev. fixes ...
david-sarah [Wed, 13 Jun 2012 16:51:35 +0000 (16:51 +0000)]
Clarify documentation of RIStorageServer.slot_testv_and_readv_and_writev. fixes #1744

12 years agointroweb: the Subscribed Clients list shows tubids, not serverids
Brian Warner [Tue, 12 Jun 2012 21:37:27 +0000 (14:37 -0700)]
introweb: the Subscribed Clients list shows tubids, not serverids

Improve the column headers to make it clear that this list shows Tub
IDs. (we can't show pubkey-based serverids because clients don't give
those to us: only servers provide pubkeys). This should be the only
place in the whole webapi that shows TubIDs for modern (V2-introducer)
nodes.

12 years agoDisplay serverids consistently as 8-char pubkey, or 6-char tubid.
Brian Warner [Tue, 12 Jun 2012 21:01:24 +0000 (14:01 -0700)]
Display serverids consistently as 8-char pubkey, or 6-char tubid.

This makes it easy to distinguish between old V1-Introducer
nodes (identified by their Foolscap TubID) and new V2 nodes (identified
by their ed25519 pubkey).

This fixes a few places where we used to display a tubid even if we had
a pubkey, making it hard to visually correlate servers in two different
displays. It also cleans up the way we pass serverids to the JS-based
download timeline.

The "introweb" subscribed-clients list still shows tubids.

12 years agotest_system.py: wait for the Helper connection properly before uploading
Brian Warner [Tue, 12 Jun 2012 06:19:30 +0000 (23:19 -0700)]
test_system.py: wait for the Helper connection properly before uploading

12 years agotest_system.py: clean up control flow, reduce use of stall()
Brian Warner [Tue, 12 Jun 2012 01:22:35 +0000 (18:22 -0700)]
test_system.py: clean up control flow, reduce use of stall()

The _upload_resumable() test interrupts a Helper upload partway
through (by shutting down the Helper), then restarts the Helper and
resumes the upload. The control flow is kind of tricky: to do anything
"partway through" requires adding a hook to the Uploadable. The previous
flow depended upon a (fragile) call to self.stall(), which waits a fixed
number of seconds.

This removes one of those stall() calls (the remainder is in
test/common.py and I'll try removing it in a subsequent revision). It
also removes some now-redundant wait_for_connections() calls, since
bounce_client() doesn't fire its Deferred until the client has finished
coming back up (and uses wait_for_connections() internally to do so).

12 years agotest_system.py: fix minor typo
Brian Warner [Tue, 12 Jun 2012 01:16:36 +0000 (18:16 -0700)]
test_system.py: fix minor typo

12 years agooffloaded.py: don't drop the Deferred
Brian Warner [Tue, 12 Jun 2012 01:16:02 +0000 (18:16 -0700)]
offloaded.py: don't drop the Deferred

There was one corner case (where the client disconnects at just the
wrong time) that could have dropped a Deferred, leading to an Unhandled
Error. Clean up the control flow to avoid this case.

12 years agosetup.py and bin/tahoe-script.template: the error when we try to use Python 3 should...
david-sarah [Mon, 11 Jun 2012 03:52:16 +0000 (03:52 +0000)]
setup.py and bin/tahoe-script.template: the error when we try to use Python 3 should give the correct minimum Python version (now 2.5). refs #1658

12 years agointroducer: add sequence-numbers to announcements, ignore replays
Brian Warner [Mon, 11 Jun 2012 02:10:22 +0000 (19:10 -0700)]
introducer: add sequence-numbers to announcements, ignore replays

This will support revocation of Accounting recommendation records,
assuming the gossip-based broadcast channel isn't easily jammed.

12 years agoclient.py: rename "server key" to "node key", use old name if present
Brian Warner [Mon, 11 Jun 2012 01:14:55 +0000 (18:14 -0700)]
client.py: rename "server key" to "node key", use old name if present

This prepares for invitation-based reciprocal-permission Accounting. In
the scheme I'm developing, nodes publish "I accept shares from Y"
messages, which are assembled into a graph, and server will accept
shares from any client node reachable in this graph. For this to work,
the serverX->clientY edge must be connectable to the serverY->clientZ
edge, which means "clientY" and "serverY" must be connected. If clientY
and serverY are two distinct keys, they must be cross-signed. Life is
easier if there's just one key "Y", rather than distinct client- and
server- keys. Calling this one key "server.privkey" would be confusing.
"node.privkey" and "node.pubkey" makes more sense.

One-server-per-node is a pretty easy restriction. Originally I was
thinking that the client.key should be provided in each webapi call,
just like a filecap is, making a single node useable by multiple users
(Accounting principals), and not providing any ambient storage
authority. But I've been unable to think of a comfortable WUI for
that (at least without requiring javascript), nor a friendly way to
transfer account authority (e.g. writecaps that include storage
authority). So I'm more willing to have one-client-per-node these days.

(and note that this rename doesn't seriously preclude
many-clients-per-node or zero-clients-per-node anyways, it just makes
one-client-per-node less awkward)

12 years agonode.py: add get_private_config()
Brian Warner [Mon, 11 Jun 2012 00:46:38 +0000 (17:46 -0700)]
node.py: add get_private_config()

Also add tests for this and the pre-existing private-config methods.

12 years agoAdded docs/specifications/backends/raic.rst for ticket #1760
Patrick R McDonald [Wed, 6 Jun 2012 02:35:23 +0000 (21:35 -0500)]
Added docs/specifications/backends/raic.rst for ticket #1760

12 years agoFix text in Publish Status results. Closes #1762.
Brian Warner [Fri, 8 Jun 2012 22:21:46 +0000 (15:21 -0700)]
Fix text in Publish Status results. Closes #1762.

12 years agoCheckResults corrupt/incompatible shares now return IServers 10/head
Brian Warner [Fri, 25 May 2012 19:58:28 +0000 (12:58 -0700)]
CheckResults corrupt/incompatible shares now return IServers

DeepResultsBase also has a get_corrupt_shares(), and it is populated
from CheckResults.get_corrupt_shares(). It has been updated too, along
with get_remaining_corrupt_shares().

Remove temporary get_new_corrupt_shares() and
get_new_incompatible_shares().

12 years agoCheckResults.get_servers_responding() now returns IServers
Brian Warner [Fri, 25 May 2012 19:58:18 +0000 (12:58 -0700)]
CheckResults.get_servers_responding() now returns IServers

Remove temporary get_new_servers_responding().

12 years agoCheckResults.get_sharemap() now returns IServers
Brian Warner [Fri, 25 May 2012 19:58:02 +0000 (12:58 -0700)]
CheckResults.get_sharemap() now returns IServers

Remove temporary get_new_sharemap().

12 years agoCheckResults: pass IServer to corrupt/incompatible share locators
Brian Warner [Fri, 25 May 2012 19:57:53 +0000 (12:57 -0700)]
CheckResults: pass IServer to corrupt/incompatible share locators

Getters still return serverid. Adds temporary get_new_corrupt_shares()
and get_new_incompatible_shares().

12 years agoCheckResults: pass IServer to servers_responding=, getter returns serverid
Brian Warner [Fri, 25 May 2012 19:56:20 +0000 (12:56 -0700)]
CheckResults: pass IServer to servers_responding=, getter returns serverid

Add temporary get_new_servers_responding().

12 years agoCheckResults: pass IServer to sharemap=, but get_sharemap() returns serverids
Brian Warner [Fri, 25 May 2012 19:56:03 +0000 (12:56 -0700)]
CheckResults: pass IServer to sharemap=, but get_sharemap() returns serverids

This changes all code which feeds CheckResults(sharemap=) to provide
IServer instances, but CheckResults converts these to old-style
serverids during output, so downstream code doesn't have to change yet.

It adds a temporary get_new_sharemap(), which *does* return IServer
instances, so the immutable repairer can build new CheckResults from an
old one. This will go away when get_sharemap() is updated to return
IServer (and downstream code is updated too).

12 years agoCheckResults: internal cleanup
Brian Warner [Fri, 25 May 2012 19:55:48 +0000 (12:55 -0700)]
CheckResults: internal cleanup

replace the one-big-dictionary with normal private attributes

12 years agoCheckResults: privatize remaining attributes
Brian Warner [Fri, 25 May 2012 19:29:10 +0000 (12:29 -0700)]
CheckResults: privatize remaining attributes

12 years agoCheckResults: use fat init, add type-checking assertions
Brian Warner [Fri, 25 May 2012 07:14:46 +0000 (00:14 -0700)]
CheckResults: use fat init, add type-checking assertions

Added assertions for sharemap, servermap, servers_responding,
list_corrupt_shares, and list_incompatible_shares.

12 years agomutable/checker: refactor to make CheckResults easier to change
Brian Warner [Fri, 25 May 2012 07:14:18 +0000 (00:14 -0700)]
mutable/checker: refactor to make CheckResults easier to change

12 years agoCheckResults: replace get_data() with as_dict(), use getters in web status
Brian Warner [Fri, 25 May 2012 07:14:08 +0000 (00:14 -0700)]
CheckResults: replace get_data() with as_dict(), use getters in web status

12 years agouse the new CheckResult getters almost everywhere
Brian Warner [Fri, 25 May 2012 07:13:48 +0000 (00:13 -0700)]
use the new CheckResult getters almost everywhere

The remaining get_data() calls are either in
web.check_results.json_check_results(), or functioning as repr()s in
various unit test failure cases.

12 years agoCheckResults: replace get_data() with a bunch of individual getters
Brian Warner [Fri, 25 May 2012 07:13:32 +0000 (00:13 -0700)]
CheckResults: replace get_data() with a bunch of individual getters

12 years agochange CheckResults to use a fat set_data()
Brian Warner [Fri, 25 May 2012 07:13:23 +0000 (00:13 -0700)]
change CheckResults to use a fat set_data()

i.e. change set_data() to accept lots of parameters, instead of taking
a single dictionary with lots of keys. Also Convert all CheckResults
creators to use it.

12 years agoCheckResults: simplify self._data
Brian Warner [Fri, 25 May 2012 07:13:13 +0000 (00:13 -0700)]
CheckResults: simplify self._data

12 years agoCheckResults: start hiding .data, first step to clean it up
Brian Warner [Tue, 15 May 2012 04:57:43 +0000 (21:57 -0700)]
CheckResults: start hiding .data, first step to clean it up

The goal is to make CheckResults more strongly typed, and remove the
ambiguous ".data" field in favor of a bunch of specific counters and
sharelists, so I can changes .sharemap and .servermap to use IServer
instances instead of string serverids. By cleaning this up first, I hope
to get that task done with less debugging.

12 years agoimmutable.CiphertextFileNode.check_and_repair: simplify for refactoring
Brian Warner [Tue, 15 May 2012 04:53:49 +0000 (21:53 -0700)]
immutable.CiphertextFileNode.check_and_repair: simplify for refactoring

There were too many nested functions here, making some upcoming changes
too difficult, so let's refactor it first.

12 years agodocs/quickstart.rst: fix rst warning.
david-sarah [Fri, 1 Jun 2012 21:01:04 +0000 (21:01 +0000)]
docs/quickstart.rst: fix rst warning.