Brian Warner [Tue, 26 Mar 2013 00:57:01 +0000 (17:57 -0700)]
resurrect provisioning.py
It looks like commit 916d26e7103208fa207259d62ce453a5a8b9acd0, in
addition to making a one-line fix for #1681, also deleted
misc/operations_helpers/provisioning/provisioning.py entirely. This
brings it back.
Brian Warner [Wed, 20 Mar 2013 22:25:10 +0000 (15:25 -0700)]
client.py: throw error when reserved_space= is unparseable. Closes #1812.
This should now fail quickly (during "tahoe start"). Previously this
would silently treat an unparseable size as "0", and the only way to
discover that it had had a problem would be to look at the foolscap log,
or examine the storage-service web page for the unexpected "Reserved
Size" number.
Brian Warner [Wed, 20 Mar 2013 22:10:47 +0000 (15:10 -0700)]
Make introducer.furl unguessable. Closes #1802.
Previously, Introducers always used a swissnum of "introducer", so
anyone who could learn the (public) tubid of the introducer would be
able to connect to and use it. This changes new Introducers to use the
same randomly-generated swissnum as clients and storage servers do, so
that you absolutely must learn the introducer.furl from someone who
knows it already before you can connect.
This change also moves the location of the file that stores
introducer.furl from BASEDIR/introducer.furl to
BASEDIR/private/introducer.furl, since that's where we keep the private
things. The first time an introducer is started with the new code, it
will move any existing BASEDIR/introducer.furl into the new place.
Note that this will not change the FURL of existing introducers: it will
only affect newly created ones. When you change an introducer's FURL,
you must also update all of the nodes (clients and storage servers)
which connect to it, so upgrading it to an unguessable one isn't
something we should do automatically.
Brian Warner [Tue, 19 Mar 2013 00:40:56 +0000 (17:40 -0700)]
introducer.client: use integer seqnums, not time-based. Closes #1767.
This stores the sequence number in BASEDIR/announcement-seqnum, and
increments it each time any service is published (every service
announcement is regenerated with the new sequence number). As everyone
knows, time is an illusion, and occasionally goes backwards, so a
counter is generally safer (and reveals less information about the
node).
Later, we'll improve the introducer client to tolerate rollbacks (where,
perhaps due to a VM being restarted from an earlier checkpoint, the
stored sequence number reverts to an earlier version).
Brian Warner [Tue, 19 Mar 2013 00:30:57 +0000 (17:30 -0700)]
fileutil.write: accept mode=, and use it in Node.write_config
I want mode="w" (i.e. text, with newline conversion) for code that
writes newline-terminated strings (which should also be human readable)
to files. I like to use things like "cat .tahoe/permutation-seed"
without seeing the seed jammed together with the next command prompt.
Mike Kazantsev [Wed, 27 Feb 2013 17:14:58 +0000 (23:14 +0600)]
test_web.py: fix test_GET_DIRECTORY_html_filenode_encoding for Twisted==12.3.0. fixes #1922
twisted.web.html.escape was used to produce html-encoded string (to then look
it up in "value" attribute), but behavior of that function has changed between
Twisted 12.2.0 (simple custom implementation) and 12.3.0 (imported from stdlib
cgi module).
tahoe_check.py: tolerate missing fields in check results for LIT files/dirs. fixes #1758
Also test this case and improve some existing tests of 'tahoe check'.
Allow T (terabytes), P (petabytes), and E (exabytes), case-insensitive, in parse_abbreviated_size.
This also simplifies how case-insensitivity is handled, and fixes a corner case
where the wrong exception was raised when the size ends in "BB".
fixes #1812
Remove ResponseCache in favor of MDMFSlotReadProxy's cache. closes #1240.
This contains several merged patches. Individual messages follow, latest first:
* Fix a warning from check-miscaptures.
* In retrieve.py, explicitly test whether a key is in self.servermap.proxies
rather than catching KeyError.
* Added a new comment to the MDMF version of the test I removed, explaining
the removal of the SDMF version.
* Removed test_corrupt_all_block_hash_tree_late, since the entire block_hash_tree
is cached in the servermap for an SDMF file.
* Fixed several tests that require files larger than the servermap cache.
* Remove unused test_response_cache_memory_leak().
* Exercise the cache.
* Test infrastructure for counting cache misses on MDMF files.
* Removed the ResponseCache. Instead, the MDMFSlotReadProxy initialized
by ServerMap is kept around so Retrieve can access it. The ReadProxy
has a cache of the first 1000 bytes initially read from each share by
the ServerMap. We're able to satisfy a number of requests out of this
cache, so roundtrips are reduced from 84 to 60 in test_deepcheck_mdmf.
There is still some mystery about under what conditions the cache has
fewer than 1000 bytes. Also this breaks some existing unit tests that
depend on the inner behavior of ResponseCache.
* The servermap.proxies (a cache of SlotReadProxies) is now keyed
by (verinfo,serverid,shnum) rather than just (serverid,shnum)
* Minor cosmetic changes
* Added a test failure if the number of cache misses is too high.
Author: Andrew Miller <amiller@dappervision.com> Signed-off-by: David-Sarah Hopwood <davidsarah@jacaranda.org>
tests: add more diagnostics when code-under-test fails
This prints out which things are different when two sets are expected to be the
same. This was useful to me when debugging the code under test. Hm, this
pattern might be more generally useful...
The unnecessary request was from the upload helper to the sender, and it was
there in order to trigger the sender to deliver cleartext hashes. But we've
long since removed cleartext hashes.
Unit tests pass both before and after this change, and code-coverage shows that
the block I changed is exercised in unit tests.
Include the text of the offending distribution spec when it is ill-formed.
I got an error message from pkg_resources.py saying that a distribution spec
was ill-formed, because it ended with an '=' and nothing came after the '='.
However, the error message and stack trace didn't tell which distribution spec
it was.
With this patch, it includes the distribution spec itself in the error message.
There is no unit test of this patch, but I did test it manually by inserting a
"raise ValueError('WHATEVER')" into the code and re-running it and observing
that the new output correctly included the distribution
docs/frontends/CLI.rst: document ./* trick, and that aliases should not be a single char on Windows.
Also fix some rst2html warnings. refs #1853, #1840
src/allmydata/__init__.py: ensure that if a dependency cannot be imported, we will report that rather than reporting a confusing message about the version and location being None. fixes #1804
docs/quickstart.rst: swap two paragraphs to make it more likely a user will report import and versioning errors from 'bin/tahoe --version' if there are any, before trying to run tests. Also, change 'python setup.py test' to 'python setup.py trial' to avoid a rebuild that might change the dependencies (if setuptools decides to throw a spanner in the works).
Failing to load a crawler state pickle uses default values, but the exception clause to detect this failure is too narrow; it can also fail with EOFError or KeyError for example.
Brian Warner [Sun, 24 Jun 2012 19:10:24 +0000 (12:10 -0700)]
_auto_deps: bump foolscap to >=0.6.3 to tolerate twisted-12.0.0
The current Twisted release is 12.1.0, which (like 12.0.0 before it)
isn't compatible with foolscap-0.6.2 and earlier. We previously required
foolscap>=0.6.1, since that's all we actually need from foolscap itself.
_auto_deps specifies twisted>=11.0.0, so any system that can't meet that
will install the current Twisted (12.1.0), which will give them
something incompatible with foolscap-0.6.1 and 0.6.2 .
If we're limited to setuptools's declarative constraint language (and
can't have a function which evaluates the available dependency versions
and gives recommendations on which to change), then the only safe
approach is to make sure that any acceptable Foolscap version will be
compatible with all acceptable Twisted versions. So, bump the foolscap
dependency to >=0.6.3, which covers all currently-known
incompatibilities.
david-sarah [Sat, 23 Jun 2012 23:23:19 +0000 (23:23 +0000)]
Makefile: Add 'check-rst' target to check .rst files for errors. Also add 'doc-checks' which at the moment just runs check-rst, but might do more in future. fixes #1780
david-sarah [Sun, 17 Jun 2012 00:21:25 +0000 (00:21 +0000)]
_auto_deps.py: bump the Twisted dependency to >= 11.0.0 to avoid Twisted bugs #411 (Deferred recursion limit) and #4395 (SSH server rekeying). refs #1297 fixes #1771
david-sarah [Thu, 21 Jun 2012 20:46:27 +0000 (20:46 +0000)]
Tests for ref #1669. Also refactor the existing tests to reduce duplicated code and to fix a cut-and-paste error that caused one case (successful SDMF repair) to go untested.
projects / tahoe-lafs / tahoe-lafs.git / log