tahoe-lafs/tahoe-lafs.git
13 years agodebugprint the values of blocks and hashes thereof; make the test data and the seg...
Zooko O'Whielacronx [Fri, 30 Sep 2011 05:46:28 +0000 (05:46 +0000)]
debugprint the values of blocks and hashes thereof; make the test data and the seg size small in order to make the debugprints easy to look at

13 years agodocs/about.rst: correct the description of availability to take into account that...
david-sarah [Tue, 25 Oct 2011 13:25:50 +0000 (06:25 -0700)]
docs/about.rst: correct the description of availability to take into account that shares may not be stored on distinct servers.

13 years agomutable/retrieve: don't write() after we've been pauseProducer'ed
Brian Warner [Mon, 17 Oct 2011 00:03:47 +0000 (17:03 -0700)]
mutable/retrieve: don't write() after we've been pauseProducer'ed

This fixes a test failure found against current Twisted trunk in
test_mutable.Filenode.test_retrieve_producer_mdmf (when it uses
PausingAndStoppingConsumer). There must be some sort of race: I could
make it fail against Twisted-11.0 if I just increased the 0.5s delay in
test_download.PausingAndStoppingConsumer to about 0.6s, and could make
Twisted-trunk pass by reducing it to about 0.3s .

I fixed the test (as opposed to the bug) by replacing the delay with a
simple reliable eventually(), and adding extra asserts to fail the test
if the consumer's write() method is called while the producer is
supposed to be paused

The bug itself was that mutable.retrieve.Retrieve wasn't checking the
"stopped" flag after resuming from a pause, and thus delivered one
segment to a consumer that wasn't expecting it. I split out
stopped-flag-checking to separate function, which is now called
immediately after _check_for_paused(). I also cleaned up some Deferred
usage and whitespace.

13 years agoremove interpreter shbang lines from non-executables
Brian Warner [Fri, 14 Oct 2011 17:23:01 +0000 (10:23 -0700)]
remove interpreter shbang lines from non-executables

thanks to Greg Troxel for the catch

13 years agoTAG allmydata-tahoe-1.9.0b1 allmydata-tahoe-1.9.0b1
Brian Warner [Fri, 14 Oct 2011 05:55:32 +0000 (22:55 -0700)]
TAG allmydata-tahoe-1.9.0b1

13 years agoNEWS: mention --format, bring up-to-date
Brian Warner [Fri, 14 Oct 2011 05:55:00 +0000 (22:55 -0700)]
NEWS: mention --format, bring up-to-date

13 years agoCLI: don't deprecate --mutable, small docs fixes. refs #1561
Brian Warner [Fri, 14 Oct 2011 03:45:59 +0000 (20:45 -0700)]
CLI: don't deprecate --mutable, small docs fixes. refs #1561

Also don't accept 'tahoe mkdir --format=chk'.

13 years agoadd --format= to 'tahoe put'/'mkdir', remove --mutable-type. Closes #1561
Brian Warner [Fri, 14 Oct 2011 03:15:00 +0000 (20:15 -0700)]
add --format= to 'tahoe put'/'mkdir', remove --mutable-type. Closes #1561

13 years agoweb/filenode.py: rely on Request.notifyFinish. Closes #1366.
Brian Warner [Thu, 13 Oct 2011 20:12:19 +0000 (13:12 -0700)]
web/filenode.py: rely on Request.notifyFinish. Closes #1366.

This is safe now that tahoe depends upon Twisted>=10.1, since notifyFinish
first appeared in Twisted-9.0

13 years agodocs: fix several imprecise or inaccurate values in performance.rst
Zooko O'Whielacronx [Sun, 8 May 2011 12:42:28 +0000 (05:42 -0700)]
docs: fix several imprecise or inaccurate values in performance.rst
add cpu values for each operation
sort the list of values into the same order in each operation
refs #1398

13 years agoutil/happinessutil.py: suppress a warning from check-miscaptures. (It is not a bug...
david-sarah [Sun, 9 Oct 2011 05:21:06 +0000 (22:21 -0700)]
util/happinessutil.py: suppress a warning from check-miscaptures. (It is not a bug because the capturing function is only used by a 'map' in the same iteration.) refs #1556

13 years agomisc/coding_tools/make-canary-files.py: fix a suspicious capture reported by check...
david-sarah [Sun, 9 Oct 2011 05:05:31 +0000 (22:05 -0700)]
misc/coding_tools/make-canary-files.py: fix a suspicious capture reported by check-miscaptures (although it happens not to be a bug because the callback will be processed synchronously). refs #1556

13 years agoFix some more potential bugs in test code exposed by check-miscaptures.py. refs ...
david-sarah [Fri, 7 Oct 2011 03:38:47 +0000 (20:38 -0700)]
Fix some more potential bugs in test code exposed by check-miscaptures.py. refs #1556

13 years agoFix some potential bugs (in non-test code) exposed by check-miscaptures.py. refs...
david-sarah [Fri, 7 Oct 2011 03:24:44 +0000 (20:24 -0700)]
Fix some potential bugs (in non-test code) exposed by check-miscaptures.py. refs #1556

13 years agoFix some potential bugs in test code exposed by check-miscaptures.py. refs #1556
david-sarah [Fri, 7 Oct 2011 02:34:43 +0000 (19:34 -0700)]
Fix some potential bugs in test code exposed by check-miscaptures.py. refs #1556

13 years agooops, missed a test failure
Brian Warner [Thu, 13 Oct 2011 16:37:13 +0000 (09:37 -0700)]
oops, missed a test failure

13 years agomisc mutable-type fixes:
Brian Warner [Thu, 13 Oct 2011 16:32:29 +0000 (09:32 -0700)]
misc mutable-type fixes:

* fix tahoe.cfg control of default mutable type
* tolerate arbitrary case in [client]mutable.format value
* small docs improvements
* use get_mutable_type() as a format-is-mutable predicate
* tighten up error message

13 years agowebapi: use all-caps "SDMF"/"MDMF" acronyms in t=json response
Brian Warner [Thu, 13 Oct 2011 16:31:43 +0000 (09:31 -0700)]
webapi: use all-caps "SDMF"/"MDMF" acronyms in t=json response

docs: upcase examples of t=json output and format= input

13 years agowebapi.rst: fix whitespace (detabify) t=json examples
Brian Warner [Thu, 13 Oct 2011 16:30:56 +0000 (09:30 -0700)]
webapi.rst: fix whitespace (detabify) t=json examples

13 years agowebapi: handle format=, remove mutable-type=
Brian Warner [Thu, 13 Oct 2011 16:29:51 +0000 (09:29 -0700)]
webapi: handle format=, remove mutable-type=

* fix CLI commands (put, mkdir) to send format=, not mutable-type=
* fix tests
* test_cli: fix tests that observe t=json output, don't ignore failures in
  'tahoe put'
* fix handling of version= to make it easier to use the default
* interpret ?mutable=true&format=MDMF as MDMF, not SDMF

13 years agodocs/frontends/webapi.rst: document the format argument
kevan [Mon, 10 Oct 2011 02:55:29 +0000 (19:55 -0700)]
docs/frontends/webapi.rst: document the format argument

13 years agoTests for ref #1547
david-sarah [Sun, 2 Oct 2011 03:53:16 +0000 (20:53 -0700)]
Tests for ref #1547

13 years agoChange the file upload forms on directory and welcome pages to use a 3-way radio...
david-sarah [Sun, 2 Oct 2011 03:45:03 +0000 (20:45 -0700)]
Change the file upload forms on directory and welcome pages to use a 3-way radio button to select immutable, SDMF, or MDMF. Add '(experimental)' to the label for creating an MDMF directory. Also improve the spacing of form elements. refs #1547

13 years agotest_web.py: minor cleanups, mainly to make the first argument to shouldFail tests...
david-sarah [Sun, 2 Oct 2011 04:03:32 +0000 (21:03 -0700)]
test_web.py: minor cleanups, mainly to make the first argument to shouldFail tests consistent

13 years agoTests for ref #1552
david-sarah [Sun, 2 Oct 2011 04:00:36 +0000 (21:00 -0700)]
Tests for ref #1552

13 years agomisc/check-interfaces.py: print a warning if a .pyc or .pyo file exists without a...
david-sarah [Wed, 12 Oct 2011 23:36:09 +0000 (16:36 -0700)]
misc/check-interfaces.py: print a warning if a .pyc or .pyo file exists without a corresponding .py file.

13 years agotest/common.py: in shouldFail and shouldHTTPError, when the raised exception does...
david-sarah [Tue, 11 Oct 2011 00:22:27 +0000 (17:22 -0700)]
test/common.py: in shouldFail and shouldHTTPError, when the raised exception does not include the expected substring (or, for shouldHTTPError, when the status code is wrong), mention which test that happened in.

13 years agointerfaces.py: remove get_extension_params and set_extension_params methods from...
david-sarah [Mon, 10 Oct 2011 19:48:42 +0000 (12:48 -0700)]
interfaces.py: remove get_extension_params and set_extension_params methods from IMutableFileURI. refs #393, #1526

13 years agointerfaces.py: fix a typo in the name of IMutableSlotWriter.put_encprivkey. refs...
david-sarah [Mon, 10 Oct 2011 19:46:42 +0000 (12:46 -0700)]
interfaces.py: fix a typo in the name of IMutableSlotWriter.put_encprivkey. refs #393

13 years agomisc/simulators/hashbasedsig.py: simplify by removing unnecessary local function...
david-sarah [Fri, 7 Oct 2011 02:30:01 +0000 (19:30 -0700)]
misc/simulators/hashbasedsig.py: simplify by removing unnecessary local function that captured a variable declared in a for loop (this was not a bug, but the code was unclear). Also fix a pyflakes warning about an import. refs #1556

13 years agoimmutable/literal.py: add pauseProducing method to LiteralProducer. refs #1537
david-sarah [Mon, 3 Oct 2011 19:52:39 +0000 (12:52 -0700)]
immutable/literal.py: add pauseProducing method to LiteralProducer. refs #1537

13 years agono_network.py: Clean up whitespace around code changed by previous patch.
david-sarah [Tue, 4 Oct 2011 01:04:07 +0000 (18:04 -0700)]
no_network.py: Clean up whitespace around code changed by previous patch.

13 years agono_network.py: Fix potential bugs in some tests due to capture of slots in for loops.
david-sarah [Tue, 4 Oct 2011 01:02:31 +0000 (18:02 -0700)]
no_network.py: Fix potential bugs in some tests due to capture of slots in for loops.

13 years agodocs: fix the rst formatting of COPYING.TGPPL.rst
Zooko O'Whielacronx [Mon, 3 Oct 2011 04:33:33 +0000 (21:33 -0700)]
docs: fix the rst formatting of COPYING.TGPPL.rst
Now it renders correctly both on trac and with rst2html --verbose from docutils v0.8.1.

13 years agoMDMF: remove extension fields from caps, tolerate arbitrary ones. Fixes #1526
Brian Warner [Sat, 1 Oct 2011 23:35:53 +0000 (00:35 +0100)]
MDMF: remove extension fields from caps, tolerate arbitrary ones. Fixes #1526

The filecaps used to be produced with hints for 'k' and segsize, but they
weren't actually used, and doing so had the potential to limit how we change
those filecaps in the future. Also the parsing code had some problems dealing
with other numbers of extensions. Removing the existing fields and making the
parser tolerate (and ignore) extra ones makes MDMF more future-proof.

13 years agotest/test_runner.py: BinTahoe.test_path has rare nondeterministic failures; this...
david-sarah [Tue, 27 Sep 2011 22:53:36 +0000 (15:53 -0700)]
test/test_runner.py: BinTahoe.test_path has rare nondeterministic failures; this patch probably fixes a problem where the actual cause of failure is masked by a string conversion error.

13 years agodocs/configuration.rst: add section about the types of node, and clarify when setting...
Zooko O'Whielacronx [Mon, 26 Sep 2011 20:38:01 +0000 (13:38 -0700)]
docs/configuration.rst: add section about the types of node, and clarify when setting web.port enables web-API service. fixes #1444

13 years agoTAG allmydata-tahoe-1.9.0a2 allmydata-tahoe-1.9.0a2
Brian Warner [Sun, 25 Sep 2011 23:48:11 +0000 (16:48 -0700)]
TAG allmydata-tahoe-1.9.0a2

13 years agoNEWS: tidy up a little bit, reprioritize some items, hide some non-user-visible items
Brian Warner [Sun, 25 Sep 2011 23:35:29 +0000 (16:35 -0700)]
NEWS: tidy up a little bit, reprioritize some items, hide some non-user-visible items

13 years agodocs: fix some broken .rst links. refs #1542
david-sarah [Sun, 25 Sep 2011 05:10:01 +0000 (22:10 -0700)]
docs: fix some broken .rst links. refs #1542

13 years agomutable/publish.py: fix an unused import. refs #1542
david-sarah [Sun, 25 Sep 2011 05:22:06 +0000 (22:22 -0700)]
mutable/publish.py: fix an unused import. refs #1542

13 years agoNEWS: fix .rst formatting.
david-sarah [Sun, 25 Sep 2011 05:01:19 +0000 (22:01 -0700)]
NEWS: fix .rst formatting.

13 years agoNEWS: updates for 1.9alpha2.
david-sarah [Sun, 25 Sep 2011 04:53:43 +0000 (21:53 -0700)]
NEWS: updates for 1.9alpha2.

13 years agomutable/layout.py: make unpack_sdmf_checkstring and unpack_mdmf_checkstring more...
david-sarah [Sun, 25 Sep 2011 02:36:51 +0000 (19:36 -0700)]
mutable/layout.py: make unpack_sdmf_checkstring and unpack_mdmf_checkstring more similar, and change an assert to give a more useful message if it fails. refs #1540

13 years agomutable/publish: handle unknown mutable share formats when handling errors
kevan [Sun, 25 Sep 2011 00:43:05 +0000 (17:43 -0700)]
mutable/publish: handle unknown mutable share formats when handling errors

13 years agomutable/layout: break unpack_checkstring into unpack_mdmf_checkstring and unpack_sdmf...
kevan [Sun, 25 Sep 2011 00:41:34 +0000 (17:41 -0700)]
mutable/layout: break unpack_checkstring into unpack_mdmf_checkstring and unpack_sdmf_checkstring, add distinguisher function for checkstrings

13 years agotest/test_mutable: reenable mdmf publish surprise test
kevan [Sat, 24 Sep 2011 23:54:15 +0000 (16:54 -0700)]
test/test_mutable: reenable mdmf publish surprise test

13 years agomutable/publish: use unpack_mdmf_checkstring and unpack_sdmf_checkstring instead...
kevan [Sat, 24 Sep 2011 23:51:37 +0000 (16:51 -0700)]
mutable/publish: use unpack_mdmf_checkstring and unpack_sdmf_checkstring instead of unpack_checkstring. fixes #1540

13 years agomisc/coding_tools/check_interfaces.py: report all violations rather than only one...
david-sarah [Fri, 16 Sep 2011 22:34:50 +0000 (15:34 -0700)]
misc/coding_tools/check_interfaces.py: report all violations rather than only one for a given class, by including a forked version of verifyClass. refs #1474

13 years agomisc/coding_tools/check_interfaces.py: use os.walk instead of FilePath, since this...
david-sarah [Fri, 16 Sep 2011 21:26:33 +0000 (14:26 -0700)]
misc/coding_tools/check_interfaces.py: use os.walk instead of FilePath, since this script shouldn't really depend on Twisted. refs #1474

13 years agomisc/coding_tools/check-interfaces.py: reduce false-positives by adding Dummy* to...
david-sarah [Fri, 16 Sep 2011 21:26:24 +0000 (14:26 -0700)]
misc/coding_tools/check-interfaces.py: reduce false-positives by adding Dummy* to the set of excluded classnames, and bench-* to the set of excluded basenames. refs #1474

13 years agoAdd a script 'misc/coding_tools/check-interfaces.py' that checks whether zope interfa...
david-sarah [Thu, 15 Sep 2011 16:15:32 +0000 (09:15 -0700)]
Add a script 'misc/coding_tools/check-interfaces.py' that checks whether zope interfaces are enforced. Also add 'check-interfaces', 'version-and-path', and 'code-checks' targets to the Makefile. fixes #1474

13 years agomutable/publish.py: copy the self.writers dict before iterating over it, since we...
david-sarah [Sat, 24 Sep 2011 21:12:08 +0000 (14:12 -0700)]
mutable/publish.py: copy the self.writers dict before iterating over it, since we remove elements from it during the iteration. refs #393

13 years agomutable/publish.py: simplify by refactoring self.outstanding to self.num_outstanding...
david-sarah [Sat, 24 Sep 2011 20:50:04 +0000 (13:50 -0700)]
mutable/publish.py: simplify by refactoring self.outstanding to self.num_outstanding. refs #393

13 years agotest_mutable.py: update SkipTest message for test_publish_surprise_mdmf to reference...
david-sarah [Fri, 23 Sep 2011 21:16:22 +0000 (14:16 -0700)]
test_mutable.py: update SkipTest message for test_publish_surprise_mdmf to reference the right ticket number. refs #1540.

13 years agocontrol.py: unbreak speed-test: overwrite() wants a MutableData, not str
Brian Warner [Fri, 23 Sep 2011 07:37:48 +0000 (00:37 -0700)]
control.py: unbreak speed-test: overwrite() wants a MutableData, not str

Really, all the upload/modify APIs should take a string or a filehandle, and
internally wrap it as needed. Callers should not need to be aware of
Uploadable() or MutableData() classes.

13 years agotest_mutable.py: skip test_publish_surprise_mdmf, which is causing an error. refs...
david-sarah [Tue, 20 Sep 2011 18:33:19 +0000 (11:33 -0700)]
test_mutable.py: skip test_publish_surprise_mdmf, which is causing an error. refs #1534, #393

13 years agotest/test_mutable: write publish surprise test for MDMF, rename existing test_publish...
kevan [Sun, 18 Sep 2011 00:36:57 +0000 (17:36 -0700)]
test/test_mutable: write publish surprise test for MDMF, rename existing test_publish_surprise to clarify that it is for SDMF

13 years agotest/test_mutable: refactor publish surprise test into common test fixture, rewrite...
kevan [Sun, 18 Sep 2011 00:35:33 +0000 (17:35 -0700)]
test/test_mutable: refactor publish surprise test into common test fixture, rewrite test_publish_surprise to use test fixture

13 years agomutable/publish: add errback immediately after write, don't consume errors from other...
kevan [Sat, 17 Sep 2011 23:47:08 +0000 (16:47 -0700)]
mutable/publish: add errback immediately after write, don't consume errors from other parts of the publisher

13 years ago.darcs-boringfile: minor cleanups.
david-sarah [Tue, 20 Sep 2011 15:49:18 +0000 (08:49 -0700)]
.darcs-boringfile: minor cleanups.

13 years agouri.py: fix two interface violations in verifier URI classes. refs #1474
david-sarah [Tue, 20 Sep 2011 03:01:56 +0000 (20:01 -0700)]
uri.py: fix two interface violations in verifier URI classes. refs #1474

13 years agoMake platform-detection code tolerate linux-3.0, patch by zooko.
Brian Warner [Thu, 15 Sep 2011 20:26:20 +0000 (13:26 -0700)]
Make platform-detection code tolerate linux-3.0, patch by zooko.

Otherwise address-autodetection can't find ifconfig. refs #1536

13 years agotest_web.py: fix a bug in _count_leases that was causing us to check only the lease...
david-sarah [Thu, 15 Sep 2011 18:51:26 +0000 (11:51 -0700)]
test_web.py: fix a bug in _count_leases that was causing us to check only the lease count of one share file, not of all share files as intended.

13 years agodocs: insert a newline at the beginning of known_issues.rst to see if this makes...
Zooko O'Whielacronx [Wed, 14 Sep 2011 06:47:28 +0000 (23:47 -0700)]
docs: insert a newline at the beginning of known_issues.rst to see if this makes it render more nicely in trac

13 years agodocs: remove the coding: utf-8 declaration at the to of known_issues.rst, since the...
Zooko O'Whielacronx [Wed, 14 Sep 2011 05:57:13 +0000 (22:57 -0700)]
docs: remove the coding: utf-8 declaration at the to of known_issues.rst, since the trac rendering doesn't hide it

13 years agodocs: more cleanup of known_issues.rst -- now it passes "rst2html --verbose" without...
Zooko O'Whielacronx [Wed, 14 Sep 2011 05:54:19 +0000 (22:54 -0700)]
docs: more cleanup of known_issues.rst -- now it passes "rst2html --verbose" without comment

13 years agodocs: more formatting improvements to known_issues.rst
Zooko O'Whielacronx [Wed, 14 Sep 2011 05:16:39 +0000 (22:16 -0700)]
docs: more formatting improvements to known_issues.rst

13 years agodocs: reformatting of known_issues.rst
Zooko O'Whielacronx [Wed, 14 Sep 2011 05:02:40 +0000 (22:02 -0700)]
docs: reformatting of known_issues.rst

13 years agodocs: fix formatting error in docs/known_issues.rst
Zooko O'Whielacronx [Wed, 14 Sep 2011 04:59:09 +0000 (21:59 -0700)]
docs: fix formatting error in docs/known_issues.rst

13 years agomerge Tahoe-LAFS v1.8.3 release announcement with trunk
Zooko O'Whielacronx [Tue, 13 Sep 2011 21:05:44 +0000 (14:05 -0700)]
merge Tahoe-LAFS v1.8.3 release announcement with trunk

13 years agodocs: release notes for Tahoe-LAFS v1.8.3
Zooko O'Whielacronx [Tue, 13 Sep 2011 16:58:26 +0000 (09:58 -0700)]
docs: release notes for Tahoe-LAFS v1.8.3

13 years agotests: bump up the timeout in this test that fails on FreeStorm's CentOS in order...
Zooko O'Whielacronx [Tue, 13 Sep 2011 02:42:55 +0000 (19:42 -0700)]
tests: bump up the timeout in this test that fails on FreeStorm's CentOS in order to see if it is just very slow

13 years agointerfaces: document that the 'fills-holes-with-zero-bytes' key should be used to...
david-sarah [Tue, 13 Sep 2011 00:28:43 +0000 (17:28 -0700)]
interfaces: document that the 'fills-holes-with-zero-bytes' key should be used to detect whether a storage server has that behavior. refs #1528

13 years agoCREDITS: more CREDITS for Kevan and David-Sarah
Zooko O'Whielacronx [Mon, 12 Sep 2011 22:33:57 +0000 (15:33 -0700)]
CREDITS: more CREDITS for Kevan and David-Sarah

13 years agomerge NEWS about the mutable file bounds fixes with NEWS about work-in-progress
Zooko O'Whielacronx [Tue, 13 Sep 2011 20:55:21 +0000 (13:55 -0700)]
merge NEWS about the mutable file bounds fixes with NEWS about work-in-progress

13 years agodoc: add NEWS item about fixes to potential palimpsest issues in mutable files
Zooko O'Whielacronx [Mon, 12 Sep 2011 22:33:29 +0000 (15:33 -0700)]
doc: add NEWS item about fixes to potential palimpsest issues in mutable files
ref. #1528

13 years agomerge the NEWS about the security fix (#1528) with the work-in-progress NEWS
Zooko O'Whielacronx [Tue, 13 Sep 2011 20:51:53 +0000 (13:51 -0700)]
merge the NEWS about the security fix (#1528) with the work-in-progress NEWS

13 years agodoc: add NEWS entry about the issue which allows unauthorized deletion of shares
Zooko O'Whielacronx [Mon, 12 Sep 2011 22:32:46 +0000 (15:32 -0700)]
doc: add NEWS entry about the issue which allows unauthorized deletion of shares
ref. #1528

13 years agodoc: add entry in known_issues.rst about the issue which allows unauthorized deletion...
Zooko O'Whielacronx [Mon, 12 Sep 2011 22:31:35 +0000 (15:31 -0700)]
doc: add entry in known_issues.rst about the issue which allows unauthorized deletion of shares
ref. #1528

13 years agostorage: more paranoid handling of bounds and palimpsests in mutable share files
Zooko O'Whielacronx [Mon, 12 Sep 2011 22:26:55 +0000 (15:26 -0700)]
storage: more paranoid handling of bounds and palimpsests in mutable share files
* storage server ignores requests to extend shares by sending a new_length
* storage server fills exposed holes (created by sending a write vector whose offset begins after the end of the current data) with 0 to avoid "palimpsest" exposure of previous contents
* storage server zeroes out lease info at the old location when moving it to a new location
ref. #1528

13 years agostorage: test that the storage server ignores requests to extend shares by sending...
Zooko O'Whielacronx [Mon, 12 Sep 2011 22:25:54 +0000 (15:25 -0700)]
storage: test that the storage server ignores requests to extend shares by sending a new_length, and that the storage server fills exposed holes with 0 to avoid "palimpsest" exposure of previous contents
ref. #1528

13 years agoimmutable: prevent clients from reading past the end of share data, which would allow...
Zooko O'Whielacronx [Mon, 12 Sep 2011 22:24:58 +0000 (15:24 -0700)]
immutable: prevent clients from reading past the end of share data, which would allow them to learn the cancellation secret
Declare explicitly that we prevent this problem in the server's version dict.
fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)

13 years agostorage: remove the storage server's "remote_cancel_lease" function
Zooko O'Whielacronx [Mon, 12 Sep 2011 22:23:31 +0000 (15:23 -0700)]
storage: remove the storage server's "remote_cancel_lease" function
We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file.
fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)

13 years agostorage: test that the storage server does *not* have a "remote_cancel_lease" function
Zooko O'Whielacronx [Mon, 12 Sep 2011 22:23:24 +0000 (15:23 -0700)]
storage: test that the storage server does *not* have a "remote_cancel_lease" function
We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file.
ref. #1528

13 years agoimmutable: test whether the server allows clients to read past the end of share data...
Zooko O'Whielacronx [Mon, 12 Sep 2011 22:12:01 +0000 (15:12 -0700)]
immutable: test whether the server allows clients to read past the end of share data, which would allow them to learn the cancellation secret
Also test whether the server explicitly declares that it prevents this problem.
ref #1528

13 years agoRetrieve._activate_enough_peers: rewrite Verify logic
Brian Warner [Mon, 5 Sep 2011 19:36:26 +0000 (12:36 -0700)]
Retrieve._activate_enough_peers: rewrite Verify logic

13 years agoRetrieve: implement/test stopProducing
Brian Warner [Mon, 5 Sep 2011 19:02:42 +0000 (12:02 -0700)]
Retrieve: implement/test stopProducing

13 years agomove DownloadStopped from download.common to interfaces
Brian Warner [Sun, 4 Sep 2011 02:59:06 +0000 (19:59 -0700)]
move DownloadStopped from download.common to interfaces

13 years agoretrieve.py: remove vestigal self._validated_readers
Brian Warner [Mon, 5 Sep 2011 18:54:04 +0000 (11:54 -0700)]
retrieve.py: remove vestigal self._validated_readers

13 years agoRetrieve: rewrite flow-control: use a top-level loop() to catch all errors
Brian Warner [Mon, 5 Sep 2011 18:38:44 +0000 (11:38 -0700)]
Retrieve: rewrite flow-control: use a top-level loop() to catch all errors

This ought to close the potential for dropped errors and hanging downloads.
Verify needs to be examined, I may have broken it, although all tests pass.

13 years agoRetrieve: merge _validate_active_prefixes into _add_active_peers
Brian Warner [Mon, 5 Sep 2011 08:31:18 +0000 (01:31 -0700)]
Retrieve: merge _validate_active_prefixes into _add_active_peers

13 years agoRetrieve: remove the initial prefix-is-still-good check
Brian Warner [Mon, 5 Sep 2011 08:26:33 +0000 (01:26 -0700)]
Retrieve: remove the initial prefix-is-still-good check

This check needs to be done with each fetch from the storage server, to
detect when someone has changed the share (i.e. our servermap goes stale).
Doing it just once at the beginning of retrieve isn't enough: a write might
occur after the first segment but before the second, etc.

_try_to_validate_prefix() was not removed: it will be used by the future
check-with-each-fetch code.

test_mutable.Roundtrip.test_corrupt_all_seqnum_late was disabled, since it
fails until this check is brought back. (the corruption it applies only
touches the prefix, not the block data, so the check-less retrieve actually
tolerates it). Don't forget to re-enable it once the check is brought back.

13 years agoMDMFSlotReadProxy: remove the queue
Brian Warner [Mon, 5 Sep 2011 19:04:08 +0000 (12:04 -0700)]
MDMFSlotReadProxy: remove the queue

This is a neat trick to reduce Foolscap overhead, but the need for an
explicit flush() complicates the Retrieve path and makes it prone to
lost-progress bugs.

Also change test_mutable.FakeStorageServer to tolerate multiple reads of the
same share in a row, a limitation exposed by turning off the queue.

13 years agorearrange Retrieve: first step, shouldn't change order of execution
Brian Warner [Sun, 4 Sep 2011 03:21:18 +0000 (20:21 -0700)]
rearrange Retrieve: first step, shouldn't change order of execution

13 years agoCLI: test_cli.py -- remove an unnecessary call in test_mkdir_mutable_type. refs ...
david-sarah [Tue, 6 Sep 2011 18:37:30 +0000 (11:37 -0700)]
CLI: test_cli.py -- remove an unnecessary call in test_mkdir_mutable_type. refs #1527

13 years agoCLI: improve test for 'tahoe mkdir --mutable-type='. refs #1527
david-sarah [Tue, 6 Sep 2011 18:30:20 +0000 (11:30 -0700)]
CLI: improve test for 'tahoe mkdir --mutable-type='. refs #1527

13 years agoCLI: make the --mutable-type option value for 'tahoe put' and 'tahoe mkdir' case...
david-sarah [Mon, 5 Sep 2011 02:09:22 +0000 (19:09 -0700)]
CLI: make the --mutable-type option value for 'tahoe put' and 'tahoe mkdir' case-insensitive, and change --help for these commands accordingly. fixes #1527

13 years agocli: make --mutable-type imply --mutable in 'tahoe put'
Kevan Carstensen [Sat, 3 Sep 2011 19:09:20 +0000 (12:09 -0700)]
cli: make --mutable-type imply --mutable in 'tahoe put'

13 years agoSFTP: add a comment about a subtle interaction between OverwriteableFileConsumer...
david-sarah [Sat, 3 Sep 2011 22:23:04 +0000 (15:23 -0700)]
SFTP: add a comment about a subtle interaction between OverwriteableFileConsumer and GeneralSFTPFile, and test the case it is commenting on.