Kevan Carstensen [Fri, 14 May 2010 20:35:16 +0000 (13:35 -0700)]
Remove a comment that no longer makes sense.
Zooko O'Whielacronx [Fri, 14 May 2010 04:34:58 +0000 (21:34 -0700)]
docs: update docs/architecture.txt to more fully and correctly explain the upload procedure
Kevan Carstensen [Fri, 14 May 2010 00:49:17 +0000 (17:49 -0700)]
Fix up the behavior of #778, per reviewers' comments
- Make some important utility functions clearer and more thoroughly
documented.
- Assert in upload.servers_of_happiness that the buckets attributes
of PeerTrackers passed to it are mutually disjoint.
- Get rid of some silly non-Pythonisms that I didn't see when I first
wrote these patches.
- Make sure that should_add_server returns true when queried about a
shnum that it doesn't know about yet.
- Change Tahoe2PeerSelector.preexisting_shares to map a shareid to a set
of peerids, alter dependencies to deal with that.
- Remove upload.should_add_servers, because it is no longer necessary
- Move upload.shares_of_happiness and upload.shares_by_server to a utility
file.
- Change some points in Tahoe2PeerSelector.
- Compute servers_of_happiness using a bipartite matching algorithm that
we know is optimal instead of an ad-hoc greedy algorithm that isn't.
- Change servers_of_happiness to just take a sharemap as an argument,
change its callers to merge existing_shares and used_peers before
calling it.
- Change an error message in the encoder to be more appropriate for
servers of happiness.
- Clarify the wording of an error message in immutable/upload.py
- Refactor a happiness failure message to happinessutil.py, and make
immutable/upload.py and immutable/encode.py use it.
- Move the word "only" as far to the right as possible in failure
messages.
- Use a better definition of progress during peer selection.
- Do read-only peer share detection queries in parallel, not sequentially.
- Clean up logging semantics; print the query statistics whenever an
upload is unsuccessful, not just in one case.
Kevan Carstensen [Wed, 30 Dec 2009 22:03:44 +0000 (15:03 -0700)]
Alter the error message when an upload fails, per some comments in #778.
When I first implemented #778, I just altered the error messages to refer to
servers where they referred to shares. The resulting error messages weren't
very good. These are a bit better.
Kevan Carstensen [Sat, 5 Dec 2009 05:30:37 +0000 (22:30 -0700)]
Change "UploadHappinessError" to "UploadUnhappinessError"
Kevan Carstensen [Mon, 23 Nov 2009 01:24:05 +0000 (18:24 -0700)]
Alter the error message returned when peer selection fails
The Tahoe2PeerSelector returned either NoSharesError or NotEnoughSharesError
for a variety of error conditions that weren't informatively described by them.
This patch creates a new error, UploadHappinessError, replaces uses of
NoSharesError and NotEnoughSharesError with it, and alters the error message
raised with the errors to be more in line with the new servers_of_happiness
behavior. See ticket #834 for more information.
Kevan Carstensen [Wed, 18 Nov 2009 02:45:42 +0000 (19:45 -0700)]
Eliminate overcounting iof servers_of_happiness in Tahoe2PeerSelector; also reorganize some things.
Kevan Carstensen [Mon, 16 Nov 2009 22:24:59 +0000 (15:24 -0700)]
Change stray "shares_of_happiness" to "servers_of_happiness"
Kevan Carstensen [Mon, 16 Nov 2009 20:28:05 +0000 (13:28 -0700)]
Alter Tahoe2PeerSelector to make sure that it recognizes existing shares on readonly servers, fixing an issue in #778
Kevan Carstensen [Wed, 4 Nov 2009 12:12:22 +0000 (05:12 -0700)]
Alter 'immutable/encode.py' and 'immutable/upload.py' to use servers_of_happiness instead of shares_of_happiness.
Kevan Carstensen [Wed, 4 Nov 2009 04:32:41 +0000 (21:32 -0700)]
Alter the signature of set_shareholders in IEncoder to add a 'servermap' parameter, which gives IEncoders enough information to perform a sane check for servers_of_happiness.
Kevan Carstensen [Thu, 24 Sep 2009 04:19:32 +0000 (21:19 -0700)]
Alter CiphertextDownloader to work with servers_of_happiness
Kevan Carstensen [Fri, 14 May 2010 01:25:42 +0000 (18:25 -0700)]
Revisions of the #778 tests, per reviewers' comments
- Fix comments and confusing naming.
- Add tests for the new error messages suggested by David-Sarah
and Zooko.
- Alter existing tests for new error messages.
- Make sure that the tests continue to work with the trunk.
- Add a test for a mutual disjointedness assertion that I added to
upload.servers_of_happiness.
- Fix the comments to correctly reflect read-onlyness
- Add a test for an edge case in should_add_server
- Add an assertion to make sure that share redistribution works as it
should
- Alter tests to work with revised servers_of_happiness semantics
- Remove tests for should_add_server, since that function no longer exists.
- Alter tests to know about merge_peers, and to use it before calling
servers_of_happiness.
- Add tests for merge_peers.
- Add Zooko's puzzles to the tests.
- Edit encoding tests to expect the new kind of failure message.
- Edit tests to expect error messages with the word "only" moved as far
to the right as possible.
- Extended and cleaned up some helper functions.
- Changed some tests to call more appropriate helper functions.
- Added a test for the failing redistribution algorithm
- Added a test for the progress message
- Added a test for the upper bound on readonly peer share discovery.
Kevan Carstensen [Thu, 7 Jan 2010 19:13:25 +0000 (12:13 -0700)]
Alter various unit tests to work with the new happy behavior
Kevan Carstensen [Sat, 5 Dec 2009 05:34:53 +0000 (22:34 -0700)]
Replace "UploadHappinessError" with "UploadUnhappinessError" in tests.
Kevan Carstensen [Mon, 23 Nov 2009 02:20:08 +0000 (19:20 -0700)]
Add tests for the behavior described in #834.
Kevan Carstensen [Mon, 16 Nov 2009 20:23:34 +0000 (13:23 -0700)]
Re-work 'test_upload.py' to be more readable; add more tests for #778
Kevan Carstensen [Mon, 9 Nov 2009 01:37:35 +0000 (18:37 -0700)]
Test Tahoe2PeerSelector to make sure that it recognizeses existing shares on readonly servers
Kevan Carstensen [Wed, 4 Nov 2009 12:28:49 +0000 (05:28 -0700)]
Add more tests for comment:53 in ticket #778
Kevan Carstensen [Wed, 4 Nov 2009 12:13:24 +0000 (05:13 -0700)]
Add a test for upload.shares_by_server
Kevan Carstensen [Wed, 4 Nov 2009 04:42:32 +0000 (21:42 -0700)]
Minor tweak to an existing test -- make the first server read-write, instead of read-only
Kevan Carstensen [Wed, 4 Nov 2009 04:36:02 +0000 (21:36 -0700)]
Alter tests to use the new form of set_shareholders
"Kevan Carstensen" [Fri, 30 Oct 2009 09:19:08 +0000 (02:19 -0700)]
Refactor some behavior into a mixin, and add tests for the behavior described in #778
Kevan Carstensen [Sun, 18 Oct 2009 01:30:13 +0000 (18:30 -0700)]
Alter NoNetworkGrid to allow the creation of readonly servers for testing purposes.
kevan [Fri, 14 May 2010 00:38:52 +0000 (17:38 -0700)]
Update 'docs/architecture.txt' to reflect readonly share discovery
Kevan Carstensen [Wed, 28 Apr 2010 00:24:55 +0000 (17:24 -0700)]
Alter the wording in docs/architecture.txt to more accurately describe the servers_of_happiness behavior.
"Kevan Carstensen" [Sat, 5 Dec 2009 04:40:05 +0000 (21:40 -0700)]
Alter wording in 'interfaces.py' to be correct wrt #778
Kevan Carstensen [Sat, 5 Dec 2009 04:38:13 +0000 (21:38 -0700)]
Update 'docs/configuration.txt' to reflect the servers_of_happiness behavior.
david-sarah [Tue, 11 May 2010 18:03:00 +0000 (11:03 -0700)]
Clarify quickstart instructions for installing pywin32
Zooko O'Whielacronx [Mon, 10 May 2010 06:37:29 +0000 (23:37 -0700)]
web: add a simple test that you can load directory.xhtml
Zooko O'Whielacronx [Mon, 10 May 2010 06:36:15 +0000 (23:36 -0700)]
setup: fix typos in misc/show-tool-versions.py
Zooko O'Whielacronx [Mon, 10 May 2010 06:29:55 +0000 (23:29 -0700)]
setup: show code-coverage tool versions in show-tools-versions.py
Zooko O'Whielacronx [Tue, 4 May 2010 09:43:40 +0000 (02:43 -0700)]
docs: update README, mv it to README.txt, update setup.py
david-sarah [Tue, 4 May 2010 16:10:43 +0000 (09:10 -0700)]
Dependency on Windmill test framework is not needed yet.
Zooko O'Whielacronx [Tue, 4 May 2010 09:06:28 +0000 (02:06 -0700)]
tests: pass z to tar so that BSD tar will know to ungzip
Zooko O'Whielacronx [Tue, 4 May 2010 06:16:53 +0000 (23:16 -0700)]
setup: update comments and URLs in setup.cfg
Zooko O'Whielacronx [Tue, 4 May 2010 04:56:43 +0000 (21:56 -0700)]
setup: reorder and extend the show-tool-versions script, the better to glean information about our new buildslaves
Francois Deppierraz [Fri, 30 Apr 2010 18:56:09 +0000 (11:56 -0700)]
CLI: Support for https url in option --node-url
This patch modifies the regular expression used for verifying of '--node-url'
parameter. Support for accessing a Tahoe gateway over HTTPS was already
present, thanks to Python's urllib.
Brian Warner [Wed, 28 Apr 2010 05:08:03 +0000 (22:08 -0700)]
backupdb.did_create_directory: use REPLACE INTO, not INSERT INTO + ignore error
This handles the case where we upload a new tahoe directory for a
previously-processed local directory, possibly creating a new dircap (if the
metadata had changed). Now we replace the old dirhash->dircap record. The
previous behavior left the old record in place (with the old dircap and
timestamps), so we'd never stop creating new directories and never converge
on a null backup.
Brian Warner [Sat, 24 Apr 2010 23:30:03 +0000 (16:30 -0700)]
"tahoe webopen": add --info flag, to get ?t=info
Also fix some trailing whitespace.
Zooko O'Whielacronx [Wed, 21 Apr 2010 16:57:08 +0000 (09:57 -0700)]
docs: install.html http-equiv refresh to quickstart.html
Zooko O'Whielacronx [Wed, 21 Apr 2010 15:57:57 +0000 (08:57 -0700)]
docs: install.html -> quickstart.html
It is not called "installing" because that implies that it is going to change the configuration of your operating system. It is not called "building" because that implies that you need developer tools like a compiler. Also I added a stern warning against looking at the "InstallDetails" wiki page, which I have renamed to "AdvancedInstall".
david-sarah [Fri, 16 Apr 2010 22:09:35 +0000 (15:09 -0700)]
Fix another typo in tahoe_storagespace munin plugin
david-sarah [Fri, 16 Apr 2010 19:04:04 +0000 (12:04 -0700)]
Add dependency on windmill >= 1.3
Zooko O'Whielacronx [Wed, 14 Apr 2010 23:25:21 +0000 (16:25 -0700)]
licensing: phrase the OpenSSL-exemption in the vocabulary of copyright instead of computer technology, and replicate the exemption from the GPL to the TGPPL
freestorm77 [Sun, 21 Feb 2010 21:36:26 +0000 (14:36 -0700)]
munin-tahoe_storagespace
Plugin configuration rename
Zooko O'Whielacronx [Tue, 9 Mar 2010 18:44:15 +0000 (10:44 -0800)]
setup: add licensing declaration for setuptools (noticed by the FSF compliance folks)
Zooko O'Whielacronx [Tue, 9 Mar 2010 16:37:36 +0000 (08:37 -0800)]
setup: fix error in licensing declaration from Shawn Willden, as noted by the FSF compliance division
Zooko O'Whielacronx [Thu, 4 Mar 2010 01:56:16 +0000 (17:56 -0800)]
CREDITS to Jacob Appelbaum
jacob [Thu, 4 Mar 2010 01:38:58 +0000 (17:38 -0800)]
desert-island-build-with-proper-versions
Zooko O'Whielacronx [Wed, 3 Mar 2010 23:19:02 +0000 (15:19 -0800)]
docs: a few small edits to try to guide newcomers through the docs
These edits were suggested by my watching over Jake Appelbaum's shoulder as he completely ignored/skipped/missed install.html and also as he decided that debian.txt wouldn't help him with basic installation. Then I threw in a few docs edits that have been sitting around in my sandbox asking to be committed for months.
david-sarah [Sun, 28 Feb 2010 06:23:14 +0000 (22:23 -0800)]
TAG allmydata-tahoe-1.6.1
david-sarah [Sun, 28 Feb 2010 06:19:41 +0000 (22:19 -0800)]
Change install.html to reference 1.6.1 instead of 1.6.0
Zooko O'Whielacronx [Sun, 28 Feb 2010 06:14:39 +0000 (22:14 -0800)]
docs: fix the asymptotic network performance of mutable file download in performance.txt, rename the howto-make-a-release file
david-sarah [Sat, 27 Feb 2010 06:55:51 +0000 (22:55 -0800)]
Change code that gives a base32 SI or an empty string to be more straightforward. (#948)
david-sarah [Thu, 25 Feb 2010 04:18:24 +0000 (20:18 -0800)]
Additional test for DIR2-LIT directories in test_web.py, fixed version (#948)
david-sarah [Wed, 24 Feb 2010 08:15:42 +0000 (00:15 -0800)]
Updates to NEWS for 1.6.1
david-sarah [Wed, 24 Feb 2010 08:02:20 +0000 (00:02 -0800)]
Additional fixes for DIR2-LIT More Info page and deep-check/manifest operations (#948)
david-sarah [Wed, 24 Feb 2010 07:54:33 +0000 (23:54 -0800)]
directories: add DIR2-LIT directories to test_deepcheck.py (#948)
david-sarah [Wed, 24 Feb 2010 04:33:45 +0000 (20:33 -0800)]
dirnode: add tests of literal dirnodes (current and fix for #948)
david-sarah [Mon, 22 Feb 2010 03:36:52 +0000 (19:36 -0800)]
Additional fix for abbrev_si, with test
david-sarah [Mon, 22 Feb 2010 02:53:52 +0000 (18:53 -0800)]
Additions to test_web.py for #948
david-sarah [Mon, 22 Feb 2010 02:45:04 +0000 (18:45 -0800)]
Change direct accesses to an_uri.storage_index to calls to .get_storage_index() (fixes #948)
david-sarah [Wed, 24 Feb 2010 03:02:31 +0000 (19:02 -0800)]
Tweak to 'tahoe ls --help' output (#837)
david-sarah [Wed, 24 Feb 2010 02:59:13 +0000 (18:59 -0800)]
Test behaviour of 'tahoe ls' for unknown objects (#837)
david-sarah [Sat, 20 Feb 2010 06:13:13 +0000 (22:13 -0800)]
Improve behaviour of 'tahoe ls' for unknown objects, addressing kevan's comments
Zooko O'Whielacronx [Wed, 24 Feb 2010 06:57:55 +0000 (22:57 -0800)]
docs: update relnotes.txt for v1.6.1
Zooko O'Whielacronx [Wed, 24 Feb 2010 06:52:31 +0000 (22:52 -0800)]
docs: NEWS and relnotes-short.txt and CREDITS for v1.6.1
Brian Warner [Wed, 24 Feb 2010 04:47:57 +0000 (23:47 -0500)]
misc/coverage.el: improve filename matching
Brian Warner [Wed, 24 Feb 2010 04:46:37 +0000 (23:46 -0500)]
test_util.py: improve coverage of util.time_format
Brian Warner [Wed, 24 Feb 2010 04:38:13 +0000 (23:38 -0500)]
docs/performance.txt: split out CPU from network, expand on mutable costs
Brian Warner [Tue, 23 Feb 2010 21:04:02 +0000 (16:04 -0500)]
docs/FTP: the Twisted patch (t3462) has landed, will be in the next release
Kevan Carstensen [Sat, 20 Feb 2010 21:07:13 +0000 (13:07 -0800)]
Change OphandleTable to use a deterministic clock, so we can test it
To test the changes for #577, we need a deterministic way to simulate
the passage of long periods of time. twisted.internet.task.Clock seems,
from my Googling, to be the way to go for this functionality. I changed
a few things so that OphandleTable would use twisted.internet.task.Clock
when testing:
* WebishServer.__init___ now takes an optional 'clock' parameter,
* which it passes to the root.Root instance it creates.
* root.Root.__init__ now takes an optional 'clock' parameter, which it
passes to the OphandleTable.__init__ method.
* OphandleTable.__init__ now takes an optional 'clock' parameter. If
it is provided, and it isn't None, its callLater method will be used
to schedule ophandle expirations (as opposed to using
reactor.callLater, which is what OphandleTable does normally).
* The WebMixin object in test_web.py now sets a self.clock parameter,
which is a twisted.internet.task.Clock that it feeds to the
WebishServer it creates.
Tests using the WebMixin can control the passage of time in
OphandleTable by accessing self.clock.
Kevan Carstensen [Sun, 21 Feb 2010 01:04:55 +0000 (17:04 -0800)]
Add tests for the ophandle expiration behavior in #577
Kevan Carstensen [Sun, 21 Feb 2010 01:07:16 +0000 (17:07 -0800)]
Update docs/frontends/webapi.txt to reflect the new expiration times in #577
Kevan Carstensen [Sun, 21 Feb 2010 01:05:12 +0000 (17:05 -0800)]
Increase ophandle expiration times, per #577
david-sarah [Sat, 6 Feb 2010 01:38:55 +0000 (17:38 -0800)]
More cleanups to test_cli using new utilities for reading and writing files.
david-sarah [Sat, 6 Feb 2010 01:37:27 +0000 (17:37 -0800)]
Fix race conditions and missing callback in allmydata.test.test_cli.Cp.test_copy_using_filecap, add utilities for one-liner reading and writing of files, and fix cases in test_cli where files were not being closed after writing.
Zooko O'Whielacronx [Mon, 15 Feb 2010 05:08:44 +0000 (21:08 -0800)]
setup: comment-out the dependency on pycrypto, see #953
Kevan Carstensen [Fri, 12 Feb 2010 06:21:37 +0000 (22:21 -0800)]
Add tests for #939
Kevan Carstensen [Thu, 11 Feb 2010 02:43:18 +0000 (18:43 -0800)]
Alter CLI utilities to handle nonexistent aliases better
Peter Secor [Sat, 6 Feb 2010 05:43:14 +0000 (21:43 -0800)]
adding pycrypto to the auto dependencies
Peter Secor [Sat, 6 Feb 2010 16:53:20 +0000 (08:53 -0800)]
docs running.html - "tahoe run ." does not work with the current installation, replaced with "tahoe start ."
Brian Warner [Mon, 8 Feb 2010 00:20:10 +0000 (16:20 -0800)]
web/storage.py: display total-seen on the last-complete-cycle line. For #940.
Brian Warner [Thu, 28 Jan 2010 17:39:04 +0000 (09:39 -0800)]
code coverage: replace figleaf with coverage.py, should work on py2.6 now.
It still lacks the right HTML report (the builtin report is very pretty, but
lacks the "lines uncovered" numbers that I want), and the half-finished
delta-from-last-run measurements.
david-sarah [Tue, 2 Feb 2010 06:12:56 +0000 (22:12 -0800)]
More comprehensive changes and ticket references for NEWS
Zooko O'Whielacronx [Tue, 2 Feb 2010 06:58:52 +0000 (22:58 -0800)]
docs: install.html: link into Python 2.5.5 download page
Zooko O'Whielacronx [Tue, 2 Feb 2010 06:11:25 +0000 (22:11 -0800)]
TAG allmydata-tahoe-1.6.0
Zooko O'Whielacronx [Tue, 2 Feb 2010 05:38:42 +0000 (21:38 -0800)]
docs: install.html: recommend Python 2.5 (because I can build extension modules for it with mingw), architecture.txt: point out that our Proof of Retrievability feature is client-side-only
Brian Warner [Tue, 2 Feb 2010 05:53:04 +0000 (21:53 -0800)]
architecture.txt: remove trailing whitespace, wrap lines: no content changes
Zooko O'Whielacronx [Tue, 2 Feb 2010 05:48:32 +0000 (21:48 -0800)]
docs: a couple of small edits to release notes (thanks Peter)
Zooko O'Whielacronx [Tue, 2 Feb 2010 05:38:31 +0000 (21:38 -0800)]
docs: CREDITS: where due
Zooko O'Whielacronx [Tue, 2 Feb 2010 05:27:50 +0000 (21:27 -0800)]
docs: a few small edits to performance.txt and README
Kevan Carstensen [Tue, 2 Feb 2010 00:59:14 +0000 (16:59 -0800)]
Fill in 'docs/performance.txt' with some performance information
Zooko O'Whielacronx [Tue, 2 Feb 2010 04:51:17 +0000 (20:51 -0800)]
docs: a few edits to architecture.txt, most significantly highlighting "future work" to avoid confusing it with the current version, and adding a "future work" about a random-sampling Proof of Retrievability verifier
Zooko O'Whielacronx [Tue, 2 Feb 2010 04:32:22 +0000 (20:32 -0800)]
docs: a few edits and updates to relnotes.txt, relnotes-short.txt, and NEWS in preparation for v1.6.0
david-sarah [Tue, 2 Feb 2010 01:52:38 +0000 (17:52 -0800)]
Document leakage of cap URLs via phishing filters in known_issues.txt
Zooko O'Whielacronx [Mon, 1 Feb 2010 18:18:09 +0000 (10:18 -0800)]
docs: updates to relnotes.txt, NEWS, architecture, historical_known_issues, install.html, etc.
Zooko O'Whielacronx [Mon, 1 Feb 2010 06:16:10 +0000 (22:16 -0800)]
immutable: downloader accepts notifications of buckets even if those notifications arrive after he has begun downloading shares.
This can be useful if one of the ones that he has already begun downloading fails. See #287 for discussion. This fixes part of #287 which part was a regression caused by #928, namely this fixes fail-over in case a share is corrupted (or the server returns an error or disconnects). This does not fix the related issue mentioned in #287 if a server hangs and doesn't reply to requests for blocks.
Zooko O'Whielacronx [Mon, 1 Feb 2010 05:23:23 +0000 (21:23 -0800)]
tests: don't require tahoe to run with no noise if we are using an old twisted that emits DeprecationWarnings