docs: introducing "provider-independent security"; Nathan Wilcox gave me good advice on how to be more specific in terminology when making security claims so as not fit in with the background noise of overblown and indefensible claims that is always buzzing in the security world
docs: update NEWS, about.html, relnotes-short.txt, and known_issues.txt in preparation for v1.5.0
Especially note that strong claims of specialness that I've added, e.g. in about.html .
Brian Warner [Mon, 20 Jul 2009 15:38:03 +0000 (11:38 -0400)]
Touch up #705 changes:
webapi.txt: clarify replace=only-files argument, mention replace= on POST t=uri
test_cli.py: insert whitespace between logical operations
web.common.parse_replace_arg: make it case-insensitive, to match the docs
introduction/storage-broker: add assertions of serverids being the right type and of internal consistency between the number of storage servers known and the number of storage servers
Brian Warner [Thu, 16 Jul 2009 23:01:20 +0000 (18:01 -0500)]
upload: fix #758 recursion-loop in peer-selection when servers report errors.
The bug was in the code that handles a third-or-later pass, and was
previously untested.
Brian Warner [Wed, 15 Jul 2009 06:45:10 +0000 (23:45 -0700)]
Allow tests to pass with -OO by turning some AssertionErrors (the ones that
we actually exercise during tests) into more specific exceptions, so they
don't get optimized away. The best rule to follow is probably this: if an
exception is worth testing, then it's part of the API, and AssertionError
should never be part of the API. Closes #749.
Brian Warner [Sun, 12 Jul 2009 23:47:50 +0000 (00:47 +0100)]
dirnode.py: security bug: also use child writecap to derive child enc key,
not just the dirnode writecap. The previous code (which only hashed the
dirnode writecap) would use the same key for all children, which is very bad.
This is the correct implementation of #750.
directories: make the IV for the writecaps in directory entries be computed from the secure hash of the writecap itself
This makes encoding of directory entries deterministic, and it is also a tad faster on Macbook Pro than getting a random IV with os.urandom(16).
directories: make initialization of the download cache lazy
If you open up a directory containing thousands of files, it currently computes the cache filename and checks for the cache file on disk immediately for each immutble file in that directory. With this patch, it delays those steps until you try to do something with an immutable file that could use the cache.
directories: rename internal data member download_cache to download_cache_dirman (benchmarks set this internal member and use the new name, so changing this makes the bench_dirnode.py work again)
directories: keep track of your position as you decode netstring after netstring from an input buffer instead of copying the trailing part
This makes decoding linear in the number of netstrings instead of O(N^2).
directories: in bench_dirnode.py, use a real CacheDirectoryManager instead of a fake one (because CacheDirectoryManager is a significant user of CPU and/or time)
test/benchmark: benchmark the time to pack and unpack dirnodes
See also the prof_benchmarks() function in this file which will run the benchmarks under profiling.
Brian Warner [Fri, 3 Jul 2009 07:28:04 +0000 (00:28 -0700)]
clean up debian packaging: we have control files for etch/lenny/sid, and
everything else uses one of those. Add dependency on python-pysqlite2 for
platforms that use py2.4 by default. Update foolscap dependency to 0.4.1.
Brian Warner [Fri, 3 Jul 2009 01:07:49 +0000 (18:07 -0700)]
Tolerate unknown URI types in directory structures. Part of #683.
The idea is that future versions of Tahoe will add new URI types that this
version won't recognize, but might store them in directories that we *can*
read. We should handle these "objects from the future" as best we can.
Previous releases of Tahoe would just explode. With this change, we'll
continue to be able to work with everything else in the directory.
The code change is to wrap anything we don't recognize as an UnknownNode
instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how
to render these (mostly by leaving fields blank), deep-check knows to skip
over them, deep-stats counts them in "count-unknown". You can rename and
delete these things, but you can't add new ones (because we wouldn't know how
to generate a readcap to put into the dirnode's rocap slot, and because this
lets us catch typos better).
Brian Warner [Wed, 1 Jul 2009 01:13:43 +0000 (18:13 -0700)]
mutable repairer: skip repair of readcaps instead of throwing an exception.
This should improve the behavior of #625 a bit: at least all the files will
get repaired.
Brian Warner [Mon, 29 Jun 2009 22:31:24 +0000 (15:31 -0700)]
use 522-bit RSA keys in all unit tests (except one)
This reduces the total test time on my laptop from 400s to 283s.
* src/allmydata/test/test_system.py (SystemTest.test_mutable._test_debug):
Remove assertion about container_size/data_size, this changes with keysize
and was too variable anyways.
* src/allmydata/mutable/filenode.py (MutableFileNode.create): add keysize=
* src/allmydata/dirnode.py (NewDirectoryNode.create): same
* src/allmydata/client.py (Client.DEFAULT_MUTABLE_KEYSIZE): add default,
this overrides the one in MutableFileNode
Brian Warner [Fri, 26 Jun 2009 06:20:22 +0000 (23:20 -0700)]
mutable/filenode.py: set _writekey to None, rather than leaving it missing
This will at least turn the really really weird error when a repair of a
readonly mutable file is attempted into a merely really weird assertion that
mentions "repair currently requires a writecap".