Mark Berger [Tue, 23 Jul 2013 16:53:22 +0000 (12:53 -0400)]
Adds _servers_with_shares to ServermapUpdater
When calculating the query boundary for updates to mutable files,
instead of using servers that used to have shares, use servers we
have added to the servermap. This way the querying process won't finish
until we have finished interacting with the servers that have shares.
This fixes the race condition which sometimes caused the querying process
to finish before the updater was done talking to servers with shares.
Brian Warner [Wed, 21 Jan 2015 18:31:31 +0000 (10:31 -0800)]
Appease req.setHeader by passing bytes, not ints.
twisted.web.http.Request.setHeader() really wants a "bytes" object, but
we've been passing integers like len(body). Twisted-12.3 started to
complain about this (with a DeprecationWarning), but the warning is
usually silenced because py2.7 disables deprecations by default.
This fixes Tahoe's misbehavior, but others remain (in Nevow, at least).
I plan to set up some tooling to run tests with
PYTHONWARNINGS=default::DeprecationWarning and collect others. We won't
be able to fix the ones that occur outside of Tahoe, but at least we
should be able to fix our own.
Nathan Wilcox [Wed, 21 Jan 2015 00:31:56 +0000 (16:31 -0800)]
Introduce icons with distinct shape for connection status display on the front wui page...
This replaces the status display which was only distinct by color which is a usability issue for color-blind users. This commit includes test coverage by way of pattern matching on rendered templates. The PNG icons are conversions of original SVG source which I've included and placed in the public domain.
Daira Hopwood [Tue, 20 Jan 2015 18:52:02 +0000 (10:52 -0800)]
dependecy specs: tolerate new PEP440 semantics too
The latest setuptools (version 8) changed the way dependency
specifications ("I can handle libfoo version 2 or 3, but not 4") are
interpreted. The new version follows PEP440, which is simpler but
somewhat less expressive. Tahoe's _auto_deps.py now uses dep-specs which
are correctly parsed by both old and new setuptools.
Fixes ticket:2354.
* Restrict the requirements in _auto_deps.py to work with either the old
or PEP 440 semantics.
* Update check_requirement and tests to take account of changes for PEP
440 compatibility.
* Fix an error message.
* Remove a superfluous TODO.
Leif Ryge [Thu, 20 Nov 2014 22:46:20 +0000 (22:46 +0000)]
add "Available" column to welcome page (#648)
add get_available_space() to NativeStorageServer
It uses a new 'available-space' key in the server's v1 version dict, or falls
back to 'maximum-immutable-share-size' (which presently always has the same
value but could have a different meaning in the future).
Daira Hopwood [Fri, 21 Nov 2014 08:38:37 +0000 (08:38 +0000)]
setup.py: set distribution.metadata.version from _version.py if we couldn't find it from git.
This simplifies reporting when we don't have the version. Also print the normalized version. refs #2340
Brian Warner [Wed, 23 May 2012 07:19:27 +0000 (00:19 -0700)]
'tahoe start': stop using the contents of .tac files
Instead of constructing a sys.argv for 'twistd' that reads the node's
.tac file, we construct arguments that tell twistd to use a special
in-memory-only plugin that creates the desired node instance directly.
We still use the name of the .tac file to decide which kind of instance
to make (Client, IntroducerNode, KeyGenerator, StatsGatherer), but never
actually read the contents of the .tac file. Later improvements could
change this to look inside the tahoe.cfg for a nodetype= directive, etc.
This also makes it easy to have "tahoe start BASEDIR" pass the rest of
its arguments on to twistd, so e.g. "tahoe start BASEDIR --nodaemon
--profile=prof.out" does what you'd expect "twistd --nodaemon
--profile=prof.out" to do. "tahoe run BASEDIR" is thus simply aliased to
"tahoe start BASEDIR --nodaemon". This removes the need to special-case
--profile and --syslog.
I also removed some of the default logging behavior:
after:
'tahoe start' = 'twistd --logfile BASEDIR logs/twistd.log'
unless --logfile, --nodaemon, or --syslog are passed
'tahoe start --profile' invalid, use 'tahoe start --profile=OUTPUT'
'tahoe run' = 'twistd --nodaemon'
so log messages go to stdout
This finally enables 'tahoe run' to work with all node types, including
the key-generator and stats-gatherer.
It gets 'tahoe start' one step closer to accepting --reactor= . To
actually accomplish this will require this file, the enclosing
__init_.py files, and everything they import to avoid importing the
reactor. (if anything imports twisted.internet.reactor before
startstop_node.start() gets to run, then --reactor= comes too late).
That will take a lot of work, and requires lazy-loading of many core
libraries (foolscap.logging in particular), and removing a lot of code
from src/allmydata/__init__.py .
Daira Hopwood [Tue, 7 Oct 2014 00:40:19 +0000 (01:40 +0100)]
misc/build_helpers/test-osx-pkg.py: script to test the OS X pkg.
After extracting the contents of the package, this script looks at the
output of 'tahoe --version-and-path' to see if the modules are installed
and invoked from the right path.
zancas [Tue, 7 Oct 2014 23:36:03 +0000 (17:36 -0600)]
exercise all branches of new helper "get_filenode_metadata"
Note fix following issues from origial commit:
refactor unittests, fix style, add test
(0) use CommonFixture as mixin to increase DRYness
(1) self.failUnlessIn('size', metadata.keys()) --> self.failUnlessIn('size', metdata)
(2) test_size_is_not_None --> test_size_is_0 AND test_size_is_1000
Daira Hopwood [Tue, 7 Oct 2014 18:28:53 +0000 (19:28 +0100)]
Specify the setuptools egg filename explicitly rather than globbing for it.
(Note that if the version changes in future, this will fail loudly so we'll know we have to change it.)
Brian Warner [Mon, 29 Sep 2014 15:43:20 +0000 (08:43 -0700)]
travis: remove the os.statvfs debug command
According to https://github.com/travis-ci/travis-ci/issues/2788 (which
was WONTFIXED), roughly half the travis-ci buildserver fleet runs off
tmpfs volumes, which have different notions of space-free and space-used
than real filesystems. We've adapted our tests to avoid relying upon
space-used>0, and no longer need this call to investigate the problem.
Solution was very simple to implement, no content disposition header was
necessary.
Tested with both Firefox and Chrome, using binary image file stored in
folder, as well as with text data using LIT cap.
Brian Warner [Thu, 11 Sep 2014 20:13:42 +0000 (13:13 -0700)]
Avoid Popen() of executables that don't exist
The stdlib 'subprocess' module in python-2.7.4 through 2.7.7 suffers
from http://bugs.python.org/issue18851 which causes unrelated file
descriptors to be closed when `subprocess.call()` fails the `exec()`,
such as when the executable being invoked does not actually exist. There
appears to be some randomness involved. This was fixed in python-2.7.8.
Tahoe's iputil.py uses subprocess.call on many different "ifconfig"-type
executables, most of which don't exist on any given platform (added in
git commit 8e31d66cd0b). This results in a lot of file-descriptor
closing, which (at least during unit tests) tends to clobber important
things like Tub TCP sockets. This seems to be the root cause behind
ticket:2121, in which normal code tries to close already-closed sockets,
crashing the unit tests. Since different platforms have different
ifconfigs, some platforms will experience more failed execs than others,
so this bug could easily behave differently on linux vs freebsd, as well
as working normally on python-2.7.8 or 2.7.4.
This patch inserts a guard to make sure that os.path.isfile() is true
before allowing Popen.call() to try executing the target. This ought to
be enough to avoid the bug. It changes both iputil.py and
allmydata.__init__ (which uses Popen for calling "lsb_release"), which
are all the places where 'subprocess' is used outside of unit tests.
Other potential fixes: use the 'subprocess32' module from PyPI (which is
a bug-free backport of the Python3 stdlib subprocess module, but would
introduce a new dependency), or require python >= 2.7.8 (but this would
rule out development/deployment on the current OS-X 10.9 release, which
ships with 2.7.5, as well as other distributions like Ubuntu 14.04 LTS).
I believe this closes ticket:2121, and given the apparent relationship
between 2121 and 2023, I think it also closes ticket:2023 (although
since 2023 doesn't have copies of the failing log files, it's hard to
tell). I'm hoping that this will tide us over until 1.11 is released, at
which point we can execute on the plan to remove iputil.py entirely by
changing the way that nodes learn their externally-facing IP address.
Brian Warner [Fri, 12 Sep 2014 19:55:52 +0000 (12:55 -0700)]
test_disk_stats: tolerate used==0 for a Travis worker bug
Some Travis-CI workers report persistently empty disks, causing spurious
test failures. It's not really that important to assert used>0, so this
relaxes the test.
Add a tooltip to explain what SDMF means. Cannot find a definition for MDMF; I presume "Medium" but at the risk of being wrong, I don't want to just blindly make that suggested change.
Brian Warner [Tue, 9 Sep 2014 00:28:26 +0000 (17:28 -0700)]
Remove 'trialcoverage' plugin and support code
Closes ticket:2281 (trac).
This removes src/allmydata/test/trial_coverage.py, which was a
in-process way to run trial tests under the "coverage" code-coverage
tool. These days, the preferred way to do this is with "coverage run",
although the actual invocation is a bit messy because of the way
bin/trial uses subprocess.call() to invoke the real entrypoint script
with the right PYTHONPATH (see #1698 for details). Hopefully this will
be improved to use a simpler "coverage run .." command in the future.
This patch also removes twisted/plugins/allmydata_trial.py, which
enabled the "--reporter=bwverbose-coverage" option. Finally it modifies
setup.py to stop looking for that option and adding "trialcoverage" to
the dependencies list, which gets us closer to removing "setup_requires"
entirely.
projects / tahoe-lafs / tahoe-lafs.git / log