tahoe-lafs/tahoe-lafs.git
16 years agoremove unused import (thanks, pyflakes)
Zooko O'Whielacronx [Fri, 1 Feb 2008 00:00:59 +0000 (17:00 -0700)]
remove unused import (thanks, pyflakes)

16 years agostorage: make two levels of share directories so as not to exceed certain filesystems...
Zooko O'Whielacronx [Thu, 31 Jan 2008 23:26:28 +0000 (16:26 -0700)]
storage: make two levels of share directories so as not to exceed certain filesystems's limitations on directory size
The filesystem which gets my vote for most undeservedly popular is ext3, and it has a hard limit of 32,000 entries in a directory.  Many other filesystems (even ones that I like more than I like ext3) have either hard limits or bad performance consequences or weird edge cases when you get too many entries in a single directory.

This patch makes it so that there is a layer of intermediate directories between the "shares" directory and the actual storage-index directory (the one whose name contains the entire storage index (z-base-32 encoded) and which contains one or more share files named by their share number).

The intermediate directories are named by the first 14 bits of the storage index, which means there are at most 16384 of them.  (This also means that the intermediate directory names are not a leading prefix of the storage-index directory names -- to do that would have required us to have intermediate directories limited to either 1024 (2-char), which is too few, or 32768 (3-chars of a full 5 bits each), which would overrun ext3's funny hard limit of 32,000.))

This closes #150, and please see the "convertshares.py" script attached to #150 to convert your old tahoe-0.7.0 storage/shares directory into a new tahoe-0.8.0 storage/shares directory.

16 years agotarballs: stick with setup.py's sdist command, which gives us bz2/gz/zip but not...
Brian Warner [Thu, 31 Jan 2008 20:52:48 +0000 (13:52 -0700)]
tarballs: stick with setup.py's sdist command, which gives us bz2/gz/zip but not rz/7z/lrz. Closes #298.

16 years agooffloaded.py: fix logging a bit
Brian Warner [Thu, 31 Jan 2008 20:45:01 +0000 (13:45 -0700)]
offloaded.py: fix logging a bit

16 years agoadd 'tarballs' target, to generate compressed source tarballs
Brian Warner [Thu, 31 Jan 2008 03:45:14 +0000 (20:45 -0700)]
add 'tarballs' target, to generate compressed source tarballs

16 years agostats: add a simple stats gathering system
robk-tahoe [Thu, 31 Jan 2008 03:11:07 +0000 (20:11 -0700)]
stats: add a simple stats gathering system

We have a desire to collect runtime statistics from multiple nodes primarily
for server monitoring purposes.   This implements a simple implementation of
such a system, as a skeleton to build more sophistication upon.

Each client now looks for a 'stats_gatherer.furl' config file.  If it has
been configured to use a stats gatherer, then it instantiates internally
a StatsProvider.  This is a central place for code which wishes to offer
stats up for monitoring to report them to, either by calling
stats_provider.count('stat.name', value) to increment a counter, or by
registering a class as a stats producer with sp.register_producer(obj).

The StatsProvider connects to the StatsGatherer server and provides its
provider upon startup.  The StatsGatherer is then responsible for polling
the attached providers periodically to retrieve the data provided.
The provider queries each registered producer when the gatherer queries
the provider.  Both the internal 'counters' and the queried 'stats' are
then reported to the gatherer.

This provides a simple gatherer app, (c.f. make stats-gatherer-run)
which prints its furl and listens for incoming connections.  Once a
minute, the gatherer polls all connected providers, and writes the
retrieved data into a pickle file.

Also included is a munin plugin which knows how to read the gatherer's
stats.pickle and output data munin can interpret.  this plugin,
tahoe-stats.py can be symlinked as multiple different names within
munin's 'plugins' directory, and inspects argv to determine which
data to display, doing a lookup in a table within that file.
It looks in the environment for 'statsfile' to determine the path to
the gatherer's stats.pickle.  An example plugins-conf.d file is
provided.

16 years agoupload: oops, fix breakage after removing upload_file/upload_data/etc
Brian Warner [Thu, 31 Jan 2008 02:41:43 +0000 (19:41 -0700)]
upload: oops, fix breakage after removing upload_file/upload_data/etc

16 years agoremove upload.upload_(data,filename,filehandle) convenience functions
Brian Warner [Thu, 31 Jan 2008 02:03:19 +0000 (19:03 -0700)]
remove upload.upload_(data,filename,filehandle) convenience functions

16 years agoupload: Data should use convergence by default
Brian Warner [Thu, 31 Jan 2008 02:02:56 +0000 (19:02 -0700)]
upload: Data should use convergence by default

16 years agoupload-helper: avoid duplicate uploads: check the grid to see if the file already...
Brian Warner [Thu, 31 Jan 2008 01:49:02 +0000 (18:49 -0700)]
upload-helper: avoid duplicate uploads: check the grid to see if the file already exists

16 years agolog.py: update log.err() to take advantage of current foolscap's log.err
Brian Warner [Thu, 31 Jan 2008 01:47:23 +0000 (18:47 -0700)]
log.py: update log.err() to take advantage of current foolscap's log.err

16 years agoinstall-details.html: debian 3.1 is better known as 'sarge'
Brian Warner [Thu, 31 Jan 2008 00:03:42 +0000 (17:03 -0700)]
install-details.html: debian 3.1 is better known as 'sarge'

16 years agoconfwiz: update to record install and uninstall events.
robk-tahoe [Wed, 30 Jan 2008 23:52:07 +0000 (16:52 -0700)]
confwiz: update to record install and uninstall events.

16 years agomake content-hash-key encryption a parameter of uploading
Zooko O'Whielacronx [Wed, 30 Jan 2008 19:24:50 +0000 (12:24 -0700)]
make content-hash-key encryption a parameter of uploading
fixes #293

16 years agodocs: add note that Debian 3.1 seems to have the same problem as Ubuntu Dapper with...
Zooko O'Whielacronx [Wed, 30 Jan 2008 19:21:17 +0000 (12:21 -0700)]
docs: add note that Debian 3.1 seems to have the same problem as Ubuntu Dapper with regard to Nevow and Twisted version compatibility

16 years agotests: add a test that nevow can find its default css file
Zooko O'Whielacronx [Wed, 30 Jan 2008 18:41:58 +0000 (11:41 -0700)]
tests: add a test that nevow can find its default css file

16 years agoMakefile: touch .built on every build, so other targets can depend upon .built and...
Brian Warner [Wed, 30 Jan 2008 08:32:57 +0000 (01:32 -0700)]
Makefile: touch .built on every build, so other targets can depend upon .built and avoid redundant rebuilds

16 years agospeedcheck: track SSK creation time separately
Brian Warner [Wed, 30 Jan 2008 03:44:32 +0000 (20:44 -0700)]
speedcheck: track SSK creation time separately

16 years agodocs/dirnodes.txt: rewrite to reflect 0.7.0's RSA-based SDMF dirnodes
Brian Warner [Wed, 30 Jan 2008 02:13:58 +0000 (19:13 -0700)]
docs/dirnodes.txt: rewrite to reflect 0.7.0's RSA-based SDMF dirnodes

16 years agowebish: remove 'URI-link' from directory page, now that we only use URI-links
Brian Warner [Wed, 30 Jan 2008 01:11:09 +0000 (18:11 -0700)]
webish: remove 'URI-link' from directory page, now that we only use URI-links

16 years agowebish: link to directory URIs rather than a child path. Addresses #103.
Brian Warner [Wed, 30 Jan 2008 01:04:32 +0000 (18:04 -0700)]
webish: link to directory URIs rather than a child path. Addresses #103.

16 years agoupload: add log message when AssistedUploader is done
Brian Warner [Wed, 30 Jan 2008 00:38:12 +0000 (17:38 -0700)]
upload: add log message when AssistedUploader is done

16 years ago_auto_deps.py: update comment
Brian Warner [Tue, 29 Jan 2008 20:53:21 +0000 (13:53 -0700)]
_auto_deps.py: update comment

16 years ago_auto_deps: tolerate DistributionNotFound (but not VersionConflict), to accomodate...
Brian Warner [Tue, 29 Jan 2008 20:52:37 +0000 (13:52 -0700)]
_auto_deps: tolerate DistributionNotFound (but not VersionConflict), to accomodate distributions (i.e. gutsy) which provide our dependencies but don't include .egg-info files

16 years agoupload: fix 'skipping_ahead' log message
Brian Warner [Tue, 29 Jan 2008 02:13:36 +0000 (19:13 -0700)]
upload: fix 'skipping_ahead' log message

16 years agooffloaded: when uploading a file that failed to upload before, ask for the last byte...
Brian Warner [Tue, 29 Jan 2008 02:05:43 +0000 (19:05 -0700)]
offloaded: when uploading a file that failed to upload before, ask for the last byte of ciphertext, so the reader is prepared to give us the plaintext hashes

16 years agoWriteBucketProxy: improve __repr__
Brian Warner [Tue, 29 Jan 2008 01:53:51 +0000 (18:53 -0700)]
WriteBucketProxy: improve __repr__

16 years agoupload.py: the 'skipping encryption' message was emitted exactly backwards
Brian Warner [Tue, 29 Jan 2008 01:38:38 +0000 (18:38 -0700)]
upload.py: the 'skipping encryption' message was emitted exactly backwards

16 years agoconfwiz: revise layout
robk-tahoe [Mon, 28 Jan 2008 21:36:03 +0000 (14:36 -0700)]
confwiz: revise layout

fix the make-confwiz-match-installer-size changes, to eliminate some weird
layout/rendering bugs.  also tweaked the layout slightly to add space between
the warning label and the newsletter subscribe checkbox.

16 years agotweak config wizard window size
robk-tahoe [Mon, 28 Jan 2008 21:07:13 +0000 (14:07 -0700)]
tweak config wizard window size

adjust the confiwiz frames to display at a size comparable to the innosetup
installer window

16 years agowebish: upload+localdir=missing should give an error
Brian Warner [Mon, 28 Jan 2008 21:48:06 +0000 (14:48 -0700)]
webish: upload+localdir=missing should give an error

16 years agowebish: oops, unit tests don't have an Uploader, don't rely upon it for helper-status...
Brian Warner [Mon, 28 Jan 2008 21:03:29 +0000 (14:03 -0700)]
webish: oops, unit tests don't have an Uploader, don't rely upon it for helper-status display

16 years agodisplay the Helper FURL and our connection status on the welcome page. Closes #285.
Brian Warner [Mon, 28 Jan 2008 20:56:22 +0000 (13:56 -0700)]
display the Helper FURL and our connection status on the welcome page. Closes #285.

16 years agoCHK upload helper: don't let one failed upload prevent us from trying again
Brian Warner [Mon, 28 Jan 2008 19:58:13 +0000 (12:58 -0700)]
CHK upload helper: don't let one failed upload prevent us from trying again

16 years agomacapp.py: cosmetic, remove trailing whitespace
Brian Warner [Mon, 28 Jan 2008 19:46:54 +0000 (12:46 -0700)]
macapp.py: cosmetic, remove trailing whitespace

16 years agoencode.py: update logging levels
Brian Warner [Mon, 28 Jan 2008 19:15:27 +0000 (12:15 -0700)]
encode.py: update logging levels

16 years agoencode.py: don't allow a shareholder which dies in start() to kill the whole upload
Brian Warner [Mon, 28 Jan 2008 19:14:48 +0000 (12:14 -0700)]
encode.py: don't allow a shareholder which dies in start() to kill the whole upload

16 years agoencode.py: don't record BAD log event unless there is actually a problem
Brian Warner [Mon, 28 Jan 2008 18:59:10 +0000 (11:59 -0700)]
encode.py: don't record BAD log event unless there is actually a problem

16 years agoconfwiz: use get_config call to backend
robk-tahoe [Sat, 26 Jan 2008 02:01:32 +0000 (19:01 -0700)]
confwiz: use get_config call to backend

this will write an arbitrary number of config files, instead of being restricted
to just the introducer.furl, based on the response of the php backend.
the get_config is passed username/password

16 years agodocs: edit "grid of storage servers" section with Amber
Zooko O'Whielacronx [Mon, 28 Jan 2008 18:48:21 +0000 (11:48 -0700)]
docs: edit "grid of storage servers" section with Amber

16 years agodocs: edit architecture.txt with Amber's help
Zooko O'Whielacronx [Mon, 28 Jan 2008 18:33:46 +0000 (11:33 -0700)]
docs: edit architecture.txt with Amber's help

16 years agoMakefile: split mac 'make .dmg' and 'upload' into separate steps
Brian Warner [Fri, 25 Jan 2008 23:29:13 +0000 (16:29 -0700)]
Makefile: split mac 'make .dmg' and 'upload' into separate steps

16 years agooffloaded upload: avoid tail-recursion problem that would break large files
Brian Warner [Fri, 25 Jan 2008 04:51:34 +0000 (21:51 -0700)]
offloaded upload: avoid tail-recursion problem that would break large files

16 years agoencode.py: improve error message when segment lengths come out wrong
Brian Warner [Fri, 25 Jan 2008 04:51:09 +0000 (21:51 -0700)]
encode.py: improve error message when segment lengths come out wrong

16 years agooffloaded uploader: don't use a huge amount of memory when skipping over previously...
Brian Warner [Fri, 25 Jan 2008 00:25:33 +0000 (17:25 -0700)]
offloaded uploader: don't use a huge amount of memory when skipping over previously-uploaded data

16 years agobetter mac .app icon
robk-tahoe [Fri, 25 Jan 2008 03:23:47 +0000 (20:23 -0700)]
better mac .app icon

16 years agoReworked mac gui to not lock up upon launch
robk-tahoe [Fri, 25 Jan 2008 03:00:28 +0000 (20:00 -0700)]
Reworked mac gui to not lock up upon launch

Previously, once the node itself was launched, the UI event loop was no longer
running.  This meant that the app would sit around seemingly 'wedged' and being
reported as 'Not Responding' by the os.

This chnages that by actually implementing a wxPython gui which is left running
while the reactor, and the node within it, is launched in another thread.
Beyond 'quit' -> reactor.stop, there are no interactions between the threads.

The ui provides 'open web root' and 'open account page' actions, both in the
file menu, and in the (right click) dock icon menu.

Something weird in the handling of wxpython's per-frame menubar stuff seems to
mean that the menu bar only displays the file menu and about etc (i.e. the items
from the wx menubar) if the focus changes from and back to the app while the
frame the menubar belongs to is displayed.  Hence a splash frame comes up at
startup to provide an opportunity.

It also seems that, in the case that the file menu is not available, that one
can induce it to reappear by choosing 'about' from the dock menu, and then
closing the about window.

16 years agocleanup mac and windows build code
robk-tahoe [Thu, 24 Jan 2008 04:06:41 +0000 (21:06 -0700)]
cleanup mac and windows build code

this moves some of the code common to both windows and mac builds into the
allmydata module hierarchy, and cleans up the windows and mac build directories
to import the code from there.

16 years agotahoe_fuse: cmdline args & system test: Allow nonstandard client basedirs to be speci...
nejucomo [Mon, 21 Jan 2008 03:56:27 +0000 (20:56 -0700)]
tahoe_fuse: cmdline args & system test: Allow nonstandard client basedirs to be specified and update the system tests to use this feature...

The commandline option handling of the version of python-fuse I use is arcane.  This is an ugly hack.

16 years agotahoe_fuse: rename decorators for clarity.
nejucomo [Mon, 21 Jan 2008 03:26:57 +0000 (20:26 -0700)]
tahoe_fuse: rename decorators for clarity.

16 years agoSmall log output change.
nejucomo [Mon, 21 Jan 2008 03:18:53 +0000 (20:18 -0700)]
Small log output change.

16 years agotahoe_fuse: system test: webapi connection: bug fix and small log output change.
nejucomo [Mon, 21 Jan 2008 03:10:31 +0000 (20:10 -0700)]
tahoe_fuse: system test: webapi connection: bug fix and small log output change.

16 years agotahoe_fuse: system test: Add FIXME comments.
nejucomo [Mon, 21 Jan 2008 03:06:19 +0000 (20:06 -0700)]
tahoe_fuse: system test: Add FIXME comments.

16 years agotahoe_fuse: system test: Manage multiple clients for test grid... System test setup...
nejucomo [Mon, 21 Jan 2008 03:02:20 +0000 (20:02 -0700)]
tahoe_fuse: system test: Manage multiple clients for test grid...  System test setup is almost complete.

This is a little convoluted because of the "layer" design, but it appears
to function correctly and do properly ordered cleanup.

Before system test setup is complete, tahoe_fuse.py needs to be modified
to allow arbitrary client base directories.

16 years agotahoe_fuse: system test: Attempt to create a dirnode to place in <basedir>/private...
nejucomo [Mon, 21 Jan 2008 01:47:47 +0000 (18:47 -0700)]
tahoe_fuse: system test: Attempt to create a dirnode to place in <basedir>/private/root_dir.cap, but this fails because the network is too small...

This patch also factors out the "polling_operation" pattern.

16 years agotahoe_fuse: system test: Launch the fuse interface.
nejucomo [Mon, 21 Jan 2008 00:55:51 +0000 (17:55 -0700)]
tahoe_fuse: system test: Launch the fuse interface.

16 years agotahoe_fuse: system test: factor out some cleanup code.
nejucomo [Mon, 21 Jan 2008 00:54:48 +0000 (17:54 -0700)]
tahoe_fuse: system test: factor out some cleanup code.

16 years agotahoe_fuse: system test: Copy the introducer.furl with a possible race condition...
nejucomo [Mon, 21 Jan 2008 00:09:44 +0000 (17:09 -0700)]
tahoe_fuse: system test: Copy the introducer.furl with a possible race condition due to timeout.

16 years agoA start at adding a system test for tahoe_fuse. Incomplete...
nejucomo [Sun, 20 Jan 2008 23:54:56 +0000 (16:54 -0700)]
A start at adding a system test for tahoe_fuse.  Incomplete...

16 years agoRemove a redundant assertion for clarity.
nejucomo [Sat, 19 Jan 2008 07:20:32 +0000 (00:20 -0700)]
Remove a redundant assertion for clarity.

16 years agoChange stdout to rudimentary file-based logging.
nejucomo [Sat, 19 Jan 2008 07:18:54 +0000 (00:18 -0700)]
Change stdout to rudimentary file-based logging.

16 years agoRename the unittest script for tahoe-fuse.
nejucomo [Sat, 19 Jan 2008 07:16:12 +0000 (00:16 -0700)]
Rename the unittest script for tahoe-fuse.

16 years agofurther fixes to windows build (pkg_resources / web templates)
robk-tahoe [Thu, 24 Jan 2008 01:52:43 +0000 (18:52 -0700)]
further fixes to windows build (pkg_resources / web templates)

now that web templates are found via pkg_resources, then the windows build
should in fact _use_ pkg_resources, rather than exclude it from the build
to prevent nevow exploding upon import due to the zip provider exception,
so that the pkgreshook can do install location based lookups

16 years agofix windows build's packaging of web templates
robk-tahoe [Thu, 24 Jan 2008 01:23:37 +0000 (18:23 -0700)]
fix windows build's packaging of web templates

the recent changes to webish's template lookup (to use nevow.util hence
pkg_resources) to support the mac build, needs these changes to the windows
build in match the new lookup path

16 years agoeliminate startup spam for resources that can't be found
robk-tahoe [Thu, 24 Jan 2008 01:22:23 +0000 (18:22 -0700)]
eliminate startup spam for resources that can't be found

remove debug messages (and traceback) from node output in the case that the
pkg resources hook can't find a requested file. it will now silently return
the empty string for files that can't be resolved

16 years agofix tahoe script installation logic
robk-tahoe [Thu, 24 Jan 2008 01:05:56 +0000 (18:05 -0700)]
fix tahoe script installation logic

refine the logic in the .app which tries to install the 'tahoe' script.

now it will do nothing if 'tahoe' is found anywhere on the user's path,
and only if it's not present will it try to install it in each of the
candidate paths (/usr/local/bin ~/bin ~/Library/bin) which are on the
user's path

16 years agoencode.py: log the contents of the uri_extension block
Brian Warner [Thu, 24 Jan 2008 01:08:04 +0000 (18:08 -0700)]
encode.py: log the contents of the uri_extension block

16 years agotest_upload.py: implement remote_abort on our fake BucketWriter
Brian Warner [Thu, 24 Jan 2008 01:07:34 +0000 (18:07 -0700)]
test_upload.py: implement remote_abort on our fake BucketWriter

16 years agofix windows build
robk-tahoe [Wed, 23 Jan 2008 22:17:27 +0000 (15:17 -0700)]
fix windows build

having changed the web template retrieval to use nevow.util.resource_filename
(and hence through pkg_resources when available) that adds a requirement that
py2exe be given a hint to induce it to include the allmydata.web module so that
it becomes importable.

16 years agomac build: fixed permission problem on upload .dmg
robk-tahoe [Wed, 23 Jan 2008 21:51:18 +0000 (14:51 -0700)]
mac build: fixed permission problem on upload .dmg

16 years agosetup: put back "chmod +x bin/tahoe" in the build target
Zooko O'Whielacronx [Wed, 23 Jan 2008 23:40:20 +0000 (16:40 -0700)]
setup: put back "chmod +x bin/tahoe" in the build target

16 years ago_auto_deps.py: relax our simplejson dependency to 1.4, since I think it works and...
Brian Warner [Wed, 23 Jan 2008 20:03:09 +0000 (13:03 -0700)]
_auto_deps.py: relax our simplejson dependency to 1.4, since I think it works and because that's what feisty offers

16 years agosetup: weaken the requirement on zope.interface from >= 3.1.0 to "any"
Zooko O'Whielacronx [Wed, 23 Jan 2008 18:26:04 +0000 (11:26 -0700)]
setup: weaken the requirement on zope.interface from >= 3.1.0 to "any"
We've never heard of a version of zope.interface that *wasn't* compatible, and there is a bug in Ubuntu's packaging of zope.interface which causes it to report its version number as 0.0.0:

https://bugs.launchpad.net/zope.interface/+bug/185418

16 years agosetup: loosen our requirement on pycryptopp from >= 0.2.9 to >= 0.2.8
Zooko O'Whielacronx [Wed, 23 Jan 2008 18:00:35 +0000 (11:00 -0700)]
setup: loosen our requirement on pycryptopp from >= 0.2.9 to >= 0.2.8
Again, tahoecs2 has pycryptopp v0.2.8, and reviewing the pycryptopp change history shows that there were no important bugfixes added since 0.2.8.

16 years agotests: it is okay to leave a src/allmydata/_auto_deps.py lying around after a build
Zooko O'Whielacronx [Wed, 23 Jan 2008 17:43:37 +0000 (10:43 -0700)]
tests: it is okay to leave a src/allmydata/_auto_deps.py lying around after a build

16 years agohave mac app write a tahoe upon startup
robk-tahoe [Wed, 23 Jan 2008 03:35:01 +0000 (20:35 -0700)]
have mac app write a tahoe upon startup

upon startup, the .app will look in '/usr/local/bin', '~/bin', '~/Library/bin'
if it finds one of these dirs, and can write into it, and there isn't already
a 'tahoe' present, it will write a small bach script which will launch the
binary contained within the .app bundle

this allows the .app bundle to offer the services of the 'tahoe' script
easily and simply

16 years agofix build breakage caused by auto_deps setuptools stuff
robk-tahoe [Wed, 23 Jan 2008 02:32:55 +0000 (19:32 -0700)]
fix build breakage caused by auto_deps setuptools stuff

zooko recently added a runtime check, via setuptools, that specific versions of various
packages were reported as available through setuptools at runtime.

however exe and app builds run with collected egg contents, not linked against entire
eggs, i.e. the code is transcluded into a single library.zip

thus setuptools reports that those specific version cannot be reported as available,
though they are in fact available built into the library

this disables that runtime check if the app is running 'frozen'

16 years agoadd mac native build
robk-tahoe [Wed, 23 Jan 2008 02:32:26 +0000 (19:32 -0700)]
add mac native build

This patch adds support for a mac native build.

At the moment it's a fairly simple .app - i.e. so simple as to be unacceptable
for a shipping product, but ok for testing and experiment at this point.

notably once launched, the app's ui does not respond at all, although its dock
icon does allow it to be force-quit.

this produces a single .app bundle, which when run will look for a node basedir
in ~/.tahoe.  If one is not found, one will be created in ~/Library/Application
Support/Allmydata Tahoe, and that will be symlinked to ~/.tahoe

if the basedir is lacking basic config (introducer.furl and root_dir.cap) then
the wx config wizard will be launched to log into an account and to set up
those files.

if a webport file is not found, the default value of 8123 will be written into
it.

once the node has started running, a webbrowser will be opened to the webish
interface at the users root_dir

note that, once configured, the node runs as the main thread of the .app,
no daemonisation is done, twistd is not involved.

the binary itself, from within the .app bundle, i.e.
"Allmydata Tahoe.app/Contents/MacOS/Allmydata Tahoe"
can be used from the command line and functions as the 'tahoe' executable
would in a unix environment, with one exception - when launched with no args
it triggers the default behaviour of running a node, and if necessary config
wizard, as if the user had launched the .app

one other gotcha to be aware of is that symlinking to this binary from some
other place in ones $PATH will most likely not work. when I tried this,
something - wx I believe - exploded, since it seems to use argv[0] to figure
out where necessary libraries reside and fails if argv[0] isn't in the .app
bundle.  it's pretty easy to set up a script a la
    #!/bin/bash
    /Blah/blah/blah/Allmydata\ Tahoe.app/Contents/MacOS/Allmydata\ Tahoe "${@}"

16 years agotweak webish to use resource_filename to find css and html files
robk-tahoe [Wed, 23 Jan 2008 00:44:58 +0000 (17:44 -0700)]
tweak webish to use resource_filename to find css and html files

using sibpath to find web template files relative to source code is functional
when running from source environments, but not especially flexible when running
from bundled built environments.  the more 'orthodox' mechanism, pkg_resources,
in theory at least, knows how to find resource files in various environments.

this makes the 'web' directory in allmydata into an actual allmydata.web module
(since pkg_resources looks for files relative to a named module, and that module
must be importable) and uses pkg_resources.resource_filename to find the files
therein.

16 years agosetup: make find_trial self-contained so that we don't have a bootstrapping problem...
Zooko O'Whielacronx [Wed, 23 Jan 2008 17:04:26 +0000 (10:04 -0700)]
setup: make find_trial self-contained so that we don't have a bootstrapping problem -- if allmydata can't be imported we still want to be able to run find_trial

16 years agosetup: loosen requirement on simplejson from 1.7.3 to 1.7.1
Zooko O'Whielacronx [Wed, 23 Jan 2008 16:54:20 +0000 (09:54 -0700)]
setup: loosen requirement on simplejson from 1.7.3 to 1.7.1
Since apparently 1.7.1 is what we use on tahoecs2, and it works.

16 years agodocs: edit install.html to point to about.html
Zooko O'Whielacronx [Wed, 23 Jan 2008 15:08:10 +0000 (08:08 -0700)]
docs: edit install.html to point to about.html

16 years agosetup: src/allmydata/_auto_deps.py is boring
Zooko O'Whielacronx [Wed, 23 Jan 2008 00:37:22 +0000 (17:37 -0700)]
setup: src/allmydata/_auto_deps.py is boring

16 years agosetup: loosen our version requirement on zfec to require >= 1.1 instead of >= 1.3
Zooko O'Whielacronx [Wed, 23 Jan 2008 00:35:38 +0000 (17:35 -0700)]
setup: loosen our version requirement on zfec to require >= 1.1 instead of >= 1.3
I see that we have .deb's only for v1.1.

16 years agosetup: require specific versions of dependencies, both at run-time (if pkg_resources...
Zooko O'Whielacronx [Wed, 23 Jan 2008 00:24:33 +0000 (17:24 -0700)]
setup: require specific versions of dependencies, both at run-time (if pkg_resources is available) and at build-time, and make there be only once place where we specify those versions
Using pkg_resources.require() like this also apparently allows people to install multiple different versions of packages on their system and tahoe (if pkg_resources is available to it) will import the version of the package that it requires.  I haven't tested this feature.

16 years agosetup: for reasons that I do not understand "show-eggspath" gives me a GNUmake error...
Zooko O'Whielacronx [Wed, 23 Jan 2008 00:22:38 +0000 (17:22 -0700)]
setup: for reasons that I do not understand "show-eggspath" gives me a GNUmake error unless I move it down a couple of stanzas (until after the stanza that sets PYTHONPATH)

16 years agosetup: use setuptools (if it is present) at run-time to give a specific error message...
Zooko O'Whielacronx [Wed, 23 Jan 2008 00:42:54 +0000 (17:42 -0700)]
setup: use setuptools (if it is present) at run-time to give a specific error message on startup if a too-old version of a dependency is installed

16 years agosetup: remove some things from .darcs-boringfile which are no longer boring since...
Zooko O'Whielacronx [Wed, 23 Jan 2008 00:40:23 +0000 (17:40 -0700)]
setup: remove some things from .darcs-boringfile which are no longer boring since we no longer use them

16 years agosetup: remove the --always-copy option, because it causes setuptools to ignore system...
Zooko O'Whielacronx [Tue, 22 Jan 2008 21:05:04 +0000 (14:05 -0700)]
setup: remove the --always-copy option, because it causes setuptools to ignore system and development apps

16 years agosetup: remove the hatch-eggs make script since the setup.cfg accomplishes it, and...
Zooko O'Whielacronx [Tue, 22 Jan 2008 20:47:48 +0000 (13:47 -0700)]
setup: remove the hatch-eggs make script since the setup.cfg accomplishes it, and make windows-exe depend on .built

16 years agosetup: add a setup.cfg file which instructs setuptools to install all eggs in unzippe...
Zooko O'Whielacronx [Tue, 22 Jan 2008 20:46:47 +0000 (13:46 -0700)]
setup: add a setup.cfg file which instructs setuptools to install all eggs in unzipped form and to always copy them into the target directory (even if they are already installed somewhere else on the path that setuptools searches, which includes the CWD)

16 years agosetup: include cli.exe in the bootstrap setptools egg so that it will work on Windows...
Zooko O'Whielacronx [Tue, 22 Jan 2008 19:33:55 +0000 (12:33 -0700)]
setup: include cli.exe in the bootstrap setptools egg so that it will work on Windows (also include gui.exe just in case)

16 years agooffloaded: reinstate fix for windows tests
robk-tahoe [Mon, 21 Jan 2008 22:25:15 +0000 (15:25 -0700)]
offloaded: reinstate fix for windows tests

in a discussion the other day, brian had asked me to try removing this fix, since
it leads to double-closing the reader.  since on my windows box, the test failures
I'd experienced were related to the ConnectionLost exception problem, and this
close didn't see to make a difference to test results, I agreed.

turns out that the buildbot's environment does fail without this fix, even with
the exception fix, as I'd kind of expected.

it makes sense, because the reader (specifically the file handle) must be closed
before it can be unlinked. at any rate, I'm reinstating this, in order to fix the
windows build

16 years agoUNDO: offloaded: close reader before removing its file
robk-tahoe [Fri, 18 Jan 2008 00:36:28 +0000 (17:36 -0700)]
UNDO: offloaded: close reader before removing its file

unlinking a file before closing it is not portable. it works on unix, but fails
since an open file holds a lock on windows.

this closes the reader before trying to unlink the encoding file within the
CHKUploadHelper.

16 years agoupdate confwiz to include account creation ui
robk-tahoe [Mon, 21 Jan 2008 22:13:10 +0000 (15:13 -0700)]
update confwiz to include account creation ui

this changes the confwiz so that hitting the 'create account' button, rather than
opening a webbrowser to the register page, instead provides a simple account creation
ui directly, along with changes to the backend (native_client.php) to support that.

also added a 'connecting...' message so the user sees a response when they hit
login or create account, since the getBasedir call can sometimes take up to ~5min
(which is unacceptable for a user product, but this at least somewhat ameliorates
the issue of the ui locking up and not giving the user any feedback while it's
happening)

16 years agosetup: bundle setuptools_darcs-1.1.7
Zooko O'Whielacronx [Tue, 22 Jan 2008 18:01:36 +0000 (11:01 -0700)]
setup: bundle setuptools_darcs-1.1.7
fixes #263

16 years agosetup: use a customized version of ez_setup.py which bootstraps from Python-version...
Zooko O'Whielacronx [Tue, 22 Jan 2008 18:00:56 +0000 (11:00 -0700)]
setup: use a customized version of ez_setup.py which bootstraps from Python-version-agnostic setuptools bootstrap eggs

16 years agosetup: add a setuptools bootstrap egg that works on all versions of Python
Zooko O'Whielacronx [Tue, 22 Jan 2008 18:00:12 +0000 (11:00 -0700)]
setup: add a setuptools bootstrap egg that works on all versions of Python
For versions of Python >= 2.3.

16 years agosetup: update some docs, metadata, and docstrings
Zooko O'Whielacronx [Tue, 22 Jan 2008 17:22:51 +0000 (10:22 -0700)]
setup: update some docs, metadata, and docstrings

16 years agosetup: it is okay to leave src/allmydata_tahoe.egg-info in place
Zooko O'Whielacronx [Tue, 22 Jan 2008 16:35:54 +0000 (09:35 -0700)]
setup: it is okay to leave src/allmydata_tahoe.egg-info in place
This directory allows programs to programmatically identify tahoe and its version number while "running from source" -- i.e. using ./setup.py develop.