tahoe-lafs/tahoe-lafs.git
16 years agomore minor architecture.txt changes
Brian Warner [Thu, 14 Feb 2008 03:20:43 +0000 (20:20 -0700)]
more minor architecture.txt changes

16 years agoarchitecture.txt: fix some things that have changed a lot in recent releases
Brian Warner [Thu, 14 Feb 2008 03:14:29 +0000 (20:14 -0700)]
architecture.txt: fix some things that have changed a lot in recent releases

16 years agowindows installer: remove uninstall tracking, add welcome page
robk-tahoe [Thu, 14 Feb 2008 01:17:16 +0000 (18:17 -0700)]
windows installer: remove uninstall tracking, add welcome page

in justin's testing, the uninstall tracking was hanging the uninstall
process (on vista) for now, until we see enough need for it to warrant
more detailed testing/debugging/tweaks, I'm simply disabling the call
to confwiz --uninstall

also this adds a 'welcome page' to the install process. once the user
has installed the windows build, then the installer will open a web
browser to the 'welcome page' on the website ('/welcome_install')

16 years agomutable.py: add comments to justify initial-read size choices
Brian Warner [Thu, 14 Feb 2008 00:41:20 +0000 (17:41 -0700)]
mutable.py: add comments to justify initial-read size choices

16 years agodocs/dirnodes.txt: add notes on dirnode sizes
Brian Warner [Thu, 14 Feb 2008 00:40:45 +0000 (17:40 -0700)]
docs/dirnodes.txt: add notes on dirnode sizes

16 years agomutable.txt: fix offset math in the SDMF layout
Brian Warner [Thu, 14 Feb 2008 00:39:06 +0000 (17:39 -0700)]
mutable.txt: fix offset math in the SDMF layout

16 years agoswitch from base62 to base32 for storage indices, switch from z-base-32 to rfc 3548...
Zooko O'Whielacronx [Fri, 15 Feb 2008 02:27:47 +0000 (19:27 -0700)]
switch from base62 to base32 for storage indices, switch from z-base-32 to rfc 3548 base-32 for everything, separate out base32 encoding from idlib

16 years agodocs: link straight to the release tar.gz in install.html
Zooko O'Whielacronx [Thu, 14 Feb 2008 18:54:14 +0000 (11:54 -0700)]
docs: link straight to the release tar.gz in install.html

16 years agodocs: update relnotes.txt for v0.8.0!
Zooko O'Whielacronx [Thu, 14 Feb 2008 16:00:26 +0000 (09:00 -0700)]
docs: update relnotes.txt for v0.8.0!

16 years agodocs: update relnotes, running.html
Zooko O'Whielacronx [Thu, 14 Feb 2008 15:54:34 +0000 (08:54 -0700)]
docs: update relnotes, running.html

16 years agodocs: beginning of update to relnotes.txt for v0.8
Zooko O'Whielacronx [Thu, 14 Feb 2008 00:43:02 +0000 (17:43 -0700)]
docs: beginning of update to relnotes.txt for v0.8

16 years agomac: shebang usr bin env python
Zooko O'Whielacronx [Wed, 13 Feb 2008 23:17:14 +0000 (16:17 -0700)]
mac: shebang usr bin env python
This makes it so that I can run "./setup.py" on the command-line, instead of "python ./setup.py".

16 years ago'tahoe catalog-shares': add SDMF filesize to the output, update misc/find-share-anoma...
Brian Warner [Wed, 13 Feb 2008 22:12:06 +0000 (15:12 -0700)]
'tahoe catalog-shares': add SDMF filesize to the output, update misc/find-share-anomalies.py to match

16 years agoadd test coverage for the /stats web page
Brian Warner [Wed, 13 Feb 2008 20:57:39 +0000 (13:57 -0700)]
add test coverage for the /stats web page

16 years agomutable.py: reject shares with different k/N than we expect. Quick fix for #312:...
Brian Warner [Wed, 13 Feb 2008 20:34:20 +0000 (13:34 -0700)]
mutable.py: reject shares with different k/N than we expect. Quick fix for #312: avoids data corruption but has availability problems.

16 years agotest_web: remove test_GET_DIRURL_large, it failed to exercise the problem in #237...
Brian Warner [Wed, 13 Feb 2008 20:08:26 +0000 (13:08 -0700)]
test_web: remove test_GET_DIRURL_large, it failed to exercise the problem in #237 and consumes a disproportionate amount of time

16 years agosetup: remove older bundled pycryptopp-0.2.10
Zooko O'Whielacronx [Wed, 13 Feb 2008 22:23:15 +0000 (15:23 -0700)]
setup: remove older bundled pycryptopp-0.2.10

16 years agosetup: bundle pycryptopp-0.3.0
Zooko O'Whielacronx [Wed, 13 Feb 2008 21:57:15 +0000 (14:57 -0700)]
setup: bundle pycryptopp-0.3.0

16 years agodocs: 10 blocks by default, not 12
amber [Wed, 13 Feb 2008 11:57:19 +0000 (04:57 -0700)]
docs: 10 blocks by default, not 12

16 years agowebish.py: fix for #237: when listing large directories, insert a turn break once...
Brian Warner [Wed, 13 Feb 2008 04:28:52 +0000 (21:28 -0700)]
webish.py: fix for #237: when listing large directories, insert a turn break once every 100 children, to work around non-optimized tail recursion Deferreds

16 years agodocs: edits and updates to architecture.txt, with Amber
Zooko O'Whielacronx [Wed, 13 Feb 2008 17:24:52 +0000 (10:24 -0700)]
docs: edits and updates to architecture.txt, with Amber

16 years agosetup: remove older bundled version of setuptools_darcs
Zooko O'Whielacronx [Wed, 13 Feb 2008 15:03:40 +0000 (08:03 -0700)]
setup: remove older bundled version of setuptools_darcs

16 years agosetup: remove another older bundled setuptools
Zooko O'Whielacronx [Wed, 13 Feb 2008 15:01:40 +0000 (08:01 -0700)]
setup: remove another older bundled setuptools

16 years agosetup: remove older bundled setuptools
Zooko O'Whielacronx [Wed, 13 Feb 2008 14:59:24 +0000 (07:59 -0700)]
setup: remove older bundled setuptools

16 years agosetup: remove older bundled version of darcsver
Zooko O'Whielacronx [Wed, 13 Feb 2008 14:57:22 +0000 (07:57 -0700)]
setup: remove older bundled version of darcsver

16 years agofix a few unused imports and suchlike, discovered by pyflakes
Zooko O'Whielacronx [Wed, 13 Feb 2008 14:38:08 +0000 (07:38 -0700)]
fix a few unused imports and suchlike, discovered by pyflakes

16 years agowebish: flogify the remaining log messages
Brian Warner [Wed, 13 Feb 2008 02:32:33 +0000 (19:32 -0700)]
webish: flogify the remaining log messages

16 years agowebish: censor all caps before logging the HTTP request, to preserve user privacy
Brian Warner [Wed, 13 Feb 2008 02:31:23 +0000 (19:31 -0700)]
webish: censor all caps before logging the HTTP request, to preserve user privacy

16 years agodownload status: refactor into a separate object, so we don't need to keep the Downlo...
Brian Warner [Wed, 13 Feb 2008 02:01:03 +0000 (19:01 -0700)]
download status: refactor into a separate object, so we don't need to keep the Download itself around for a long time

16 years agointroducer: remove encoding-parameter config, for now
Brian Warner [Wed, 13 Feb 2008 01:59:54 +0000 (18:59 -0700)]
introducer: remove encoding-parameter config, for now

16 years agouse base62 encoding for storage indexes, on disk and in verifier caps, and in logging...
Zooko O'Whielacronx [Wed, 13 Feb 2008 03:48:37 +0000 (20:48 -0700)]
use base62 encoding for storage indexes, on disk and in verifier caps, and in logging and diagnostic tools
base62 encoding fits more information into alphanumeric chars while avoiding the troublesome non-alphanumeric chars of base64 encoding.  In particular, this allows us to work around the ext3 "32,000 entries in a directory" limit while retaining the convenient property that the intermediate directory names are leading prefixes of the storage index file names.

16 years agowebish: add /status page to display current uploads/downloads
Brian Warner [Tue, 12 Feb 2008 22:40:05 +0000 (15:40 -0700)]
webish: add /status page to display current uploads/downloads

16 years agomake current upload/download status objects available from the client
Brian Warner [Tue, 12 Feb 2008 22:39:45 +0000 (15:39 -0700)]
make current upload/download status objects available from the client

16 years agoadd download-status objects, to track download progress
Brian Warner [Tue, 12 Feb 2008 22:38:39 +0000 (15:38 -0700)]
add download-status objects, to track download progress

16 years agoadd upload-status objects, to track upload progress
Brian Warner [Tue, 12 Feb 2008 22:36:05 +0000 (15:36 -0700)]
add upload-status objects, to track upload progress

16 years agomisc/find-share-anomalies.py: tool to analyze 'tahoe catalog-shares' output and look...
Brian Warner [Tue, 12 Feb 2008 02:53:36 +0000 (19:53 -0700)]
misc/find-share-anomalies.py: tool to analyze 'tahoe catalog-shares' output and look for problems

16 years agowebish: add edge metadata to t=json output, including timestamps
Brian Warner [Tue, 12 Feb 2008 02:14:10 +0000 (19:14 -0700)]
webish: add edge metadata to t=json output, including timestamps

16 years agocatalog-shares: add expiration time to output
Brian Warner [Tue, 12 Feb 2008 01:44:54 +0000 (18:44 -0700)]
catalog-shares: add expiration time to output

16 years agoadd 'tahoe catalog-shares' tool, to make a one-line summary of each share file. This...
Brian Warner [Tue, 12 Feb 2008 01:17:01 +0000 (18:17 -0700)]
add 'tahoe catalog-shares' tool, to make a one-line summary of each share file. This can help do cross-server correlation of sharefiles, looking for anomalies

16 years agotest_dirnode.py: improve error messages in case timestamp tests fail
Brian Warner [Tue, 12 Feb 2008 00:37:51 +0000 (17:37 -0700)]
test_dirnode.py: improve error messages in case timestamp tests fail

16 years agotest_dirnode.py: simplejson-1.7.1 incorrectly rounds floats to two decimal places...
Brian Warner [Tue, 12 Feb 2008 00:35:17 +0000 (17:35 -0700)]
test_dirnode.py: simplejson-1.7.1 incorrectly rounds floats to two decimal places. Don't let this bug flunk the timestamp test.

16 years agotest_system.py: refactor bounce_client, probably make it stop failing on cygwin
Brian Warner [Mon, 11 Feb 2008 22:26:58 +0000 (15:26 -0700)]
test_system.py: refactor bounce_client, probably make it stop failing on cygwin

16 years agowebish: display timestamps
Brian Warner [Mon, 11 Feb 2008 22:13:18 +0000 (15:13 -0700)]
webish: display timestamps

16 years agotest_dirnode.py: assert that we update mtime and preserve ctime
Brian Warner [Mon, 11 Feb 2008 22:12:55 +0000 (15:12 -0700)]
test_dirnode.py: assert that we update mtime and preserve ctime

16 years agodirnode.py: add metadata= to add_file(), add tests
Brian Warner [Mon, 11 Feb 2008 21:53:28 +0000 (14:53 -0700)]
dirnode.py: add metadata= to add_file(), add tests

16 years agotest_dirnode.py: add diag output to test-ctime/mtime tests
Brian Warner [Mon, 11 Feb 2008 21:13:07 +0000 (14:13 -0700)]
test_dirnode.py: add diag output to test-ctime/mtime tests

16 years agodirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more...
Brian Warner [Sat, 9 Feb 2008 01:43:47 +0000 (18:43 -0700)]
dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.

16 years agodocs/install-details.html: wrap to 80 cols, no content changes
Brian Warner [Fri, 8 Feb 2008 04:50:04 +0000 (21:50 -0700)]
docs/install-details.html: wrap to 80 cols, no content changes

16 years agotest_system: remove the hackish debug_interrupt= attribute magic used to exercise...
Brian Warner [Fri, 8 Feb 2008 03:15:37 +0000 (20:15 -0700)]
test_system: remove the hackish debug_interrupt= attribute magic used to exercise interrupted-upload resumption, instead just make the Uploadable bounce the helper halfway through the upload

16 years agodirnodes.txt: update docs a bit, we don't yet do choose-latest-version, just choose...
Brian Warner [Fri, 8 Feb 2008 03:14:05 +0000 (20:14 -0700)]
dirnodes.txt: update docs a bit, we don't yet do choose-latest-version, just choose-any-version

16 years agowebapi.txt: document POST /uri?t=mkdir
Brian Warner [Fri, 8 Feb 2008 03:10:28 +0000 (20:10 -0700)]
webapi.txt: document POST /uri?t=mkdir

16 years agotest_system.py: remove that ugly debug_stash_RemoteencryptedUploadable hack, now...
Brian Warner [Fri, 8 Feb 2008 00:27:30 +0000 (17:27 -0700)]
test_system.py: remove that ugly debug_stash_RemoteencryptedUploadable hack, now that UploadResults give us a better approach

16 years agoupload: if we lose the helper, go back to doing direct-to-server uploads instead...
Brian Warner [Fri, 8 Feb 2008 00:26:59 +0000 (17:26 -0700)]
upload: if we lose the helper, go back to doing direct-to-server uploads instead of causing all subsequent uploads to fail

16 years agoupload.EncryptAnUploadable: use 50KiB read chunks (instead of 50KB), to match the...
Brian Warner [Fri, 8 Feb 2008 00:10:13 +0000 (17:10 -0700)]
upload.EncryptAnUploadable: use 50KiB read chunks (instead of 50KB), to match the chunksize requested by the upload helper

16 years agotest_upload.py: hush pyflakes
Brian Warner [Thu, 7 Feb 2008 03:04:31 +0000 (20:04 -0700)]
test_upload.py: hush pyflakes

16 years agotest_upload.py: add test to exercise CHK hashing variations
Brian Warner [Thu, 7 Feb 2008 03:03:35 +0000 (20:03 -0700)]
test_upload.py: add test to exercise CHK hashing variations

16 years agotest_cli: oops, need to update this when the CHK hash changes
Brian Warner [Thu, 7 Feb 2008 02:58:53 +0000 (19:58 -0700)]
test_cli: oops, need to update this when the CHK hash changes

16 years agochange encryption-key hash to include encoding parameters. This is a minor compatibil...
Brian Warner [Thu, 7 Feb 2008 02:50:47 +0000 (19:50 -0700)]
change encryption-key hash to include encoding parameters. This is a minor compatibility break: CHK files encoded (with convergence) before and after this will have different keys and ciphertexts. Also switched to SHA-256d for both the data-to-key hash and the key-to-storageindex hash

16 years agohashutil: add tagged_hash_256d and tagged_hasher_256d
Brian Warner [Thu, 7 Feb 2008 02:36:43 +0000 (19:36 -0700)]
hashutil: add tagged_hash_256d and tagged_hasher_256d

16 years agoupload: rework passing of default encoding parameters: move more responsibility into...
Brian Warner [Thu, 7 Feb 2008 01:39:03 +0000 (18:39 -0700)]
upload: rework passing of default encoding parameters: move more responsibility into BaseUploadable

16 years agooffloaded: oops, need more tricks to make the unit tests pass
Brian Warner [Thu, 7 Feb 2008 00:51:11 +0000 (17:51 -0700)]
offloaded: oops, need more tricks to make the unit tests pass

16 years agohelper: return full uri-extension data to the client, so it can compare encoding...
Brian Warner [Thu, 7 Feb 2008 00:30:58 +0000 (17:30 -0700)]
helper: return full uri-extension data to the client, so it can compare encoding parameters

16 years agointerfaces: remove spurious line that counted against the figleaf coverage
Brian Warner [Wed, 6 Feb 2008 23:41:26 +0000 (16:41 -0700)]
interfaces: remove spurious line that counted against the figleaf coverage

16 years agotest_util: add full coverage for allmydata.util.deferredutil
Brian Warner [Wed, 6 Feb 2008 23:41:04 +0000 (16:41 -0700)]
test_util: add full coverage for allmydata.util.deferredutil

16 years agowindows: track uninstalls
robk-tahoe [Wed, 6 Feb 2008 21:12:49 +0000 (14:12 -0700)]
windows: track uninstalls

the confwiz and the native_client backend both gained hooks to track
uninstall events.  however that somehow didn't make it to the uninstaller :-)

16 years agotahoe dump-share: remove --filename option, just take it from argv
Brian Warner [Wed, 6 Feb 2008 21:15:33 +0000 (14:15 -0700)]
tahoe dump-share: remove --filename option, just take it from argv

16 years agotest_system: deferred-handling logic in test_upload_and_download was broken
Brian Warner [Wed, 6 Feb 2008 21:05:11 +0000 (14:05 -0700)]
test_system: deferred-handling logic in test_upload_and_download was broken

16 years agodump-share: tweak formatting a little bit, to make dumping multiple shares in a row...
Brian Warner [Wed, 6 Feb 2008 20:37:43 +0000 (13:37 -0700)]
dump-share: tweak formatting a little bit, to make dumping multiple shares in a row easier to read

16 years agoadd 'tahoe find-shares' command, to locate share files on a local node's disk
Brian Warner [Wed, 6 Feb 2008 20:19:51 +0000 (13:19 -0700)]
add 'tahoe find-shares' command, to locate share files on a local node's disk

16 years agodump-cap: include UEB_hash in output
Brian Warner [Wed, 6 Feb 2008 19:48:19 +0000 (12:48 -0700)]
dump-cap: include UEB_hash in output

16 years agodownload.py: make logging safe in ValidatedBucket
Brian Warner [Wed, 6 Feb 2008 09:50:34 +0000 (02:50 -0700)]
download.py: make logging safe in ValidatedBucket

16 years agowebish: more upload stats: total encode-and-push rate, already-in-grid existence...
Brian Warner [Wed, 6 Feb 2008 09:39:01 +0000 (02:39 -0700)]
webish: more upload stats: total encode-and-push rate, already-in-grid existence check time

16 years agowebish: add when_done= to POST /uri?t=upload . I did not add a 'recent uploads' secti...
Brian Warner [Wed, 6 Feb 2008 09:38:16 +0000 (02:38 -0700)]
webish: add when_done= to POST /uri?t=upload . I did not add a 'recent uploads' section to the welcome page, but I think the new upload-results page provides the desired data

16 years agoUploadResults: add more helper timing stats (ciphertext fetch times)
Brian Warner [Wed, 6 Feb 2008 09:12:35 +0000 (02:12 -0700)]
UploadResults: add more helper timing stats (ciphertext fetch times)

16 years agoadd upload-results timing info for helper uploads. This changes the Helper protocol...
Brian Warner [Wed, 6 Feb 2008 08:52:25 +0000 (01:52 -0700)]
add upload-results timing info for helper uploads. This changes the Helper protocol, and introduces a compatibility break

16 years agoadd upload timings and rates to the POST /uri?t=upload results page
Brian Warner [Wed, 6 Feb 2008 07:41:51 +0000 (00:41 -0700)]
add upload timings and rates to the POST /uri?t=upload results page

16 years agowebish: make POST /uri?t=upload deposit you on an 'Upload Results' page
Brian Warner [Wed, 6 Feb 2008 06:01:37 +0000 (23:01 -0700)]
webish: make POST /uri?t=upload deposit you on an 'Upload Results' page

16 years agowebish.py: refactor /uri handlers, one rend.Page class per operation
Brian Warner [Wed, 6 Feb 2008 05:38:20 +0000 (22:38 -0700)]
webish.py: refactor /uri handlers, one rend.Page class per operation

16 years agotest_web.py: hush pyflakes
Brian Warner [Wed, 6 Feb 2008 05:37:51 +0000 (22:37 -0700)]
test_web.py: hush pyflakes

16 years agowebish: add PUT /uri?mutable=true
Brian Warner [Wed, 6 Feb 2008 05:18:02 +0000 (22:18 -0700)]
webish: add PUT /uri?mutable=true

16 years agowebish: add POST /uri?t=upload&mutable=true
Brian Warner [Wed, 6 Feb 2008 05:10:22 +0000 (22:10 -0700)]
webish: add POST /uri?t=upload&mutable=true

16 years agowebish: add upload/view-uri forms (not associated with any particular directory)...
Brian Warner [Wed, 6 Feb 2008 04:44:40 +0000 (21:44 -0700)]
webish: add upload/view-uri forms (not associated with any particular directory) to the welcome page. Document POST /uri?t=upload .

16 years agoupload: return an UploadResults instance (with .uri) instead of just a URI
Brian Warner [Wed, 6 Feb 2008 04:01:38 +0000 (21:01 -0700)]
upload: return an UploadResults instance (with .uri) instead of just a URI

16 years agowebish: add storage-consumed estimate on welcome page
Brian Warner [Wed, 6 Feb 2008 03:29:39 +0000 (20:29 -0700)]
webish: add storage-consumed estimate on welcome page

16 years agostorage: change service name from 'storageserver' to 'storage'
Brian Warner [Wed, 6 Feb 2008 03:28:59 +0000 (20:28 -0700)]
storage: change service name from 'storageserver' to 'storage'

16 years agowebish: display tahoe import path on the welcome page, to help figure out where the...
Brian Warner [Wed, 6 Feb 2008 03:08:49 +0000 (20:08 -0700)]
webish: display tahoe import path on the welcome page, to help figure out where the code is coming from

16 years agorefactor node startup, remove tub_ready()
Brian Warner [Wed, 6 Feb 2008 02:58:38 +0000 (19:58 -0700)]
refactor node startup, remove tub_ready()

16 years agosetup: fix the md5sum of the bundled setuptools egg
Zooko O'Whielacronx [Wed, 6 Feb 2008 19:35:29 +0000 (12:35 -0700)]
setup: fix the md5sum of the bundled setuptools egg

16 years agodocs/testgrid/introducer.furl: update to new introducer
Brian Warner [Wed, 6 Feb 2008 01:48:26 +0000 (18:48 -0700)]
docs/testgrid/introducer.furl: update to new introducer

16 years agowebish: condense display of nickname a little bit
Brian Warner [Wed, 6 Feb 2008 01:29:28 +0000 (18:29 -0700)]
webish: condense display of nickname a little bit

16 years agowebish: show nickname too
Brian Warner [Wed, 6 Feb 2008 01:26:05 +0000 (18:26 -0700)]
webish: show nickname too

16 years agowebish: add extra introducer data (version, timestamps) to Welcome page
Brian Warner [Wed, 6 Feb 2008 00:32:27 +0000 (17:32 -0700)]
webish: add extra introducer data (version, timestamps) to Welcome page

16 years agotest_introducer.py: increase timeouts on poll() calls
Brian Warner [Tue, 5 Feb 2008 23:37:58 +0000 (16:37 -0700)]
test_introducer.py: increase timeouts on poll() calls

16 years agobundle foolscap-0.2.4 instead of 0.2.3
Brian Warner [Tue, 5 Feb 2008 22:27:41 +0000 (15:27 -0700)]
bundle foolscap-0.2.4 instead of 0.2.3

16 years agobump foolscap dependency to 0.2.4, since we now use log.err
Brian Warner [Tue, 5 Feb 2008 22:27:14 +0000 (15:27 -0700)]
bump foolscap dependency to 0.2.4, since we now use log.err

16 years agodocs/configuration.txt: wrap to 80 cols
Brian Warner [Tue, 5 Feb 2008 21:35:12 +0000 (14:35 -0700)]
docs/configuration.txt: wrap to 80 cols

16 years agodocs/configuration.txt: document nickname, no_storage, readonly_storage
Brian Warner [Tue, 5 Feb 2008 21:33:29 +0000 (14:33 -0700)]
docs/configuration.txt: document nickname, no_storage, readonly_storage

16 years agointroducer: remove remaining bits of 'push-to-myself' flags. The uploading/downloadin...
Brian Warner [Tue, 5 Feb 2008 21:16:01 +0000 (14:16 -0700)]
introducer: remove remaining bits of 'push-to-myself' flags. The uploading/downloading node is no longer special.

16 years agointroducer: remove PeerCountObserver, tests are managing with purely poll-for-connect...
Brian Warner [Tue, 5 Feb 2008 21:15:49 +0000 (14:15 -0700)]
introducer: remove PeerCountObserver, tests are managing with purely poll-for-connected approachers

16 years agobig introducer refactoring: separate publish+subscribe. Addresses #271.
Brian Warner [Tue, 5 Feb 2008 20:05:13 +0000 (13:05 -0700)]
big introducer refactoring: separate publish+subscribe. Addresses #271.