1 User visible changes in Tahoe-LAFS. -*- outline -*-
7 *** Immutable Directories
9 Tahoe can now create and handle immutable directories. These are read just
10 like normal directories, but are "deep-immutable", meaning that all their
11 children (and everything reachable from those children) must be immutable
12 objects (i.e. immutable/literal files, and other immutable directories).
14 These directories must be created in a single webapi call, which provides all
15 of the children at once (instead of the usual create/add/add sequence, since
16 they cannot be changed after creation). They have URIs that start with
17 "URI:DIR2-CHK:" or "URI:DIR2-LIT:", and are described on the human-facing web
18 interface (aka the "WUI") with a "DIR-IMM" abbreviation (as opposed to "DIR"
19 for the usual read-write directories and "DIR-RO" for read-only directories).
21 Tahoe releases before 1.6.0 cannot read the contents of an immutable
22 directory. 1.5.0 will tolerate their presence in a directory listing (and
23 display it as an "unknown node"). 1.4.1 and earlier cannot tolerate them: a
24 DIR-IMM child in any directory will prevent the listing of that directory.
26 Immutable directories are repairable, just like normal immutable files.
28 The webapi "POST t=mkdir-immutable" call is used to create immutable
29 directories. See docs/frontends/webapi.txt for details.
31 *** "tahoe backup" now creates immutable directories, backupdb has dircache
33 The "tahoe backup" command has been enhanced to create immutable directories
34 (in previous releases, it created read-only mutable directories). This is
35 significantly faster, since it does not need to create an RSA keypair for
36 each new directory. Also "DIR-IMM" immutable directories are repairable,
37 unlike "DIR-RO" read-only mutable directories (at least in this release: a
38 future Tahoe release should be able to repair DIR-RO).
40 In addition, the backupdb (used by "tahoe backup" to remember what it has
41 already copied) has been enhanced to store information about existing
42 immutable directories. This allows it to re-use directories that have moved
43 but still contain identical contents, or which have been deleted and later
44 replaced. (the 1.5.0 "tahoe backup" command could only re-use directories
45 that were in the same place as they were in the immediately previous backup).
46 With this change, the backup process no longer needs to read the previous
47 snapshot out of the Tahoe grid, reducing the network load considerably.
49 A "null backup" (in which nothing has changed since the previous backup) will
50 require only two Tahoe-side operations: one to add an Archives/$TIMESTAMP
51 entry, and a second to update the Latest/ link. On the local disk side, it
52 will readdir() all your local directories and stat() all your local files.
54 If you've been using "tahoe backup" for a while, you will notice that your
55 first use of it after upgrading to 1.6.0 may take a long time: it must create
56 proper immutable versions of all the old read-only mutable directories. This
57 process won't take as long as the initial backup (where all the file contents
58 had to be uploaded too): it will require time proportional to the number and
59 size of your directories. After this initial pass, all subsequent passes
60 should take a tiny fraction of the time.
62 As noted above, Tahoe versions earlier than 1.5.0 cannot read immutable
65 The "tahoe backup" command has been improved to skip over unreadable objects
66 (like device files, named pipes, and files with permissions that prevent the
67 command from reading their contents), instead of throwing an exception and
68 terminating the backup process. It also skips over symlinks, because these
69 cannot be represented faithfully in the Tahoe-side filesystem. A warning
70 message will be emitted each time something is skipped. (#729, #850, #641)
72 *** "create-node" command added, "create-client" now implies --no-storage
74 The basic idea behind Tahoe's client+server and client-only processes is that
75 you are creating a general-purpose Tahoe "node" process, which has several
76 components activated (or not). Storage service is one of these optional
77 components, as is the Helper, FTP server, and SFTP server. (Client/webapi
78 functionality is nominally on this list, but it is always active: a future
79 release will make it optional). The special-purpose servers remain separate
80 (introducer, key-generator, stats-gatherer).
82 So now "tahoe create-node" will create a Tahoe node process, and after
83 creation you can edit its tahoe.cfg to enable or disable the desired
84 services. It is a more general-purpose replacement for "tahoe create-client".
85 The default configuration has storage service enabled. For convenience, the
86 "--no-storage" argument makes a tahoe.cfg file that disables storage service.
88 "tahoe create-client" has been changed to create a Tahoe node without a
89 storage service. It is equivalent to "tahoe create-node --no-storage". This
90 helps to reduce the confusion surrounding the use of a command with "client"
91 in its name to create a storage *server*. Use "tahoe create-client" to create
92 a purely client-side node. If you want to offer storage to the grid, use
93 "tahoe create-node" instead.
95 In the future, other services will be added to the node, and they will be
96 controlled through options in tahoe.cfg . The most important of these
97 services may get additional --enable-XYZ or --disable-XYZ arguments to "tahoe
102 The webapi acquired a new "t=mkdir-with-children" command, to create and
103 populate a directory in a single call. This is significantly faster than
104 using separate "t=mkdir" and "t=set-children" operations (it uses one
105 gateway-to-grid roundtrip, instead of three or four).
107 The t=set-children (note the hyphen) operation is now documented in
108 docs/frontends/webapi.txt, and is the new preferred spelling of the old
109 t=set_children (with an underscore). The underscore version remains for
110 backwards compatibility.
112 The tracebacks produced by errors in CLI tools should now be in plain text,
113 instead of HTML (which is unreadable outside of a browser). (#646)
115 The [storage]reserved_space configuration knob (which causes the storage
116 server to refuse shares when available disk space drops below a threshold)
117 should work on windows now, not just unix. (#637)
119 "tahoe cp" should now exit with status "1" if it cannot figure out a suitable
120 target filename, such as when you copy from a bare filecap. (#761)
122 "tahoe get" no longer creates a zero-length file upon error. (#121)
124 "tahoe ls" can now list single files. (#457)
126 "tahoe deep-check --repair" should tolerate repair failures now, instead of
127 halting traversal. (#874, #786)
129 Many small packaging improvements were made to facilitate the "tahoe-lafs"
130 package being added to Ubuntu's "Karmic Koala" 9.10 release. Several
131 mac/win32 binary libraries were removed, some figleaf code-coverage files
132 were removed, a bundled copy of darcsver-1.2.1 was removed, and additional
133 licensing text was added.
135 Several DeprecationWarnings for python2.6 were silenced. (#859)
138 * Release 1.5.0 (2009-08-01)
142 Uploads of immutable files now use pipelined writes, improving upload speed
143 slightly (10%) over high-latency connections. (#392)
145 Processing large directories has been sped up, by removing a O(N^2) algorithm
146 from the dirnode decoding path and retaining unmodified encrypted entries.
149 The human-facing web interface (aka the "WUI") received a significant CSS
150 makeover by Kevin Reid, making it much prettier and easier to read. The WUI
151 "check" and "deep-check" forms now include a "Renew Lease" checkbox,
152 mirroring the CLI --add-lease option, so leases can be added or renewed from
155 The CLI "tahoe mv" command now refuses to overwrite directories. (#705)
157 The CLI "tahoe webopen" command, when run without arguments, will now bring
158 up the "Welcome Page" (node status and mkdir/upload forms).
160 The 3.5MB limit on mutable files was removed, so it should be possible to
161 upload arbitrarily-sized mutable files. Note, however, that the data format
162 and algorithm remains the same, so using mutable files still requires
163 bandwidth, computation, and RAM in proportion to the size of the mutable file.
166 This version of Tahoe-LAFS will tolerate directory entries that contain filecap
167 formats which it does not recognize: files and directories from the future.
168 This should improve the user experience (for 1.5.0 users) when we add new cap
169 formats in the future. Previous versions would fail badly, preventing the user
170 from seeing or editing anything else in those directories. These unrecognized
171 objects can be renamed and deleted, but obviously not read or written. Also
172 they cannot generally be copied. (#683)
176 deep-check-and-repair now tolerates read-only directories, such as the ones
177 produced by the "tahoe backup" CLI command. Read-only directories and mutable
178 files are checked, but not repaired. Previous versions threw an exception
179 when attempting the repair and failed to process the remaining contents. We
180 cannot yet repair these read-only objects, but at least this version allows
181 the rest of the check+repair to proceed. (#625)
183 A bug in 1.4.1 which caused a server to be listed multiple times (and
184 frequently broke all connections to that server) was fixed. (#653)
186 The plaintext-hashing code was removed from the Helper interface, removing
187 the Helper's ability to mount a partial-information-guessing attack. (#722)
189 ** Platform/packaging changes
191 Tahoe-LAFS now runs on NetBSD, OpenBSD, ArchLinux, and NixOS, and on an
192 embedded system based on an ARM CPU running at 266 MHz.
194 Unit test timeouts have been raised to allow the tests to complete on
195 extremely slow platforms like embedded ARM-based NAS boxes, which may take
196 several hours to run the test suite. An ARM-specific data-corrupting bug in
197 an older version of Crypto++ (5.5.2) was identified: ARM-users are encouraged
198 to use recent Crypto++/pycryptopp which avoids this problem.
200 Tahoe-LAFS now requires a SQLite library, either the sqlite3 that comes
201 built-in with python2.5/2.6, or the add-on pysqlite2 if you're using
202 python2.4. In the previous release, this was only needed for the "tahoe backup"
203 command: now it is mandatory.
205 Several minor documentation updates were made.
207 To help get Tahoe-LAFS into Linux distributions like Fedora and Debian,
208 packaging improvements are being made in both Tahoe-LAFS and related libraries
209 like pycryptopp and zfec.
211 The Crypto++ library included in the pycryptopp package has been upgraded to
212 version 5.6.0 of Crypto++, which includes a more efficient implementation of
213 SHA-256 in assembly for x86 or amd64 architectures.
215 ** dependency updates
218 no python-2.4.0 or 2.4.1 (2.4.2 is good)
219 (they contained a bug in base64.b32decode)
220 avoid python-2.6 on windows with mingw: compiler issues
221 python2.4 requires pysqlite2 (2.5,2.6 does not)
226 * Release 1.4.1 (2009-04-13)
228 ** Garbage Collection
230 The big feature for this release is the implementation of garbage collection,
231 allowing Tahoe storage servers to delete shares for old deleted files. When
232 enabled, this uses a "mark and sweep" process: clients are responsible for
233 updating the leases on their shares (generally by running "tahoe deep-check
234 --add-lease"), and servers are allowed to delete any share which does not
235 have an up-to-date lease. The process is described in detail in
236 docs/garbage-collection.txt .
238 The server must be configured to enable garbage-collection, by adding
239 directives to the [storage] section that define an age limit for shares. The
240 default configuration will not delete any shares.
242 Both servers and clients should be upgraded to this release to make the
243 garbage-collection as pleasant as possible. 1.2.0 servers have code to
244 perform the update-lease operation but it suffers from a fatal bug, while
245 1.3.0 servers have update-lease but will return an exception for unknown
246 storage indices, causing clients to emit an Incident for each exception,
247 slowing the add-lease process down to a crawl. 1.1.0 servers did not have the
248 add-lease operation at all.
250 ** Security/Usability Problems Fixed
252 A super-linear algorithm in the Merkle Tree code was fixed, which previously
253 caused e.g. download of a 10GB file to take several hours before the first
254 byte of plaintext could be produced. The new "alacrity" is about 2 minutes. A
255 future release should reduce this to a few seconds by fixing ticket #442.
257 The previous version permitted a small timing attack (due to our use of
258 strcmp) against the write-enabler and lease-renewal/cancel secrets. An
259 attacker who could measure response-time variations of approximatly 3ns
260 against a very noisy background time of about 15ms might be able to guess
261 these secrets. We do not believe this attack was actually feasible. This
262 release closes the attack by first hashing the two strings to be compared
263 with a random secret.
267 In most cases, HTML tracebacks will only be sent if an "Accept: text/html"
268 header was provided with the HTTP request. This will generally cause browsers
269 to get an HTMLized traceback but send regular text/plain tracebacks to
270 non-browsers (like the CLI clients). More errors have been mapped to useful
273 The streaming webapi operations (deep-check and manifest) now have a way to
274 indicate errors (an output line that starts with "ERROR" instead of being
275 legal JSON). See docs/frontends/webapi.txt for details.
277 The storage server now has its own status page (at /storage), linked from the
278 Welcome page. This page shows progress and results of the two new
279 share-crawlers: one which merely counts shares (to give an estimate of how
280 many files/directories are being stored in the grid), the other examines
281 leases and reports how much space would be freed if GC were enabled. The page
282 also shows how much disk space is present, used, reserved, and available for
283 the Tahoe server, and whether the server is currently running in "read-write"
284 mode or "read-only" mode.
286 When a directory node cannot be read (perhaps because of insufficent shares),
287 a minimal webapi page is created so that the "more-info" links (including a
288 Check/Repair operation) will still be accessible.
290 A new "reliability" page was added, with the beginnings of work on a
291 statistical loss model. You can tell this page how many servers you are using
292 and their independent failure probabilities, and it will tell you the
293 likelihood that an arbitrary file will survive each repair period. The
294 "numpy" package must be installed to access this page. A partial paper,
295 written by Shawn Willden, has been added to docs/proposed/lossmodel.lyx .
299 "tahoe check" and "tahoe deep-check" now accept an "--add-lease" argument, to
300 update a lease on all shares. This is the "mark" side of garbage collection.
302 In many cases, CLI error messages have been improved: the ugly HTMLized
303 traceback has been replaced by a normal python traceback.
305 "tahoe deep-check" and "tahoe manifest" now have better error reporting.
306 "tahoe cp" is now non-verbose by default.
308 "tahoe backup" now accepts several "--exclude" arguments, to ignore certain
309 files (like editor temporary files and version-control metadata) during
312 On windows, the CLI now accepts local paths like "c:\dir\file.txt", which
313 previously was interpreted as a Tahoe path using a "c:" alias.
315 The "tahoe restart" command now uses "--force" by default (meaning it will
316 start a node even if it didn't look like there was one already running).
318 The "tahoe debug consolidate" command was added. This takes a series of
319 independent timestamped snapshot directories (such as those created by the
320 allmydata.com windows backup program, or a series of "tahoe cp -r" commands)
321 and creates new snapshots that used shared read-only directories whenever
322 possible (like the output of "tahoe backup"). In the most common case (when
323 the snapshots are fairly similar), the result will use significantly fewer
324 directories than the original, allowing "deep-check" and similar tools to run
325 much faster. In some cases, the speedup can be an order of magnitude or more.
326 This tool is still somewhat experimental, and only needs to be run on large
327 backups produced by something other than "tahoe backup", so it was placed
328 under the "debug" category.
330 "tahoe cp -r --caps-only tahoe:dir localdir" is a diagnostic tool which,
331 instead of copying the full contents of files into the local directory,
332 merely copies their filecaps. This can be used to verify the results of a
333 "consolidation" operation.
337 The codebase no longer rauses RuntimeError as a kind of assert(). Specific
338 exception classes were created for each previous instance of RuntimeError.
340 Many unit tests were changed to use a non-network test harness, speeding them
343 Deep-traversal operations (manifest and deep-check) now walk individual
344 directories in alphabetical order. Occasional turn breaks are inserted to
345 prevent a stack overflow when traversing directories with hundreds of
348 The experimental SFTP server had its path-handling logic changed slightly, to
349 accomodate more SFTP clients, although there are still issues (#645).
352 * Release 1.3.0 (2009-02-13)
354 ** Checker/Verifier/Repairer
356 The primary focus of this release has been writing a checker / verifier /
357 repairer for files and directories. "Checking" is the act of asking storage
358 servers whether they have a share for the given file or directory: if there
359 are not enough shares available, the file or directory will be
360 unrecoverable. "Verifying" is the act of downloading and cryptographically
361 asserting that the server's share is undamaged: it requires more work
362 (bandwidth and CPU) than checking, but can catch problems that simple
363 checking cannot. "Repair" is the act of replacing missing or damaged shares
366 This release includes a full checker, a partial verifier, and a partial
367 repairer. The repairer is able to handle missing shares: new shares are
368 generated and uploaded to make up for the missing ones. This is currently the
369 best application of the repairer: to replace shares that were lost because of
370 server departure or permanent drive failure.
372 The repairer in this release is somewhat able to handle corrupted shares. The
375 * Immutable verifier is incomplete: not all shares are used, and not all
376 fields of those shares are verified. Therefore the immutable verifier has
377 only a moderate chance of detecting corrupted shares.
378 * The mutable verifier is mostly complete: all shares are examined, and most
379 fields of the shares are validated.
380 * The storage server protocol offers no way for the repairer to replace or
381 delete immutable shares. If corruption is detected, the repairer will
382 upload replacement shares to other servers, but the corrupted shares will
384 * read-only directories and read-only mutable files must be repaired by
385 someone who holds the write-cap: the read-cap is insufficient. Moreover,
386 the deep-check-and-repair operation will halt with an error if it attempts
387 to repair one of these read-only objects.
388 * Some forms of corruption can cause both download and repair operations to
389 fail. A future release will fix this, since download should be tolerant of
390 any corruption as long as there are at least 'k' valid shares, and repair
391 should be able to fix any file that is downloadable.
393 If the downloader, verifier, or repairer detects share corruption, the
394 servers which provided the bad shares will be notified (via a file placed in
395 the BASEDIR/storage/corruption-advisories directory) so their operators can
396 manually delete the corrupted shares and investigate the problem. In
397 addition, the "incident gatherer" mechanism will automatically report share
398 corruption to an incident gatherer service, if one is configured. Note that
399 corrupted shares indicate hardware failures, serious software bugs, or malice
400 on the part of the storage server operator, so a corrupted share should be
401 considered highly unusual.
403 By periodically checking/repairing all files and directories, objects in the
404 Tahoe filesystem remain resistant to recoverability failures due to missing
405 and/or broken servers.
407 This release includes a wapi mechanism to initiate checks on individual
408 files and directories (with or without verification, and with or without
409 automatic repair). A related mechanism is used to initiate a "deep-check" on
410 a directory: recursively traversing the directory and its children, checking
411 (and/or verifying/repairing) everything underneath. Both mechanisms can be
412 run with an "output=JSON" argument, to obtain machine-readable check/repair
413 status results. These results include a copy of the filesystem statistics
414 from the "deep-stats" operation (including total number of files, size
415 histogram, etc). If repair is possible, a "Repair" button will appear on the
418 The client web interface now features some extra buttons to initiate check
419 and deep-check operations. When these operations finish, they display a
420 results page that summarizes any problems that were encountered. All
421 long-running deep-traversal operations, including deep-check, use a
422 start-and-poll mechanism, to avoid depending upon a single long-lived HTTP
423 connection. docs/frontends/webapi.txt has details.
427 The "tahoe backup" command is new in this release, which creates efficient
428 versioned backups of a local directory. Given a local pathname and a target
429 Tahoe directory, this will create a read-only snapshot of the local directory
430 in $target/Archives/$timestamp. It will also create $target/Latest, which is
431 a reference to the latest such snapshot. Each time you run "tahoe backup"
432 with the same source and target, a new $timestamp snapshot will be added.
433 These snapshots will share directories that have not changed since the last
434 backup, to speed up the process and minimize storage requirements. In
435 addition, a small database is used to keep track of which local files have
436 been uploaded already, to avoid uploading them a second time. This
437 drastically reduces the work needed to do a "null backup" (when nothing has
438 changed locally), making "tahoe backup' suitable to run from a daily cronjob.
440 Note that the "tahoe backup" CLI command must be used in conjunction with a
441 1.3.0-or-newer Tahoe client node; there was a bug in the 1.2.0 webapi
442 implementation that would prevent the last step (create $target/Latest) from
447 The 12GiB (approximate) immutable-file-size limitation is lifted. This
448 release knows how to handle so-called "v2 immutable shares", which permit
449 immutable files of up to about 18 EiB (about 3*10^14). These v2 shares are
450 created if the file to be uploaded is too large to fit into v1 shares. v1
451 shares are created if the file is small enough to fit into them, so that
452 files created with tahoe-1.3.0 can still be read by earlier versions if they
453 are not too large. Note that storage servers also had to be changed to
454 support larger files, and this release is the first release in which they are
455 able to do that. Clients will detect which servers are capable of supporting
456 large files on upload and will not attempt to upload shares of a large file
457 to a server which doesn't support it.
461 Tahoe now includes experimental FTP and SFTP servers. When configured with a
462 suitable method to translate username+password into a root directory cap, it
463 provides simple access to the virtual filesystem. Remember that FTP is
464 completely unencrypted: passwords, filenames, and file contents are all sent
465 over the wire in cleartext, so FTP should only be used on a local (127.0.0.1)
466 connection. This feature is still in development: there are no unit tests
467 yet, and behavior with respect to Unicode filenames is uncertain. Please see
468 docs/frontends/FTP-and-SFTP.txt for configuration details. (#512, #531)
472 This release adds the 'tahoe create-alias' command, which is a combination of
473 'tahoe mkdir' and 'tahoe add-alias'. This also allows you to start using a
474 new tahoe directory without exposing its URI in the argv list, which is
475 publicly visible (through the process table) on most unix systems. Thanks to
476 Kevin Reid for bringing this issue to our attention.
478 The single-argument form of "tahoe put" was changed to create an unlinked
479 file. I.e. "tahoe put bar.txt" will take the contents of a local "bar.txt"
480 file, upload them to the grid, and print the resulting read-cap; the file
481 will not be attached to any directories. This seemed a bit more useful than
482 the previous behavior (copy stdin, upload to the grid, attach the resulting
483 file into your default tahoe: alias in a child named 'bar.txt').
485 "tahoe put" was also fixed to handle mutable files correctly: "tahoe put
486 bar.txt URI:SSK:..." will read the contents of the local bar.txt and use them
487 to replace the contents of the given mutable file.
489 The "tahoe webopen" command was modified to accept aliases. This means "tahoe
490 webopen tahoe:" will cause your web browser to open to a "wui" page that
491 gives access to the directory associated with the default "tahoe:" alias. It
492 should also accept leading slashes, like "tahoe webopen tahoe:/stuff".
494 Many esoteric debugging commands were moved down into a "debug" subcommand:
497 tahoe debug dump-share
498 tahoe debug find-shares
499 tahoe debug catalog-shares
500 tahoe debug corrupt-share
502 The last command ("tahoe debug corrupt-share") flips a random bit of the
503 given local sharefile. This is used to test the file verifying/repairing
504 code, and obviously should not be used on user data.
506 The cli might not correctly handle arguments which contain non-ascii
507 characters in Tahoe v1.3 (although depending on your platform it
508 might, especially if your platform can be configured to pass such
509 characters on the command-line in utf-8 encoding). See
510 http://allmydata.org/trac/tahoe/ticket/565 for details.
514 The "default webapi port", used when creating a new client node (and in the
515 getting-started documentation), was changed from 8123 to 3456, to reduce
516 confusion when Tahoe accessed through a Firefox browser on which the
517 "Torbutton" extension has been installed. Port 8123 is occasionally used as a
518 Tor control port, so Torbutton adds 8123 to Firefox's list of "banned ports"
519 to avoid CSRF attacks against Tor. Once 8123 is banned, it is difficult to
520 diagnose why you can no longer reach a Tahoe node, so the Tahoe default was
521 changed. Note that 3456 is reserved by IANA for the "vat" protocol, but there
522 are argueably more Torbutton+Tahoe users than vat users these days. Note that
523 this will only affect newly-created client nodes. Pre-existing client nodes,
524 created by earlier versions of tahoe, may still be listening on 8123.
526 All deep-traversal operations (start-manifest, start-deep-size,
527 start-deep-stats, start-deep-check) now use a start-and-poll approach,
528 instead of using a single (fragile) long-running synchronous HTTP connection.
529 All these "start-" operations use POST instead of GET. The old "GET
530 manifest", "GET deep-size", and "POST deep-check" operations have been
533 The new "POST start-manifest" operation, when it finally completes, results
534 in a table of (path,cap), instead of the list of verifycaps produced by the
535 old "GET manifest". The table is available in several formats: use
536 output=html, output=text, or output=json to choose one. The JSON output also
537 includes stats, and a list of verifycaps and storage-index strings.
539 The "return_to=" and "when_done=" arguments have been removed from the
540 t=check and deep-check operations.
542 The top-level status page (/status) now has a machine-readable form, via
543 "/status/?t=json". This includes information about the currently-active
544 uploads and downloads, which may be useful for frontends that wish to display
545 progress information. There is no easy way to correlate the activities
546 displayed here with recent wapi requests, however.
548 Any files in BASEDIR/public_html/ (configurable) will be served in response
549 to requests in the /static/ portion of the URL space. This will simplify the
550 deployment of javascript-based frontends that can still access wapi calls
551 by conforming to the (regrettable) "same-origin policy".
553 The welcome page now has a "Report Incident" button, which is tied into the
554 "Incident Gatherer" machinery. If the node is attached to an incident
555 gatherer (via log_gatherer.furl), then pushing this button will cause an
556 Incident to be signalled: this means recent log events are aggregated and
557 sent in a bundle to the gatherer. The user can push this button after
558 something strange takes place (and they can provide a short message to go
559 along with it), and the relevant data will be delivered to a centralized
560 incident-gatherer for later processing by operations staff.
562 The "HEAD" method should now work correctly, in addition to the usual "GET",
563 "PUT", and "POST" methods. "HEAD" is supposed to return exactly the same
564 headers as "GET" would, but without any of the actual response body data. For
565 mutable files, this now does a brief mapupdate (to figure out the size of the
566 file that would be returned), without actually retrieving the file's
569 The "GET" operation on files can now support the HTTP "Range:" header,
570 allowing requests for partial content. This allows certain media players to
571 correctly stream audio and movies out of a Tahoe grid. The current
572 implementation uses a disk-based cache in BASEDIR/private/cache/download ,
573 which holds the plaintext of the files being downloaded. Future
574 implementations might not use this cache. GET for immutable files now returns
577 Each file and directory now has a "Show More Info" web page, which contains
578 much of the information that was crammed into the directory page before. This
579 includes readonly URIs, storage index strings, object type, buttons to
580 control checking/verifying/repairing, and deep-check/deep-stats buttons (for
581 directories). For mutable files, the "replace contents" upload form has been
582 moved here too. As a result, the directory page is now much simpler and
583 cleaner, and several potentially-misleading links (like t=uri) are now gone.
585 Slashes are discouraged in Tahoe file/directory names, since they cause
586 problems when accessing the filesystem through the wapi. However, there are
587 a couple of accidental ways to generate such names. This release tries to
588 make it easier to correct such mistakes by escaping slashes in several
589 places, allowing slashes in the t=info and t=delete commands, and in the
590 source (but not the target) of a t=rename command.
594 Tahoe's dependencies have been extended to require the "[secure_connections]"
595 feature from Foolscap, which will cause pyOpenSSL to be required and/or
596 installed. If OpenSSL and its development headers are already installed on
597 your system, this can occur automatically. Tahoe now uses pollreactor
598 (instead of the default selectreactor) to work around a bug between pyOpenSSL
599 and the most recent release of Twisted (8.1.0). This bug only affects unit
600 tests (hang during shutdown), and should not impact regular use.
602 The Tahoe source code tarballs now come in two different forms: regular and
603 "sumo". The regular tarball contains just Tahoe, nothing else. When building
604 from the regular tarball, the build process will download any unmet
605 dependencies from the internet (starting with the index at PyPI) so it can
606 build and install them. The "sumo" tarball contains copies of all the
607 libraries that Tahoe requires (foolscap, twisted, zfec, etc), so using the
608 "sumo" tarball should not require any internet access during the build
609 process. This can be useful if you want to build Tahoe while on an airplane,
610 a desert island, or other bandwidth-limited environments.
612 Similarly, allmydata.org now hosts a "tahoe-deps" tarball which contains the
613 latest versions of all these dependencies. This tarball, located at
614 http://allmydata.org/source/tahoe/deps/tahoe-deps.tar.gz, can be unpacked in
615 the tahoe source tree (or in its parent directory), and the build process
616 should satisfy its downloading needs from it instead of reaching out to PyPI.
617 This can be useful if you want to build Tahoe from a darcs checkout while on
618 that airplane or desert island.
620 Because of the previous two changes ("sumo" tarballs and the "tahoe-deps"
621 bundle), most of the files have been removed from misc/dependencies/ . This
622 brings the regular Tahoe tarball down to 2MB (compressed), and the darcs
623 checkout (without history) to about 7.6MB. A full darcs checkout will still
624 be fairly large (because of the historical patches which included the
625 dependent libraries), but a 'lazy' one should now be small.
627 The default "make" target is now an alias for "setup.py build", which itself
628 is an alias for "setup.py develop --prefix support", with some extra work
629 before and after (see setup.cfg). Most of the complicated platform-dependent
630 code in the Makefile was rewritten in Python and moved into setup.py,
631 simplifying things considerably.
633 Likewise, the "make test" target now delegates most of its work to "setup.py
634 test", which takes care of getting PYTHONPATH configured to access the tahoe
635 code (and dependencies) that gets put in support/lib/ by the build_tahoe
636 step. This should allow unit tests to be run even when trial (which is part
637 of Twisted) wasn't already installed (in this case, trial gets installed to
638 support/bin because Twisted is a dependency of Tahoe).
640 Tahoe is now compatible with the recently-released Python 2.6 , although it
641 is recommended to use Tahoe on Python 2.5, on which it has received more
642 thorough testing and deployment.
644 Tahoe is now compatible with simplejson-2.0.x . The previous release assumed
645 that simplejson.loads always returned unicode strings, which is no longer the
648 ** Grid Management Tools
650 Several tools have been added or updated in the misc/ directory, mostly munin
651 plugins that can be used to monitor a storage grid.
653 The misc/spacetime/ directory contains a "disk watcher" daemon (startable
654 with 'tahoe start'), which can be configured with a set of HTTP URLs
655 (pointing at the wapi '/statistics' page of a bunch of storage servers),
656 and will periodically fetch disk-used/disk-available information from all the
657 servers. It keeps this information in an Axiom database (a sqlite-based
658 library available from divmod.org). The daemon computes time-averaged rates
659 of disk usage, as well as a prediction of how much time is left before the
660 grid is completely full.
662 The misc/munin/ directory contains a new set of munin plugins
663 (tahoe_diskleft, tahoe_diskusage, tahoe_doomsday) which talk to the
664 disk-watcher and provide graphs of its calculations.
666 To support the disk-watcher, the Tahoe statistics component (visible through
667 the wapi at the /statistics/ URL) now includes disk-used and disk-available
668 information. Both are derived through an equivalent of the unix 'df' command
669 (i.e. they ask the kernel for the number of free blocks on the partition that
670 encloses the BASEDIR/storage directory). In the future, the disk-available
671 number will be further influenced by the local storage policy: if that policy
672 says that the server should refuse new shares when less than 5GB is left on
673 the partition, then "disk-available" will report zero even though the kernel
676 The 'tahoe_overhead' munin plugin interacts with an allmydata.com-specific
677 server which reports the total of the 'deep-size' reports for all active user
678 accounts, compares this with the disk-watcher data, to report on overhead
679 percentages. This provides information on how much space could be recovered
680 once Tahoe implements some form of garbage collection.
682 ** Configuration Changes: single INI-format tahoe.cfg file
684 The Tahoe node is now configured with a single INI-format file, named
685 "tahoe.cfg", in the node's base directory. Most of the previous
686 multiple-separate-files are still read for backwards compatibility (the
687 embedded SSH debug server and the advertised_ip_addresses files are the
688 exceptions), but new directives will only be added to tahoe.cfg . The "tahoe
689 create-client" command will create a tahoe.cfg for you, with sample values
690 commented out. (ticket #518)
692 tahoe.cfg now has controls for the foolscap "keepalive" and "disconnect"
695 tahoe.cfg now has controls for the encoding parameters: "shares.needed" and
696 "shares.total" in the "[client]" section. The default parameters are still
699 The inefficient storage 'sizelimit' control (which established an upper bound
700 on the amount of space that a storage server is allowed to consume) has been
701 replaced by a lightweight 'reserved_space' control (which establishes a lower
702 bound on the amount of remaining space). The storage server will reject all
703 writes that would cause the remaining disk space (as measured by a '/bin/df'
704 equivalent) to drop below this value. The "[storage]reserved_space="
705 tahoe.cfg parameter controls this setting. (note that this only affects
706 immutable shares: it is an outstanding bug that reserved_space does not
707 prevent the allocation of new mutable shares, nor does it prevent the growth
708 of existing mutable shares).
712 Clients now declare which versions of the protocols they support. This is
713 part of a new backwards-compatibility system:
714 http://allmydata.org/trac/tahoe/wiki/Versioning .
716 The version strings for human inspection (as displayed on the Welcome web
717 page, and included in logs) now includes a platform identifer (frequently
718 including a linux distribution name, processor architecture, etc).
720 Several bugs have been fixed, including one that would cause an exception (in
721 the logs) if a wapi download operation was cancelled (by closing the TCP
722 connection, or pushing the "stop" button in a web browser).
724 Tahoe now uses Foolscap "Incidents", writing an "incident report" file to
725 logs/incidents/ each time something weird occurs. These reports are available
726 to an "incident gatherer" through the flogtool command. For more details,
727 please see the Foolscap logging documentation. An incident-classifying plugin
728 function is provided in misc/incident-gatherer/classify_tahoe.py .
730 If clients detect corruption in shares, they now automatically report it to
731 the server holding that share, if it is new enough to accept the report.
732 These reports are written to files in BASEDIR/storage/corruption-advisories .
734 The 'nickname' setting is now defined to be a UTF-8 -encoded string, allowing
737 The 'tahoe start' command will now accept a --syslog argument and pass it
738 through to twistd, making it easier to launch non-Tahoe nodes (like the
739 cpu-watcher) and have them log to syslogd instead of a local file. This is
740 useful when running a Tahoe node out of a USB flash drive.
742 The Mac GUI in src/allmydata/gui/ has been improved.
745 * Release 1.2.0 (2008-07-21)
749 This release makes the immutable-file "ciphertext hash tree" mandatory.
750 Previous releases allowed the uploader to decide whether their file would
751 have an integrity check on the ciphertext or not. A malicious uploader could
752 use this to create a readcap that would download as one file or a different
753 one, depending upon which shares the client fetched first, with no errors
754 raised. There are other integrity checks on the shares themselves, preventing
755 a storage server or other party from violating the integrity properties of
756 the read-cap: this failure was only exploitable by the uploader who gives you
757 a carefully constructed read-cap. If you download the file with Tahoe 1.2.0
758 or later, you will not be vulnerable to this problem. #491
760 This change does not introduce a compatibility issue, because all existing
761 versions of Tahoe will emit the ciphertext hash tree in their shares.
765 Tahoe now requires Foolscap-0.2.9 . It also requires pycryptopp 0.5 or newer,
766 since earlier versions had a bug that interacted with specific compiler
767 versions that could sometimes result in incorrect encryption behavior. Both
768 packages are included in the Tahoe source tarball in misc/dependencies/ , and
769 should be built automatically when necessary.
773 Web API directory pages should now contain properly-slash-terminated links to
774 other directories. They have also stopped using absolute links in forms and
775 pages (which interfered with the use of a front-end load-balancing proxy).
777 The behavior of the "Check This File" button changed, in conjunction with
778 larger internal changes to file checking/verification. The button triggers an
779 immediate check as before, but the outcome is shown on its own page, and does
780 not get stored anywhere. As a result, the web directory page no longer shows
781 historical checker results.
783 A new "Deep-Check" button has been added, which allows a user to initiate a
784 recursive check of the given directory and all files and directories
785 reachable from it. This can cause quite a bit of work, and has no
786 intermediate progress information or feedback about the process. In addition,
787 the results of the deep-check are extremely limited. A later release will
788 improve this behavior.
790 The web server's behavior with respect to non-ASCII (unicode) filenames in
791 the "GET save=true" operation has been improved. To achieve maximum
792 compatibility with variously buggy web browsers, the server does not try to
793 figure out the character set of the inbound filename. It just echoes the same
794 bytes back to the browser in the Content-Disposition header. This seems to
795 make both IE7 and Firefox work correctly.
797 ** Checker/Verifier/Repairer
799 Tahoe is slowly acquiring convenient tools to check up on file health,
800 examine existing shares for errors, and repair files that are not fully
801 healthy. This release adds a mutable checker/verifier/repairer, although
802 testing is very limited, and there are no web interfaces to trigger repair
803 yet. The "Check" button next to each file or directory on the wapi page
804 will perform a file check, and the "deep check" button on each directory will
805 recursively check all files and directories reachable from there (which may
806 take a very long time).
808 Future releases will improve access to this functionality.
810 ** Operations/Packaging
812 A "check-grid" script has been added, along with a Makefile target. This is
813 intended (with the help of a pre-configured node directory) to check upon the
814 health of a Tahoe grid, uploading and downloading a few files. This can be
815 used as a monitoring tool for a deployed grid, to be run periodically and to
816 signal an error if it ever fails. It also helps with compatibility testing,
817 to verify that the latest Tahoe code is still able to handle files created by
820 The munin plugins from misc/munin/ are now copied into any generated debian
821 packages, and are made executable (and uncompressed) so they can be symlinked
822 directly from /etc/munin/plugins/ .
824 Ubuntu "Hardy" was added as a supported debian platform, with a Makefile
825 target to produce hardy .deb packages. Some notes have been added to
826 docs/debian.txt about building Tahoe on a debian/ubuntu system.
828 Storage servers now measure operation rates and latency-per-operation, and
829 provides results through the /statistics web page as well as the stats
830 gatherer. Munin plugins have been added to match.
834 Tahoe nodes now use Foolscap "incident logging" to record unusual events to
835 their NODEDIR/logs/incidents/ directory. These incident files can be examined
836 by Foolscap logging tools, or delivered to an external log-gatherer for
837 further analysis. Note that Tahoe now requires Foolscap-0.2.9, since 0.2.8
838 had a bug that complained about "OSError: File exists" when trying to create
839 the incidents/ directory for a second time.
841 If no servers are available when retrieving a mutable file (like a
842 directory), the node now reports an error instead of hanging forever. Earlier
843 releases would not only hang (causing the wapi directory listing to get
844 stuck half-way through), but the internal dirnode serialization would cause
845 all subsequent attempts to retrieve or modify the same directory to hang as
848 A minor internal exception (reported in logs/twistd.log, in the
849 "stopProducing" method) was fixed, which complained about "self._paused_at
850 not defined" whenever a file download was stopped from the web browser end.
853 * Release 1.1.0 (2008-06-11)
855 ** CLI: new "alias" model
857 The new CLI code uses an scp/rsync -like interface, in which directories in
858 the Tahoe storage grid are referenced by a colon-suffixed alias. The new
860 tahoe cp local.txt tahoe:virtual.txt
863 More functionality is available through the CLI: creating unlinked files and
864 directories, recursive copy in or out of the storage grid, hardlinks, and
865 retrieving the raw read- or write- caps through the 'ls' command. Please read
866 docs/CLI.txt for complete details.
868 ** wapi: new pages, new commands
870 Several new pages were added to the web API:
872 /helper_status : to describe what a Helper is doing
873 /statistics : reports node uptime, CPU usage, other stats
874 /file : for easy file-download URLs, see #221
875 /cap == /uri : future compatibility
877 The localdir=/localfile= and t=download operations were removed. These
878 required special configuration to enable anyways, but this feature was a
879 security problem, and was mostly obviated by the new "cp -r" command.
881 Several new options to the GET command were added:
883 t=deep-size : add up the size of all immutable files reachable from the directory
884 t=deep-stats : return a JSON-encoded description of number of files, size
885 distribution, total size, etc
887 POST is now preferred over PUT for most operations which cause side-effects.
889 Most wapi calls now accept overwrite=, and default to overwrite=true .
891 "POST /uri/DIRCAP/parent/child?t=mkdir" is now the preferred API to create
892 multiple directories at once, rather than ...?t=mkdir-p .
894 PUT to a mutable file ("PUT /uri/MUTABLEFILECAP", "PUT /uri/DIRCAP/child")
895 will modify the file in-place.
897 ** more munin graphs in misc/munin/
902 mutable files published/retrieved
911 setuptools (now required at runtime)
913 ** New Mutable-File Code
915 The mutable-file handling code (mostly used for directories) has been
916 completely rewritten. The new scheme has a better API (with a modify()
917 method) and is less likely to lose data when several uncoordinated writers
918 change a file at the same time.
920 In addition, a single Tahoe process will coordinate its own writes. If you
921 make two concurrent directory-modifying wapi calls to a single tahoe node,
922 it will internally make one of them wait for the other to complete. This
923 prevents auto-collision (#391).
925 The new mutable-file code also detects errors during publish better. Earlier
926 releases might believe that a mutable file was published when in fact it
931 The node now monitors its own CPU usage, as a percentage, measured every 60
932 seconds. 1/5/15 minute moving averages are available on the /statistics web
933 page and via the stats-gathering interface.
935 Clients now accelerate reconnection to all servers after being offline
936 (#374). When a client is offline for a long time, it scales back reconnection
937 attempts to approximately once per hour, so it may take a while to make the
938 first attempt, but once any attempt succeeds, the other server connections
939 will be retried immediately.
941 A new "offloaded KeyGenerator" facility can be configured, to move RSA key
942 generation out from, say, a wapi node, into a separate process. RSA keys
943 can take several seconds to create, and so a wapi node which is being used
944 for directory creation will be unavailable for anything else during this
945 time. The Key Generator process will pre-compute a small pool of keys, to
946 speed things up further. This also takes better advantage of multi-core CPUs,
949 The node will only use a potentially-slow "du -s" command at startup (to
950 measure how much space has been used) if the "sizelimit" parameter has been
951 configured (to limit how much space is used). Large storage servers should
952 turn off sizelimit until a later release improves the space-management code,
953 since "du -s" on a terabyte filesystem can take hours.
955 The Introducer now allows new announcements to replace old ones, to avoid
956 buildups of obsolete announcements.
958 Immutable files are limited to about 12GiB (when using the default 3-of-10
959 encoding), because larger files would be corrupted by the four-byte
960 share-size field on the storage servers (#439). A later release will remove
961 this limit. Earlier releases would allow >12GiB uploads, but the resulting
962 file would be unretrievable.
964 The docs/ directory has been rearranged, with old docs put in
965 docs/historical/ and not-yet-implemented ones in docs/proposed/ .
967 The Mac OS-X FUSE plugin has a significant bug fix: earlier versions would
968 corrupt writes that used seek() instead of writing the file in linear order.
969 The rsync tool is known to perform writes in this order. This has been fixed.