1 NEW VERSION RELEASED -- Allmydata-Tahoe version 0.6
3 We are pleased to announce the release of version 0.6 of allmydata.org
4 "Tahoe", a secure, decentralized storage grid under a free-software
5 licence. This is the successor to v0.5.1, which was released
6 August 23, 2007 (see [1]).
8 Since v0.5.1 we've made the following changes:
10 * Distribute shares more evenly onto servers -- this makes files more
11 reliable when there are few servers. (ticket #132)
13 * Package Tahoe with setuptools/easy_install. This makes it so that
14 other libraries that Tahoe depends upon get automatically installed
15 when Tahoe is installed. It also means that people who have Python
16 and the easy_install tool can execute "easy_install
17 allmydata-tahoe" on the command-line (including on Windows), and it
18 will download and install Tahoe. (tickets #82, 93, 130)
20 * Memory usage during download now remains low, even if your node is
21 streaming the downloaded content to a slow web browser over
24 * Shares have a version number in them so that in the future we can
25 upgrade the share format without losing old data. (ticket #90)
27 * improved logging, thanks to Arno
29 * Shares now contain leases, which gives us the information to
30 compute which shares are safe to delete, but we haven't yet
31 implemented deletion itself. Eventually, this will enable client
32 quota tracking. (tickets #119, #67)
35 We also fixed other bugs and implemented other improvements. For
36 complete details, see this web page which shows all ticket changes,
37 repository checkins, and wiki changes from August 24 to today,
40 Allmydata.org Tahoe v0.6 is incompatible with Allmydata.org Tahoe
41 v0.5.1 because of the share format version number and the leases.
46 With Tahoe, you can store your files in a distributed way across a set
47 of computers, such that if some of the computers fail or become
48 unavailable, you can still retrieve your data from the remaining
49 computers. You can also securely share your files with other users.
51 This release is targeted at hackers and users who are willing to use a
52 text-oriented web user interface, or a command-line user interface.
53 (Or a RESTful API. Just telnet to localhost and type HTTP requests to
56 Because this software is new, it is not yet recommended for storage of
57 highly confidential data nor for important data which is not otherwise
58 backed up. Given that caveat, this software works and there are no
59 known security flaws which would compromise confidentiality or data
62 This release of Tahoe is suitable for the "friendnet" use case [3].
63 It is easy to set up a private grid which is securely shared among a
64 specific, limited set of friends. Files uploaded to this shared grid
65 will be available to all friends, even when some of the computers are
66 unavailable. It is also easy to encrypt individual files and
67 directories so that only designated recipients can read them.
72 Tahoe is offered under the GNU General Public License (v2 or later),
73 with the added permission that, if you become obligated to release a
74 derived work under this licence (as per section 2.b), you may delay
75 the fulfillment of this obligation for up to 12 months. If you are
76 obligated to release code under section 2.b of this licence, you are
77 obligated to release it under these same terms, including the 12-month
83 Tahoe works on Linux, Mac OS X, Windows, Cygwin, and Solaris. For
84 installation instructions please see the README [4].
89 Once installed, create a "client node". Instruct this client node to
90 connect to a specific "introducer node" by means of config files in
91 the client node's working directory. To join a grid, copy in the
92 .furl files for that grid. To create a private grid, run your own
93 introducer, and copy its .furl files. See the README for step-by-step
96 Each client node can run a local webserver (enabled by writing the
97 desired port number into a file called 'webport'). The welcome page
98 of this webserver shows the node's status, including which introducer
99 is being used and which other nodes are connected.
101 Links from the welcome page lead to other pages that give access to a
102 virtual filesystem, in which each directory is represented by a
103 separate page. Each directory page shows a list of the files
104 available there, with download links, and forms to upload new files.
106 USAGE - command-line interface
108 Run "allmydata-tahoe ls [VIRTUAL PATH NAME]" to list the contents of a
109 virtual directory. Run "allmydata-tahoe get [VIRTUAL FILE NAME]
110 [LOCAL FILE NAME]" to download a file. Run "allmydata-tahoe put
111 [LOCAL FILE NAME] [VIRTUAL FILE NAME]" to upload a file. Run
112 "allmydata-tahoe rm [VIRTUAL PATH NAME]" to unlink a file or directory
113 in the virtual drive.
117 You can control the filesystem through the RESTful web API [5]. Other
118 ways to access the filesystem are planned: please see the
119 roadmap.txt [6] for some plans.
122 HACKING AND COMMUNITY
124 Please join the mailing list [7] to discuss the ideas behind Tahoe and
125 extensions of and uses of Tahoe. Patches that extend and improve
126 Tahoe are gratefully accepted -- roadmap.txt [6] shows the next
127 improvements that we plan to make and CREDITS [8] lists the names of
128 people who've contributed to the project. The wiki Dev page [9]
129 collects various hacking resources including revision history
130 browsing, automated test results (including code coverage), automated
131 performance tests, graphs of how many people are using the public test
132 grid for how many files, and more.
137 Each peer maintains a connection to each other peer. A single
138 distinct server called an "introducer" is used to discover other peers
139 with which to connect.
141 To store a file, the file is encrypted and erasure coded, and each
142 resulting share is uploaded to a different peer. The secure hash of
143 the encrypted file and the encryption key are packed into a URI,
144 knowledge of which is necessary and sufficient to recover the file.
146 To fetch a file, starting with the URI, a subset of shares is
147 downloaded from peers, the file is reconstructed from the shares, and
150 A single distinct server called a "vdrive server" maintains a global
151 mapping from pathnames/filenames to URIs.
153 We are acutely aware of the limitations on decentralization and
154 scalability inherent in this version. In particular, the
155 completely-connected property of the grid and the requirement of a
156 single distinct introducer and vdrive server limits the possible size
157 of the grid. We have plans to loosen these limitations (see
158 roadmap.txt). Currently it should be noted that the grid already
159 depends as little as possible on the accessibility and correctness of
160 the introduction server and the vdrive server. Also note that the
161 choice of which servers to use is easily configured -- you can set up
162 a private grid for you and your friends as easily as connecting to our
166 SOFTWARE ARCHITECTURE
168 Tahoe is a "from the ground-up" rewrite, inspired by Allmydata's
169 existing consumer backup service. It is primarily written in the
170 Python programming language.
172 Tahoe is based on the Foolscap library [10] which provides a remote
173 object protocol inspired by the capability-secure "E" programming
174 language [11]. Foolscap allows us to express the intended behavior of
175 the distributed grid directly in object-oriented terms while relying
176 on a well-engineered, secure transport layer.
178 The network layer is provided by the Twisted library [12].
179 Computationally intensive operations are performed in native compiled
180 code, such as the "zfec" library for fast erasure coding (also
181 available separately: [13]).
186 Tahoe is sponsored by Allmydata, Inc. [14], a provider of consumer
187 backup services. Allmydata, Inc. contributes hardware, software,
188 ideas, bug reports, suggestions, demands, and money (employing several
189 allmydata.org Tahoe hackers and allowing them to spend part of their
190 work time on the next-generation, free-software project). We are
194 Zooko O'Whielacronx and Brian Warner
195 on behalf of the allmydata.org Tahoe team
197 Boulder, Colorado and San Francisco, California
200 [1] http://allmydata.org/trac/tahoe/browser/relnotes.txt?rev=1154
201 [2] http://allmydata.org/trac/tahoe/timeline?from=2007-09-24&daysback=30&changeset=on&milestone=on&ticket=on&ticket_details=on&wiki=on&update=Update
202 [3] http://allmydata.org/trac/tahoe/wiki/UseCases
203 [4] http://allmydata.org/trac/tahoe/browser/README?rev=1333
204 [5] http://allmydata.org/trac/tahoe/browser/docs/webapi.txt?rev=1151
205 [6] http://allmydata.org/trac/tahoe/browser/roadmap.txt
206 [7] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
207 [8] http://allmydata.org/trac/tahoe/browser/CREDITS?rev=1270
208 [9] http://allmydata.org/trac/tahoe/wiki/Dev
209 [10] http://twistedmatrix.com/trac/wiki/FoolsCap
210 [11] http://erights.org/
211 [12] http://twistedmatrix.com/
212 [13] http://allmydata.org/source/zfec/zfec/
213 [14] http://allmydata.com