1 This document has six sections:
3 1. the basic API for how to programmatically control your tahoe node
5 3. safety and security issues
6 4. features for controlling your tahoe node from a standard web browser
7 5. debugging and testing features
8 6. XML-RPC (coming soon)
11 1. the basic API for how to programmatically control your tahoe node
13 a. connecting to the tahoe node
15 Writing "8123" into $NODEDIR/webport causes the node to run a webserver on
16 port 8123. Writing "tcp:8123:interface=127.0.0.1" into $NODEDIR/webport does
17 the same but binds to the loopback interface, ensuring that only the programs
18 on the local host can connect. Using
19 "ssl:8123:privateKey=mykey.pem:certKey=cert.pem" would run an SSL server. See
20 twisted.application.strports for more details.
22 This webport can be set when the node is created by passing a --webport
23 option to the 'tahoe create-client' command. By default, the node listens on
24 port 8123, on the loopback (127.0.0.1) interface.
28 The node provides some small number of "virtual drives". In the 0.5 release,
29 this number is two: the first is the global shared vdrive, the second is the
30 private non-shared vdrive. We will call the global one "global", and we will
31 refer to the second one by "$PRIVATE_VDRIVE_URI", to show that to use it you
32 have to insert the specific URI for that private vdrive.
34 For the purpose of this document, let us assume that the vdrives currently
35 contain the following directories and files:
39 global/Documents/notes.txt
42 $PRIVATE_VDRIVE_URI/Pictures/
43 $PRIVATE_VDRIVE_URI/Pictures/tractors.jpg
44 $PRIVATE_VDRIVE_URI/Pictures/family/
45 $PRIVATE_VDRIVE_URI/Pictures/family/bobby.jpg
47 Within the webserver, there is a tree of resources. The top-level "vdrive"
48 resource gives access to files and directories in all of the user's virtual
49 drives. For example, the URL that corresponds to notes.txt would be:
51 http://localhost:8123/vdrive/global/Documents/notes.txt
53 and the URL for tractors.jpg would be:
55 http://localhost:8123/uri/$PRIVATE_VDRIVE_URI/Pictures/tractors.jpg
57 In addition, each directory has a corresponding URL. The Pictures URL is:
59 http://localhost:8123/uri/$PRIVATE_VDRIVE_URI/Pictures
63 From the "URIs" chapter in architecture.txt, recall that each file and
64 directory has a unique "URI". This is a string which provides a secure
65 reference to the file or directory: if you know the URI, you can retrieve
66 (and possibly modify) the object. If you don't know the URI, you cannot
69 A separate top-level namespace ("uri/" instead of "vdrive/") is used to
70 access to files and directories directly by URI, rather than by going through
71 the pathnames in the vdrive.
73 For example, this identifies a file or directory:
75 http://localhost:8123/uri/$URI
77 And this identifies a file or directory named "tractors.jpg" in a
78 subdirectory "Pictures" of the identified directory:
80 http://localhost:8123/uri/$URI/Pictures/tractors.jpg
82 In the following examples, "$URL" is a shorthand for a URL like the ones
83 above, either with "vdrive/" and a vdrive name as the top level and a
84 sequence of slash-separated pathnames following, or with "uri/" as the top
85 level, followed by a URI, optionally followed by a sequence of
86 slash-separated pathnames.
89 Now, what can we do with these URLs? By varying the HTTP method
90 (GET/PUT/POST/DELETE) and by appending a type-indicating query argument, we
91 control what we want to do with the data and how it should be presented.
93 d. examining files or directories
99 This returns machine-parseable information about the indicated file or
100 directory in the HTTP response body. The JSON always contains a list, and
101 the first element of the list is always a flag that indicates whether the
102 referenced object is a file or a directory.
104 If it is a file, then the information includes file size and URI, like
107 [ 'filenode', { 'ro_uri': file_uri,
110 If it is a directory, then it includes information about the children of
111 this directory, as a mapping from child name to a set of metadata about the
112 child (the same data that would appear in a corresponding GET?t=json of the
113 child itself). Like this:
115 [ 'dirnode', { 'rw_uri': read_write_uri,
116 'ro_uri': read_only_uri,
117 'children': children } ]
119 In the above example, 'children' is a dictionary in which the keys are
120 child names and the values depend upon whether the child is a file or a
123 'foo.txt': [ 'filenode', { 'ro_uri': uri, 'size': bytes } ]
124 'subdir': [ 'dirnode', { 'rw_uri': rwuri, 'ro_uri': rouri } ]
126 note that the value is the same as the JSON representation of the child
127 object (except that directories do not recurse -- the "children" entry of
128 the child is omitted).
130 Then the rw_uri field will be present in the information about a directory
131 if and only if you have read-write access to that directory,
133 e. downloading a file
137 out: file contents or dir metadata
139 save=<boolean> - If true add header "Content-Disposition: attachment"
141 If the indicated object is a file, then this simply retrieves the contents
142 of the file. The file's contents are provided in the body of the HTTP
145 If the indicated object a directory, then this returns an HTML page,
146 intended to be displayed to a human by a web browser, which contains HREF
147 links to all files and directories reachable from this directory. These
148 HREF links do not have a t= argument, meaning that a human who follows them
149 will get pages also meant for a human. It also contains forms to upload new
150 files, and to delete files and directories. These forms use POST methods to
153 You can add the "save=true" argument, which adds a 'Content-Disposition:
154 attachment' header to prompt most web browsers to save the file to disk
155 rather than attempting to display it.
157 A filename (from which a MIME type can be derived, for use in the
158 Content-Type header) can be specified using a 'filename=' query argument.
159 This is especially useful if the $URL does not end with the name of the
160 file (e.g. if it ends with the URI of the file instead). This filename is
161 also the one used if the 'save=true' argument is set. For example:
163 GET http://localhost:8123/uri/$TRACTORS_URI?filename=tractors.jpg
167 PUT http://localhost:8123/uri
172 Upload a file, using the data from the HTTP request body, and returning
173 the resulting URI as the HTTP response body. This does not make the file
174 visible from the virtual drive -- to do that, see section 1.h. below, or
175 the convenience method in section 2.a..
177 POST http://localhost:8123/uri?t=upload
179 This action also uploads a file without attaching it to a virtual drive
180 directory, but can be used from an HTML form. The response is an HTML page
181 that describes the results of the upload, including the resulting URI (but
182 also including information about which peers were used, etc). If a
183 when_done=URL argument is provided, the reponse is a redirect to the given
184 URL instead of the upload-results page.
186 POST http://localhost:8123/uri?t=upload&mutable=true
188 This action also uploads a file without attaching it to a virtual drive
189 directory, but creates a mutable file (SSK) instead of an immutable one.
190 The response contains the new URI that was created.
192 PUT http://localhost:8123/uri?mutable=true
194 This second form also accepts data from the HTTP request body, but creates
195 a mutable file (SSK) instead of an immutable one (CHK). The response
196 contains the new URI that was created.
199 g. creating a new directory
201 PUT http://localhost:8123/uri?t=mkdir
204 out: directory write cap
206 Create a new empty directory and return its URI as the HTTP response body.
207 This does not make the newly created directory visible from the virtual
208 drive, but you can use section 1.h. to attach it, or the convenience method
211 h. attaching a file or directory as the child of an extant directory
216 out: the same child cap
218 replace=<boolean> - If true, overwrite existing contents.
220 This attaches a child (either a file or a directory) to the given directory
221 $URL is required to indicate a directory as the second-to-last element and
222 the desired filename as the last element, for example:
224 PUT http://localhost:8123/uri/$URI_OF_SOME_DIR/Pictures/tractors.jpg
225 PUT http://localhost:8123/uri/$URI_OF_SOME_DIR/tractors.jpg
226 PUT http://localhost:8123/uri/$PRIVATE_VDRIVE_URI/Pictures/tractors.jpg
228 (Note that a URI_OF_SOME_DIR and a PRIVATE_VDRIVE_URI are each just
229 separate URIs, and there is nothing special about the latter except that it
230 is useful to put all of the user's top-level files and directories into one
231 place, so we choose to use that particular directory to be the user's main
234 The URI of the child is provided in the body of the HTTP request,
235 and this same URI is returned in the response body.
237 There is an optional "?replace=" param whose value can be "true", "t", "1",
238 "false", "f", or "0" (case-insensitive), and which defaults to "true". If
239 the indicated directory already contains the given child name, then if
240 replace is true then the value of that name is changed to be the new URI.
241 If replace is false then an HTTP 409 "Conflict" error is returned.
243 This can be used to attach a shared directory (a directory that other
244 people can read or write) to the vdrive. Intermediate directories, if any,
245 are created on-demand.
247 i. removing a name from a directory
251 This removes the given name from the given directory. $URL is required to
252 indicate a directory as the second-to-last element and the name to remove
253 from that directory as the last element, just as in section 1.g..
255 Note that this does not actually delete the resource that the name points
256 to from the tahoe grid -- it only removes this name in this directory. If
257 there are other names in this directory or in other directories that point
258 to the resource, then it will remain accessible through those paths. Even
259 if all names pointing to this resource are removed from their parent
260 directories, then if someone is in possession of the URI of this resource
261 they can continue to access the resource through the URI. Only if a person
262 is not in possession of the URI, and they do not have access to any
263 directories which contain names pointing to this resource, are they
264 prevented from accessing the resource. (This behavior is very similar to
265 the way hardlinks and anonymous files work in traditional unix
268 2. convenience methods
270 a. uploading a file and attaching it to the vdrive
278 200 - File updated. [FIXME: Is this true yet?]
279 201 - File created. [FIXME: Is this true yet?]
281 Upload a file and link it into the the vdrive at the location specified by
282 $URI. The last item in the $URI must be a filename, and the second-to-last
283 item must identify a directory.
285 It will create intermediate directories as necessary. The file's contents
286 are taken from the body of the HTTP request. For convenience, the HTTP
287 response contains the URI that results from uploading the file, although
288 the client is not obligated to do anything with the URI. According to the
289 HTTP/1.1 specification (rfc2616), this should return a 200 (OK) code when
290 modifying an existing file, and a 201 (Created) code when creating a new
291 file. (TODO: as of 0.5, the web server only returns 200, never 201).
293 To use this, run 'curl -T localfile http://localhost:8123/vdrive/global/newfile'
295 3. safety and security issues -- names vs. URIs
297 The vdrive provides a mutable filesystem, but the ways that the filesystem
298 can change are limited. The only thing that can change is that the mapping
299 from child names to child objects that each directory contains can be changed
300 by adding a new child name pointing to an object, removing an existing child
301 name, or changing an existing child name to point to a different object.
303 Obviously if you query tahoe for information about the filesystem and then
304 act upon the filesystem (such as by getting a listing of the contents of a
305 directory and then adding a file to the directory), then the filesystem might
306 have been changed after you queried it and before you acted upon it.
307 However, if you use the URI instead of the pathname of an object when you act
308 upon the object, then the only change that can happen is when the object is a
309 directory then the set of child names it has might be different. If, on the
310 other hand, you act upon the object using its pathname, then a different
311 object might be in that place, which can result in more kinds of surprises.
313 For example, suppose you are writing code which recursively downloads the
314 contents of a directory. The first thing your code does is fetch the listing
315 of the contents of the directory. For each child that it fetched, if that
316 child is a file then it downloads the file, and if that child is a directory
317 then it recurses into that directory. Now, if the download and the recurse
318 actions are performed using the child's name, then the results might be
319 wrong, because for example a child name that pointed to a sub-directory when
320 you listed the directory might have been changed to point to a file (in which
321 case your attempt to recurse into it would result in an error and the file
322 would be skipped), or a child name that pointed to a file when you listed the
323 directory might now point to a sub-directory (in which case your attempt to
324 download the child would result in a file containing HTML text describing the
327 If your recursive algorithm uses the uri of the child instead of the name of
328 the child, then those kinds of mistakes just can't happen. Note that both the
329 child's name and the child's URI are included in the results of listing the
330 parent directory, so it isn't any harder to use the URI for this purpose.
332 In general, use names if you want "whatever object (whether file or
333 directory) is found by following this name (or sequence of names) when my
334 request reaches the server". Use URIs if you want "this particular object".
336 4. features for controlling your tahoe node from a standard web browser
340 GET http://localhost:8123/uri?uri=$URI
342 This causes a redirect to /uri/$URI, and retains any additional query
343 arguments (like filename= or save=). This is for the convenience of web
344 forms which allow the user to paste in a URI (obtained through some
345 out-of-band channel, like IM or email).
347 Note that this form merely redirects to the specific file or directory
348 indicated by the URI: unlike the GET /uri/$URI form, you cannot traverse to
349 children by appending additional path segments to the URL.
351 b. web page offering rename
353 GET $URL?t=rename-form&name=$CHILDNAME
355 This provides a useful facility to browser-based user interfaces. It
356 returns a page containing a form targetting the "POST $URL t=rename"
357 functionality described below, with the provided $CHILDNAME present in the
358 'from_name' field of that form. I.e. this presents a form offering to
359 rename $CHILDNAME, requesting the new name, and submitting POST rename.
365 name=childname (optional)
368 This instructs the node to upload a file into the given directory. We need
369 this because forms are the only way for a web browser to upload a file
370 (browsers do not know how to do PUT or DELETE). The file's contents and the
371 new child name will be included in the form's arguments. This can only be
372 used to upload a single file at a time. To avoid confusion, name= is not
373 allowed to contain a slash (a 400 Bad Request error will result). The
374 response is the file read-cap (URI) of the resulting file.
379 name=childname (optional)
383 This instructs the node to upload a file into the given directory, using a
384 mutable file (SSK) rather than the usual immutable file (CHK). As a result,
385 further operations to the same $URL will not cause the identity of the file
386 to change. The response is the file write-cap (URI) of the resulting
394 This is used to replace the existing (mutable) file's contents with new
395 ones. It may only be used when $URL refers to a mutable file, as created by
396 POST $URL?t=upload&mutable=true, or PUT /uri?t=mutable . The name
397 associated with the uploaded file is ignored. TODO: rethink this, it's kind
405 This instructs the node to create a new empty directory. The name of the
406 new child directory will be included in the form's arguments.
414 This instructs the node to attach a child that is referenced by URI (just
415 like the PUT $URL?t=uri method). The name and URI of the new child
416 will be included in the form's arguments.
423 This instructs the node to delete a file from the given directory. The name
424 of the child to be deleted will be included in the form's arguments.
429 from_name=oldchildname
432 This instructs the node to rename a child within the given directory. The
433 child specified by 'from_name' is removed, and reattached as a child named
434 for 'to_name'. This is unconditional and will replace any child already
435 present under 'to_name', akin to 'mv -f' in unix parlance.
441 This triggers the FileChecker to determine the current "health" of the
442 given file, by counting how many shares are available. The results will be
443 displayed on the directory page containing this file.
446 5. debugging and testing features
448 GET $URL?t=download&localfile=$LOCALPATH
449 GET $URL?t=download&localdir=$LOCALPATH
451 The localfile= form instructs the node to download the given file and write
452 it into the local filesystem at $LOCALPATH. The localdir= form instructs
453 the node to recursively download everything from the given directory and
454 below into the local filesystem. To avoid surprises, the localfile= form
455 will signal an error if $URL actually refers to a directory, likewise if
456 localdir= is used with a $URL that refers to a file.
458 This request will only be accepted from an HTTP client connection
459 originating at 127.0.0.1 . This request is most useful when the client node
460 and the HTTP client are operated by the same user. $LOCALPATH should be an
463 This form is only implemented for testing purposes, because of a trivially
464 easy attack: any web server that the local browser visits could serve an
465 IMG tag that causes the local node to modify the local filesystem.
466 Therefore this form is only enabled if you create a file named
467 'webport_allow_localfile' in the node's base directory.
469 PUT $NEWURL?t=upload&localfile=$LOCALPATH
470 PUT $NEWURL?t=upload&localdir=$LOCALPATH
472 This uploads a file or directory from the node's local filesystem to the
473 vdrive. As with "GET $URL?t=download&localfile=$LOCALPATH", this request
474 will only be accepted from an HTTP connection originating from 127.0.0.1 .
476 The localfile= form expects that $LOCALPATH will point to a file on the
477 node's local filesystem, and causes the node to upload that one file into
478 the vdrive at the given location. Any parent directories will be created in
479 the vdrive as necessary.
481 The localdir= form expects that $LOCALPATH will point to a directory on the
482 node's local filesystem, and it causes the node to perform a recursive
483 upload of the directory into the vdrive at the given location, creating
484 parent directories as necessary. When the operation is complete, the
485 directory referenced by $NEWURL will contain all of the files and
486 directories that were present in $LOCALPATH, so this is equivalent to the
489 mkdir -p $NEWURL; cp -r $LOCALPATH/* $NEWURL/
491 Note that the "curl" utility can be used to provoke this sort of recursive
492 upload, since the -T option will make it use an HTTP 'PUT':
494 curl -T /dev/null 'http://localhost:8123/vdrive/global/newdir?t=upload&localdir=/home/user/directory-to-upload'
496 This form is only implemented for testing purposes, because any attacker's
497 web server that a local browser visits could serve an IMG tag that causes
498 the local node to modify the local filesystem. Therefore this form is only
499 enabled if you create a file named 'webport_allow_localfile' in the node's
504 Return an HTML-formatted manifest of the given directory, for debugging.
506 6. XMLRPC (coming soon)
508 http://localhost:8123/xmlrpc
510 This resource provides an XMLRPC server on which all of the previous
511 operations can be expressed as function calls taking a "pathname" argument.
512 This is provided for applications that want to think of everything in terms
515 listdir(vdrivename, path) -> dict of (childname -> (stuff))
516 put(vdrivename, path, contents) -> URI
517 get(vdrivename, path) -> contents
518 mkdir(vdrivename, path) -> URI
519 put_localfile(vdrivename, path, localfilename) -> URI
520 get_localfile(vdrivename, path, localfilename)
521 put_localdir(vdrivename, path, localdirname) # recursive
522 get_localdir(vdrivename, path, localdirname) # recursive
523 put_uri(vdrivename, path, URI)