+.. -*- coding: utf-8-with-signature -*-
+
==========================
The Tahoe REST-ful Web API
==========================
2. `Basic Concepts: GET, PUT, DELETE, POST`_
3. `URLs`_
- 1. `Child Lookup`_
+ 1. `Child Lookup`_
4. `Slow Operations, Progress, and Cancelling`_
5. `Programmatic Operations`_
1. `Reading a file`_
2. `Writing/Uploading a File`_
3. `Creating a New Directory`_
- 4. `Getting Information About A File Or Directory (as JSON)`_
+ 4. `Getting Information About a File Or Directory (as JSON)`_
5. `Attaching an Existing File or Directory by its read- or write-cap`_
6. `Adding Multiple Files or Directories to a Parent Directory at Once`_
7. `Unlinking a File or Directory`_
6. `Browser Operations: Human-Oriented Interfaces`_
- 1. `Viewing A Directory (as HTML)`_
+ 1. `Viewing a Directory (as HTML)`_
2. `Viewing/Downloading a File`_
- 3. `Getting Information About A File Or Directory (as HTML)`_
+ 3. `Getting Information About a File Or Directory (as HTML)`_
4. `Creating a Directory`_
5. `Uploading a File`_
- 6. `Attaching An Existing File Or Directory (by URI)`_
- 7. `Unlinking A Child`_
- 8. `Renaming A Child`_
- 9. `Other Utilities`_
- 10. `Debugging and Testing Features`_
+ 6. `Attaching an Existing File Or Directory (by URI)`_
+ 7. `Unlinking a Child`_
+ 8. `Renaming a Child`_
+ 9. `Relinking ("Moving") a Child`_
+ 10. `Other Utilities`_
+ 11. `Debugging and Testing Features`_
7. `Other Useful Pages`_
8. `Static Files in /public_html`_
This string is actually a Twisted "strports" specification, meaning you can
get more control over the interface to which the server binds by supplying
additional arguments. For more details, see the documentation on
-`twisted.application.strports
-<http://twistedmatrix.com/documents/current/api/twisted.application.strports.html>`_.
+`twisted.application.strports`_.
Writing "tcp:3456:interface=127.0.0.1" into the web.port line does the same
but binds to the loopback interface, ensuring that only the programs on the
option to the 'tahoe create-node' command. By default, the node listens on
port 3456, on the loopback (127.0.0.1) interface.
+.. _twisted.application.strports: https://twistedmatrix.com/documents/current/api/twisted.application.strports.html
+
Basic Concepts: GET, PUT, DELETE, POST
======================================
-As described in `docs/architecture.rst <../architecture.rst>`_, each file
-and directory in a Tahoe virtual filesystem is referenced by an identifier
-that combines the designation of the object with the authority to do something
-with it (such as read or modify the contents). This identifier is called a
-"read-cap" or "write-cap", depending upon whether it enables read-only or
-read-write access. These "caps" are also referred to as URIs (which may be
-confusing because they are not currently `RFC3986
-<http://tools.ietf.org/html/rfc3986>`_-compliant URIs).
+As described in `docs/architecture.rst`_, each file and directory in a Tahoe
+virtual filesystem is referenced by an identifier that combines the
+designation of the object with the authority to do something with it (such as
+read or modify the contents). This identifier is called a "read-cap" or
+"write-cap", depending upon whether it enables read-only or read-write
+access. These "caps" are also referred to as URIs (which may be confusing
+because they are not currently RFC3986_-compliant URIs).
The Tahoe web-based API is "REST-ful", meaning it implements the concepts of
"REpresentational State Transfer": the original scheme by which the World
``text/*``, or text/html (or if there is no Accept header), HTML tracebacks will
be generated.
+.. _RFC3986: https://tools.ietf.org/html/rfc3986
+.. _docs/architecture.rst: ../architecture.rst
+
URLs
====
http://127.0.0.1:3456/uri/ + $CAP
-So, to access the directory named above (which happens to be the
-publically-writeable sample directory on the Tahoe test grid, described at
-http://allmydata.org/trac/tahoe/wiki/TestGrid), the URL would be::
+So, to access the directory named above, the URL would be::
http://127.0.0.1:3456/uri/URI%3ADIR2%3Adjrdkfawoqihigoett4g6auz6a%3Ajx5mplfpwexnoqff7y5e4zjus4lidm76dcuarpct7cckorh2dpgq/
/uri/$DIRCAP/
/uri/$FILECAP
+/cap can be used as a synonym for /uri. If interoperability with older web-API
+servers is required, /uri should be used.
Child Lookup
------------
operations that are intended for web browsers.
-Reading A File
+Reading a File
--------------
``GET /uri/$FILECAP``
``GET /uri/$DIRCAP/[SUBDIRS../]FILENAME``
This will retrieve the contents of the given file. The HTTP response body
- will contain the sequence of bytes that make up the file.
+ will contain the sequence of bytes that make up the file. The "Range:"
+ header can be used to restrict which portions of the file are returned (see
+ RFC 2616 section 14.35.1 "Byte Ranges"), however Tahoe only supports a
+ single "bytes" range and never provides a `multipart/byteranges` response.
To view files in a web browser, you may want more control over the
Content-Type and Content-Disposition headers. Please see the next section
purpose.
-Writing/Uploading A File
+Writing/Uploading a File
------------------------
``PUT /uri/$FILECAP``
file is replaced with the data from the HTTP request body. For an
immutable file, the "offset" parameter is not valid.
- When creating a new file, if "mutable=true" is in the query arguments, the
- operation will create a mutable file instead of an immutable one.
+ When creating a new file, you can control the type of file created by
+ specifying a format= argument in the query string. format=MDMF creates an
+ MDMF mutable file. format=SDMF creates an SDMF mutable file. format=CHK
+ creates an immutable file. The value of the format argument is
+ case-insensitive. If no format is specified, the newly-created file will be
+ immutable (but see below).
+
+ For compatibility with previous versions of Tahoe-LAFS, the web-API will
+ also accept a mutable=true argument in the query string. If mutable=true is
+ given, then the new file will be mutable, and its format will be the default
+ mutable file format, as configured by the [client]mutable.format option of
+ tahoe.cfg on the Tahoe-LAFS node hosting the webapi server. Use of
+ mutable=true is discouraged; new code should use format= instead of
+ mutable=true (unless it needs to be compatible with web-API servers older
+ than v1.9.0). If neither format= nor mutable=true are given, the
+ newly-created file will be immutable.
This returns the file-cap of the resulting file. If a new file was created
by this method, the HTTP response code (as dictated by rfc2616) will be set
attach the file into the filesystem. No directories will be modified by
this operation. The file-cap is returned as the body of the HTTP response.
- If "mutable=true" is in the query arguments, the operation will create a
- mutable file, and return its write-cap in the HTTP respose. The default is
- to create an immutable file, returning the read-cap as a response. If
- you create a mutable file, you can also use the "mutable-type" query
- parameter. If "mutable-type=sdmf", then the mutable file will be created
- in the old SDMF mutable file format. This is desirable for files that
- need to be read by old clients. If "mutable-type=mdmf", then the file
- will be created in the new MDMF mutable file format. MDMF mutable files
- can be downloaded more efficiently, and modified in-place efficiently,
- but are not compatible with older versions of Tahoe-LAFS. If no
- "mutable-type" argument is given, the file is created in whatever
- format was configured in tahoe.cfg.
-
-
-Creating A New Directory
+ This method accepts format= and mutable=true as query string arguments, and
+ interprets those arguments in the same way as the linked forms of PUT
+ described immediately above.
+
+Creating a New Directory
------------------------
``POST /uri?t=mkdir``
filesystem. The "PUT" operation is provided for backwards compatibility:
new code should use POST.
+ This supports a format= argument in the query string. The format=
+ argument, if specified, controls the format of the directory. format=MDMF
+ indicates that the directory should be stored as an MDMF file; format=SDMF
+ indicates that the directory should be stored as an SDMF file. The value of
+ the format= argument is case-insensitive. If no format= argument is
+ given, the directory's format is determined by the default mutable file
+ format, as configured on the Tahoe-LAFS node responding to the request.
+
``POST /uri?t=mkdir-with-children``
Create a new directory, populated with a set of child nodes, and return its
write-cap as the HTTP response body. The new directory is not attached to
any other directory: the returned write-cap is the only reference to it.
+ The format of the directory can be controlled with the format= argument in
+ the query string, as described above.
+
Initial children are provided as the body of the POST form (this is more
efficient than doing separate mkdir and set_children operations). If the
body is empty, the new directory will be empty. If not empty, the body will
{
"Fran\u00e7ais": [ "filenode", {
"ro_uri": "URI:CHK:...",
- "size": bytes,
"metadata": {
"ctime": 1202777696.7564139,
"mtime": 1202777696.7564139,
The metadata may have a "no-write" field. If this is set to true in the
metadata of a link, it will not be possible to open that link for writing
- via the SFTP frontend; see `<FTP-and-SFTP.rst>`_ for details.
- Also, if the "no-write" field is set to true in the metadata of a link to
- a mutable child, it will cause the link to be diminished to read-only.
+ via the SFTP frontend; see FTP-and-SFTP.rst_ for details. Also, if the
+ "no-write" field is set to true in the metadata of a link to a mutable
+ child, it will cause the link to be diminished to read-only.
Note that the web-API-using client application must not provide the
"Content-Type: multipart/form-data" header that usually accompanies HTML
If the final directory is created, it will be empty.
+ This accepts a format= argument in the query string, which controls the
+ format of the named target directory, if it does not already exist. format=
+ is interpreted in the same way as in the POST /uri?t=mkdir form. Note that
+ format= only controls the format of the named target directory;
+ intermediate directories, if created, are created based on the default
+ mutable type, as configured on the Tahoe-LAFS server responding to the
+ request.
+
This operation will return an error if a blocking file is present at any of
the parent names, preventing the server from creating the necessary parent
directory; or if it would require changing an immutable directory.
intermediate mutable directories as necessary. If the final directory is
created, it will be populated with initial children from the POST request
body, as described above.
+
+ This accepts a format= argument in the query string, which controls the
+ format of the target directory, if the target directory is created as part
+ of the operation. format= is interpreted in the same way as in the POST/
+ uri?t=mkdir-with-children operation. Note that format= only controls the
+ format of the named target directory; intermediate directories, if created,
+ are created using the default mutable type setting, as configured on the
+ Tahoe-LAFS server responding to the request.
This operation will return an error if a blocking file is present at any of
the parent names, preventing the server from creating the necessary parent
Create a new empty mutable directory and attach it to the given existing
directory. This will create additional intermediate directories as necessary.
+ This accepts a format= argument in the query string, which controls the
+ format of the named target directory, if it does not already exist. format=
+ is interpreted in the same way as in the POST /uri?t=mkdir form. Note that
+ format= only controls the format of the named target directory;
+ intermediate directories, if created, are created based on the default
+ mutable type, as configured on the Tahoe-LAFS server responding to the
+ request.
+
This operation will return an error if a blocking file is present at any of
the parent names, preventing the server from creating the necessary parent
directory, or if it would require changing any immutable directory.
Like /uri/$DIRCAP/[SUBDIRS../]?t=mkdir&name=NAME, but the new directory will
be populated with initial children via the POST request body. This command
will create additional intermediate mutable directories as necessary.
-
+
+ This accepts a format= argument in the query string, which controls the
+ format of the target directory, if the target directory is created as part
+ of the operation. format= is interpreted in the same way as in the POST/
+ uri?t=mkdir-with-children operation. Note that format= only controls the
+ format of the named target directory; intermediate directories, if created,
+ are created using the default mutable type setting, as configured on the
+ Tahoe-LAFS server responding to the request.
+
This operation will return an error if a blocking file is present at any of
the parent names, preventing the server from creating the necessary parent
directory; or if it would require changing an immutable directory; or if
This operation will return an error if the parent directory is immutable,
or already has a child named NAME.
+.. _FTP-and-SFTP.rst: FTP-and-SFTP.rst
+
-Getting Information About A File Or Directory (as JSON)
+Getting Information About a File Or Directory (as JSON)
-------------------------------------------------------
``GET /uri/$FILECAP?t=json``
GET /uri/$FILECAP?t=json :
[ "filenode", {
- "ro_uri": file_uri,
- "verify_uri": verify_uri,
- "size": bytes,
- "mutable": false
- } ]
+ "ro_uri": file_uri,
+ "verify_uri": verify_uri,
+ "size": bytes,
+ "mutable": false,
+ "format": "CHK"
+ } ]
If it is a capability to a directory followed by a path from that directory
to a file, then the information also includes metadata from the link to the
GET /uri/$DIRCAP/[SUBDIRS../]FILENAME?t=json
[ "filenode", {
- "ro_uri": file_uri,
- "verify_uri": verify_uri,
- "size": bytes,
- "mutable": false,
- "metadata": {
- "ctime": 1202777696.7564139,
- "mtime": 1202777696.7564139,
- "tahoe": {
- "linkcrtime": 1202777696.7564139,
- "linkmotime": 1202777696.7564139
- } } } ]
+ "ro_uri": file_uri,
+ "verify_uri": verify_uri,
+ "size": bytes,
+ "mutable": false,
+ "format": "CHK",
+ "metadata": {
+ "ctime": 1202777696.7564139,
+ "mtime": 1202777696.7564139,
+ "tahoe": {
+ "linkcrtime": 1202777696.7564139,
+ "linkmotime": 1202777696.7564139
+ } } } ]
If it is a directory, then it includes information about the children of
this directory, as a mapping from child name to a set of data about the
GET /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=json :
[ "dirnode", {
- "rw_uri": read_write_uri,
- "ro_uri": read_only_uri,
- "verify_uri": verify_uri,
- "mutable": true,
- "children": {
- "foo.txt": [ "filenode", {
- "ro_uri": uri,
- "size": bytes,
- "metadata": {
- "ctime": 1202777696.7564139,
- "mtime": 1202777696.7564139,
- "tahoe": {
- "linkcrtime": 1202777696.7564139,
- "linkmotime": 1202777696.7564139
- } } } ],
- "subdir": [ "dirnode", {
- "rw_uri": rwuri,
- "ro_uri": rouri,
- "metadata": {
- "ctime": 1202778102.7589991,
- "mtime": 1202778111.2160511,
- "tahoe": {
- "linkcrtime": 1202777696.7564139,
- "linkmotime": 1202777696.7564139
- } } } ]
- } } ]
+ "rw_uri": read_write_uri,
+ "ro_uri": read_only_uri,
+ "verify_uri": verify_uri,
+ "mutable": true,
+ "format": "SDMF",
+ "children": {
+ "foo.txt": [ "filenode",
+ {
+ "ro_uri": uri,
+ "size": bytes,
+ "metadata": {
+ "ctime": 1202777696.7564139,
+ "mtime": 1202777696.7564139,
+ "tahoe": {
+ "linkcrtime": 1202777696.7564139,
+ "linkmotime": 1202777696.7564139
+ } } } ],
+ "subdir": [ "dirnode",
+ {
+ "rw_uri": rwuri,
+ "ro_uri": rouri,
+ "metadata": {
+ "ctime": 1202778102.7589991,
+ "mtime": 1202778111.2160511,
+ "tahoe": {
+ "linkcrtime": 1202777696.7564139,
+ "linkmotime": 1202777696.7564139
+ } } } ]
+ } } ]
In the above example, note how 'children' is a dictionary in which the keys
are child names and the values depend upon whether the child is a file or a
GET /uri/$UNKNOWNCAP?t=json :
[ "unknown", {
- "ro_uri": unknown_read_uri
- } ]
+ "ro_uri": unknown_read_uri
+ } ]
GET /uri/$DIRCAP/[SUBDIRS../]UNKNOWNCHILDNAME?t=json :
[ "unknown", {
- "rw_uri": unknown_write_uri,
- "ro_uri": unknown_read_uri,
- "mutable": true,
- "metadata": {
- "ctime": 1202777696.7564139,
- "mtime": 1202777696.7564139,
- "tahoe": {
- "linkcrtime": 1202777696.7564139,
- "linkmotime": 1202777696.7564139
- } } } ]
+ "rw_uri": unknown_write_uri,
+ "ro_uri": unknown_read_uri,
+ "mutable": true,
+ "metadata": {
+ "ctime": 1202777696.7564139,
+ "mtime": 1202777696.7564139,
+ "tahoe": {
+ "linkcrtime": 1202777696.7564139,
+ "linkmotime": 1202777696.7564139
+ } } } ]
As in the case of file nodes, the metadata will only be present when the
capability is to a directory followed by a path. The "mutable" field is also
existing "tahoe" metadata is preserved. The metadata["tahoe"] value is
reserved for metadata generated by the tahoe node itself. The only two keys
currently placed here are "linkcrtime" and "linkmotime". For details, see
- the section above entitled "Get Information About A File Or Directory (as
+ the section above entitled "Getting Information About a File Or Directory (as
JSON)", in the "About the metadata" subsection.
Note that this command was introduced with the name "set_children", which
descriptions below display the most significant arguments as URL query args.
-Viewing A Directory (as HTML)
+Viewing a Directory (as HTML)
-----------------------------
``GET /uri/$DIRCAP/[SUBDIRS../]``
this form can *only* be used with file caps; it is an error to use a
directory cap after the /named/ prefix.
+ URLs may also use /file/$FILECAP/FILENAME as a synonym for
+ /named/$FILECAP/FILENAME.
-Getting Information About A File Or Directory (as HTML)
+Getting Information About a File Or Directory (as HTML)
-------------------------------------------------------
``GET /uri/$FILECAP?t=info``
* access caps (URIs): verify-cap, read-cap, write-cap (for mutable objects)
* check/verify/repair form
* deep-check/deep-size/deep-stats/manifest (for directories)
- * replace-conents form (for mutable files)
+ * replace-contents form (for mutable files)
Creating a Directory
/uri/$DIRCAP page. There is a "create directory" button on the Welcome page
to invoke this action.
+ This accepts a format= argument in the query string. Refer to the
+ documentation of the PUT /uri?t=mkdir operation in `Creating A
+ New Directory`_ for information on the behavior of the format= argument.
+
If "redirect_to_result=true" is not provided (or is given a value of
"false"), then the HTTP response body will simply be the write-cap of the
new directory.
This creates a new empty directory as a child of the designated SUBDIR. This
will create additional intermediate directories as necessary.
+ This accepts a format= argument in the query string. Refer to the
+ documentation of POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir&name=CHILDNAME in
+ `Creating a New Directory`_ for information on the behavior of the format=
+ argument.
+
If a "when_done=URL" argument is provided, the HTTP response will cause the
web browser to redirect to the given URL. This provides a convenient way to
return the browser to the directory that was just modified. Without a
about which storage servers were used for the upload, how long each
operation took, etc.
- If a "mutable=true" argument is provided, the operation will create a
- mutable file, and the response body will contain the write-cap instead of
- the upload results page. The default is to create an immutable file,
- returning the upload results page as a response. If you create a
- mutable file, you may choose to specify the format of that mutable file
- with the "mutable-type" parameter. If "mutable-type=mdmf", then the
- file will be created as an MDMF mutable file. If "mutable-type=sdmf",
- then the file will be created as an SDMF mutable file. If no value is
- specified, the file will be created in whatever format is specified in
- tahoe.cfg.
-
+ This accepts format= and mutable=true query string arguments. Refer to
+ `Writing/Uploading a File`_ for information on the behavior of format= and
+ mutable=true.
``POST /uri/$DIRCAP/[SUBDIRS../]?t=upload``
/uri/$DIRCAP/[SUBDIRS../]", it is likely that the parent directory will
already exist.
- If a "mutable=true" argument is provided, any new file that is created will
- be a mutable file instead of an immutable one. <input type="checkbox"
- name="mutable" /> will give the user a way to set this option.
+ This accepts format= and mutable=true query string arguments. Refer to
+ `Writing/Uploading a File`_ for information on the behavior of format= and
+ mutable=true.
If a "when_done=URL" argument is provided, the HTTP response will cause the
web browser to redirect to the given URL. This provides a convenient way to
This accepts the same replace= argument as POST t=upload.
-Unlinking A Child
+Unlinking a Child
-----------------
``POST /uri/$DIRCAP/[SUBDIRS../]?t=delete&name=CHILDNAME``
-``POST /uri/$DIRCAP/[SUBDIRS../]?t=unlink&name=CHILDNAME``
+``POST /uri/$DIRCAP/[SUBDIRS../]?t=unlink&name=CHILDNAME`` (Tahoe >= v1.9)
This instructs the node to remove a child object (file or subdirectory) from
the given directory, which must be mutable. Note that the entire subtree is
be used.
-Renaming A Child
+Renaming a Child
----------------
``POST /uri/$DIRCAP/[SUBDIRS../]?t=rename&from_name=OLD&to_name=NEW``
same child-cap under the new name, except that it preserves metadata. This
operation cannot move the child to a different directory.
- This operation will replace any existing child of the new name, making it
- behave like the UNIX "``mv -f``" command.
+ The default behavior is to overwrite any existing link at the destination
+ (replace=true). To prevent this (and make the operation return an error
+ instead of overwriting), add a "replace=false" argument. With replace=false,
+ this operation will return an HTTP 409 "Conflict" error if the destination
+ is not the same link as the source and there is already a link at the
+ destination, rather than overwriting the existing link. To allow the
+ operation to overwrite a link to a file, but return an HTTP 409 error when
+ trying to overwrite a link to a directory, use "replace=only-files" (this
+ behavior is closer to the traditional UNIX "mv" command). Note that "true",
+ "t", and "1" are all synonyms for "True"; "false", "f", and "0" are synonyms
+ for "False"; and the parameter is case-insensitive.
+
+
+Relinking ("Moving") a Child
+----------------------------
+
+``POST /uri/$DIRCAP/[SUBDIRS../]?t=relink&from_name=OLD&to_dir=$NEWDIRCAP/[NEWSUBDIRS../]&to_name=NEW``
+ ``[&replace=true|false|only-files]`` (Tahoe >= v1.10)
+
+ This instructs the node to move a child of the given source directory, into
+ a different directory and/or to a different name. The command is named
+ ``relink`` because what it does is add a new link to the child from the new
+ location, then remove the old link. Nothing is actually "moved": the child
+ is still reachable through any path from which it was formerly reachable,
+ and the storage space occupied by its ciphertext is not affected.
+
+ The source and destination directories must be writeable. If {{{to_dir}}} is
+ not present, the child link is renamed within the same directory. If
+ {{{to_name}}} is not present then it defaults to {{{from_name}}}. If the
+ destination link (directory and name) is the same as the source link, the
+ operation has no effect.
+
+ Metadata from the source directory entry is preserved. Multiple levels of
+ descent in the source and destination paths are supported.
+
+ This operation will return an HTTP 404 "Not Found" error if
+ ``$DIRCAP/[SUBDIRS../]``, the child being moved, or the destination
+ directory does not exist. It will return an HTTP 400 "Bad Request" error
+ if any entry in the source or destination paths is not a directory.
+
+ The default behavior is to overwrite any existing link at the destination
+ (replace=true). To prevent this (and make the operation return an error
+ instead of overwriting), add a "replace=false" argument. With replace=false,
+ this operation will return an HTTP 409 "Conflict" error if the destination
+ is not the same link as the source and there is already a link at the
+ destination, rather than overwriting the existing link. To allow the
+ operation to overwrite a link to a file, but return an HTTP 409 error when
+ trying to overwrite a link to a directory, use "replace=only-files" (this
+ behavior is closer to the traditional UNIX "mv" command). Note that "true",
+ "t", and "1" are all synonyms for "True"; "false", "f", and "0" are synonyms
+ for "False"; and the parameter is case-insensitive.
+
+ When relinking into a different directory, for safety, the child link is
+ not removed from the old directory until it has been successfully added to
+ the new directory. This implies that in case of a crash or failure, the
+ link to the child will not be lost, but it could be linked at both the old
+ and new locations.
+
+ The source link should not be the same as any link (directory and child name)
+ in the ``to_dir`` path. This restriction is not enforced, but it may be
+ enforced in a future version. If it were violated then the result would be
+ to create a cycle in the directory structure that is not necessarily reachable
+ from the root of the destination path (``$NEWDIRCAP``), which could result in
+ data loss, as described in ticket `#943`_.
+
+.. _`#943`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/943
+
Other Utilities
---------------
functionality described above, with the provided $CHILDNAME present in the
'from_name' field of that form. I.e. this presents a form offering to
rename $CHILDNAME, requesting the new name, and submitting POST rename.
+ This same URL format can also be used with "move-form" with the expected
+ results.
``GET /uri/$DIRCAP/[SUBDIRS../]CHILDNAME?t=uri``
dictionary with the following keys::
storage-index: a base32-encoded string with the objects's storage index,
- or an empty string for LIT files
+ or an empty string for LIT files
summary: a string, with a one-line summary of the stats of the file
results: a dictionary that describes the state of the file. For LIT files,
- this dictionary has only the 'healthy' key, which will always be
- True. For distributed files, this dictionary has the following
- keys:
- count-shares-good: the number of good shares that were found
- count-shares-needed: 'k', the number of shares required for recovery
- count-shares-expected: 'N', the number of total shares generated
- count-good-share-hosts: this was intended to be the number of distinct
- storage servers with good shares. It is currently
- (as of Tahoe-LAFS v1.8.0) computed incorrectly;
- see ticket #1115.
- count-wrong-shares: for mutable files, the number of shares for
- versions other than the 'best' one (highest
- sequence number, highest roothash). These are
- either old ...
- count-recoverable-versions: for mutable files, the number of
- recoverable versions of the file. For
- a healthy file, this will equal 1.
- count-unrecoverable-versions: for mutable files, the number of
- unrecoverable versions of the file.
- For a healthy file, this will be 0.
- count-corrupt-shares: the number of shares with integrity failures
- list-corrupt-shares: a list of "share locators", one for each share
- that was found to be corrupt. Each share locator
- is a list of (serverid, storage_index, sharenum).
- needs-rebalancing: (bool) True if there are multiple shares on a single
- storage server, indicating a reduction in reliability
- that could be resolved by moving shares to new
- servers.
- servers-responding: list of base32-encoded storage server identifiers,
- one for each server which responded to the share
- query.
- healthy: (bool) True if the file is completely healthy, False otherwise.
- Healthy files have at least N good shares. Overlapping shares
- do not currently cause a file to be marked unhealthy. If there
- are at least N good shares, then corrupt shares do not cause the
- file to be marked unhealthy, although the corrupt shares will be
- listed in the results (list-corrupt-shares) and should be manually
- removed to wasting time in subsequent downloads (as the
- downloader rediscovers the corruption and uses alternate shares).
- Future compatibility: the meaning of this field may change to
- reflect whether the servers-of-happiness criterion is met
- (see ticket #614).
- sharemap: dict mapping share identifier to list of serverids
- (base32-encoded strings). This indicates which servers are
- holding which shares. For immutable files, the shareid is
- an integer (the share number, from 0 to N-1). For
- immutable files, it is a string of the form
- 'seq%d-%s-sh%d', containing the sequence number, the
- roothash, and the share number.
+ this dictionary has only the 'healthy' key, which will always be
+ True. For distributed files, this dictionary has the following
+ keys:
+ count-happiness: the servers-of-happiness level of the file, as
+ defined in `docs/specifications/servers-of-happiness.rst`_.
+ count-shares-good: the number of good shares that were found
+ count-shares-needed: 'k', the number of shares required for recovery
+ count-shares-expected: 'N', the number of total shares generated
+ count-good-share-hosts: the number of distinct storage servers with
+ good shares. Note that a high value does not
+ necessarily imply good share distribution,
+ because some of these servers may only hold
+ duplicate shares.
+ count-wrong-shares: for mutable files, the number of shares for
+ versions other than the 'best' one (highest
+ sequence number, highest roothash). These are
+ either old, or created by an uncoordinated or
+ not fully successful write.
+ count-recoverable-versions: for mutable files, the number of
+ recoverable versions of the file. For
+ a healthy file, this will equal 1.
+ count-unrecoverable-versions: for mutable files, the number of
+ unrecoverable versions of the file.
+ For a healthy file, this will be 0.
+ count-corrupt-shares: the number of shares with integrity failures
+ list-corrupt-shares: a list of "share locators", one for each share
+ that was found to be corrupt. Each share locator
+ is a list of (serverid, storage_index, sharenum).
+ servers-responding: list of base32-encoded storage server identifiers,
+ one for each server which responded to the share
+ query.
+ healthy: (bool) True if the file is completely healthy, False otherwise.
+ Healthy files have at least N good shares. Overlapping shares
+ do not currently cause a file to be marked unhealthy. If there
+ are at least N good shares, then corrupt shares do not cause the
+ file to be marked unhealthy, although the corrupt shares will be
+ listed in the results (list-corrupt-shares) and should be manually
+ removed to wasting time in subsequent downloads (as the
+ downloader rediscovers the corruption and uses alternate shares).
+ Future compatibility: the meaning of this field may change to
+ reflect whether the servers-of-happiness criterion is met
+ (see ticket #614).
+ sharemap: dict mapping share identifier to list of serverids
+ (base32-encoded strings). This indicates which servers are
+ holding which shares. For immutable files, the shareid is
+ an integer (the share number, from 0 to N-1). For
+ immutable files, it is a string of the form
+ 'seq%d-%s-sh%d', containing the sequence number, the
+ roothash, and the share number.
+
+Before Tahoe-LAFS v1.11, the `results` dictionary also had a `needs-rebalancing`
+field, but that has been removed since it was computed incorrectly.
+
+.. _`docs/specifications/servers-of-happiness.rst`: ../specifications/servers-of-happiness.rst
+
``POST $URL?t=start-deep-check`` (must add &ophandle=XYZ)
machine-readable JSON dictionary with the following keys::
finished: a boolean, True if the operation is complete, else False. Some
- of the remaining keys may not be present until the operation
- is complete.
+ of the remaining keys may not be present until the operation
+ is complete.
root-storage-index: a base32-encoded string with the storage index of the
- starting point of the deep-check operation
+ starting point of the deep-check operation
count-objects-checked: count of how many objects were checked. Note that
- non-distributed objects (i.e. small immutable LIT
- files) are not checked, since for these objects,
- the data is contained entirely in the URI.
+ non-distributed objects (i.e. small immutable LIT
+ files) are not checked, since for these objects,
+ the data is contained entirely in the URI.
count-objects-healthy: how many of those objects were completely healthy
count-objects-unhealthy: how many were damaged in some way
count-corrupt-shares: how many shares were found to have corruption,
- summed over all objects examined
+ summed over all objects examined
list-corrupt-shares: a list of "share identifiers", one for each share
- that was found to be corrupt. Each share identifier
- is a list of (serverid, storage_index, sharenum).
+ that was found to be corrupt. Each share identifier
+ is a list of (serverid, storage_index, sharenum).
list-unhealthy-files: a list of (pathname, check-results) tuples, for
- each file that was not fully healthy. 'pathname' is
- a list of strings (which can be joined by "/"
- characters to turn it into a single string),
- relative to the directory on which deep-check was
- invoked. The 'check-results' field is the same as
- that returned by t=check&output=JSON, described
- above.
+ each file that was not fully healthy. 'pathname' is
+ a list of strings (which can be joined by "/"
+ characters to turn it into a single string),
+ relative to the directory on which deep-check was
+ invoked. The 'check-results' field is the same as
+ that returned by t=check&output=JSON, described
+ above.
stats: a dictionary with the same keys as the t=start-deep-stats command
- (described below)
+ (described below)
``POST $URL?t=stream-deep-check``
will contain the following keys::
storage-index: a base32-encoded string with the objects's storage index,
- or an empty string for LIT files
+ or an empty string for LIT files
repair-attempted: (bool) True if repair was attempted
repair-successful: (bool) True if repair was attempted and the file was
- fully healthy afterwards. False if no repair was
- attempted, or if a repair attempt failed.
+ fully healthy afterwards. False if no repair was
+ attempted, or if a repair attempt failed.
pre-repair-results: a dictionary that describes the state of the file
- before any repair was performed. This contains exactly
- the same keys as the 'results' value of the t=check
- response, described above.
+ before any repair was performed. This contains exactly
+ the same keys as the 'results' value of the t=check
+ response, described above.
post-repair-results: a dictionary that describes the state of the file
- after any repair was performed. If no repair was
- performed, post-repair-results and pre-repair-results
- will be the same. This contains exactly the same keys
- as the 'results' value of the t=check response,
- described above.
+ after any repair was performed. If no repair was
+ performed, post-repair-results and pre-repair-results
+ will be the same. This contains exactly the same keys
+ as the 'results' value of the t=check response,
+ described above.
``POST $URL?t=start-deep-check&repair=true`` (must add &ophandle=XYZ)
finished: (bool) True if the operation has completed, else False
root-storage-index: a base32-encoded string with the storage index of the
- starting point of the deep-check operation
+ starting point of the deep-check operation
count-objects-checked: count of how many objects were checked
count-objects-healthy-pre-repair: how many of those objects were completely
- healthy, before any repair
+ healthy, before any repair
count-objects-unhealthy-pre-repair: how many were damaged in some way
count-objects-healthy-post-repair: how many of those objects were completely
- healthy, after any repair
+ healthy, after any repair
count-objects-unhealthy-post-repair: how many were damaged in some way
count-repairs-attempted: repairs were attempted on this many objects.
count-repairs-successful: how many repairs resulted in healthy objects
count-repairs-unsuccessful: how many repairs resulted did not results in
- completely healthy objects
+ completely healthy objects
count-corrupt-shares-pre-repair: how many shares were found to have
- corruption, summed over all objects
- examined, before any repair
+ corruption, summed over all objects
+ examined, before any repair
count-corrupt-shares-post-repair: how many shares were found to have
- corruption, summed over all objects
- examined, after any repair
+ corruption, summed over all objects
+ examined, after any repair
list-corrupt-shares: a list of "share identifiers", one for each share
- that was found to be corrupt (before any repair).
- Each share identifier is a list of (serverid,
- storage_index, sharenum).
+ that was found to be corrupt (before any repair).
+ Each share identifier is a list of (serverid,
+ storage_index, sharenum).
list-remaining-corrupt-shares: like list-corrupt-shares, but mutable shares
- that were successfully repaired are not
- included. These are shares that need
- manual processing. Since immutable shares
- cannot be modified by clients, all corruption
- in immutable shares will be listed here.
+ that were successfully repaired are not
+ included. These are shares that need
+ manual processing. Since immutable shares
+ cannot be modified by clients, all corruption
+ in immutable shares will be listed here.
list-unhealthy-files: a list of (pathname, check-results) tuples, for
- each file that was not fully healthy. 'pathname' is
- relative to the directory on which deep-check was
- invoked. The 'check-results' field is the same as
- that returned by t=check&repair=true&output=JSON,
- described above.
+ each file that was not fully healthy. 'pathname' is
+ relative to the directory on which deep-check was
+ invoked. The 'check-results' field is the same as
+ that returned by t=check&repair=true&output=JSON,
+ described above.
stats: a dictionary with the same keys as the t=start-deep-stats command
- (described below)
+ (described below)
``POST $URL?t=stream-deep-check&repair=true``
verifycaps: list of (printable) verify cap strings
storage-index: list of (base32) storage index strings
stats: a dictionary with the same keys as the t=start-deep-stats command
- (described below)
+ (described below)
``POST $DIRURL?t=start-deep-size`` (must add &ophandle=XYZ)
size-literal-files: same, for LIT files
size-directories: size of directories (includes size-literal-files)
size-files-histogram: list of (minsize, maxsize, count) buckets,
- with a histogram of filesizes, 5dB/bucket,
- for both literal and immutable files
+ with a histogram of filesizes, 5dB/bucket,
+ for both literal and immutable files
largest-directory: number of children in the largest directory
largest-immutable-file: number of bytes in the largest CHK file
progress-hash (float): 1.0 when the file has been hashed
progress-ciphertext (float): 1.0 when the file has been encrypted.
progress-encode-push (float): 1.0 when the file has been encoded and
- pushed to the storage servers. For helper
- uploads, the ciphertext value climbs to 1.0
- first, then encoding starts. For unassisted
- uploads, ciphertext and encode-push progress
- will climb at the same pace.
+ pushed to the storage servers. For helper
+ uploads, the ciphertext value climbs to 1.0
+ first, then encoding starts. For unassisted
+ uploads, ciphertext and encode-push progress
+ will climb at the same pace.
The "download" op-dict will contain the following keys::
implementation hashes synchronously, so clients will probably never see
progress-hash!=1.0).
-``GET /provisioning/``
-
- This page provides a basic tool to predict the likely storage and bandwidth
- requirements of a large Tahoe grid. It provides forms to input things like
- total number of users, number of files per user, average file size, number
- of servers, expansion ratio, hard drive failure rate, etc. It then provides
- numbers like how many disks per server will be needed, how many read
- operations per second should be expected, and the likely MTBF for files in
- the grid. This information is very preliminary, and the model upon which it
- is based still needs a lot of work.
-
``GET /helper_status/``
If the node is running a helper (i.e. if [helper]enabled is set to True in
and serve it from a configurable directory which defaults to
$BASEDIR/public_html . This is configured by setting the "[node]web.static"
value in $BASEDIR/tahoe.cfg . If this is left at the default value of
-"public_html", then http://localhost:3456/static/subdir/foo.html will be
+"public_html", then http://127.0.0.1:3456/static/subdir/foo.html will be
served with the contents of the file $BASEDIR/public_html/subdir/foo.html .
This can be useful to serve a javascript application which provides a
web requests themselves).
For more details, please see the "Consistency vs Availability" and "The Prime
-Coordination Directive" sections of `mutable.rst <../specifications/mutable.rst>`_.
+Coordination Directive" sections of mutable.rst_.
+
+.. _mutable.rst: ../specifications/mutable.rst
Access Blacklist
request may arrive while the file is only halfway written, and the partial
file may be incorrectly parsed.
-The blacklist is applied to all access paths (including FTP, SFTP, and CLI
+The blacklist is applied to all access paths (including SFTP, FTP, and CLI
operations), not just the web-API. The blacklist also applies to directories.
If a directory is blacklisted, the gateway will refuse access to both that
directory and any child files/directories underneath it, when accessed via
.. [1] URLs and HTTP and UTF-8, Oh My
HTTP does not provide a mechanism to specify the character set used to
- encode non-ASCII names in URLs
- (`RFC3986#2.1 <http://tools.ietf.org/html/rfc3986#section-2.1>`_).
- We prefer the convention that the ``filename=`` argument shall be a
- URL-escaped UTF-8 encoded Unicode string.
- For example, suppose we want to provoke the server into using a filename of
- "f i a n c e-acute e" (i.e. f i a n c U+00E9 e). The UTF-8 encoding of this
- is 0x66 0x69 0x61 0x6e 0x63 0xc3 0xa9 0x65 (or "fianc\\xC3\\xA9e", as python's
- ``repr()`` function would show). To encode this into a URL, the non-printable
- characters must be escaped with the urlencode ``%XX`` mechanism, giving
- us "fianc%C3%A9e". Thus, the first line of the HTTP request will be
- "``GET /uri/CAP...?save=true&filename=fianc%C3%A9e HTTP/1.1``". Not all
- browsers provide this: IE7 by default uses the Latin-1 encoding, which is
- "fianc%E9e" (although it has a configuration option to send URLs as UTF-8).
+ encode non-ASCII names in URLs (`RFC3986#2.1`_). We prefer the convention
+ that the ``filename=`` argument shall be a URL-escaped UTF-8 encoded Unicode
+ string. For example, suppose we want to provoke the server into using a
+ filename of "f i a n c e-acute e" (i.e. f i a n c U+00E9 e). The UTF-8
+ encoding of this is 0x66 0x69 0x61 0x6e 0x63 0xc3 0xa9 0x65 (or
+ "fianc\\xC3\\xA9e", as python's ``repr()`` function would show). To encode
+ this into a URL, the non-printable characters must be escaped with the
+ urlencode ``%XX`` mechanism, giving us "fianc%C3%A9e". Thus, the first line
+ of the HTTP request will be "``GET
+ /uri/CAP...?save=true&filename=fianc%C3%A9e HTTP/1.1``". Not all browsers
+ provide this: IE7 by default uses the Latin-1 encoding, which is "fianc%E9e"
+ (although it has a configuration option to send URLs as UTF-8).
The response header will need to indicate a non-ASCII filename. The actual
mechanism to do this is not clear. For ASCII filenames, the response header
(note, the last four bytes of that line, not including the newline, are
0xC3 0xA9 0x65 0x22)
- `RFC2231#4 <http://tools.ietf.org/html/rfc2231#section-4>`_
- (dated 1997): suggests that the following might work, and
- `some developers have reported <http://markmail.org/message/dsjyokgl7hv64ig3>`_
- that it is supported by Firefox (but not IE7)::
+ `RFC2231#4`_ (dated 1997): suggests that the following might work, and `some
+ developers have reported`_ that it is supported by Firefox (but not IE7)::
#2: Content-Disposition: attachment; filename*=utf-8''fianc%C3%A9e
- My reading of `RFC2616#19.5.1 <http://tools.ietf.org/html/rfc2616#section-19.5.1>`_
- (which defines Content-Disposition) says that the filename= parameter is
- defined to be wrapped in quotes (presumably to allow spaces without breaking
- the parsing of subsequent parameters), which would give us::
+ My reading of `RFC2616#19.5.1`_ (which defines Content-Disposition) says
+ that the filename= parameter is defined to be wrapped in quotes (presumably
+ to allow spaces without breaking the parsing of subsequent parameters),
+ which would give us::
#3: Content-Disposition: attachment; filename*=utf-8''"fianc%C3%A9e"
into the response header, rather than enforcing the UTF-8 convention. This
means it does not try to decode the filename from the URL argument, nor does
it encode the filename into the response header.
+
+.. _RFC3986#2.1: https://tools.ietf.org/html/rfc3986#section-2.1
+.. _RFC2231#4: https://tools.ietf.org/html/rfc2231#section-4
+.. _some developers have reported: http://markmail.org/message/dsjyokgl7hv64ig3
+.. _RFC2616#19.5.1: https://tools.ietf.org/html/rfc2616#section-19.5.1
+
projects / tahoe-lafs / tahoe-lafs.git / blobdiff