1 .. -*- coding: utf-8-with-signature -*-
3 ===========================
4 The Tahoe-LAFS CLI commands
5 ===========================
8 2. `CLI Command Overview`_
13 4. `Filesystem Manipulation`_
15 1. `Starting Directories`_
16 2. `Command Syntax Summary`_
17 3. `Command Examples`_
19 5. `Storage Grid Maintenance`_
26 Tahoe-LAFS provides a single executable named "``tahoe``", which can be used to
27 create and manage client/server nodes, manipulate the filesystem, and perform
28 several debugging/maintenance tasks.
30 This executable lives in the source tree at "``bin/tahoe``". Once you've done a
31 build (by running "``make``" or "``python setup.py build``"), ``bin/tahoe`` can
32 be run in-place: if it discovers that it is being run from within a Tahoe-LAFS
33 source tree, it will modify ``sys.path`` as necessary to use all the source code
34 and dependent libraries contained in that tree.
36 If you've installed Tahoe-LAFS (using "``make install``" or
37 "``python setup.py install``", or by installing a binary package), then the
38 ``tahoe`` executable will be available somewhere else, perhaps in
39 ``/usr/bin/tahoe``. In this case, it will use your platform's normal
40 PYTHONPATH search path to find the Tahoe-LAFS code and other libraries.
46 The "``tahoe``" tool provides access to three categories of commands.
48 * node management: create a client/server node, start/stop/restart it
49 * filesystem manipulation: list files, upload, download, unlink, rename
50 * debugging: unpack cap-strings, examine share files
52 To get a list of all commands, just run "``tahoe``" with no additional
53 arguments. "``tahoe --help``" might also provide something useful.
55 Running "``tahoe --version``" will display a list of version strings, starting
56 with the "allmydata" module (which contains the majority of the Tahoe-LAFS
57 functionality) and including versions for a number of dependent libraries,
58 like Twisted, Foolscap, pycryptopp, and zfec. "``tahoe --version-and-path``"
59 will also show the path from which each library was imported.
61 On Unix systems, the shell expands filename wildcards (``'*'`` and ``'?'``)
62 before the program is able to read them, which may produce unexpected results
63 for many ``tahoe`` comands. We recommend, if you use wildcards, to start the
64 path with "``./``", for example "``tahoe cp -r ./* somewhere:``". This
65 prevents the expanded filename from being interpreted as an option or as an
66 alias, allowing filenames that start with a dash or contain colons to be
69 On Windows, a single letter followed by a colon is treated as a drive
70 specification rather than an alias (and is invalid unless a local path is
71 allowed in that context). Wildcards cannot be used to specify multiple
72 filenames to ``tahoe`` on Windows.
77 As of Tahoe-LAFS v1.7.0 (v1.8.0 on Windows), the ``tahoe`` tool supports
78 non-ASCII characters in command lines and output. On Unix, the command-line
79 arguments are assumed to use the character encoding specified by the
80 current locale (usually given by the ``LANG`` environment variable).
82 If a name to be output contains control characters or characters that
83 cannot be represented in the encoding used on your terminal, it will be
84 quoted. The quoting scheme used is similar to `POSIX shell quoting`_: in
85 a "double-quoted" string, backslashes introduce escape sequences (like
86 those in Python strings), but in a 'single-quoted' string all characters
87 stand for themselves. This quoting is only used for output, on all
88 operating systems. Your shell interprets any quoting or escapes used on
91 .. _`POSIX shell quoting`: http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html
97 "``tahoe create-node [NODEDIR]``" is the basic make-a-new-node command. It
98 creates a new directory and populates it with files that will allow the
99 "``tahoe start``" command to use it later on. This command creates nodes that
100 have client functionality (upload/download files), web API services
101 (controlled by the '[node]web.port' configuration), and storage services
102 (unless ``--no-storage`` is specified).
104 NODEDIR defaults to ``~/.tahoe/`` , and newly-created nodes default to
105 publishing a web server on port 3456 (limited to the loopback interface, at
106 127.0.0.1, to restrict access to other programs on the same host). All of the
107 other "``tahoe``" subcommands use corresponding defaults (with the exception
108 that "``tahoe run``" defaults to running a node in the current directory).
110 "``tahoe create-client [NODEDIR]``" creates a node with no storage service.
111 That is, it behaves like "``tahoe create-node --no-storage [NODEDIR]``".
112 (This is a change from versions prior to v1.6.0.)
114 "``tahoe create-introducer [NODEDIR]``" is used to create the Introducer node.
115 This node provides introduction services and nothing else. When started, this
116 node will produce a ``private/introducer.furl`` file, which should be
117 published to all clients.
119 "``tahoe create-key-generator [NODEDIR]``" is used to create a special
120 "key-generation" service, which allows a client to offload their RSA key
121 generation to a separate process. Since RSA key generation takes several
122 seconds, and must be done each time a directory is created, moving it to a
123 separate process allows the first process (perhaps a busy web-API server) to
124 continue servicing other requests. The key generator exports a FURL that can
125 be copied into a node to enable this functionality.
127 "``tahoe run [NODEDIR]``" will start a previously-created node in the foreground.
129 "``tahoe start [NODEDIR]``" will launch a previously-created node. It will
130 launch the node into the background, using the standard Twisted "``twistd``"
131 daemon-launching tool. On some platforms (including Windows) this command is
132 unable to run a daemon in the background; in that case it behaves in the
133 same way as "``tahoe run``".
135 "``tahoe stop [NODEDIR]``" will shut down a running node.
137 "``tahoe restart [NODEDIR]``" will stop and then restart a running node. This
138 is most often used by developers who have just modified the code and want to
139 start using their changes.
142 Filesystem Manipulation
143 =======================
145 These commands let you exmaine a Tahoe-LAFS filesystem, providing basic
146 list/upload/download/unlink/rename/mkdir functionality. They can be used as
147 primitives by other scripts. Most of these commands are fairly thin wrappers
148 around web-API calls, which are described in `<webapi.rst>`__.
150 By default, all filesystem-manipulation commands look in ``~/.tahoe/`` to
151 figure out which Tahoe-LAFS node they should use. When the CLI command makes
152 web-API calls, it will use ``~/.tahoe/node.url`` for this purpose: a running
153 Tahoe-LAFS node that provides a web-API port will write its URL into this
154 file. If you want to use a node on some other host, just create ``~/.tahoe/``
155 and copy that node's web-API URL into this file, and the CLI commands will
156 contact that node instead of a local one.
158 These commands also use a table of "aliases" to figure out which directory
159 they ought to use a starting point. This is explained in more detail below.
164 As described in `docs/architecture.rst <../architecture.rst>`__, the
165 Tahoe-LAFS distributed filesystem consists of a collection of directories
166 and files, each of which has a "read-cap" or a "write-cap" (also known as
167 a URI). Each directory is simply a table that maps a name to a child file
168 or directory, and this table is turned into a string and stored in a
169 mutable file. The whole set of directory and file "nodes" are connected
170 together into a directed graph.
172 To use this collection of files and directories, you need to choose a
173 starting point: some specific directory that we will refer to as a
174 "starting directory". For a given starting directory, the
175 "``ls [STARTING_DIR]``" command would list the contents of this directory,
176 the "``ls [STARTING_DIR]/dir1``" command would look inside this directory
177 for a child named "``dir1``" and list its contents,
178 "``ls [STARTING_DIR]/dir1/subdir2``" would look two levels deep, etc.
180 Note that there is no real global "root" directory, but instead each
181 starting directory provides a different, possibly overlapping
182 perspective on the graph of files and directories.
184 Each Tahoe-LAFS node remembers a list of starting points, called "aliases",
185 which are short Unicode strings that stand in for a directory read- or
186 write- cap. They are stored (encoded as UTF-8) in the file
187 ``NODEDIR/private/aliases`` . If you use the command line "``tahoe ls``"
188 without any "[STARTING_DIR]" argument, then it will use the default alias,
189 which is ``tahoe:``, therefore "``tahoe ls``" has the same effect as
190 "``tahoe ls tahoe:``". The same goes for the other commands that can
191 reasonably use a default alias: ``get``, ``put``, ``mkdir``, ``mv``, and
194 For backwards compatibility with Tahoe-LAFS v1.0, if the ``tahoe:`` alias
195 is not found in ``~/.tahoe/private/aliases``, the CLI will use the contents
196 of ``~/.tahoe/private/root_dir.cap`` instead. Tahoe-LAFS v1.0 had only a
197 single starting point, and stored it in this ``root_dir.cap`` file, so v1.1
198 and later will use it if necessary. However, once you've set a ``tahoe:``
199 alias with "``tahoe set-alias``", that will override anything in the old
200 ``root_dir.cap`` file.
202 The Tahoe-LAFS CLI commands use the same path syntax as ``scp`` and
203 ``rsync`` -- an optional ``ALIAS:`` prefix, followed by the pathname or
204 filename. Some commands (like "``tahoe cp``") use the lack of an alias to
205 mean that you want to refer to a local file, instead of something from the
206 Tahoe-LAFS filesystem. Another way to indicate this is to start the
207 pathname with "./", "~/", "~username/", or "/".
209 When you're dealing a single starting directory, the ``tahoe:`` alias is
210 all you need. But when you want to refer to something that isn't yet
211 attached to the graph rooted at that starting directory, you need to
212 refer to it by its capability. The way to do that is either to use its
213 capability directory as an argument on the command line, or to add an
214 alias to it, with the "``tahoe add-alias``" command. Once you've added an
215 alias, you can use that alias as an argument to commands.
217 The best way to get started with Tahoe-LAFS is to create a node, start it,
218 then use the following command to create a new directory and set it as your
221 tahoe create-alias tahoe
223 After that you can use "``tahoe ls tahoe:``" and
224 "``tahoe cp local.txt tahoe:``", and both will refer to the directory that
227 SECURITY NOTE: For users of shared systems
228 ``````````````````````````````````````````
230 Another way to achieve the same effect as the above "``tahoe create-alias``"
233 tahoe add-alias tahoe `tahoe mkdir`
235 However, command-line arguments are visible to other users (through the
236 ``ps`` command or ``/proc`` filesystem, or the Windows Process Explorer tool),
237 so if you are using a Tahoe-LAFS node on a shared host, your login neighbors
238 will be able to see (and capture) any directory caps that you set up with the
239 "``tahoe add-alias``" command.
241 The "``tahoe create-alias``" command avoids this problem by creating a new
242 directory and putting the cap into your aliases file for you. Alternatively,
243 you can edit the ``NODEDIR/private/aliases`` file directly, by adding a line
246 fun: URI:DIR2:ovjy4yhylqlfoqg2vcze36dhde:4d4f47qko2xm5g7osgo2yyidi5m4muyo2vjjy53q4vjju2u55mfa
248 By entering the dircap through the editor, the command-line arguments are
249 bypassed, and other users will not be able to see them. Once you've added the
250 alias, no other secrets are passed through the command line, so this
251 vulnerability becomes less significant: they can still see your filenames and
252 other arguments you type there, but not the caps that Tahoe-LAFS uses to permit
253 access to your files and directories.
256 Command Syntax Summary
257 ----------------------
259 ``tahoe add-alias ALIAS[:] DIRCAP``
261 ``tahoe create-alias ALIAS[:]``
263 ``tahoe list-aliases``
271 ``tahoe webopen [PATH]``
273 ``tahoe put [--mutable] [FROMLOCAL|-]``
275 ``tahoe put [--mutable] FROMLOCAL|- TOPATH``
277 ``tahoe put [FROMLOCAL|-] mutable-file-writecap``
279 ``tahoe get FROMPATH [TOLOCAL|-]``
281 ``tahoe cp [-r] FROMPATH TOPATH``
285 ``tahoe mv FROMPATH TOPATH``
287 ``tahoe ln FROMPATH TOPATH``
289 ``tahoe backup FROMLOCAL TOPATH``
291 In these summaries, ``PATH``, ``TOPATH`` or ``FROMPATH`` can be one of:
293 * ``[SUBDIRS/]FILENAME`` for a path relative to the default ``tahoe:`` alias;
294 * ``ALIAS:[SUBDIRS/]FILENAME`` for a path relative to another alias;
295 * ``DIRCAP/[SUBDIRS/]FILENAME`` or ``DIRCAP:./[SUBDIRS/]FILENAME`` for a path
296 relative to a directory cap.
298 See `CLI Command Overview`_ above for information on using wildcards with
299 local paths, and different treatment of colons between Unix and Windows.
301 ``FROMLOCAL`` or ``TOLOCAL`` is a path in the local filesystem.
307 ``tahoe add-alias ALIAS[:] DIRCAP``
309 An example would be::
311 tahoe add-alias fun URI:DIR2:ovjy4yhylqlfoqg2vcze36dhde:4d4f47qko2xm5g7osgo2yyidi5m4muyo2vjjy53q4vjju2u55mfa
313 This creates an alias ``fun:`` and configures it to use the given directory
314 cap. Once this is done, "``tahoe ls fun:``" will list the contents of this
315 directory. Use "``tahoe add-alias tahoe DIRCAP``" to set the contents of the
316 default ``tahoe:`` alias.
318 Since Tahoe-LAFS v1.8.2, the alias name can be given with or without the
321 On Windows, the alias should not be a single character, because it would be
322 confused with the drive letter of a local path.
324 ``tahoe create-alias fun``
326 This combines "``tahoe mkdir``" and "``tahoe add-alias``" into a single step.
328 ``tahoe list-aliases``
330 This displays a table of all configured aliases.
334 This creates a new empty unlinked directory, and prints its write-cap to
335 stdout. The new directory is not attached to anything else.
337 ``tahoe mkdir subdir``
339 ``tahoe mkdir /subdir``
341 This creates a new empty directory and attaches it below the root directory
342 of the default ``tahoe:`` alias with the name "``subdir``".
352 All four list the root directory of the default ``tahoe:`` alias.
356 This lists a subdirectory of your filesystem.
360 ``tahoe webopen tahoe:``
362 ``tahoe webopen tahoe:subdir/``
364 ``tahoe webopen subdir/``
366 This uses the python 'webbrowser' module to cause a local web browser to
367 open to the web page for the given directory. This page offers interfaces to
368 add, download, rename, and unlink files and subdirectories in that directory.
369 If no alias or path is given, this command opens the root directory of the
370 default ``tahoe:`` alias.
372 ``tahoe put file.txt``
374 ``tahoe put ./file.txt``
376 ``tahoe put /tmp/file.txt``
378 ``tahoe put ~/file.txt``
380 These upload the local file into the grid, and prints the new read-cap to
381 stdout. The uploaded file is not attached to any directory. All one-argument
382 forms of "``tahoe put``" perform an unlinked upload.
388 These also perform an unlinked upload, but the data to be uploaded is taken
391 ``tahoe put file.txt uploaded.txt``
393 ``tahoe put file.txt tahoe:uploaded.txt``
395 These upload the local file and add it to your ``tahoe:`` root with the name
398 ``tahoe put file.txt subdir/foo.txt``
400 ``tahoe put - subdir/foo.txt``
402 ``tahoe put file.txt tahoe:subdir/foo.txt``
404 ``tahoe put file.txt DIRCAP/foo.txt``
406 ``tahoe put file.txt DIRCAP/subdir/foo.txt``
408 These upload the named file and attach them to a subdirectory of the given
409 root directory, under the name "``foo.txt``". When a directory write-cap is
410 given, you can use either ``/`` (as shown above) or ``:./`` to separate it
411 from the following path. When the source file is named "``-``", the contents
412 are taken from stdin.
414 ``tahoe put file.txt --mutable``
416 Create a new (SDMF) mutable file, fill it with the contents of ``file.txt``,
417 and print the new write-cap to stdout.
419 ``tahoe put file.txt MUTABLE-FILE-WRITECAP``
421 Replace the contents of the given mutable file with the contents of
422 ``file.txt`` and print the same write-cap to stdout.
424 ``tahoe cp file.txt tahoe:uploaded.txt``
426 ``tahoe cp file.txt tahoe:``
428 ``tahoe cp file.txt tahoe:/``
430 ``tahoe cp ./file.txt tahoe:``
432 These upload the local file and add it to your ``tahoe:`` root with the name
435 ``tahoe cp tahoe:uploaded.txt downloaded.txt``
437 ``tahoe cp tahoe:uploaded.txt ./downloaded.txt``
439 ``tahoe cp tahoe:uploaded.txt /tmp/downloaded.txt``
441 ``tahoe cp tahoe:uploaded.txt ~/downloaded.txt``
443 This downloads the named file from your ``tahoe:`` root, and puts the result on
444 your local filesystem.
446 ``tahoe cp tahoe:uploaded.txt fun:stuff.txt``
448 This copies a file from your ``tahoe:`` root to a different directory, set up
449 earlier with "``tahoe add-alias fun DIRCAP``" or "``tahoe create-alias fun``".
451 ``tahoe cp -r ~/my_dir/ tahoe:``
453 This copies the folder ``~/my_dir/`` and all its children to the grid, creating
454 the new folder ``tahoe:my_dir``. Note that the trailing slash is not required:
455 all source arguments which are directories will be copied into new
456 subdirectories of the target.
458 The behavior of ``tahoe cp``, like the regular UNIX ``/bin/cp``, is subtly
459 different depending upon the exact form of the arguments. In particular:
461 * Trailing slashes indicate directories, but are not required.
462 * If the target object does not already exist:
463 * and if the source is a single file, it will be copied into the target;
464 * otherwise, the target will be created as a directory.
465 * If there are multiple sources, the target must be a directory.
466 * If the target is a pre-existing file, the source must be a single file.
467 * If the target is a directory, each source must be a named file, a named
468 directory, or an unnamed directory. It is not possible to copy an unnamed
469 file (e.g. a raw filecap) into a directory, as there is no way to know what
470 the new file should be named.
473 ``tahoe unlink uploaded.txt``
475 ``tahoe unlink tahoe:uploaded.txt``
477 This unlinks a file from your ``tahoe:`` root (that is, causes there to no
478 longer be an entry ``uploaded.txt`` in the root directory that points to it).
479 Note that this does not delete the file from the grid.
480 For backward compatibility, ``tahoe rm`` is accepted as a synonym for
483 ``tahoe mv uploaded.txt renamed.txt``
485 ``tahoe mv tahoe:uploaded.txt tahoe:renamed.txt``
487 These rename a file within your ``tahoe:`` root directory.
489 ``tahoe mv uploaded.txt fun:``
491 ``tahoe mv tahoe:uploaded.txt fun:``
493 ``tahoe mv tahoe:uploaded.txt fun:uploaded.txt``
495 These move a file from your ``tahoe:`` root directory to the directory
496 set up earlier with "``tahoe add-alias fun DIRCAP``" or
497 "``tahoe create-alias fun``".
499 ``tahoe backup ~ work:backups``
501 This command performs a versioned backup of every file and directory
502 underneath your "``~``" home directory, placing an immutable timestamped
503 snapshot in e.g. ``work:backups/Archives/2009-02-06_04:00:05Z/`` (note that
504 the timestamp is in UTC, hence the "Z" suffix), and a link to the latest
505 snapshot in work:backups/Latest/ . This command uses a small SQLite database
506 known as the "backupdb", stored in ``~/.tahoe/private/backupdb.sqlite``, to
507 remember which local files have been backed up already, and will avoid
508 uploading files that have already been backed up (except occasionally that
509 will randomly upload them again if it has been awhile since had last been
510 uploaded, just to make sure that the copy of it on the server is still good).
511 It compares timestamps and filesizes when making this comparison. It also
512 re-uses existing directories which have identical contents. This lets it
513 run faster and reduces the number of directories created.
515 If you reconfigure your client node to switch to a different grid, you
516 should delete the stale backupdb.sqlite file, to force "``tahoe backup``"
517 to upload all files to the new grid.
519 The fact that "tahoe backup" checks timestamps on your local files and
520 skips ones that don't appear to have been changed is one of the major
521 differences between "tahoe backup" and "tahoe cp -r". The other major
522 difference is that "tahoe backup" keeps links to all of the versions that
523 have been uploaded to the grid, so you can navigate among old versions
524 stored in the grid. In contrast, "tahoe cp -r" unlinks the previous
525 version from the grid directory and links the new version into place,
526 so unless you have a link to the older version stored somewhere else,
527 you'll never be able to get back to it.
529 ``tahoe backup --exclude=*~ ~ work:backups``
531 Same as above, but this time the backup process will ignore any
532 filename that will end with '~'. ``--exclude`` will accept any standard
533 Unix shell-style wildcards, as implemented by the
534 `Python fnmatch module <http://docs.python.org/library/fnmatch.html>`__.
535 You may give multiple ``--exclude`` options. Please pay attention that
536 the pattern will be matched against any level of the directory tree;
537 it's still impossible to specify absolute path exclusions.
539 ``tahoe backup --exclude-from=/path/to/filename ~ work:backups``
541 ``--exclude-from`` is similar to ``--exclude``, but reads exclusion
542 patterns from ``/path/to/filename``, one per line.
544 ``tahoe backup --exclude-vcs ~ work:backups``
546 This command will ignore any file or directory name known to be used by
547 version control systems to store metadata. The excluded names are:
568 Storage Grid Maintenance
569 ========================
571 ``tahoe manifest tahoe:``
573 ``tahoe manifest --storage-index tahoe:``
575 ``tahoe manifest --verify-cap tahoe:``
577 ``tahoe manifest --repair-cap tahoe:``
579 ``tahoe manifest --raw tahoe:``
581 This performs a recursive walk of the given directory, visiting every file
582 and directory that can be reached from that point. It then emits one line to
583 stdout for each object it encounters.
585 The default behavior is to print the access cap string (like ``URI:CHK:..``
586 or ``URI:DIR2:..``), followed by a space, followed by the full path name.
588 If ``--storage-index`` is added, each line will instead contain the object's
589 storage index. This (string) value is useful to determine which share files
590 (on the server) are associated with this directory tree. The ``--verify-cap``
591 and ``--repair-cap`` options are similar, but emit a verify-cap and repair-cap,
592 respectively. If ``--raw`` is provided instead, the output will be a
593 JSON-encoded dictionary that includes keys for pathnames, storage index
594 strings, and cap strings. The last line of the ``--raw`` output will be a JSON
595 encoded deep-stats dictionary.
597 ``tahoe stats tahoe:``
599 This performs a recursive walk of the given directory, visiting every file
600 and directory that can be reached from that point. It gathers statistics on
601 the sizes of the objects it encounters, and prints a summary to stdout.
607 For a list of all debugging commands, use "``tahoe debug``". For more detailed
608 help on any of these commands, use "``tahoe debug COMMAND --help``".
610 "``tahoe debug find-shares STORAGEINDEX NODEDIRS..``" will look through one or
611 more storage nodes for the share files that are providing storage for the
614 "``tahoe debug catalog-shares NODEDIRS..``" will look through one or more
615 storage nodes and locate every single share they contain. It produces a report
616 on stdout with one line per share, describing what kind of share it is, the
617 storage index, the size of the file is used for, etc. It may be useful to
618 concatenate these reports from all storage hosts and use it to look for
621 "``tahoe debug dump-share SHAREFILE``" will take the name of a single share file
622 (as found by "``tahoe find-shares``") and print a summary of its contents to
623 stdout. This includes a list of leases, summaries of the hash tree, and
624 information from the UEB (URI Extension Block). For mutable file shares, it
625 will describe which version (seqnum and root-hash) is being stored in this
628 "``tahoe debug dump-cap CAP``" will take any Tahoe-LAFS URI and unpack it
629 into separate pieces. The most useful aspect of this command is to reveal the
630 storage index for any given URI. This can be used to locate the share files
631 that are holding the encoded+encrypted data for this file.
633 "``tahoe debug repl``" will launch an interactive Python interpreter in which
634 the Tahoe-LAFS packages and modules are available on ``sys.path`` (e.g. by using
635 '``import allmydata``'). This is most useful from a source tree: it simply sets
636 the PYTHONPATH correctly and runs the Python executable.
638 "``tahoe debug corrupt-share SHAREFILE``" will flip a bit in the given
639 sharefile. This can be used to test the client-side verification/repair code.
640 Obviously, this command should not be used during normal operation.
642 "``tahoe debug trial [OPTIONS] [TESTSUITE]``" will run the tests specified by
643 TESTSUITE (defaulting to the whole Tahoe test suite), using Twisted Trial.