1 ===========================
2 The Tahoe-LAFS CLI commands
3 ===========================
6 2. `CLI Command Overview`_
8 4. `Filesystem Manipulation`_
10 1. `Starting Directories`_
11 2. `Command Syntax Summary`_
12 3. `Command Examples`_
14 5. `Storage Grid Maintenance`_
21 Tahoe-LAFS provides a single executable named "``tahoe``", which can be used to
22 create and manage client/server nodes, manipulate the filesystem, and perform
23 several debugging/maintenance tasks.
25 This executable lives in the source tree at "``bin/tahoe``". Once you've done a
26 build (by running "``make``" or "``python setup.py build``"), ``bin/tahoe`` can
27 be run in-place: if it discovers that it is being run from within a Tahoe-LAFS
28 source tree, it will modify ``sys.path`` as necessary to use all the source code
29 and dependent libraries contained in that tree.
31 If you've installed Tahoe-LAFS (using "``make install``" or
32 "``python setup.py install``", or by installing a binary package), then the
33 ``tahoe`` executable will be available somewhere else, perhaps in
34 ``/usr/bin/tahoe``. In this case, it will use your platform's normal
35 PYTHONPATH search path to find the Tahoe-LAFS code and other libraries.
41 The "``tahoe``" tool provides access to three categories of commands.
43 * node management: create a client/server node, start/stop/restart it
44 * filesystem manipulation: list files, upload, download, delete, rename
45 * debugging: unpack cap-strings, examine share files
47 To get a list of all commands, just run "``tahoe``" with no additional
48 arguments. "``tahoe --help``" might also provide something useful.
50 Running "``tahoe --version``" will display a list of version strings, starting
51 with the "allmydata" module (which contains the majority of the Tahoe-LAFS
52 functionality) and including versions for a number of dependent libraries,
53 like Twisted, Foolscap, pycryptopp, and zfec.
59 "``tahoe create-node [NODEDIR]``" is the basic make-a-new-node command. It
60 creates a new directory and populates it with files that will allow the
61 "``tahoe start``" command to use it later on. This command creates nodes that
62 have client functionality (upload/download files), web API services
63 (controlled by the 'webport' file), and storage services (unless
64 ``--no-storage`` is specified).
66 NODEDIR defaults to ``~/.tahoe/`` , and newly-created nodes default to
67 publishing a web server on port 3456 (limited to the loopback interface, at
68 127.0.0.1, to restrict access to other programs on the same host). All of the
69 other "``tahoe``" subcommands use corresponding defaults (with the exception
70 that "``tahoe run``" defaults to running a node in the current directory).
72 "``tahoe create-client [NODEDIR]``" creates a node with no storage service.
73 That is, it behaves like "``tahoe create-node --no-storage [NODEDIR]``".
74 (This is a change from versions prior to v1.6.0.)
76 "``tahoe create-introducer [NODEDIR]``" is used to create the Introducer node.
77 This node provides introduction services and nothing else. When started, this
78 node will produce an ``introducer.furl`` file, which should be published to all
81 "``tahoe create-key-generator [NODEDIR]``" is used to create a special
82 "key-generation" service, which allows a client to offload their RSA key
83 generation to a separate process. Since RSA key generation takes several
84 seconds, and must be done each time a directory is created, moving it to a
85 separate process allows the first process (perhaps a busy web-API server) to
86 continue servicing other requests. The key generator exports a FURL that can
87 be copied into a node to enable this functionality.
89 "``tahoe run [NODEDIR]``" will start a previously-created node in the foreground.
91 "``tahoe start [NODEDIR]``" will launch a previously-created node. It will
92 launch the node into the background, using the standard Twisted "``twistd``"
93 daemon-launching tool. On some platforms (including Windows) this command is
94 unable to run a daemon in the background; in that case it behaves in the
95 same way as "``tahoe run``".
97 "``tahoe stop [NODEDIR]``" will shut down a running node.
99 "``tahoe restart [NODEDIR]``" will stop and then restart a running node. This
100 is most often used by developers who have just modified the code and want to
101 start using their changes.
104 Filesystem Manipulation
105 =======================
107 These commands let you exmaine a Tahoe-LAFS filesystem, providing basic
108 list/upload/download/delete/rename/mkdir functionality. They can be used as
109 primitives by other scripts. Most of these commands are fairly thin wrappers
110 around web-API calls, which are described in `<webapi.rst>`_.
112 By default, all filesystem-manipulation commands look in ``~/.tahoe/`` to
113 figure out which Tahoe-LAFS node they should use. When the CLI command makes
114 web-API calls, it will use ``~/.tahoe/node.url`` for this purpose: a running
115 Tahoe-LAFS node that provides a web-API port will write its URL into this
116 file. If you want to use a node on some other host, just create ``~/.tahoe/``
117 and copy that node's web-API URL into this file, and the CLI commands will
118 contact that node instead of a local one.
120 These commands also use a table of "aliases" to figure out which directory
121 they ought to use a starting point. This is explained in more detail below.
123 As of Tahoe-LAFS v1.7 (v1.7.1 on Windows), passing non-ASCII characters to the
124 CLI should work. On Unix, the command-line arguments are assumed to use the
125 character encoding specified by the current locale.
130 As described in `docs/architecture.rst <../architecture.rst>`_, the
131 Tahoe-LAFS distributed filesystem consists of a collection of directories
132 and files, each of which has a "read-cap" or a "write-cap" (also known as
133 a URI). Each directory is simply a table that maps a name to a child file
134 or directory, and this table is turned into a string and stored in a
135 mutable file. The whole set of directory and file "nodes" are connected
136 together into a directed graph.
138 To use this collection of files and directories, you need to choose a
139 starting point: some specific directory that we will refer to as a
140 "starting directory". For a given starting directory, the
141 "``ls [STARTING_DIR]``" command would list the contents of this directory,
142 the "``ls [STARTING_DIR]/dir1``" command would look inside this directory
143 for a child named "``dir1``" and list its contents,
144 "``ls [STARTING_DIR]/dir1/subdir2``" would look two levels deep, etc.
146 Note that there is no real global "root" directory, but instead each
147 starting directory provides a different, possibly overlapping
148 perspective on the graph of files and directories.
150 Each Tahoe-LAFS node remembers a list of starting points, called "aliases",
151 which are short Unicode strings that stand in for a directory read- or
152 write- cap. They are stored (encoded as UTF-8) in the file
153 ``NODEDIR/private/aliases`` . If you use the command line "``tahoe ls``"
154 without any "[STARTING_DIR]" argument, then it will use the default alias,
155 which is ``tahoe:``, therefore "``tahoe ls``" has the same effect as
156 "``tahoe ls tahoe:``". The same goes for the other commands that can
157 reasonably use a default alias: ``get``, ``put``, ``mkdir``, ``mv``, and
160 For backwards compatibility with Tahoe-LAFS v1.0, if the ``tahoe:`` alias
161 is not found in ``~/.tahoe/private/aliases``, the CLI will use the contents
162 of ``~/.tahoe/private/root_dir.cap`` instead. Tahoe-LAFS v1.0 had only a
163 single starting point, and stored it in this ``root_dir.cap`` file, so v1.1
164 and later will use it if necessary. However, once you've set a ``tahoe:``
165 alias with "``tahoe set-alias``", that will override anything in the old
166 ``root_dir.cap`` file.
168 The Tahoe-LAFS CLI commands use the same path syntax as ``scp`` and
169 ``rsync`` -- an optional ``ALIAS:`` prefix, followed by the pathname or
170 filename. Some commands (like "``tahoe cp``") use the lack of an alias to
171 mean that you want to refer to a local file, instead of something from the
172 Tahoe-LAFS filesystem. [TODO] Another way to indicate this is to start
173 the pathname with a dot, slash, or tilde.
175 When you're dealing a single starting directory, the ``tahoe:`` alias is
176 all you need. But when you want to refer to something that isn't yet
177 attached to the graph rooted at that starting directory, you need to
178 refer to it by its capability. The way to do that is either to use its
179 capability directory as an argument on the command line, or to add an
180 alias to it, with the "``tahoe add-alias``" command. Once you've added an
181 alias, you can use that alias as an argument to commands.
183 The best way to get started with Tahoe-LAFS is to create a node, start it,
184 then use the following command to create a new directory and set it as your
187 tahoe create-alias tahoe
189 After that you can use "``tahoe ls tahoe:``" and
190 "``tahoe cp local.txt tahoe:``", and both will refer to the directory that
193 SECURITY NOTE: For users of shared systems
194 ``````````````````````````````````````````
196 Another way to achieve the same effect as the above "``tahoe create-alias``"
199 tahoe add-alias tahoe `tahoe mkdir`
201 However, command-line arguments are visible to other users (through the
202 ``ps`` command or ``/proc`` filesystem, or the Windows Process Explorer tool),
203 so if you are using a Tahoe-LAFS node on a shared host, your login neighbors
204 will be able to see (and capture) any directory caps that you set up with the
205 "``tahoe add-alias``" command.
207 The "``tahoe create-alias``" command avoids this problem by creating a new
208 directory and putting the cap into your aliases file for you. Alternatively,
209 you can edit the ``NODEDIR/private/aliases`` file directly, by adding a line
212 fun: URI:DIR2:ovjy4yhylqlfoqg2vcze36dhde:4d4f47qko2xm5g7osgo2yyidi5m4muyo2vjjy53q4vjju2u55mfa
214 By entering the dircap through the editor, the command-line arguments are
215 bypassed, and other users will not be able to see them. Once you've added the
216 alias, no other secrets are passed through the command line, so this
217 vulnerability becomes less significant: they can still see your filenames and
218 other arguments you type there, but not the caps that Tahoe-LAFS uses to permit
219 access to your files and directories.
222 Command Syntax Summary
223 ----------------------
225 ``tahoe add-alias ALIAS DIRCAP``
227 ``tahoe create-alias ALIAS``
229 ``tahoe list-aliases``
237 ``tahoe webopen [PATH]``
239 ``tahoe put [--mutable] [FROMLOCAL|-]``
241 ``tahoe put [--mutable] FROMLOCAL|- TOPATH``
243 ``tahoe put [FROMLOCAL|-] mutable-file-writecap``
245 ``tahoe get FROMPATH [TOLOCAL|-]``
247 ``tahoe cp [-r] FROMPATH TOPATH``
251 ``tahoe mv FROMPATH TOPATH``
253 ``tahoe ln FROMPATH TOPATH``
255 ``tahoe backup FROMLOCAL TOPATH``
257 In these summaries, ``PATH``, ``TOPATH`` or ``FROMPATH`` can be one of::
259 * ``[SUBDIRS/]FILENAME`` for a path relative to the default ``tahoe:`` alias;
260 * ``ALIAS:[SUBDIRS/]FILENAME`` for a path relative to another alias;
261 * ``DIRCAP/[SUBDIRS/]FILENAME`` or ``DIRCAP:./[SUBDIRS/]FILENAME`` for a
262 path relative to a directory cap.
268 ``tahoe add-alias ALIAS DIRCAP``
270 An example would be::
272 tahoe add-alias fun URI:DIR2:ovjy4yhylqlfoqg2vcze36dhde:4d4f47qko2xm5g7osgo2yyidi5m4muyo2vjjy53q4vjju2u55mfa
274 This creates an alias ``fun:`` and configures it to use the given directory
275 cap. Once this is done, "``tahoe ls fun:``" will list the contents of this
276 directory. Use "``tahoe add-alias tahoe DIRCAP``" to set the contents of the
277 default ``tahoe:`` alias.
279 ``tahoe create-alias fun``
281 This combines "``tahoe mkdir``" and "``tahoe add-alias``" into a single step.
283 ``tahoe list-aliases``
285 This displays a table of all configured aliases.
289 This creates a new empty unlinked directory, and prints its write-cap to
290 stdout. The new directory is not attached to anything else.
292 ``tahoe mkdir subdir``
294 ``tahoe mkdir /subdir``
296 This creates a new empty directory and attaches it below the root directory
297 of the default ``tahoe:`` alias with the name "``subdir``".
307 All four list the root directory of the default ``tahoe:`` alias.
311 This lists a subdirectory of your filesystem.
315 ``tahoe webopen tahoe:``
317 ``tahoe webopen tahoe:subdir/``
319 ``tahoe webopen subdir/``
321 This uses the python 'webbrowser' module to cause a local web browser to
322 open to the web page for the given directory. This page offers interfaces to
323 add, download, rename, and delete files in the directory. If no alias or
324 path is given, this command opens the root directory of the default
327 ``tahoe put file.txt``
329 ``tahoe put ./file.txt``
331 ``tahoe put /tmp/file.txt``
333 ``tahoe put ~/file.txt``
335 These upload the local file into the grid, and prints the new read-cap to
336 stdout. The uploaded file is not attached to any directory. All one-argument
337 forms of "``tahoe put``" perform an unlinked upload.
343 These also perform an unlinked upload, but the data to be uploaded is taken
346 ``tahoe put file.txt uploaded.txt``
348 ``tahoe put file.txt tahoe:uploaded.txt``
350 These upload the local file and add it to your ``tahoe:`` root with the name
353 ``tahoe put file.txt subdir/foo.txt``
355 ``tahoe put - subdir/foo.txt``
357 ``tahoe put file.txt tahoe:subdir/foo.txt``
359 ``tahoe put file.txt DIRCAP/foo.txt``
361 ``tahoe put file.txt DIRCAP/subdir/foo.txt``
363 These upload the named file and attach them to a subdirectory of the given
364 root directory, under the name "``foo.txt``". When a directory write-cap is
365 given, you can use either ``/`` (as shown above) or ``:./`` to separate it
366 from the following path. When the source file is named "``-``", the contents
367 are taken from stdin.
369 ``tahoe put file.txt --mutable``
371 Create a new mutable file, fill it with the contents of ``file.txt``, and
372 print the new write-cap to stdout.
374 ``tahoe put file.txt MUTABLE-FILE-WRITECAP``
376 Replace the contents of the given mutable file with the contents of ``file.txt``
377 and prints the same write-cap to stdout.
379 ``tahoe cp file.txt tahoe:uploaded.txt``
381 ``tahoe cp file.txt tahoe:``
383 ``tahoe cp file.txt tahoe:/``
385 ``tahoe cp ./file.txt tahoe:``
387 These upload the local file and add it to your ``tahoe:`` root with the name
390 ``tahoe cp tahoe:uploaded.txt downloaded.txt``
392 ``tahoe cp tahoe:uploaded.txt ./downloaded.txt``
394 ``tahoe cp tahoe:uploaded.txt /tmp/downloaded.txt``
396 ``tahoe cp tahoe:uploaded.txt ~/downloaded.txt``
398 This downloads the named file from your ``tahoe:`` root, and puts the result on
399 your local filesystem.
401 ``tahoe cp tahoe:uploaded.txt fun:stuff.txt``
403 This copies a file from your ``tahoe:`` root to a different directory, set up
404 earlier with "``tahoe add-alias fun DIRCAP``" or "``tahoe create-alias fun``".
406 ``tahoe rm uploaded.txt``
408 ``tahoe rm tahoe:uploaded.txt``
410 This deletes a file from your ``tahoe:`` root.
412 ``tahoe mv uploaded.txt renamed.txt``
414 ``tahoe mv tahoe:uploaded.txt tahoe:renamed.txt``
416 These rename a file within your ``tahoe:`` root directory.
418 ``tahoe mv uploaded.txt fun:``
420 ``tahoe mv tahoe:uploaded.txt fun:``
422 ``tahoe mv tahoe:uploaded.txt fun:uploaded.txt``
424 These move a file from your ``tahoe:`` root directory to the directory
425 set up earlier with "``tahoe add-alias fun DIRCAP``" or
426 "``tahoe create-alias fun``".
428 ``tahoe backup ~ work:backups``
430 This command performs a full versioned backup of every file and directory
431 underneath your "``~``" home directory, placing an immutable timestamped
432 snapshot in e.g. ``work:backups/Archives/2009-02-06_04:00:05Z/`` (note that
433 the timestamp is in UTC, hence the "Z" suffix), and a link to the latest
434 snapshot in work:backups/Latest/ . This command uses a small SQLite database
435 known as the "backupdb", stored in ``~/.tahoe/private/backupdb.sqlite``, to
436 remember which local files have been backed up already, and will avoid
437 uploading files that have already been backed up. It compares timestamps and
438 filesizes when making this comparison. It also re-uses existing directories
439 which have identical contents. This lets it run faster and reduces the
440 number of directories created.
442 If you reconfigure your client node to switch to a different grid, you
443 should delete the stale backupdb.sqlite file, to force "``tahoe backup``"
444 to upload all files to the new grid.
446 ``tahoe backup --exclude=*~ ~ work:backups``
448 Same as above, but this time the backup process will ignore any
449 filename that will end with '~'. ``--exclude`` will accept any standard
450 Unix shell-style wildcards, as implemented by the
451 `Python fnmatch module <http://docs.python.org/library/fnmatch.html>`_.
452 You may give multiple ``--exclude`` options. Please pay attention that
453 the pattern will be matched against any level of the directory tree;
454 it's still impossible to specify absolute path exclusions.
456 ``tahoe backup --exclude-from=/path/to/filename ~ work:backups``
458 ``--exclude-from`` is similar to ``--exclude``, but reads exclusion
459 patterns from ``/path/to/filename``, one per line.
461 ``tahoe backup --exclude-vcs ~ work:backups``
463 This command will ignore any file or directory name known to be used by
464 version control systems to store metadata. The excluded names are:
485 Storage Grid Maintenance
486 ========================
488 ``tahoe manifest tahoe:``
490 ``tahoe manifest --storage-index tahoe:``
492 ``tahoe manifest --verify-cap tahoe:``
494 ``tahoe manifest --repair-cap tahoe:``
496 ``tahoe manifest --raw tahoe:``
498 This performs a recursive walk of the given directory, visiting every file
499 and directory that can be reached from that point. It then emits one line to
500 stdout for each object it encounters.
502 The default behavior is to print the access cap string (like ``URI:CHK:..``
503 or ``URI:DIR2:..``), followed by a space, followed by the full path name.
505 If ``--storage-index`` is added, each line will instead contain the object's
506 storage index. This (string) value is useful to determine which share files
507 (on the server) are associated with this directory tree. The ``--verify-cap``
508 and ``--repair-cap`` options are similar, but emit a verify-cap and repair-cap,
509 respectively. If ``--raw`` is provided instead, the output will be a
510 JSON-encoded dictionary that includes keys for pathnames, storage index
511 strings, and cap strings. The last line of the ``--raw`` output will be a JSON
512 encoded deep-stats dictionary.
514 ``tahoe stats tahoe:``
516 This performs a recursive walk of the given directory, visiting every file
517 and directory that can be reached from that point. It gathers statistics on
518 the sizes of the objects it encounters, and prints a summary to stdout.
524 For a list of all debugging commands, use "``tahoe debug``".
526 "``tahoe debug find-shares STORAGEINDEX NODEDIRS..``" will look through one or
527 more storage nodes for the share files that are providing storage for the
530 "``tahoe debug catalog-shares NODEDIRS..``" will look through one or more
531 storage nodes and locate every single share they contain. It produces a report
532 on stdout with one line per share, describing what kind of share it is, the
533 storage index, the size of the file is used for, etc. It may be useful to
534 concatenate these reports from all storage hosts and use it to look for
537 "``tahoe debug dump-share SHAREFILE``" will take the name of a single share file
538 (as found by "``tahoe find-shares``") and print a summary of its contents to
539 stdout. This includes a list of leases, summaries of the hash tree, and
540 information from the UEB (URI Extension Block). For mutable file shares, it
541 will describe which version (seqnum and root-hash) is being stored in this
544 "``tahoe debug dump-cap CAP``" will take any Tahoe-LAFS URI and unpack it
545 into separate pieces. The most useful aspect of this command is to reveal the
546 storage index for any given URI. This can be used to locate the share files
547 that are holding the encoded+encrypted data for this file.
549 "``tahoe debug repl``" will launch an interactive Python interpreter in which
550 the Tahoe-LAFS packages and modules are available on ``sys.path`` (e.g. by using
551 '``import allmydata``'). This is most useful from a source tree: it simply sets
552 the PYTHONPATH correctly and runs the Python executable.
554 "``tahoe debug corrupt-share SHAREFILE``" will flip a bit in the given
555 sharefile. This can be used to test the client-side verification/repair code.
556 Obviously, this command should not be used during normal operation.