1 =================================
2 Tahoe-LAFS FTP and SFTP Frontends
3 =================================
5 1. `FTP/SFTP Background`_
6 2. `Tahoe-LAFS Support`_
7 3. `Creating an Account File`_
8 4. `Configuring FTP Access`_
9 5. `Configuring SFTP Access`_
11 7. `Immutable and mutable files`_
18 FTP is the venerable internet file-transfer protocol, first developed in
19 1971. The FTP server usually listens on port 21. A separate connection is
20 used for the actual data transfers, either in the same direction as the
21 initial client-to-server connection (for PORT mode), or in the reverse
22 direction (for PASV) mode. Connections are unencrypted, so passwords, file
23 names, and file contents are visible to eavesdroppers.
25 SFTP is the modern replacement, developed as part of the SSH "secure shell"
26 protocol, and runs as a subchannel of the regular SSH connection. The SSH
27 server usually listens on port 22. All connections are encrypted.
29 Both FTP and SFTP were developed assuming a UNIX-like server, with accounts
30 and passwords, octal file modes (user/group/other, read/write/execute), and
31 ctime/mtime timestamps.
36 All Tahoe-LAFS client nodes can run a frontend FTP server, allowing regular
37 FTP clients (like /usr/bin/ftp, ncftp, and countless others) to access the
38 virtual filesystem. They can also run an SFTP server, so SFTP clients (like
39 /usr/bin/sftp, the sshfs FUSE plugin, and others) can too. These frontends
40 sit at the same level as the web-API interface.
42 Since Tahoe-LAFS does not use user accounts or passwords, the FTP/SFTP
43 servers must be configured with a way to first authenticate a user (confirm
44 that a prospective client has a legitimate claim to whatever authorities we
45 might grant a particular user), and second to decide what directory cap
46 should be granted to the authenticated username. A username and password is
47 used for this purpose. (The SFTP protocol is also capable of using client RSA
48 or DSA public keys, but this is not currently implemented.)
50 Tahoe-LAFS provides two mechanisms to perform this user-to-cap mapping. The
51 first is a simple flat file with one account per line. The second is an
52 HTTP-based login mechanism, backed by simple PHP script and a database. The
53 latter form is used by allmydata.com to provide secure access to customer
56 Creating an Account File
57 ========================
59 To use the first form, create a file (probably in
60 BASEDIR/private/ftp.accounts) in which each non-comment/non-blank line is a
61 space-separated line of (USERNAME, PASSWORD, ROOTCAP), like so::
63 % cat BASEDIR/private/ftp.accounts
64 # This is a password line, (username, password, cap)
65 alice password URI:DIR2:ioej8xmzrwilg772gzj4fhdg7a:wtiizszzz2rgmczv4wl6bqvbv33ag4kvbr6prz3u6w3geixa6m6a
66 bob sekrit URI:DIR2:6bdmeitystckbl9yqlw7g56f4e:serp5ioqxnh34mlbmzwvkp3odehsyrr7eytt5f64we3k9hhcrcja
68 Future versions of Tahoe-LAFS may support using client public keys for SFTP.
69 The words "ssh-rsa" and "ssh-dsa" after the username are reserved to specify
70 the public key format, so users cannot have a password equal to either of
73 Now add an 'accounts.file' directive to your tahoe.cfg file, as described in
76 accounts.url Directive
77 ======================
79 The accounts.url directive allows access requests to be controlled by an
80 HTTP-based login service, useful for centralized deployments. This was
81 used by AllMyData to provide web-based file access, where the service
82 used a simple PHP script and database lookups to map an account email
83 address and password into a tahoe rootcap. The service will receive a
84 multipart/form-data POST, just like one created with a <form> and <input>
85 fields, with three parameters:
87 * action: "authenticate" (this is a static string, for backwards
88 compatibility with the old AllMyData authentication service)
89 * email: USERNAME (Tahoe has no notion of email addresses, but the
90 authentication service uses them as account names, so the interface
91 presents this argument as "email" rather than "username").
94 And should return a single string that either contains a Tahoe rootcap
95 (URI:DIR2:...), or "0" to indicate a login failure.
97 Tahoe-LAFS recommends the service be secure, preferably localhost-only. This
98 makes it harder for attackers to brute force the password or use DNS
99 poisoning to cause the Tahoe-LAFS gateway to talk with the wrong server,
100 thereby revealing the usernames and passwords.
102 Configuring FTP Access
103 ======================
105 To enable the FTP server with an accounts file, add the following lines to
106 the BASEDIR/tahoe.cfg file::
110 port = tcp:8021:interface=127.0.0.1
111 accounts.file = private/ftp.accounts
113 The FTP server will listen on the given port number and on the loopback
114 interface only. The "accounts.file" pathname will be interpreted relative to
117 To enable the FTP server with an account server instead, provide the URL of
118 that server in an "accounts.url" directive::
122 port = tcp:8021:interface=127.0.0.1
123 accounts.url = https://example.com/login
125 You can provide both accounts.file and accounts.url, although it probably
126 isn't very useful except for testing.
128 FTP provides no security, and so your password or caps could be eavesdropped
129 if you connect to the FTP server remotely. The examples above include
130 ":interface=127.0.0.1" in the "port" option, which causes the server to only
131 accept connections from localhost.
133 Configuring SFTP Access
134 =======================
136 The Tahoe-LAFS SFTP server requires a host keypair, just like the regular SSH
137 server. It is important to give each server a distinct keypair, to prevent
138 one server from masquerading as different one. The first time a client
139 program talks to a given server, it will store the host key it receives, and
140 will complain if a subsequent connection uses a different key. This reduces
141 the opportunity for man-in-the-middle attacks to just the first connection.
143 Exercise caution when connecting to the SFTP server remotely. The AES
144 implementation used by the SFTP code does not have defenses against timing
145 attacks. The code for encrypting the SFTP connection was not written by the
146 Tahoe-LAFS team, and we have not reviewed it as carefully as we have reviewed
147 the code for encrypting files and directories in Tahoe-LAFS itself. If you
148 can connect to the SFTP server (which is provided by the Tahoe-LAFS gateway)
149 only from a client on the same host, then you would be safe from any problem
150 with the SFTP connection security. The examples given below enforce this
151 policy by including ":interface=127.0.0.1" in the "port" option, which causes
152 the server to only accept connections from localhost.
154 You will use directives in the tahoe.cfg file to tell the SFTP code where to
155 find these keys. To create one, use the ``ssh-keygen`` tool (which comes with
156 the standard openssh client distribution)::
159 % ssh-keygen -f private/ssh_host_rsa_key
161 The server private key file must not have a passphrase.
163 Then, to enable the SFTP server with an accounts file, add the following
164 lines to the BASEDIR/tahoe.cfg file::
168 port = tcp:8022:interface=127.0.0.1
169 host_pubkey_file = private/ssh_host_rsa_key.pub
170 host_privkey_file = private/ssh_host_rsa_key
171 accounts.file = private/ftp.accounts
173 The SFTP server will listen on the given port number and on the loopback
174 interface only. The "accounts.file" pathname will be interpreted relative to
177 Or, to use an account server instead, do this::
181 port = tcp:8022:interface=127.0.0.1
182 host_pubkey_file = private/ssh_host_rsa_key.pub
183 host_privkey_file = private/ssh_host_rsa_key
184 accounts.url = https://example.com/login
186 You can provide both accounts.file and accounts.url, although it probably
187 isn't very useful except for testing.
189 For further information on SFTP compatibility and known issues with various
190 clients and with the sshfs filesystem, see
191 `<http://tahoe-lafs.org/trac/tahoe-lafs/wiki/SftpFrontend>`_.
196 The Tahoe-LAFS SFTP server requires the Twisted "Conch" component (a "conch"
197 is a twisted shell, get it?). Many Linux distributions package the Conch code
198 separately: debian puts it in the "python-twisted-conch" package. Conch
199 requires the "pycrypto" package, which is a Python+C implementation of many
200 cryptographic functions (the debian package is named "python-crypto").
202 Note that "pycrypto" is different than the "pycryptopp" package that
203 Tahoe-LAFS uses (which is a Python wrapper around the C++ -based Crypto++
204 library, a library that is frequently installed as /usr/lib/libcryptopp.a, to
205 avoid problems with non-alphanumerics in filenames).
207 Immutable and Mutable Files
208 ===========================
210 All files created via SFTP (and FTP) are immutable files. However, files can
211 only be created in writeable directories, which allows the directory entry to
212 be relinked to a different file. Normally, when the path of an immutable file
213 is opened for writing by SFTP, the directory entry is relinked to another
214 file with the newly written contents when the file handle is closed. The old
215 file is still present on the grid, and any other caps to it will remain
216 valid. (See `docs/garbage-collection.rst <../garbage-collection.rst>`_ for
217 how to reclaim the space used by files that are no longer needed.)
219 The 'no-write' metadata field of a directory entry can override this
220 behaviour. If the 'no-write' field holds a true value, then a permission
221 error will occur when trying to write to the file, even if it is in a
222 writeable directory. This does not prevent the directory entry from being
223 unlinked or replaced.
225 When using sshfs, the 'no-write' field can be set by clearing the 'w' bits in
226 the Unix permissions, for example using the command 'chmod 444
227 path/to/file'. Note that this does not mean that arbitrary combinations of
228 Unix permissions are supported. If the 'w' bits are cleared on a link to a
229 mutable file or directory, that link will become read-only.
231 If SFTP is used to write to an existing mutable file, it will publish a new
232 version when the file handle is closed.
237 Mutable files are not supported by the FTP frontend (`ticket #680
238 <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/680>`_). Currently, a directory
239 containing mutable files cannot even be listed over FTP.
241 The FTP frontend sometimes fails to report errors, for example if an upload
242 fails because it does meet the "servers of happiness" threshold (`ticket
243 #1081 <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1081>`_). Upload errors
244 also may not be reported when writing files using SFTP via sshfs (`ticket
245 #1059 <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1059>`_).
247 Non-ASCII filenames are not supported by FTP (`ticket #682
248 <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/682>`_). They can be used with
249 SFTP only if the client encodes filenames as UTF-8 (`ticket #1089
250 <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1089>`_).
252 The gateway node may hang or consume 100% CPU if the client tries to rekey.
253 (`ticket #1297 <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1297>`_). This
254 is due to `a bug in Twisted <http://twistedmatrix.com/trac/ticket/4395>`_
255 which was fixed in Twisted 11.0 (released 3-April-2011).
257 For options to disable rekeying in various clients in order to work around
258 this issue, and for other known issues in SFTP, see
259 `<http://tahoe-lafs.org/trac/tahoe-lafs/wiki/SftpFrontend>`_.