3 =================================
4 Tahoe-LAFS SFTP and FTP Frontends
5 =================================
7 1. `SFTP/FTP Background`_
8 2. `Tahoe-LAFS Support`_
9 3. `Creating an Account File`_
10 4. `Running An Account Server (accounts.url)`_
11 5. `Configuring SFTP Access`_
12 6. `Configuring FTP Access`_
14 8. `Immutable and Mutable Files`_
21 FTP is the venerable internet file-transfer protocol, first developed in
22 1971. The FTP server usually listens on port 21. A separate connection is
23 used for the actual data transfers, either in the same direction as the
24 initial client-to-server connection (for PORT mode), or in the reverse
25 direction (for PASV) mode. Connections are unencrypted, so passwords, file
26 names, and file contents are visible to eavesdroppers.
28 SFTP is the modern replacement, developed as part of the SSH "secure shell"
29 protocol, and runs as a subchannel of the regular SSH connection. The SSH
30 server usually listens on port 22. All connections are encrypted.
32 Both FTP and SFTP were developed assuming a UNIX-like server, with accounts
33 and passwords, octal file modes (user/group/other, read/write/execute), and
34 ctime/mtime timestamps.
36 We recommend SFTP over FTP, because the protocol is better, and the server
37 implementation in Tahoe-LAFS is more complete. See `Known Issues`_, below,
43 All Tahoe-LAFS client nodes can run a frontend SFTP server, allowing regular
44 SFTP clients (like ``/usr/bin/sftp``, the ``sshfs`` FUSE plugin, and many
45 others) to access the virtual filesystem. They can also run an FTP server,
46 so FTP clients (like ``/usr/bin/ftp``, ``ncftp``, and others) can too. These
47 frontends sit at the same level as the web-API interface.
49 Since Tahoe-LAFS does not use user accounts or passwords, the SFTP/FTP
50 servers must be configured with a way to first authenticate a user (confirm
51 that a prospective client has a legitimate claim to whatever authorities we
52 might grant a particular user), and second to decide what directory cap
53 should be used as the root directory for a log-in by the authenticated user.
54 A username and password is used for this purpose. (The SFTP protocol is also
55 capable of using client RSA or DSA public keys, but this is not currently
56 implemented in Tahoe-LAFS.)
58 Tahoe-LAFS provides two mechanisms to perform this user-to-cap mapping. The
59 first is a simple flat file with one account per line. The second is an
60 HTTP-based login mechanism, backed by simple PHP script and a database. The
61 latter form is used by allmydata.com to provide secure access to customer
64 Creating an Account File
65 ========================
67 To use the first form, create a file (for example ``BASEDIR/private/accounts``)
68 in which each non-comment/non-blank line is a space-separated line of
69 (USERNAME, PASSWORD, ROOTCAP), like so::
71 % cat BASEDIR/private/accounts
72 # This is a password line, (username, password, cap)
73 alice password URI:DIR2:ioej8xmzrwilg772gzj4fhdg7a:wtiizszzz2rgmczv4wl6bqvbv33ag4kvbr6prz3u6w3geixa6m6a
74 bob sekrit URI:DIR2:6bdmeitystckbl9yqlw7g56f4e:serp5ioqxnh34mlbmzwvkp3odehsyrr7eytt5f64we3k9hhcrcja
76 Future versions of Tahoe-LAFS may support using client public keys for SFTP.
77 The words "ssh-rsa" and "ssh-dsa" after the username are reserved to specify
78 the public key format, so users cannot have a password equal to either of
81 Now add an ``accounts.file`` directive to your ``tahoe.cfg`` file, as described in
84 Running An Account Server (accounts.url)
85 ========================================
87 The accounts.url directive allows access requests to be controlled by an
88 HTTP-based login service, useful for centralized deployments. This was used
89 by AllMyData to provide web-based file access, where the service used a
90 simple PHP script and database lookups to map an account email address and
91 password to a Tahoe-LAFS directory cap. The service will receive a
92 multipart/form-data POST, just like one created with a <form> and <input>
93 fields, with three parameters:
95 • action: "authenticate" (this is a static string, for backwards
96 compatibility with the old AllMyData authentication service)
97 • email: USERNAME (Tahoe-LAFS has no notion of email addresses, but the
98 authentication service uses them as account names, so the interface
99 presents this argument as "email" rather than "username").
102 It should return a single string that either contains a Tahoe-LAFS directory
103 cap (URI:DIR2:...), or "0" to indicate a login failure.
105 Tahoe-LAFS recommends the service be secure, preferably localhost-only. This
106 makes it harder for attackers to brute force the password or use DNS
107 poisoning to cause the Tahoe-LAFS gateway to talk with the wrong server,
108 thereby revealing the usernames and passwords.
110 Configuring SFTP Access
111 =======================
113 The Tahoe-LAFS SFTP server requires a host keypair, just like the regular SSH
114 server. It is important to give each server a distinct keypair, to prevent
115 one server from masquerading as different one. The first time a client
116 program talks to a given server, it will store the host key it receives, and
117 will complain if a subsequent connection uses a different key. This reduces
118 the opportunity for man-in-the-middle attacks to just the first connection.
120 Exercise caution when connecting to the SFTP server remotely. The AES
121 implementation used by the SFTP code does not have defenses against timing
122 attacks. The code for encrypting the SFTP connection was not written by the
123 Tahoe-LAFS team, and we have not reviewed it as carefully as we have reviewed
124 the code for encrypting files and directories in Tahoe-LAFS itself. If you
125 can connect to the SFTP server (which is provided by the Tahoe-LAFS gateway)
126 only from a client on the same host, then you would be safe from any problem
127 with the SFTP connection security. The examples given below enforce this
128 policy by including ":interface=127.0.0.1" in the "port" option, which causes
129 the server to only accept connections from localhost.
131 You will use directives in the tahoe.cfg file to tell the SFTP code where to
132 find these keys. To create one, use the ``ssh-keygen`` tool (which comes with
133 the standard OpenSSH client distribution)::
136 % ssh-keygen -f private/ssh_host_rsa_key
138 The server private key file must not have a passphrase.
140 Then, to enable the SFTP server with an accounts file, add the following
141 lines to the BASEDIR/tahoe.cfg file::
145 port = tcp:8022:interface=127.0.0.1
146 host_pubkey_file = private/ssh_host_rsa_key.pub
147 host_privkey_file = private/ssh_host_rsa_key
148 accounts.file = private/accounts
150 The SFTP server will listen on the given port number and on the loopback
151 interface only. The "accounts.file" pathname will be interpreted relative to
154 Or, to use an account server instead, do this::
158 port = tcp:8022:interface=127.0.0.1
159 host_pubkey_file = private/ssh_host_rsa_key.pub
160 host_privkey_file = private/ssh_host_rsa_key
161 accounts.url = https://example.com/login
163 You can provide both accounts.file and accounts.url, although it probably
164 isn't very useful except for testing.
166 For further information on SFTP compatibility and known issues with various
167 clients and with the sshfs filesystem, see wiki:SftpFrontend_
169 .. _wiki:SftpFrontend: https://tahoe-lafs.org/trac/tahoe-lafs/wiki/SftpFrontend
171 Configuring FTP Access
172 ======================
174 To enable the FTP server with an accounts file, add the following lines to
175 the BASEDIR/tahoe.cfg file::
179 port = tcp:8021:interface=127.0.0.1
180 accounts.file = private/accounts
182 The FTP server will listen on the given port number and on the loopback
183 interface only. The "accounts.file" pathname will be interpreted relative to
186 To enable the FTP server with an account server instead, provide the URL of
187 that server in an "accounts.url" directive::
191 port = tcp:8021:interface=127.0.0.1
192 accounts.url = https://example.com/login
194 You can provide both accounts.file and accounts.url, although it probably
195 isn't very useful except for testing.
197 FTP provides no security, and so your password or caps could be eavesdropped
198 if you connect to the FTP server remotely. The examples above include
199 ":interface=127.0.0.1" in the "port" option, which causes the server to only
200 accept connections from localhost.
205 The Tahoe-LAFS SFTP server requires the Twisted "Conch" component (a "conch"
206 is a twisted shell, get it?). Many Linux distributions package the Conch code
207 separately: debian puts it in the "python-twisted-conch" package. Conch
208 requires the "pycrypto" package, which is a Python+C implementation of many
209 cryptographic functions (the debian package is named "python-crypto").
211 Note that "pycrypto" is different than the "pycryptopp" package that
212 Tahoe-LAFS uses (which is a Python wrapper around the C++ -based Crypto++
213 library, a library that is frequently installed as /usr/lib/libcryptopp.a, to
214 avoid problems with non-alphanumerics in filenames).
216 Immutable and Mutable Files
217 ===========================
219 All files created via SFTP (and FTP) are immutable files. However, files can
220 only be created in writeable directories, which allows the directory entry to
221 be relinked to a different file. Normally, when the path of an immutable file
222 is opened for writing by SFTP, the directory entry is relinked to another
223 file with the newly written contents when the file handle is closed. The old
224 file is still present on the grid, and any other caps to it will remain
225 valid. (See `docs/garbage-collection.rst`_ for how to reclaim the space used
226 by files that are no longer needed.)
228 The 'no-write' metadata field of a directory entry can override this
229 behaviour. If the 'no-write' field holds a true value, then a permission
230 error will occur when trying to write to the file, even if it is in a
231 writeable directory. This does not prevent the directory entry from being
232 unlinked or replaced.
234 When using sshfs, the 'no-write' field can be set by clearing the 'w' bits in
235 the Unix permissions, for example using the command ``chmod 444 path/to/file``.
236 Note that this does not mean that arbitrary combinations of Unix permissions
237 are supported. If the 'w' bits are cleared on a link to a mutable file or
238 directory, that link will become read-only.
240 If SFTP is used to write to an existing mutable file, it will publish a new
241 version when the file handle is closed.
243 .. _docs/garbage-collection.rst: file:../garbage-collection.rst
248 Known Issues in the SFTP Frontend
249 ---------------------------------
251 Upload errors may not be reported when writing files using SFTP via sshfs
254 Non-ASCII filenames are supported with SFTP only if the client encodes
255 filenames as UTF-8 (`ticket #1089`_).
257 The gateway node may hang or consume 100% CPU if the client tries to rekey.
258 (`ticket #1297`_). This is due to a bug in Twisted (`Twisted ticket #4395`_)
259 which was fixed in Twisted 11.0 (released 3-April-2011).
261 See also wiki:SftpFrontend_.
263 .. _ticket #1059: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1059
264 .. _ticket #1089: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1089
265 .. _ticket #1297: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1297
266 .. _Twisted ticket #4395: https://twistedmatrix.com/trac/ticket/4395
268 Known Issues in the FTP Frontend
269 --------------------------------
271 Mutable files are not supported by the FTP frontend (`ticket
272 #680`_). Currently, a directory containing mutable files cannot even be
275 Non-ASCII filenames are not supported by FTP (`ticket #682`_).
277 The FTP frontend returns all timestamps as being Jan 1, 1970 (`ticket
280 The FTP frontend sometimes fails to report errors, for example if an upload
281 fails because it does meet the "servers of happiness" threshold (`ticket
284 .. _ticket #680: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/680
285 .. _ticket #682: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/682
286 .. _ticket #1081: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1081
287 .. _ticket #1688: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1688