From: Daira Hopwood Date: Tue, 8 Apr 2014 23:43:30 +0000 (+0100) Subject: Add documentation for each storage backend (rebased). X-Git-Url: https://git.rkrishnan.org/pf/content/simplejson/running.html?a=commitdiff_plain;h=c24a0b8270028fb574916df0ed2417e4bf39cc1a;p=tahoe-lafs%2Ftahoe-lafs.git Add documentation for each storage backend (rebased). Signed-off-by: Daira Hopwood --- diff --git a/docs/backends/cloud.rst b/docs/backends/cloud.rst new file mode 100644 index 00000000..e6e43db8 --- /dev/null +++ b/docs/backends/cloud.rst @@ -0,0 +1,59 @@ +================================ +Storing Shares on Cloud Services +================================ + +The Tahoe-LAFS storage server can be configured to store its shares on a +cloud storage service, rather than on the local filesystem. + + +Amazon Simple Storage Service (S3) +================================== + +S3 is a commercial storage service provided by Amazon, described at +``__. + +To enable storing shares on S3, add the following keys to the server's +``tahoe.cfg`` file: + +``[storage]`` + +``backend = cloud.s3`` + + This turns off the local filesystem backend and enables use of S3. + +``s3.access_key_id = (string, required)`` + + This identifies your Amazon Web Services access key. The access key id is + not secret, but there is a secret key associated with it. The secret key + is stored in a separate file named ``private/s3secret``. + +``s3.bucket = (string, required)`` + + This controls which bucket will be used to hold shares. The Tahoe-LAFS + storage server will only modify and access objects in the configured S3 + bucket. Multiple storage servers cannot share the same bucket. + +``s3.url = (URL string, optional)`` + + This URL tells the storage server how to access the S3 service. It + defaults to ``http://s3.amazonaws.com``, but by setting it to something + else, you may be able to use some other S3-like service if it is + sufficiently compatible. + +The system time of the storage server must be correct to within 15 minutes +in order for S3 to accept the authentication provided with requests. + + +DevPay +------ + +Optionally, Amazon `DevPay`_ may be used to delegate billing for a service +based on Tahoe-LAFS and S3 to Amazon Payments. + +If DevPay is to be used, the user token and product token (in base64 form) +must be stored in the files ``private/s3usertoken`` and ``private/s3producttoken`` +respectively. DevPay-related request headers will be sent only if these files +are present when the server is started. It is currently assumed that only one +user and product token pair is needed by a given storage server. + +.. _DevPay: http://docs.amazonwebservices.com/AmazonDevPay/latest/DevPayGettingStartedGuide/ diff --git a/docs/backends/disk.rst b/docs/backends/disk.rst new file mode 100644 index 00000000..c4e54c67 --- /dev/null +++ b/docs/backends/disk.rst @@ -0,0 +1,43 @@ +==================================== +Storing Shares on a Local Filesystem +==================================== + +The "disk" backend stores shares on the local filesystem. Versions of +Tahoe-LAFS before v1.11.0 always stored shares in this way. + +``[storage]`` + +``backend = disk`` + + This enables use of the disk backend, and is the default. + +``readonly = (boolean, optional)`` + + If ``True``, the node will run a storage server but will not accept any + shares, making it effectively read-only. Use this for storage servers + that are being decommissioned: the ``storage/`` directory could be + mounted read-only, while shares are moved to other servers. Note that + this currently only affects immutable shares. Mutable shares will be + written and modified anyway. See ticket `#390 + `__ for the current + status of this bug. The default value is ``False``. + +``reserved_space = (quantity of space, optional)`` + + If provided, this value defines how much disk space is reserved: the + storage server will not accept any share that causes the amount of free + disk space to drop below this value. (The free space is measured by a + call to ``statvfs(2)`` on Unix, or ``GetDiskFreeSpaceEx`` on Windows, and + is the space available to the user account under which the storage server + runs.) + + This string contains a number, with an optional case-insensitive scale + suffix, optionally followed by "B" or "iB". The supported scale suffixes + are "K", "M", "G", "T", "P" and "E", and a following "i" indicates to use + powers of 1024 rather than 1000. So "100MB", "100 M", "100000000B", + "100000000", and "100000kb" all mean the same thing. Likewise, "1MiB", + "1024KiB", "1024 Ki", and "1048576 B" all mean the same thing. + + "``tahoe create-node``" generates a tahoe.cfg with + "``reserved_space=1G``", but you may wish to raise, lower, or remove the + reservation to suit your needs. diff --git a/docs/configuration.rst b/docs/configuration.rst index bab80f19..7c7bdbc7 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -4,7 +4,7 @@ Configuring a Tahoe-LAFS node ============================= -1. `Node Types`_ +1. `Node Types`_ 2. `Overall Node Configuration`_ 3. `Client Configuration`_ 4. `Storage Server Configuration`_ @@ -443,35 +443,30 @@ Storage Server Configuration for clients who do not wish to provide storage service. The default value is ``True``. -``readonly = (boolean, optional)`` +``backend = (string, optional)`` - If ``True``, the node will run a storage server but will not accept any - shares, making it effectively read-only. Use this for storage servers - that are being decommissioned: the ``storage/`` directory could be - mounted read-only, while shares are moved to other servers. Note that - this currently only affects immutable shares. Mutable shares (used for - directories) will be written and modified anyway. See ticket `#390`_ for - the current status of this bug. The default value is ``False``. + Storage servers can store the data into different "backends". Clients + need not be aware of which backend is used by a server. The default + value is ``disk``. -``reserved_space = (str, optional)`` +``backend = disk`` - If provided, this value defines how much disk space is reserved: the - storage server will not accept any share that causes the amount of free - disk space to drop below this value. (The free space is measured by a - call to ``statvfs(2)`` on Unix, or ``GetDiskFreeSpaceEx`` on Windows, and - is the space available to the user account under which the storage server - runs.) + The storage server stores shares on the local filesystem (in + BASEDIR/storage/shares/). For configuration details (including how to + reserve a minimum amount of free space), see ``__. - This string contains a number, with an optional case-insensitive scale - suffix, optionally followed by "B" or "iB". The supported scale suffixes - are "K", "M", "G", "T", "P" and "E", and a following "i" indicates to use - powers of 1024 rather than 1000. So "100MB", "100 M", "100000000B", - "100000000", and "100000kb" all mean the same thing. Likewise, "1MiB", - "1024KiB", "1024 Ki", and "1048576 B" all mean the same thing. +``backend = cloud.`` - "``tahoe create-node``" generates a tahoe.cfg with - "``reserved_space=1G``", but you may wish to raise, lower, or remove the - reservation to suit your needs. + The storage server stores all shares to the cloud service specified + by . For supported services and configuration details, see + ``__. For backward compatibility, ``backend = s3`` + is equivalent to ``backend = cloud.s3``. + +``backend = debug_discard`` + + The storage server stores all shares in /dev/null. This is actually used, + for testing. It is not recommended for storage of data that you might + want to retrieve in the future. ``expire.enabled =`` @@ -483,10 +478,7 @@ Storage Server Configuration These settings control garbage collection, in which the server will delete shares that no longer have an up-to-date lease on them. Please see - garbage-collection.rst_ for full details. - -.. _#390: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/390 -.. _garbage-collection.rst: garbage-collection.rst + ``__ for full details. Running A Helper