From 7fced57aff2795726395ff2a99b6c49504cd9676 Mon Sep 17 00:00:00 2001 From: Daira Hopwood Date: Wed, 9 Apr 2014 00:43:30 +0100 Subject: [PATCH] Add documentation for each storage backend (rebased). Signed-off-by: Daira Hopwood --- docs/backends/cloud.rst | 59 +++++++++++++++++++++++++++++++++++++++++ docs/backends/disk.rst | 43 ++++++++++++++++++++++++++++++ docs/configuration.rst | 58 +++++++++++++++++----------------------- 3 files changed, 127 insertions(+), 33 deletions(-) create mode 100644 docs/backends/cloud.rst create mode 100644 docs/backends/disk.rst diff --git a/docs/backends/cloud.rst b/docs/backends/cloud.rst new file mode 100644 index 00000000..e6e43db8 --- /dev/null +++ b/docs/backends/cloud.rst @@ -0,0 +1,59 @@ +================================ +Storing Shares on Cloud Services +================================ + +The Tahoe-LAFS storage server can be configured to store its shares on a +cloud storage service, rather than on the local filesystem. + + +Amazon Simple Storage Service (S3) +================================== + +S3 is a commercial storage service provided by Amazon, described at +``__. + +To enable storing shares on S3, add the following keys to the server's +``tahoe.cfg`` file: + +``[storage]`` + +``backend = cloud.s3`` + + This turns off the local filesystem backend and enables use of S3. + +``s3.access_key_id = (string, required)`` + + This identifies your Amazon Web Services access key. The access key id is + not secret, but there is a secret key associated with it. The secret key + is stored in a separate file named ``private/s3secret``. + +``s3.bucket = (string, required)`` + + This controls which bucket will be used to hold shares. The Tahoe-LAFS + storage server will only modify and access objects in the configured S3 + bucket. Multiple storage servers cannot share the same bucket. + +``s3.url = (URL string, optional)`` + + This URL tells the storage server how to access the S3 service. It + defaults to ``http://s3.amazonaws.com``, but by setting it to something + else, you may be able to use some other S3-like service if it is + sufficiently compatible. + +The system time of the storage server must be correct to within 15 minutes +in order for S3 to accept the authentication provided with requests. + + +DevPay +------ + +Optionally, Amazon `DevPay`_ may be used to delegate billing for a service +based on Tahoe-LAFS and S3 to Amazon Payments. + +If DevPay is to be used, the user token and product token (in base64 form) +must be stored in the files ``private/s3usertoken`` and ``private/s3producttoken`` +respectively. DevPay-related request headers will be sent only if these files +are present when the server is started. It is currently assumed that only one +user and product token pair is needed by a given storage server. + +.. _DevPay: http://docs.amazonwebservices.com/AmazonDevPay/latest/DevPayGettingStartedGuide/ diff --git a/docs/backends/disk.rst b/docs/backends/disk.rst new file mode 100644 index 00000000..c4e54c67 --- /dev/null +++ b/docs/backends/disk.rst @@ -0,0 +1,43 @@ +==================================== +Storing Shares on a Local Filesystem +==================================== + +The "disk" backend stores shares on the local filesystem. Versions of +Tahoe-LAFS before v1.11.0 always stored shares in this way. + +``[storage]`` + +``backend = disk`` + + This enables use of the disk backend, and is the default. + +``readonly = (boolean, optional)`` + + If ``True``, the node will run a storage server but will not accept any + shares, making it effectively read-only. Use this for storage servers + that are being decommissioned: the ``storage/`` directory could be + mounted read-only, while shares are moved to other servers. Note that + this currently only affects immutable shares. Mutable shares will be + written and modified anyway. See ticket `#390 + `__ for the current + status of this bug. The default value is ``False``. + +``reserved_space = (quantity of space, optional)`` + + If provided, this value defines how much disk space is reserved: the + storage server will not accept any share that causes the amount of free + disk space to drop below this value. (The free space is measured by a + call to ``statvfs(2)`` on Unix, or ``GetDiskFreeSpaceEx`` on Windows, and + is the space available to the user account under which the storage server + runs.) + + This string contains a number, with an optional case-insensitive scale + suffix, optionally followed by "B" or "iB". The supported scale suffixes + are "K", "M", "G", "T", "P" and "E", and a following "i" indicates to use + powers of 1024 rather than 1000. So "100MB", "100 M", "100000000B", + "100000000", and "100000kb" all mean the same thing. Likewise, "1MiB", + "1024KiB", "1024 Ki", and "1048576 B" all mean the same thing. + + "``tahoe create-node``" generates a tahoe.cfg with + "``reserved_space=1G``", but you may wish to raise, lower, or remove the + reservation to suit your needs. diff --git a/docs/configuration.rst b/docs/configuration.rst index 7e47f4a2..112f436e 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -453,35 +453,30 @@ Storage Server Configuration for clients who do not wish to provide storage service. The default value is ``True``. -``readonly = (boolean, optional)`` - - If ``True``, the node will run a storage server but will not accept any - shares, making it effectively read-only. Use this for storage servers - that are being decommissioned: the ``storage/`` directory could be - mounted read-only, while shares are moved to other servers. Note that - this currently only affects immutable shares. Mutable shares (used for - directories) will be written and modified anyway. See ticket `#390`_ for - the current status of this bug. The default value is ``False``. - -``reserved_space = (str, optional)`` - - If provided, this value defines how much disk space is reserved: the - storage server will not accept any share that causes the amount of free - disk space to drop below this value. (The free space is measured by a - call to ``statvfs(2)`` on Unix, or ``GetDiskFreeSpaceEx`` on Windows, and - is the space available to the user account under which the storage server - runs.) - - This string contains a number, with an optional case-insensitive scale - suffix, optionally followed by "B" or "iB". The supported scale suffixes - are "K", "M", "G", "T", "P" and "E", and a following "i" indicates to use - powers of 1024 rather than 1000. So "100MB", "100 M", "100000000B", - "100000000", and "100000kb" all mean the same thing. Likewise, "1MiB", - "1024KiB", "1024 Ki", and "1048576 B" all mean the same thing. - - "``tahoe create-node``" generates a tahoe.cfg with - "``reserved_space=1G``", but you may wish to raise, lower, or remove the - reservation to suit your needs. +``backend = (string, optional)`` + + Storage servers can store the data into different "backends". Clients + need not be aware of which backend is used by a server. The default + value is ``disk``. + +``backend = disk`` + + The storage server stores shares on the local filesystem (in + BASEDIR/storage/shares/). For configuration details (including how to + reserve a minimum amount of free space), see ``__. + +``backend = cloud.`` + + The storage server stores all shares to the cloud service specified + by . For supported services and configuration details, see + ``__. For backward compatibility, ``backend = s3`` + is equivalent to ``backend = cloud.s3``. + +``backend = debug_discard`` + + The storage server stores all shares in /dev/null. This is actually used, + for testing. It is not recommended for storage of data that you might + want to retrieve in the future. ``expire.enabled =`` @@ -493,10 +488,7 @@ Storage Server Configuration These settings control garbage collection, in which the server will delete shares that no longer have an up-to-date lease on them. Please see - garbage-collection.rst_ for full details. - -.. _#390: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/390 -.. _garbage-collection.rst: garbage-collection.rst + ``__ for full details. Running A Helper -- 2.45.2