+.. -*- coding: utf-8-with-signature -*-
+
=======================
Tahoe-LAFS Architecture
=======================
Overview
========
-(See the `docs/specifications directory <specifications>`_ for more details.)
+(See the `docs/specifications directory`_ for more details.)
There are three layers: the key-value store, the filesystem, and the
application.
copies files from the local disk onto the decentralized filesystem. We later
provide read-only access to those files, allowing users to recover them.
There are several other applications built on top of the Tahoe-LAFS
-filesystem (see the `RelatedProjects
-<http://tahoe-lafs.org/trac/tahoe-lafs/wiki/RelatedProjects>`_ page of the
-wiki for a list).
+filesystem (see the RelatedProjects_ page of the wiki for a list).
+.. _docs/specifications directory: specifications
+.. _RelatedProjects: https://tahoe-lafs.org/trac/tahoe-lafs/wiki/RelatedProjects
The Key-Value Store
===================
filesystem layer (described below) adds human-meaningful names atop the
key-value layer.
-.. _`Zooko's Triangle`: http://en.wikipedia.org/wiki/Zooko%27s_triangle
+.. _`Zooko's Triangle`: https://en.wikipedia.org/wiki/Zooko%27s_triangle
Server Selection
at the end of the upload process, the appropriate upload health check fails,
the upload is considered a failure.
-The current defaults use ``k``=3, ``servers_of_happiness``=7, and ``N``=10.
-``N``=10 means that we'll try to place 10 shares. ``k``=3 means that we need
-any three shares to recover the file. ``servers_of_happiness``=7 means that
+The current defaults use ``k`` = 3, ``servers_of_happiness`` = 7, and ``N`` = 10.
+``N`` = 10 means that we'll try to place 10 shares. ``k`` = 3 means that we need
+any three shares to recover the file. ``servers_of_happiness`` = 7 means that
we'll consider an immutable file upload to be successful if we can place shares
on enough servers that there are 7 different servers, the correct functioning
of any ``k`` of which guarantee the availability of the immutable file.
-``N``=10 and ``k``=3 means there is a 3.3x expansion factor. On a small grid, you
+``N`` = 10 and ``k`` = 3 means there is a 3.3x expansion factor. On a small grid, you
should set ``N`` about equal to the number of storage servers in your grid; on a
large grid, you might set it to something smaller to avoid the overhead of
contacting every server to place a file. In either case, you should then set ``k``
times longer than downloading it again later.
Smaller expansion ratios can reduce this upload penalty, at the expense of
-reliability (see RELIABILITY, below). By using an "upload helper", this
+reliability (see `Reliability`_, below). By using an "upload helper", this
penalty is eliminated: the client does a 1x upload of encrypted data to the
helper, then the helper performs encoding and pushes the shares to the
storage servers. This is an improvement if the helper has significantly
facility with high interconnect bandwidth. In this case, the helper is placed
in the same facility, so the helper-to-storage-server bandwidth is huge.
-See `<helper.rst>`_ for details about the upload helper.
+See helper.rst_ for details about the upload helper.
+
+.. _helper.rst: helper.rst
The Filesystem Layer
least frequently enough to prevent any of the leases from expiring before the
next renewal pass.
-See `<garbage-collection.rst>`_ for further information, and for how to
-configure garbage collection.
+See garbage-collection.rst_ for further information, and for how to configure
+garbage collection.
+
+.. _garbage-collection.rst: garbage-collection.rst
File Repairer
to periodically count the number of shares still available for any given
file. A more extensive form of checking known as the File Verifier can
download the ciphertext of the target file and perform integrity checks
-(using strong hashes) to make sure the data is stil intact. When the file is
+(using strong hashes) to make sure the data is still intact. When the file is
found to have decayed below some threshold, the File Repairer can be used to
regenerate and re-upload the missing shares. These processes are conceptually
distinct (the repairer is only run if the checker/verifier decides it is
many tradeoffs.
First, some terms: the erasure-coding algorithm is described as ``k``-out-of-``N``
-(for this release, the default values are ``k``=3 and ``N``=10). Each grid will
+(for this release, the default values are ``k`` = 3 and ``N`` = 10). Each grid will
have some number of nodes; this number will rise and fall over time as nodes
join, drop out, come back, and leave forever. Files are of various sizes, some
are popular, others are unpopular. Nodes have various capacities, variable
still retaining high reliability, but large unstable grids (where nodes are
coming and going very quickly) may require more repair/verification bandwidth
than actual upload/download traffic.
-
-Tahoe-LAFS nodes that run a webserver have a page dedicated to provisioning
-decisions: this tool may help you evaluate different expansion factors and
-view the disk consumption of each. It is also acquiring some sections with
-availability/reliability numbers, as well as preliminary cost analysis data.
-This tool will continue to evolve as our analysis improves.
projects / tahoe-lafs / tahoe-lafs.git / blobdiff