From: Daira Hopwood Date: Thu, 2 Apr 2015 15:15:39 +0000 (+0100) Subject: Copyediting. X-Git-Tag: allmydata-tahoe-1.10.1a1~47 X-Git-Url: https://git.rkrishnan.org/(%5B%5E?a=commitdiff_plain;h=8fabcbe54bc557ee7bc6dc8dfa4f2cd72dcb3a6b;p=tahoe-lafs%2Ftahoe-lafs.git Copyediting. Signed-off-by: Daira Hopwood --- diff --git a/docs/proposed/magic-folder/filesystem-integration.rst b/docs/proposed/magic-folder/filesystem-integration.rst index b2642f03..eae12b5e 100644 --- a/docs/proposed/magic-folder/filesystem-integration.rst +++ b/docs/proposed/magic-folder/filesystem-integration.rst @@ -1,10 +1,10 @@ -Magic Folders local filesystem integration design -================================================= +Magic Folder local filesystem integration design +================================================ *Scope* This document describes how to integrate the local filesystem with Magic -Folders in an efficient and reliable manner. For now we ignore Remote to +Folder in an efficient and reliable manner. For now we ignore Remote to Local synchronization; the design and implementation of this is scheduled for a later time. We also ignore multiple writers for the same Magic Folder, which may or may not be supported in future. The design here will @@ -20,15 +20,15 @@ under the local directory and adds every file to a first-in first-out "scan queue". When processing the scan queue, redundant uploads are avoided by using the same mechanism the Tahoe backup command uses: we keep track of previous uploads by recording each file's metadata such as -size, CTIME and MTIME. This information is stored in a database referred -to from now on as the magic folder db. Using this recorded state, we -ensure that when Magic Folder is subsequently started, the local -directory tree can be scanned quickly by comparing current filesystem -metadata with the previously recorded metadata. Each file referenced in -the scan queue is uploaded only if its metadata differs at the time it is -processed. If a change event is detected for a file that is already -queued (and therefore will be processed later), the redundant event is -ignored. +size, ``ctime`` and ``mtime``. This information is stored in a database, +referred to from now on as the magic folder db. Using this recorded +state, we ensure that when Magic Folder is subsequently started, the +local directory tree can be scanned quickly by comparing current +filesystem metadata with the previously recorded metadata. Each file +referenced in the scan queue is uploaded only if its metadata differs at +the time it is processed. If a change event is detected for a file that +is already queued (and therefore will be processed later), the redundant +event is ignored. To implement the magic folder db, we will use an SQLite schema that initially is the existing Tahoe-LAFS backup schema. This schema may @@ -39,31 +39,33 @@ database file and separate mutex lock just for Magic Folder. This avoids usability problems related to mutual exclusion. (If a single file and lock were used, a backup would block Magic Folder updates for a long time, and a user would not be able to tell when backups are possible -because Magic Folder acquires a lock at arbitrary times.) +because Magic Folder would acquire a lock at arbitrary times.) *Eventual consistency property* -It is not possible to prevent local writes to a file while it is being -read in order to upload it. Such writes will result in temporary -inconsistency. Eventual consistency is reached when the queue of pending -uploads is empty. That is, a consistent snapshot will be achieved -eventually when local writes to the target folder cease for a -sufficiently long period of time. +During the process of reading a file in order to upload it, it is not +possible to prevent further local writes. Such writes will result in +temporary inconsistency (that is, the uploaded file will not reflect +what the contents of the local file were at any specific time). Eventual +consistency is reached when the queue of pending uploads is empty. That +is, a consistent snapshot will be achieved eventually when local writes +to the target folder cease for a sufficiently long period of time. *Detecting filesystem changes* -For the Linux implementation we will use the inotify Linux kernel +For the Linux implementation, we will use the inotify Linux kernel subsystem to gather events on the local Magic Folder directory tree. This implementation was already present in Tahoe-LAFS 1.9.0, but needs to be -changed to gather directory creation and move events, as well as events -indicating that a file has been written. +changed to gather directory creation and move events, in addition to the +events indicating that a file has been written that are gathered by the +current code. -For the Windows implementation we will use the ``ReadDirectoryChangesW`` -Win32 API. The prototype implementation simulates the inotify API in -terms of ``ReadDirectoryChangesW``, allowing most of the code to be -shared across platforms. +For the Windows implementation, we will use the ``ReadDirectoryChangesW`` +Win32 API. The prototype implementation simulates a Python interface to +the inotify API in terms of ``ReadDirectoryChangesW``, allowing most of +the code to be shared across platforms. When we detect the creation of a new directory below the local Magic Folder directory, we create it in the Tahoe-LAFS filesystem, and also @@ -75,23 +77,23 @@ is moved to be under the local Magic Folder directory. *User interface* -The Magic Folders local filesystem integration will initially have a +The Magic Folder local filesystem integration will initially have a provisional configuration file-based interface that may not be ideal from a usability perspective. Creating our local filesystem integration in -this manner will allow us to use and test it indepently of the rest of +this manner will allow us to use and test it independently of the rest of the Magic Folder software components. We will focus greater attention on user interface design as a later milestone in our development roadmap. The configuration file, ``tahoe.cfg``, must define a target local directory to be synchronized. Provisionally, this configuration will -replace the current "drop-upload" section:: +replace the current ``[drop_upload]`` section:: - [magic_folders] + [magic_folder] enabled = true local.directory = "/home/human" -When a filesystem directory is first "added" to Magic Folders, the user -needs to create the remote Tahoe-LAFS directory using ``tahoe mkdir``, +When a filesystem directory is first configured for Magic Folder, the user +needs to create the remote Tahoe-LAFS directory using ``tahoe mkdir``, and configure the Magic-Folder-enabled node with its URI (e.g. by putting it in a file ``private/magic_folder_dircap``). If there are existing files in the local directory, they will be uploaded as a result of the