From: Brian Warner Date: Tue, 24 Mar 2009 01:54:57 +0000 (-0700) Subject: docs/proposed: move old accounting docs out of the way X-Git-Tag: allmydata-tahoe-1.4.0~34 X-Git-Url: https://git.rkrishnan.org/pf/content/en/seg/?a=commitdiff_plain;h=5f2f95a51e74222f05a84b6b7028aecbd3f90c1a;p=tahoe-lafs%2Ftahoe-lafs.git docs/proposed: move old accounting docs out of the way --- diff --git a/docs/proposed/accounts-introducer.txt b/docs/proposed/accounts-introducer.txt deleted file mode 100644 index 36a5a56f..00000000 --- a/docs/proposed/accounts-introducer.txt +++ /dev/null @@ -1,134 +0,0 @@ -This is a proposal for handing accounts and quotas in Tahoe. Nothing is final -yet.. we are still evaluating the options. - - -= Account Management: Introducer-based = - -A Tahoe grid can be configured in several different modes. The simplest mode -(which is also the default) is completely permissive: all storage servers -will accept shares from all clients, and no attempt is made to keep track of -who is storing what. Access to the grid is mostly equivalent to having access -to the Introducer (or convincing one of the existing members to give you a -list of all their storage server FURLs). - -This mode, while a good starting point, does not accomodate any sort of -auditing or quota management. Even in a small friendnet, operators might like -to know how much of their storage space is being consumed by Alice, so they -might be able to ask her to cut back when overall disk usage is getting to -high. In a larger commercial deployment, a service provider needs to be able -to get accurate usage numbers so they can bill the user appropriately. In -addition, the operator may want the ability to delete all of Bob's shares -(i.e. cancel any outstanding leases) when he terminates his account. - -There are several lease-management/garbage-collection/deletion strategies -possible for a Tahoe grid, but the most efficient ones require knowledge of -lease ownership, so that renewals and expiration can take place on a -per-account basis rather than a (more numerous) per-share basis. - -== Accounts == - -To accomplish this, "Accounts" can be established in a Tahoe grid. There is -nominally one account per human user of the grid, but of course a user might -use multiple accounts, or an account might be shared between multiple users. -The Account is the smallest unit of quota and lease management. - -Accounts are created by an "Account Manager". In a commercial network there -will be just one (centralized) account manager, and all storage nodes will be -configured to require a valid account before providing storage services. In a -friendnet, each peer can run their own account manager, and servers will -accept accounts from any of the managers (this mode is permissive but allows -quota-tracking of non-malicious users). - -The account manager is free to manage the accounts as it pleases. Large -systems will probably use a database to correlate things like username, -storage consumed, billing status, etc. - -== Overview == - -The Account Manager ("AM") replaces the normal Introducer node: grids which -use an Account Manager will not run an Introducer, and the participating -nodes will not be configured with an "introducer.furl". - -Instead, each client will be configured with a different "account.furl", -which gives that client access to a specific account. These account FURLs -point to an object inside the Account Manager which exists solely for the -benefit of that one account. When the client needs access to storage servers, -it will use this account object to acquire personalized introductions to a -per-account "Personal Storage Server" facet, one per storage server node. For -example, Alice would wind up with PSS[1A] on server 1, and PSS[2A] on server -2. Bob would get PSS[1B] and PSS[2B]. - -These PSS facets provide the same remote methods as the old generic SS facet, -except that every time they create a lease object, the account information of -the holder is recorded in that lease. The client stores a list of these PSS -facet FURLs in persistent storage, and uses them in the "get_permuted_peers" -function that all uploads and downloads use to figure out who to talk to when -looking for shares or shareholders. - -Each Storage Server has a private facet that it gives to the Account Manager. -This facet allows the AM to create PSS facets for a specific account. In -particular, the AM tells the SS "please create account number 42, and tell me -the PSS FURL that I should give to the client". The SS creates an object -which remembers the account number, creates a FURL for it, and returns the -FURL. - -If there is a single central account manager, then account numbers can be -small integers. (if there are multiple ones, they need to be large random -strings to ensure uniqueness). To avoid requiring large (accounts*servers) -lookup tables, a given account should use the same identifer for all the -servers it talks to. When this can be done, the PSS and Account FURLs are -generated as MAC'ed copies of the account number. - -More specifically, the PSS FURL is a MAC'ed copy of the account number: each -SS has a private secret "S", and it creates a string "%d-%s" % (accountnum, -b2a(hash(S+accountnum))) to use as the swissnum part of the FURL. The SS uses -tub.registerNameLookupHandler to add a function that tries to validate -inbound FURLs against this scheme: if successful, it creates a new PSS object -with the account number stashed inside. This allows the server to minimize -their per-user storage requirements but still insure that PSS FURLs are -unguessable. - -Account FURLs are created by the Account Manager in a similar fashion, using -a MAC of the account number. The Account Manager can use the same account -number to index other information in a database, like account status, billing -status, etc. - -The mechanism by which Account FURLs are minted is left up to the account -manager, but the simple AM that the 'tahoe create-account-manager' command -makes has a "new-account" FURL which accepts a username and creates an -account for them. The 'tahoe create-account' command is a CLI frontend to -this facility. In a friendnet, you could publish this FURL to your friends, -allowing everyone to make their own account. In a commercial grid, this -facility would be reserved use by the same code which handles billing. - - -== Creating the Account Manager == - -The 'tahoe create-account-manager' command is used to create a simple account -manager node. When started, this node will write several FURLs to its -private/ directory, some of which should be provided to other services. - - * new-account.furl : this FURL allows the holder to create new accounts - * manage-accounts.furl : this FURL allows the holder to list and modify - all existing accounts - * serverdesk.furl : this FURL is used by storage servers to make themselves - available to all account holders - - -== Configuring the Storage Servers == - -To use an account manager, each storage server node should be given access to -the AM's serverdesk (by simply copying "serverdesk.furl" into the storage -server's base directory). In addition, it should *not* be given an -introducer.furl . The serverdesk FURL tells the SS that it should allow the -AM to create PSS facets for each account, and the lack of an introducer FURL -tells the SS to not make its generic SS facet available to anyone. The -combination means that clients must acquire PSS facets instead of using the -generic one. - -== Configuring Clients == - -Each client should be configured to use a specific account by copying their -account FURL into their basedir, in a file named "account.furl". In addition, -these client nodes should *not* have an "introducer.furl". This combination -tells the client to ask the AM for ... diff --git a/docs/proposed/accounts-pubkey.txt b/docs/proposed/accounts-pubkey.txt deleted file mode 100644 index fc483a84..00000000 --- a/docs/proposed/accounts-pubkey.txt +++ /dev/null @@ -1,662 +0,0 @@ -This is a proposal for handing accounts and quotas in Tahoe. Nothing is final -yet.. we are still evaluating the options. - - -= Accounts = - -The basic Tahoe account is defined by a DSA key pair. The holder of the -private key has the ability to consume storage in conjunction with a specific -account number. - -The Account Server has a long-term keypair. Valid accounts are marked as such -by the Account Server's signature on a "membership card", which binds a -specific pubkey to an account number and declares that this pair is a valid -account. - -Each Storage Server which participates in the AS's domain will have the AS's -pubkey in its list of valid AS keys, and will thus accept membership cards -that were signed by that AS. If the SS accepts multiple ASs, then it will -give each a distinct number, and leases will be labled with an (AS#,Account#) -pair. If there is only one AS, then leases will be labeled with just the -Account#. - -Each client node is given the FURL of their personal Account object. The -Account will accept a DSA public key and return a signed membership card that -authorizes the corresponding private key to consume storage on behalf of the -account. The client will create its own DSA keypair the first time it -connects to the Account, and will then use the resulting membership card for -all subsequent storage operations. - -== Storage Server Goals == - -The Storage Server cares about two things: - - 1: maintaining an accurate refcount on each bucket, so it can delete the - bucket when the refcount goes to zero - 2: being able to answer questions about aggregate usage per account - -The SS conceptually maintains a big matrix of lease information: one column -per account, one row per storage index. The cells contain a boolean -(has-lease or no-lease). If the grid uses per-lease timers, then each -has-lease cell also contains a lease timer. - -This matrix may be stored in a variety of ways: entries in each share file, -or items in a SQL database, according to the desired tradeoff between -complexity, robustness, read speed, and write speed. - -Each client (by virtue of their knowledge of an authorized private key) gets -to manipulate their column of this matrix in any way they like: add lease, -renew lease, delete lease. (TODO: for reconcilliation purposes, the should -also be able to enumerate leases). - -== Storage Operations == - -Side-effect-causing storage operations come in three forms: - - 1: allocate bucket / add lease to existing bucket - arguments: storage_index=, storage_server=, ueb_hash=, account= - 2: renew lease - arguments: storage_index=, storage_server=, account= - 3: cancel lease - arguments: storage_index=, storage_server=, account= - -(where lease renewal is only relevant for grids which use per-lease timers). -Clients do add-lease when they upload a file, and cancel-lease when they -remove their last reference to it. - -Storage Servers publish a "public storage port" through the introducer, which -does not actually enable storage operations, but is instead used in a -rights-amplification pattern to grant authorized parties access to a -"personal storage server facet". This personal facet is the one that -implements allocate_bucket. All clients get access to the same public storage -port, which means that we can improve the introduction mechanism later (to -use a gossip-based protocol) without affecting the authority-granting -protocols. - -The public storage port accepts signed messages asking for storage authority. -It responds by creating a personal facet and making it available to the -requester. The account number is curried into the facet, so that all -lease-creating operations will record this account number into the lease. By -restricting the nature of the personal facets that a client can access, we -restrict them to using their designated account number. - - -======================================== - -There are two kinds of signed messages: use (other names: connection, -FURLification, activation, reification, grounding, specific-making, ?), and -delegation. The FURLification message results in a FURL that points to an -object which can actually accept RIStorageServer methods. The delegation -message results in a new signed message. - -The furlification message looks like: - - (pubkey, signed(serialized({limitations}, beneficiary_furl))) - -The delegation message looks like: - - (pubkey, signed(serialized({limitations}, delegate_pubkey))) - -The limitations dict indicates what the resulting connection or delegation -can be used for. All limitations for the cert chain are applied, and the -result must be restricted to their overall minimum. - -The following limitation keys are defined: - - 'account': a number. All resulting leases must be tagged with this account - number. A chain with multiple distinct 'account' limitations is - an error (the result will not permit leases) - 'SI': a storage index (binary string). Leases may only be created for this - specific storage index, no other. - 'serverid': a peerid (binary string). Leases may only be created on the - storage server identified by this serverid. - 'UEB_hash': (binary string): Leases may only be created for shares which - contain a matching UEB_hash. Note: this limitation is a nuisance - to implement correctly: it requires that the storage server - parse the share and verify all hashes. - 'before': a timestamp (seconds since epoch). All leases must be made before - this time. In addition, all liverefs and FURLs must expire and - cease working at this time. - 'server_size': a number, measuring share size (in bytes). A storage server - which sees this message should keep track of how much storage - space has been consumed using this liveref/FURL, and throw - an exception when receiving a lease request that would bring - this total above 'server_size'. Note: this limitation is - a nuisance to implement (it works best if 'before' is used - and provides a short lifetime). - -Actually, let's merge the two, and put the type in the limitations dict. -'furl_to' and 'delegate_key' are mutually exclusive. - - 'furl_to': (string): Used only on furlification messages. This requests the - recipient to create an object which implements the given access, - then send a FURL which references this object to an - RIFURLReceiver.furl() call at the given 'furl_to' FURL. - - To reduce the number of extra roundtrips, both foolscap calls - include an extra (ignored) argument that will carry the object - being referenced by the FURL, used to pre-load the recipient's - foolscap table. In addition, the response message will contain a - nonce, to allow the same beneficiary to be used for multiple - messages: - - def process(limitations, nonce, ignored): - facet = create_storage_facet(limitations) - facet_furl = tub.registerReference(facet) - d = tub.getReference(limitations['furl_to']) - d.addCallback(lambda rref: rref.furl(facet_furl, nonce, facet)) - - The server must always send the facet/facet_furl to the furl_to - beneficiary, and never to the 'ignored' argument (even though for - well-behaved clients these will both refer to the same target). - This is to prevent a rogue server from echoing a client's signed - message to some other server, to try to steal the client's - authority. - - The facet_furl should be persistent, so to reduce storage space, - facet_furl should contain an HMAC'ed list of all limitations, and - create_storage_facet() should be deferred until the client - actually tries to use the furl. This leads to 150-200 byte base32 - swissnums. - - 'delegate_key': (binary string, a DSA pubkey). Used only on delegation - messages. This requests all observers to accept messages - signed by the given public key and to apply the associated - limitations. - -I also want to keep the message size small, so I'm going to define a custom -netstring-based encoding format for it (JSON expands binary data by about -3.5x). Each dict entry will be encoded as netstring(key)+netstring(value). -The container is responsible for providing the size of this serialized -structure. - -The actual message will then look like: - -def make_message(privkey, limitations): - message_to_sign = "".join([ netstring(k) + netstring(v) - for k,v in limitations ]) - signature = privkey.sign(message_to_sign) - pubkey = privkey.get_public_key() - msg = netstring(message_to_sign) + netstring(signature) + netstring(pubkey) - return msg - -The deserialization code MUST throw an exception if the same limitations key -appears twice, to ensure that everybody interprets the dict the same way. - -These messages are passed over foolscap connections as a single string. They -are also saved to disk in this format. Code should only store them in a -deserialized form if the signature has been verified, the cert chain -verified, and the limitations accumulated. - - -The membership card is just the following: - - membership_card = make_message(account_server_privkey, - {'account': account_number, - 'before': time.time() + 1*MONTH, - 'delegate_key': client_pubkey}) - -This card is provided on demand by the given user's Account facet, for -whatever pubkey they submit. - -When a client learns about a new storage server, they create a new receiver -object (and stash the peerid in it), and submit the following message to the -RIStorageServerWelcome.get_personal_facet() method: - - class Receiver(foolscap.Referenceable): - def remote_furl(self, facet_furl, nonce, ignored_facet): - self.stash = facet_furl - receiver = Receiver() - nonce = make_nonce() - mymsg = make_message(client_privkey, {'furl_to': receiver_furl}) - send([membership_card, mymsg], nonce, receiver) - -Note that the receiver_furl will probably not have a routeable address, but -this won't matter because the client is already attached, so foolscap can use -the existing connection. The receiver should use facet_furl in preference to -ignored_facet for consistency, but (unlike the server's use of receiver_furl) -there is no security risk in using ignored_facet (since both are coming from -the same source). - -The server will validate the cert chain (see below) and wind up with a -complete list of limitations that are to be applied to the facet it will -provide to the caller. This list must combine limitations from the entire -chain: in particular it must enforce the account= limitation from the -membership card. - -The server will then serialize this limitation dict into a string, compute a -fixed-size HMAC code using a server-private secret, then base32 encode the -(hmac+limitstring) value (and prepend a "0-" version indicator). The -resulting string is used as the swissnum portion of the FURL that is sent to -the furl_to target. - -Later, when the client tries to dereference this FURL, a -Tub.registerNameLookupHandler hook will notice the attempt, claim the "0-" -namespace, base32decode the string, check the HMAC, decode the limitation -dict, then create and return an RIStorageServer facet with these limitations. - -The client should cache the (peerid, FURL) mapping in persistent storage. -Later, when it learns about this storage server again, it will use the cached -FURL instead of signing another message. If the getReference or the storage -operation fails with StorageAuthorityExpiredError, the cache entry should be -removed and the client should sign a new message to obtain a new one. - - (security note: an evil storage server can take 'mymsg' and present it to - someone else, but other servers will only send the resulting authority to - the client's receiver_furl, so the evil server cannot benefit from this. The - receiver object has the serverid curried into it, so the evil server can - only affect the client's mapping for this one serverid, not anything else, - so the server cannot hurt the client in any way other than denying service - to itself. It might be a good idea to include serverid= in the message, but - it isn't clear that it really helps anything). - -When the client wants to use a Helper, it needs to delegate some amount of -storage authority to the helper. The first phase has the client send the -storage index to the helper, so it can query servers and decide whether the -file needs to be uploaded or not. If it decides yes, the Helper creates a new -Uploader object and a receiver object, and sends the Uploader liveref and the -receiver FURL to the client. - -The client then creates a message for the helper to use: - - helper_msg = make_message(client_privkey, {'furl_to': helper_rx_furl, - 'SI': storage_index, - 'before': time.time() + 1*DAY, #? - 'server_size': filesize/k+overhead, - }) - -The client then sends (membership_card, helper_msg) to the helper. The Helper -sends (membership_card, helper_msg) to each storage server that it needs to -use for the upload. This gives the Helper access to a limited facet on each -storage server. This facet gives the helper the authority to upload data for -a specific storage index, for a limited time, using leases that are tagged by -the user's account number. The helper cannot use the client's storage -authority for any other file. The size limit prevents the helper from storing -some other (larger) file of its own using this authority. The time -restriction allows the storage servers to expire their 'server_size' table -entry quickly, and prevents the helper from hanging on to the storage -authority indefinitely. - -The Helper only gets one furl_to target, which must be used for multiple SS -peerids. The helper's receiver must parse the FURL that gets returned to -determine which server is which. [problems: an evil server could deliver a -bogus FURL which points to a different server. The Helper might reject the -real server's good FURL as a duplicate. This allows an evil server to block -access to a good server. Queries could be sent sequentially, which would -partially mitigate this problem (an evil server could send multiple -requests). Better: if the cert-chain send message could include a nonce, -which is supposed to be returned with the FURL, then the helper could use -this to correlate sends and receives.] - -=== repair caps === - -There are three basic approaches to provide a Repairer with the storage -authority that it needs. The first is to give the Repairer complete -authority: allow it to place leases for whatever account number it wishes. -This is simple and requires the least overhead, but of course it give the -Repairer the ability to abuse everyone's quota. The second is to give the -Repairer no user authority: instead, give the repairer its own account, and -build it to keep track of which leases it is holding on behalf of one of its -customers. This repairer will slowly accumulate quota space over time, as it -creates new shares to replace ones that have decayed. Eventually, when the -client comes back online, the client should establish its own leases on these -new shares and allow the repairer to cancel its temporary ones. - -The third approach is in between the other two: give the repairer some -limited authority over the customer's account, but not enough to let it -consume the user's whole quota. - -To create the storage-authority portion of a (one-month) repair-cap, the -client creates a new DSA keypair (repair_privkey, repair_pubkey), and then -creates a signed message and bundles it into the repaircap: - - repair_msg = make_message(client_privkey, {'delegate_key': repair_pubkey, - 'SI': storage_index, - 'UEB_hash': file_ueb_hash}) - repair_cap = (verify_cap, repair_privkey, (membership_card, repair_msg)) - -This gives the holder of the repair cap a time-limited authority to upload -shares for the given storage index which contain the given data. This -prohibits the repair-cap from being used to upload or repair any other file. - -When the repairer needs to upload a new share, it will use the delegated key -to create its own signed message: - - upload_msg = make_message(repair_privkey, {'furl_to': repairer_rx_furl}) - send(membership_card, repair_msg, upload_msg) - -The biggest problem with the low-authority approaches is the expiration time -of the membership card, which limits the duration for which the repair-cap -authority is valid. It would be nice if repair-caps could last a long time, -years perhaps, so that clients can be offline for a similar period of time. -However to retain a reasonable revocation interval for users, the membership -card's before= timeout needs to be closer to a month. [it might be reasonable -to use some sort of rights-amplification: the repairer has a special cert -which allows it to remove the before= value from a chain]. - - -=== chain verification === - -The server will create a chain that starts with the AS's certificate: an -unsigned message which derives its authority from being manually placed in -the SS's configdir. The only limitation in the AS certificate will be on some -kind of meta-account, in case we want to use multiple account servers and -allow their account numbers to live in distinct number spaces (think -sub-accounts or business partners to buy storage in bulk and resell it to -users). The rest of the chain comes directly from what the client sent. - -The server walks the chain, keeping an accumulated limitations dictionary -along the way. At each step it knows the pubkey that was delegated by the -previous step. - -== client config == - -Clients are configured with an Account FURL that points to a private facet on -the Account Server. The client generates a private key at startup. It sends -the pubkey to the AS facet, which will return a signed delegate_key message -(the "membership card") that grants the client's privkey any storage -authority it wishes (as long as the account number is set to a specific -value). - -The client stores this membership card in private/membership.cert . - - -RIStorageServer messages will accept an optional account= argument. If left -unspecified, the value is taken from the limitations that were curried into -the SS facet. In all cases, the value used must meet those limitations. The -value must not be None: Helpers/Repairers or other super-powered storage -clients are obligated to specify an account number. - -== server config == - -Storage servers are configured with an unsigned root authority message. This -is like the output of make_message(account_server_privkey, {}) but has empty -'signature' and 'pubkey' strings. This root goes into -NODEDIR/storage_authority_root.cert . It is prepended to all chains that -arrive. - - [if/when we accept multiple authorities, storage_authority_root.cert will - turn into a storage_authority_root/ directory with *.cert files, and each - arriving chain will cause a search through these root certs for a matching - pubkey. The empty limitations will be replaced by {domain=X}, which is used - as a sort of meta-account.. the details depend upon whether we express - account numbers as an int (with various ranges) or as a tuple] - -The root authority message is published by the Account Server through its web -interface, and also into a local file: NODEDIR/storage_authority_root.cert . -The admin of the storage server is responsible for copying this file into -place, thus enabling clients to use storage services. - - ----------------------------------------- - --- Text beyond this point is out-of-date, and exists purely for background -- - -Each storage server offers a "public storage port", which only accepts signed -messages. The Introducer mechanism exists to give clients a reference to a -set of these public storage ports. All clients get access to the same ports. -If clients did all their work themselves, these public storage ports would be -enough, and no further code would be necessary (all storage requests would we -signed the same way). - -Fundamentally, each storage request must be signed by the account's private -key, giving the SS an authenticated Account Number to go with the request. -This is used to index the correct cell in the lease matrix. The holder of the -account privkey is allowed to manipulate their column of the matrix in any -way they like: add leases, renew leases, delete leases. (TODO: for -reconcilliation purposes, they should also be able to enumerate leases). The -storage request is sent in the form of a signed request message, accompanied -by the membership card. For example: - - req = SIGN("allocate SI=123 SSID=abc", accountprivkey) , membership_card - -> RemoteBucketWriter reference - -Upon receipt of this request, the storage server will return a reference to a -RemoteBucketWriter object, which the client can use to fill and close the -bucket. The SS must perform two DSA signature verifications before accepting -this request. The first is to validate the membership card: the Account -Server's pubkey is used to verify the membership card's signature, from which -an account pubkey and account# is extracted. The second is to validate the -request: the account pubkey is used to verify the request signature. If both -are valid, the full request (with account# and storage index) is delivered to -the internal StorageServer object. - -Note that the signed request message includes the Storage Server's node ID, -to prevent this storage server from taking the signed message and echoing to -other storage servers. Each SS will ignore any request that is not addressed -to the right SSID. Also note that the SI= and SSID= fields may contain -wildcards, if the signing client so chooses. - -== Caching Signature Verification == - -We add some complexity to this simple model to achieve two goals: to enable -fine-grained delegation of storage capabilities (specifically for renewers -and repairers), and to reduce the number of public-key crypto operations that -must be performed. - -The first enhancement is to allow the SS to cache the results of the -verification step. To do this, the client creates a signed message which asks -the SS to return a FURL of an object which can be used to execute further -operations *without* a DSA signature. The FURL is expected to contain a -MAC'ed string that contains the account# and the argument restrictions, -effectively currying a subset of arguments into the RemoteReference. Clients -which do all their operations themselves would use this to obtain a private -storage port for each public storage port, stashing the FURLs in a local -table, and then later storage operations would be done to those FURLs instead -of creating signed requests. For example: - - req = SIGN("FURL(allocate SI=* SSID=abc)", accountprivkey), membership_card - -> FURL - Tub.getReference(FURL).allocate(SI=123) -> RemoteBucketWriter reference - -== Renewers and Repairers - -A brief digression is in order, to motivate the other enhancement. The -"manifest" is a list of caps, one for each node that is reachable from the -user's root directory/directories. The client is expected to generate the -manifest on a periodic basis (perhaps once a day), and to keep track of which -files/dirnodes have been added and removed. Items which have been removed -must be explicitly dereferenced to reclaim their storage space. For grids -which use per-file lease timers, the manifest is used to drive the Renewer: a -process which renews the lease timers on a periodic basis (perhaps once a -week). The manifest can also be used to drive a Checker, which in turn feeds -work into the Repairer. - -The manifest should contain the minimum necessary authority to do its job, -which generally means it contains the "verify cap" for each node. For -immutable files, the verify cap contains the storage index and the UEB hash: -enough information to retrieve and validate the ciphertext but not enough to -decrypt it. For mutable files, the verify cap contains the storage index and -the pubkey hash, which also serves to retrieve and validate ciphertext but -not decrypt it. - -If the client does its own Renewing and Repairing, then a verifycap-based -manifest is sufficient. However, if the user wants to be able to turn their -computer off for a few months and still keep their files around, they need to -delegate this job off to some other willing node. In a commercial network, -there will be centralized (and perhaps trusted) Renewer/Repairer nodes, but -in a friendnet these may not be available, and the user will depend upon one -of their friends being willing to run this service for them while they are -away. In either of these cases, the verifycaps are not enough: the Renewer -will need additional authority to renew the client's leases, and the Repairer -will need the authority to create new shares (in the client's name) when -necessary. - -A trusted central service could be given all-account superpowers, allowing it -to exercise storage authority on behalf of all users as it pleases. If this -is the case, the verifycaps are sufficient. But if we desire to grant less -authority to the Renewer/Repairer, then we need a mechanism to attenuate this -authority. - -The usual objcap approach is to create a proxy: an intermediate object which -itself is given full authority, but which is unwilling to exercise more than -a portion of that authority in response to incoming requests. The -not-fully-trusted service is then only given access to the proxy, not the -final authority. For example: - - class Proxy(RemoteReference): - def __init__(self, original, storage_index): - self.original = original - self.storage_index = storage_index - def remote_renew_leases(self): - return self.original.renew_leases(self.storage_index) - renewer.grant(Proxy(target, "abcd")) - -But this approach interposes the proxy in the calling chain, requiring the -machine which hosts the proxy to be available and on-line at all times, which -runs opposite to our use case (turning the client off for a month). - -== Creating Attenuated Authorities == - -The other enhancement is to use more public-key operations to allow the -delegation of reduced authority to external helper services. Specifically, we -want to give then Renewer the ability to renew leases for a specific file, -rather than giving it lease-renewal power for all files. Likewise, the -Repairer should have the ability to create new shares, but only for the file -that is being repaired, not for unrelated files. - -If we do not mind giving the storage servers the ability to replay their -inbound message to other storage servers, then the client can simply generate -a signed message with a wildcard SSID= argument and leave it in the care of -the Renewer or Repairer. For example, the Renewer would get: - - SIGN("renew-lease SI=123 SSID=*", accountprivkey), membership_card - -Then, when the Renewer needed to renew a lease, it would deliver this signed -request message to the storage server. The SS would verify the signatures -just as if the message came from the original client, find them good, and -perform the desired operation. With this approach, the manifest that is -delivered to the remote Renewer process needs to include a signed -lease-renewal request for each file: we use the term "renew-cap" for this -combined (verifycap + signed lease-renewal request) message. Likewise the -"repair-cap" would be the verifycap plus a signed allocate-bucket message. A -renew-cap manifest would be enough for a remote Renewer to do its job, a -repair-cap manifest would provide a remote Repairer with enough authority, -and a cancel-cap manifest would be used for a remote Canceller (used, e.g., -to make sure that file has been dereferenced even if the client does not -stick around long enough to track down and inform all of the storage servers -involved). - -The only concern is that the SS could also take this exact same renew-lease -message and deliver it to other storage servers. This wouldn't cause a -concern for mere lease renewal, but the allocate-share message might be a bit -less comfortable (you might not want to grant the first storage server the -ability to claim space in your name on all other storage servers). - -Ideally we'd like to send a different message to each storage server, each -narrowed in scope to a single SSID, since then none of these messages would -be useful on any other SS. If the client knew the identities of all the -storage servers in the system ahead of time, it might create a whole slew of -signed messages, but a) this is a lot of signatures, only a fraction of which -will ever actually be used, and b) new servers might be introduced after the -manifest is created, particularly if we're talking about repair-caps instead -of renewal-caps. The Renewer can't generate these one-per-SSID messages from -the SSID=* message, because it doesn't have a privkey to make the correct -signatures. So without some other mechanism, we're stuck with these -relatively coarse authorities. - -If we want to limit this sort of authority, then we need to introduce a new -method. The client begins by generating a new DSA keypair. Then it signs a -message that declares the new pubkey to be valid for a specific subset of -storage operations (such as "renew-lease SI=123 SSID=*"). Then it delivers -the new privkey, the declaration message, and the membership card to the -Renewer. The renewer uses the new privkey to sign its own one-per-SSID -request message for each server, then sends the (signed request, declaration, -membership card) triple to the server. The server needs to perform three -verification checks per message: first the membership card, then the -declaration message, then the actual request message. - -== Other Enhancements == - -If a given authority is likely to be used multiple times, the same -give-me-a-FURL trick can be used to cut down on the number of public key -operations that must be performed. This is trickier with the per-SI messages. - -When storing the manifest, things like the membership card should be -amortized across a set of common entries. An isolated renew-cap needs to -contain the verifycap, the signed renewal request, and the membership card. -But a manifest with a thousand entries should only include one copy of the -membership card. - -It might be sensible to define a signed renewal request that grants authority -for a set of storage indicies, so that the signature can be shared among -several entries (to save space and perhaps processing time). The request -could include a Bloom filter of authorized SI values: when the request is -actually sent to the server, the renewer would add a list of actual SI values -to renew, and the server would accept all that are contained in the filter. - -== Revocation == - -The lifetime of the storage authority included in the manifest's renew-caps -or repair-caps will determine the lifetime of those caps. In particular, if -we implement account revocation by using time-limited membership cards -(requiring the client to get a new card once a month), then the repair-caps -won't work for more than a month, which kind of defeats the purpose. - -A related issue is the FURL-shortcut: the MAC'ed message needs to include a -validity period of some sort, and if the client tries to use a old FURL they -should get an error message that will prompt them to try and acquire a newer -one. - ------------------------------- - -The client can produce a repair-cap manifest for a specific Repairer's -pubkey, so it can produce a signed message that includes the pubkey (instead -of needing to generate a new privkey just for this purpose). The result is -not a capability, since it can only be used by the holder of the -corresponding privkey. - -So the generic form of the storage operation message is the request (which -has all the argument values filled in), followed by a chain of -authorizations. The first authorization must be signed by the Account -Server's key. Each authorization must be signed by the key mentioned in the -previous one. Each one adds a new limitation on the power of the following -ones. The actual request is bounded by all the limitations of the chain. - -The membership card is an authorization that simply limits the account number -that can be used: "op=* SI=* SSID=* account=4 signed-by=CLIENT-PUBKEY". - -So a repair manifest created for a Repairer with pubkey ABCD could consist of -a list of verifycaps plus a single authorization (using a Bloom filter to -identify the SIs that were allowed): - - SIGN("allocate SI=[bloom] SSID=* signed-by=ABCD") - -If/when the Repairer needed to allocate a share, it would use its own privkey -to sign an additional message and send the whole list to the SS: - - request=allocate SI=1234 SSID=EEFS account=4 shnum=2 - SIGN("allocate SI=1234 SSID=EEFS", ABCD) - SIGN("allocate SI=[bloom] SSID=* signed-by=ABCD", clientkey) - membership: SIGN("op=* SI=* SSID=* account=4 signed-by=clientkey", ASkey) - [implicit]: ASkey - ----------------------------------------- - -Things would be a lot simpler if the Repairer (actually the Re-Leaser) had -everybody's account authority. - -One simplifying approach: the Repairer/Re-Leaser has its own account, and the -shares it creates are leased under that account number. The R/R keeps track -of which leases it has created for whom. When the client eventually comes -back online, it is told to perform a re-leasing run, and after that occurs -the R/R can cancel its own temporary leases. - -This would effectively transfer storage quota from the original client to the -R/R over time (as shares are regenerated by the R/R while the client remains -offline). If the R/R is centrally managed, the quota mechanism can sum the -R/R's numbers with the SS's numbers when determining how much storage is -consumed by any given account. Not quite as clean as storing the exact -information in the SS's lease tables directly, but: - - * the R/R no longer needs any special account authority (it merely needs an - accurate account number, which can be supplied by giving the client a - specific facet that is bound to that account number) - * the verify-cap manifest is sufficient to perform repair - * no extra DSA keys are necessary - * account authority could be implemented with either DSA keys or personal SS - facets: i.e. we don't need the delegability aspects of DSA keys for use by - the repair mechanism (we might still want them to simplify introduction). - -I *think* this would eliminate all that complexity of chained authorization -messages. diff --git a/docs/proposed/old-accounts-introducer.txt b/docs/proposed/old-accounts-introducer.txt new file mode 100644 index 00000000..36a5a56f --- /dev/null +++ b/docs/proposed/old-accounts-introducer.txt @@ -0,0 +1,134 @@ +This is a proposal for handing accounts and quotas in Tahoe. Nothing is final +yet.. we are still evaluating the options. + + += Account Management: Introducer-based = + +A Tahoe grid can be configured in several different modes. The simplest mode +(which is also the default) is completely permissive: all storage servers +will accept shares from all clients, and no attempt is made to keep track of +who is storing what. Access to the grid is mostly equivalent to having access +to the Introducer (or convincing one of the existing members to give you a +list of all their storage server FURLs). + +This mode, while a good starting point, does not accomodate any sort of +auditing or quota management. Even in a small friendnet, operators might like +to know how much of their storage space is being consumed by Alice, so they +might be able to ask her to cut back when overall disk usage is getting to +high. In a larger commercial deployment, a service provider needs to be able +to get accurate usage numbers so they can bill the user appropriately. In +addition, the operator may want the ability to delete all of Bob's shares +(i.e. cancel any outstanding leases) when he terminates his account. + +There are several lease-management/garbage-collection/deletion strategies +possible for a Tahoe grid, but the most efficient ones require knowledge of +lease ownership, so that renewals and expiration can take place on a +per-account basis rather than a (more numerous) per-share basis. + +== Accounts == + +To accomplish this, "Accounts" can be established in a Tahoe grid. There is +nominally one account per human user of the grid, but of course a user might +use multiple accounts, or an account might be shared between multiple users. +The Account is the smallest unit of quota and lease management. + +Accounts are created by an "Account Manager". In a commercial network there +will be just one (centralized) account manager, and all storage nodes will be +configured to require a valid account before providing storage services. In a +friendnet, each peer can run their own account manager, and servers will +accept accounts from any of the managers (this mode is permissive but allows +quota-tracking of non-malicious users). + +The account manager is free to manage the accounts as it pleases. Large +systems will probably use a database to correlate things like username, +storage consumed, billing status, etc. + +== Overview == + +The Account Manager ("AM") replaces the normal Introducer node: grids which +use an Account Manager will not run an Introducer, and the participating +nodes will not be configured with an "introducer.furl". + +Instead, each client will be configured with a different "account.furl", +which gives that client access to a specific account. These account FURLs +point to an object inside the Account Manager which exists solely for the +benefit of that one account. When the client needs access to storage servers, +it will use this account object to acquire personalized introductions to a +per-account "Personal Storage Server" facet, one per storage server node. For +example, Alice would wind up with PSS[1A] on server 1, and PSS[2A] on server +2. Bob would get PSS[1B] and PSS[2B]. + +These PSS facets provide the same remote methods as the old generic SS facet, +except that every time they create a lease object, the account information of +the holder is recorded in that lease. The client stores a list of these PSS +facet FURLs in persistent storage, and uses them in the "get_permuted_peers" +function that all uploads and downloads use to figure out who to talk to when +looking for shares or shareholders. + +Each Storage Server has a private facet that it gives to the Account Manager. +This facet allows the AM to create PSS facets for a specific account. In +particular, the AM tells the SS "please create account number 42, and tell me +the PSS FURL that I should give to the client". The SS creates an object +which remembers the account number, creates a FURL for it, and returns the +FURL. + +If there is a single central account manager, then account numbers can be +small integers. (if there are multiple ones, they need to be large random +strings to ensure uniqueness). To avoid requiring large (accounts*servers) +lookup tables, a given account should use the same identifer for all the +servers it talks to. When this can be done, the PSS and Account FURLs are +generated as MAC'ed copies of the account number. + +More specifically, the PSS FURL is a MAC'ed copy of the account number: each +SS has a private secret "S", and it creates a string "%d-%s" % (accountnum, +b2a(hash(S+accountnum))) to use as the swissnum part of the FURL. The SS uses +tub.registerNameLookupHandler to add a function that tries to validate +inbound FURLs against this scheme: if successful, it creates a new PSS object +with the account number stashed inside. This allows the server to minimize +their per-user storage requirements but still insure that PSS FURLs are +unguessable. + +Account FURLs are created by the Account Manager in a similar fashion, using +a MAC of the account number. The Account Manager can use the same account +number to index other information in a database, like account status, billing +status, etc. + +The mechanism by which Account FURLs are minted is left up to the account +manager, but the simple AM that the 'tahoe create-account-manager' command +makes has a "new-account" FURL which accepts a username and creates an +account for them. The 'tahoe create-account' command is a CLI frontend to +this facility. In a friendnet, you could publish this FURL to your friends, +allowing everyone to make their own account. In a commercial grid, this +facility would be reserved use by the same code which handles billing. + + +== Creating the Account Manager == + +The 'tahoe create-account-manager' command is used to create a simple account +manager node. When started, this node will write several FURLs to its +private/ directory, some of which should be provided to other services. + + * new-account.furl : this FURL allows the holder to create new accounts + * manage-accounts.furl : this FURL allows the holder to list and modify + all existing accounts + * serverdesk.furl : this FURL is used by storage servers to make themselves + available to all account holders + + +== Configuring the Storage Servers == + +To use an account manager, each storage server node should be given access to +the AM's serverdesk (by simply copying "serverdesk.furl" into the storage +server's base directory). In addition, it should *not* be given an +introducer.furl . The serverdesk FURL tells the SS that it should allow the +AM to create PSS facets for each account, and the lack of an introducer FURL +tells the SS to not make its generic SS facet available to anyone. The +combination means that clients must acquire PSS facets instead of using the +generic one. + +== Configuring Clients == + +Each client should be configured to use a specific account by copying their +account FURL into their basedir, in a file named "account.furl". In addition, +these client nodes should *not* have an "introducer.furl". This combination +tells the client to ask the AM for ... diff --git a/docs/proposed/old-accounts-pubkey.txt b/docs/proposed/old-accounts-pubkey.txt new file mode 100644 index 00000000..fc483a84 --- /dev/null +++ b/docs/proposed/old-accounts-pubkey.txt @@ -0,0 +1,662 @@ +This is a proposal for handing accounts and quotas in Tahoe. Nothing is final +yet.. we are still evaluating the options. + + += Accounts = + +The basic Tahoe account is defined by a DSA key pair. The holder of the +private key has the ability to consume storage in conjunction with a specific +account number. + +The Account Server has a long-term keypair. Valid accounts are marked as such +by the Account Server's signature on a "membership card", which binds a +specific pubkey to an account number and declares that this pair is a valid +account. + +Each Storage Server which participates in the AS's domain will have the AS's +pubkey in its list of valid AS keys, and will thus accept membership cards +that were signed by that AS. If the SS accepts multiple ASs, then it will +give each a distinct number, and leases will be labled with an (AS#,Account#) +pair. If there is only one AS, then leases will be labeled with just the +Account#. + +Each client node is given the FURL of their personal Account object. The +Account will accept a DSA public key and return a signed membership card that +authorizes the corresponding private key to consume storage on behalf of the +account. The client will create its own DSA keypair the first time it +connects to the Account, and will then use the resulting membership card for +all subsequent storage operations. + +== Storage Server Goals == + +The Storage Server cares about two things: + + 1: maintaining an accurate refcount on each bucket, so it can delete the + bucket when the refcount goes to zero + 2: being able to answer questions about aggregate usage per account + +The SS conceptually maintains a big matrix of lease information: one column +per account, one row per storage index. The cells contain a boolean +(has-lease or no-lease). If the grid uses per-lease timers, then each +has-lease cell also contains a lease timer. + +This matrix may be stored in a variety of ways: entries in each share file, +or items in a SQL database, according to the desired tradeoff between +complexity, robustness, read speed, and write speed. + +Each client (by virtue of their knowledge of an authorized private key) gets +to manipulate their column of this matrix in any way they like: add lease, +renew lease, delete lease. (TODO: for reconcilliation purposes, the should +also be able to enumerate leases). + +== Storage Operations == + +Side-effect-causing storage operations come in three forms: + + 1: allocate bucket / add lease to existing bucket + arguments: storage_index=, storage_server=, ueb_hash=, account= + 2: renew lease + arguments: storage_index=, storage_server=, account= + 3: cancel lease + arguments: storage_index=, storage_server=, account= + +(where lease renewal is only relevant for grids which use per-lease timers). +Clients do add-lease when they upload a file, and cancel-lease when they +remove their last reference to it. + +Storage Servers publish a "public storage port" through the introducer, which +does not actually enable storage operations, but is instead used in a +rights-amplification pattern to grant authorized parties access to a +"personal storage server facet". This personal facet is the one that +implements allocate_bucket. All clients get access to the same public storage +port, which means that we can improve the introduction mechanism later (to +use a gossip-based protocol) without affecting the authority-granting +protocols. + +The public storage port accepts signed messages asking for storage authority. +It responds by creating a personal facet and making it available to the +requester. The account number is curried into the facet, so that all +lease-creating operations will record this account number into the lease. By +restricting the nature of the personal facets that a client can access, we +restrict them to using their designated account number. + + +======================================== + +There are two kinds of signed messages: use (other names: connection, +FURLification, activation, reification, grounding, specific-making, ?), and +delegation. The FURLification message results in a FURL that points to an +object which can actually accept RIStorageServer methods. The delegation +message results in a new signed message. + +The furlification message looks like: + + (pubkey, signed(serialized({limitations}, beneficiary_furl))) + +The delegation message looks like: + + (pubkey, signed(serialized({limitations}, delegate_pubkey))) + +The limitations dict indicates what the resulting connection or delegation +can be used for. All limitations for the cert chain are applied, and the +result must be restricted to their overall minimum. + +The following limitation keys are defined: + + 'account': a number. All resulting leases must be tagged with this account + number. A chain with multiple distinct 'account' limitations is + an error (the result will not permit leases) + 'SI': a storage index (binary string). Leases may only be created for this + specific storage index, no other. + 'serverid': a peerid (binary string). Leases may only be created on the + storage server identified by this serverid. + 'UEB_hash': (binary string): Leases may only be created for shares which + contain a matching UEB_hash. Note: this limitation is a nuisance + to implement correctly: it requires that the storage server + parse the share and verify all hashes. + 'before': a timestamp (seconds since epoch). All leases must be made before + this time. In addition, all liverefs and FURLs must expire and + cease working at this time. + 'server_size': a number, measuring share size (in bytes). A storage server + which sees this message should keep track of how much storage + space has been consumed using this liveref/FURL, and throw + an exception when receiving a lease request that would bring + this total above 'server_size'. Note: this limitation is + a nuisance to implement (it works best if 'before' is used + and provides a short lifetime). + +Actually, let's merge the two, and put the type in the limitations dict. +'furl_to' and 'delegate_key' are mutually exclusive. + + 'furl_to': (string): Used only on furlification messages. This requests the + recipient to create an object which implements the given access, + then send a FURL which references this object to an + RIFURLReceiver.furl() call at the given 'furl_to' FURL. + + To reduce the number of extra roundtrips, both foolscap calls + include an extra (ignored) argument that will carry the object + being referenced by the FURL, used to pre-load the recipient's + foolscap table. In addition, the response message will contain a + nonce, to allow the same beneficiary to be used for multiple + messages: + + def process(limitations, nonce, ignored): + facet = create_storage_facet(limitations) + facet_furl = tub.registerReference(facet) + d = tub.getReference(limitations['furl_to']) + d.addCallback(lambda rref: rref.furl(facet_furl, nonce, facet)) + + The server must always send the facet/facet_furl to the furl_to + beneficiary, and never to the 'ignored' argument (even though for + well-behaved clients these will both refer to the same target). + This is to prevent a rogue server from echoing a client's signed + message to some other server, to try to steal the client's + authority. + + The facet_furl should be persistent, so to reduce storage space, + facet_furl should contain an HMAC'ed list of all limitations, and + create_storage_facet() should be deferred until the client + actually tries to use the furl. This leads to 150-200 byte base32 + swissnums. + + 'delegate_key': (binary string, a DSA pubkey). Used only on delegation + messages. This requests all observers to accept messages + signed by the given public key and to apply the associated + limitations. + +I also want to keep the message size small, so I'm going to define a custom +netstring-based encoding format for it (JSON expands binary data by about +3.5x). Each dict entry will be encoded as netstring(key)+netstring(value). +The container is responsible for providing the size of this serialized +structure. + +The actual message will then look like: + +def make_message(privkey, limitations): + message_to_sign = "".join([ netstring(k) + netstring(v) + for k,v in limitations ]) + signature = privkey.sign(message_to_sign) + pubkey = privkey.get_public_key() + msg = netstring(message_to_sign) + netstring(signature) + netstring(pubkey) + return msg + +The deserialization code MUST throw an exception if the same limitations key +appears twice, to ensure that everybody interprets the dict the same way. + +These messages are passed over foolscap connections as a single string. They +are also saved to disk in this format. Code should only store them in a +deserialized form if the signature has been verified, the cert chain +verified, and the limitations accumulated. + + +The membership card is just the following: + + membership_card = make_message(account_server_privkey, + {'account': account_number, + 'before': time.time() + 1*MONTH, + 'delegate_key': client_pubkey}) + +This card is provided on demand by the given user's Account facet, for +whatever pubkey they submit. + +When a client learns about a new storage server, they create a new receiver +object (and stash the peerid in it), and submit the following message to the +RIStorageServerWelcome.get_personal_facet() method: + + class Receiver(foolscap.Referenceable): + def remote_furl(self, facet_furl, nonce, ignored_facet): + self.stash = facet_furl + receiver = Receiver() + nonce = make_nonce() + mymsg = make_message(client_privkey, {'furl_to': receiver_furl}) + send([membership_card, mymsg], nonce, receiver) + +Note that the receiver_furl will probably not have a routeable address, but +this won't matter because the client is already attached, so foolscap can use +the existing connection. The receiver should use facet_furl in preference to +ignored_facet for consistency, but (unlike the server's use of receiver_furl) +there is no security risk in using ignored_facet (since both are coming from +the same source). + +The server will validate the cert chain (see below) and wind up with a +complete list of limitations that are to be applied to the facet it will +provide to the caller. This list must combine limitations from the entire +chain: in particular it must enforce the account= limitation from the +membership card. + +The server will then serialize this limitation dict into a string, compute a +fixed-size HMAC code using a server-private secret, then base32 encode the +(hmac+limitstring) value (and prepend a "0-" version indicator). The +resulting string is used as the swissnum portion of the FURL that is sent to +the furl_to target. + +Later, when the client tries to dereference this FURL, a +Tub.registerNameLookupHandler hook will notice the attempt, claim the "0-" +namespace, base32decode the string, check the HMAC, decode the limitation +dict, then create and return an RIStorageServer facet with these limitations. + +The client should cache the (peerid, FURL) mapping in persistent storage. +Later, when it learns about this storage server again, it will use the cached +FURL instead of signing another message. If the getReference or the storage +operation fails with StorageAuthorityExpiredError, the cache entry should be +removed and the client should sign a new message to obtain a new one. + + (security note: an evil storage server can take 'mymsg' and present it to + someone else, but other servers will only send the resulting authority to + the client's receiver_furl, so the evil server cannot benefit from this. The + receiver object has the serverid curried into it, so the evil server can + only affect the client's mapping for this one serverid, not anything else, + so the server cannot hurt the client in any way other than denying service + to itself. It might be a good idea to include serverid= in the message, but + it isn't clear that it really helps anything). + +When the client wants to use a Helper, it needs to delegate some amount of +storage authority to the helper. The first phase has the client send the +storage index to the helper, so it can query servers and decide whether the +file needs to be uploaded or not. If it decides yes, the Helper creates a new +Uploader object and a receiver object, and sends the Uploader liveref and the +receiver FURL to the client. + +The client then creates a message for the helper to use: + + helper_msg = make_message(client_privkey, {'furl_to': helper_rx_furl, + 'SI': storage_index, + 'before': time.time() + 1*DAY, #? + 'server_size': filesize/k+overhead, + }) + +The client then sends (membership_card, helper_msg) to the helper. The Helper +sends (membership_card, helper_msg) to each storage server that it needs to +use for the upload. This gives the Helper access to a limited facet on each +storage server. This facet gives the helper the authority to upload data for +a specific storage index, for a limited time, using leases that are tagged by +the user's account number. The helper cannot use the client's storage +authority for any other file. The size limit prevents the helper from storing +some other (larger) file of its own using this authority. The time +restriction allows the storage servers to expire their 'server_size' table +entry quickly, and prevents the helper from hanging on to the storage +authority indefinitely. + +The Helper only gets one furl_to target, which must be used for multiple SS +peerids. The helper's receiver must parse the FURL that gets returned to +determine which server is which. [problems: an evil server could deliver a +bogus FURL which points to a different server. The Helper might reject the +real server's good FURL as a duplicate. This allows an evil server to block +access to a good server. Queries could be sent sequentially, which would +partially mitigate this problem (an evil server could send multiple +requests). Better: if the cert-chain send message could include a nonce, +which is supposed to be returned with the FURL, then the helper could use +this to correlate sends and receives.] + +=== repair caps === + +There are three basic approaches to provide a Repairer with the storage +authority that it needs. The first is to give the Repairer complete +authority: allow it to place leases for whatever account number it wishes. +This is simple and requires the least overhead, but of course it give the +Repairer the ability to abuse everyone's quota. The second is to give the +Repairer no user authority: instead, give the repairer its own account, and +build it to keep track of which leases it is holding on behalf of one of its +customers. This repairer will slowly accumulate quota space over time, as it +creates new shares to replace ones that have decayed. Eventually, when the +client comes back online, the client should establish its own leases on these +new shares and allow the repairer to cancel its temporary ones. + +The third approach is in between the other two: give the repairer some +limited authority over the customer's account, but not enough to let it +consume the user's whole quota. + +To create the storage-authority portion of a (one-month) repair-cap, the +client creates a new DSA keypair (repair_privkey, repair_pubkey), and then +creates a signed message and bundles it into the repaircap: + + repair_msg = make_message(client_privkey, {'delegate_key': repair_pubkey, + 'SI': storage_index, + 'UEB_hash': file_ueb_hash}) + repair_cap = (verify_cap, repair_privkey, (membership_card, repair_msg)) + +This gives the holder of the repair cap a time-limited authority to upload +shares for the given storage index which contain the given data. This +prohibits the repair-cap from being used to upload or repair any other file. + +When the repairer needs to upload a new share, it will use the delegated key +to create its own signed message: + + upload_msg = make_message(repair_privkey, {'furl_to': repairer_rx_furl}) + send(membership_card, repair_msg, upload_msg) + +The biggest problem with the low-authority approaches is the expiration time +of the membership card, which limits the duration for which the repair-cap +authority is valid. It would be nice if repair-caps could last a long time, +years perhaps, so that clients can be offline for a similar period of time. +However to retain a reasonable revocation interval for users, the membership +card's before= timeout needs to be closer to a month. [it might be reasonable +to use some sort of rights-amplification: the repairer has a special cert +which allows it to remove the before= value from a chain]. + + +=== chain verification === + +The server will create a chain that starts with the AS's certificate: an +unsigned message which derives its authority from being manually placed in +the SS's configdir. The only limitation in the AS certificate will be on some +kind of meta-account, in case we want to use multiple account servers and +allow their account numbers to live in distinct number spaces (think +sub-accounts or business partners to buy storage in bulk and resell it to +users). The rest of the chain comes directly from what the client sent. + +The server walks the chain, keeping an accumulated limitations dictionary +along the way. At each step it knows the pubkey that was delegated by the +previous step. + +== client config == + +Clients are configured with an Account FURL that points to a private facet on +the Account Server. The client generates a private key at startup. It sends +the pubkey to the AS facet, which will return a signed delegate_key message +(the "membership card") that grants the client's privkey any storage +authority it wishes (as long as the account number is set to a specific +value). + +The client stores this membership card in private/membership.cert . + + +RIStorageServer messages will accept an optional account= argument. If left +unspecified, the value is taken from the limitations that were curried into +the SS facet. In all cases, the value used must meet those limitations. The +value must not be None: Helpers/Repairers or other super-powered storage +clients are obligated to specify an account number. + +== server config == + +Storage servers are configured with an unsigned root authority message. This +is like the output of make_message(account_server_privkey, {}) but has empty +'signature' and 'pubkey' strings. This root goes into +NODEDIR/storage_authority_root.cert . It is prepended to all chains that +arrive. + + [if/when we accept multiple authorities, storage_authority_root.cert will + turn into a storage_authority_root/ directory with *.cert files, and each + arriving chain will cause a search through these root certs for a matching + pubkey. The empty limitations will be replaced by {domain=X}, which is used + as a sort of meta-account.. the details depend upon whether we express + account numbers as an int (with various ranges) or as a tuple] + +The root authority message is published by the Account Server through its web +interface, and also into a local file: NODEDIR/storage_authority_root.cert . +The admin of the storage server is responsible for copying this file into +place, thus enabling clients to use storage services. + + +---------------------------------------- + +-- Text beyond this point is out-of-date, and exists purely for background -- + +Each storage server offers a "public storage port", which only accepts signed +messages. The Introducer mechanism exists to give clients a reference to a +set of these public storage ports. All clients get access to the same ports. +If clients did all their work themselves, these public storage ports would be +enough, and no further code would be necessary (all storage requests would we +signed the same way). + +Fundamentally, each storage request must be signed by the account's private +key, giving the SS an authenticated Account Number to go with the request. +This is used to index the correct cell in the lease matrix. The holder of the +account privkey is allowed to manipulate their column of the matrix in any +way they like: add leases, renew leases, delete leases. (TODO: for +reconcilliation purposes, they should also be able to enumerate leases). The +storage request is sent in the form of a signed request message, accompanied +by the membership card. For example: + + req = SIGN("allocate SI=123 SSID=abc", accountprivkey) , membership_card + -> RemoteBucketWriter reference + +Upon receipt of this request, the storage server will return a reference to a +RemoteBucketWriter object, which the client can use to fill and close the +bucket. The SS must perform two DSA signature verifications before accepting +this request. The first is to validate the membership card: the Account +Server's pubkey is used to verify the membership card's signature, from which +an account pubkey and account# is extracted. The second is to validate the +request: the account pubkey is used to verify the request signature. If both +are valid, the full request (with account# and storage index) is delivered to +the internal StorageServer object. + +Note that the signed request message includes the Storage Server's node ID, +to prevent this storage server from taking the signed message and echoing to +other storage servers. Each SS will ignore any request that is not addressed +to the right SSID. Also note that the SI= and SSID= fields may contain +wildcards, if the signing client so chooses. + +== Caching Signature Verification == + +We add some complexity to this simple model to achieve two goals: to enable +fine-grained delegation of storage capabilities (specifically for renewers +and repairers), and to reduce the number of public-key crypto operations that +must be performed. + +The first enhancement is to allow the SS to cache the results of the +verification step. To do this, the client creates a signed message which asks +the SS to return a FURL of an object which can be used to execute further +operations *without* a DSA signature. The FURL is expected to contain a +MAC'ed string that contains the account# and the argument restrictions, +effectively currying a subset of arguments into the RemoteReference. Clients +which do all their operations themselves would use this to obtain a private +storage port for each public storage port, stashing the FURLs in a local +table, and then later storage operations would be done to those FURLs instead +of creating signed requests. For example: + + req = SIGN("FURL(allocate SI=* SSID=abc)", accountprivkey), membership_card + -> FURL + Tub.getReference(FURL).allocate(SI=123) -> RemoteBucketWriter reference + +== Renewers and Repairers + +A brief digression is in order, to motivate the other enhancement. The +"manifest" is a list of caps, one for each node that is reachable from the +user's root directory/directories. The client is expected to generate the +manifest on a periodic basis (perhaps once a day), and to keep track of which +files/dirnodes have been added and removed. Items which have been removed +must be explicitly dereferenced to reclaim their storage space. For grids +which use per-file lease timers, the manifest is used to drive the Renewer: a +process which renews the lease timers on a periodic basis (perhaps once a +week). The manifest can also be used to drive a Checker, which in turn feeds +work into the Repairer. + +The manifest should contain the minimum necessary authority to do its job, +which generally means it contains the "verify cap" for each node. For +immutable files, the verify cap contains the storage index and the UEB hash: +enough information to retrieve and validate the ciphertext but not enough to +decrypt it. For mutable files, the verify cap contains the storage index and +the pubkey hash, which also serves to retrieve and validate ciphertext but +not decrypt it. + +If the client does its own Renewing and Repairing, then a verifycap-based +manifest is sufficient. However, if the user wants to be able to turn their +computer off for a few months and still keep their files around, they need to +delegate this job off to some other willing node. In a commercial network, +there will be centralized (and perhaps trusted) Renewer/Repairer nodes, but +in a friendnet these may not be available, and the user will depend upon one +of their friends being willing to run this service for them while they are +away. In either of these cases, the verifycaps are not enough: the Renewer +will need additional authority to renew the client's leases, and the Repairer +will need the authority to create new shares (in the client's name) when +necessary. + +A trusted central service could be given all-account superpowers, allowing it +to exercise storage authority on behalf of all users as it pleases. If this +is the case, the verifycaps are sufficient. But if we desire to grant less +authority to the Renewer/Repairer, then we need a mechanism to attenuate this +authority. + +The usual objcap approach is to create a proxy: an intermediate object which +itself is given full authority, but which is unwilling to exercise more than +a portion of that authority in response to incoming requests. The +not-fully-trusted service is then only given access to the proxy, not the +final authority. For example: + + class Proxy(RemoteReference): + def __init__(self, original, storage_index): + self.original = original + self.storage_index = storage_index + def remote_renew_leases(self): + return self.original.renew_leases(self.storage_index) + renewer.grant(Proxy(target, "abcd")) + +But this approach interposes the proxy in the calling chain, requiring the +machine which hosts the proxy to be available and on-line at all times, which +runs opposite to our use case (turning the client off for a month). + +== Creating Attenuated Authorities == + +The other enhancement is to use more public-key operations to allow the +delegation of reduced authority to external helper services. Specifically, we +want to give then Renewer the ability to renew leases for a specific file, +rather than giving it lease-renewal power for all files. Likewise, the +Repairer should have the ability to create new shares, but only for the file +that is being repaired, not for unrelated files. + +If we do not mind giving the storage servers the ability to replay their +inbound message to other storage servers, then the client can simply generate +a signed message with a wildcard SSID= argument and leave it in the care of +the Renewer or Repairer. For example, the Renewer would get: + + SIGN("renew-lease SI=123 SSID=*", accountprivkey), membership_card + +Then, when the Renewer needed to renew a lease, it would deliver this signed +request message to the storage server. The SS would verify the signatures +just as if the message came from the original client, find them good, and +perform the desired operation. With this approach, the manifest that is +delivered to the remote Renewer process needs to include a signed +lease-renewal request for each file: we use the term "renew-cap" for this +combined (verifycap + signed lease-renewal request) message. Likewise the +"repair-cap" would be the verifycap plus a signed allocate-bucket message. A +renew-cap manifest would be enough for a remote Renewer to do its job, a +repair-cap manifest would provide a remote Repairer with enough authority, +and a cancel-cap manifest would be used for a remote Canceller (used, e.g., +to make sure that file has been dereferenced even if the client does not +stick around long enough to track down and inform all of the storage servers +involved). + +The only concern is that the SS could also take this exact same renew-lease +message and deliver it to other storage servers. This wouldn't cause a +concern for mere lease renewal, but the allocate-share message might be a bit +less comfortable (you might not want to grant the first storage server the +ability to claim space in your name on all other storage servers). + +Ideally we'd like to send a different message to each storage server, each +narrowed in scope to a single SSID, since then none of these messages would +be useful on any other SS. If the client knew the identities of all the +storage servers in the system ahead of time, it might create a whole slew of +signed messages, but a) this is a lot of signatures, only a fraction of which +will ever actually be used, and b) new servers might be introduced after the +manifest is created, particularly if we're talking about repair-caps instead +of renewal-caps. The Renewer can't generate these one-per-SSID messages from +the SSID=* message, because it doesn't have a privkey to make the correct +signatures. So without some other mechanism, we're stuck with these +relatively coarse authorities. + +If we want to limit this sort of authority, then we need to introduce a new +method. The client begins by generating a new DSA keypair. Then it signs a +message that declares the new pubkey to be valid for a specific subset of +storage operations (such as "renew-lease SI=123 SSID=*"). Then it delivers +the new privkey, the declaration message, and the membership card to the +Renewer. The renewer uses the new privkey to sign its own one-per-SSID +request message for each server, then sends the (signed request, declaration, +membership card) triple to the server. The server needs to perform three +verification checks per message: first the membership card, then the +declaration message, then the actual request message. + +== Other Enhancements == + +If a given authority is likely to be used multiple times, the same +give-me-a-FURL trick can be used to cut down on the number of public key +operations that must be performed. This is trickier with the per-SI messages. + +When storing the manifest, things like the membership card should be +amortized across a set of common entries. An isolated renew-cap needs to +contain the verifycap, the signed renewal request, and the membership card. +But a manifest with a thousand entries should only include one copy of the +membership card. + +It might be sensible to define a signed renewal request that grants authority +for a set of storage indicies, so that the signature can be shared among +several entries (to save space and perhaps processing time). The request +could include a Bloom filter of authorized SI values: when the request is +actually sent to the server, the renewer would add a list of actual SI values +to renew, and the server would accept all that are contained in the filter. + +== Revocation == + +The lifetime of the storage authority included in the manifest's renew-caps +or repair-caps will determine the lifetime of those caps. In particular, if +we implement account revocation by using time-limited membership cards +(requiring the client to get a new card once a month), then the repair-caps +won't work for more than a month, which kind of defeats the purpose. + +A related issue is the FURL-shortcut: the MAC'ed message needs to include a +validity period of some sort, and if the client tries to use a old FURL they +should get an error message that will prompt them to try and acquire a newer +one. + +------------------------------ + +The client can produce a repair-cap manifest for a specific Repairer's +pubkey, so it can produce a signed message that includes the pubkey (instead +of needing to generate a new privkey just for this purpose). The result is +not a capability, since it can only be used by the holder of the +corresponding privkey. + +So the generic form of the storage operation message is the request (which +has all the argument values filled in), followed by a chain of +authorizations. The first authorization must be signed by the Account +Server's key. Each authorization must be signed by the key mentioned in the +previous one. Each one adds a new limitation on the power of the following +ones. The actual request is bounded by all the limitations of the chain. + +The membership card is an authorization that simply limits the account number +that can be used: "op=* SI=* SSID=* account=4 signed-by=CLIENT-PUBKEY". + +So a repair manifest created for a Repairer with pubkey ABCD could consist of +a list of verifycaps plus a single authorization (using a Bloom filter to +identify the SIs that were allowed): + + SIGN("allocate SI=[bloom] SSID=* signed-by=ABCD") + +If/when the Repairer needed to allocate a share, it would use its own privkey +to sign an additional message and send the whole list to the SS: + + request=allocate SI=1234 SSID=EEFS account=4 shnum=2 + SIGN("allocate SI=1234 SSID=EEFS", ABCD) + SIGN("allocate SI=[bloom] SSID=* signed-by=ABCD", clientkey) + membership: SIGN("op=* SI=* SSID=* account=4 signed-by=clientkey", ASkey) + [implicit]: ASkey + +---------------------------------------- + +Things would be a lot simpler if the Repairer (actually the Re-Leaser) had +everybody's account authority. + +One simplifying approach: the Repairer/Re-Leaser has its own account, and the +shares it creates are leased under that account number. The R/R keeps track +of which leases it has created for whom. When the client eventually comes +back online, it is told to perform a re-leasing run, and after that occurs +the R/R can cancel its own temporary leases. + +This would effectively transfer storage quota from the original client to the +R/R over time (as shares are regenerated by the R/R while the client remains +offline). If the R/R is centrally managed, the quota mechanism can sum the +R/R's numbers with the SS's numbers when determining how much storage is +consumed by any given account. Not quite as clean as storing the exact +information in the SS's lease tables directly, but: + + * the R/R no longer needs any special account authority (it merely needs an + accurate account number, which can be supplied by giving the client a + specific facet that is bound to that account number) + * the verify-cap manifest is sufficient to perform repair + * no extra DSA keys are necessary + * account authority could be implemented with either DSA keys or personal SS + facets: i.e. we don't need the delegability aspects of DSA keys for use by + the repair mechanism (we might still want them to simplify introduction). + +I *think* this would eliminate all that complexity of chained authorization +messages.