From: Zooko O'Whielacronx Date: Mon, 30 Apr 2007 16:57:52 +0000 (-0700) Subject: extirpate all references the "queen" and "metatracker" X-Git-Tag: allmydata-tahoe-0.2.0~57 X-Git-Url: https://git.rkrishnan.org/Site/Content/Exhibitors/nxhtml.html?a=commitdiff_plain;h=d19d1058e028b932360459aaad51a25451fc2c1a;p=tahoe-lafs%2Ftahoe-lafs.git extirpate all references the "queen" and "metatracker" This is a potentially disruptive and potentially ugly change to the code base, because I renamed the object that serves in both roles from "Queen" to "IntroducerAndVdrive", which is a bit of an ugly name. However, I think that clarity is important enough in this release to make this change. All unit tests pass. I'm now darcs recording this patch in order to pull it to other machines for more testing. --- diff --git a/GNUmakefile b/GNUmakefile index 5417a82e..c99a2ae1 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -158,7 +158,7 @@ clean: clean-zfec clean-Crypto clean-foolscap rm -rf instdir create_dirs: - mkdir -p queen-basedir + mkdir -p introducer_and_vdrive-basedir mkdir -p client-basedir mkdir -p client-basedir2 mkdir -p client-basedir/storage diff --git a/README b/README index 1da96d54..cdeb5aa1 100644 --- a/README +++ b/README @@ -157,14 +157,14 @@ RUNNING: directory, inside of which you can add files to configure and control the node. Nodes also read and write files within that directory. - A mesh consists of a single central 'introducer' node and a large number of - 'client' nodes. If you are joining an existing mesh, the introducer node - will already be running, and you'll just need to create a client node. If - you're creating a brand new mesh, you'll need to create both an introducer - and a client (and then invite other people to create their own client nodes - and join your mesh). - - The introducer node is constructed by running 'allmydata-tahoe + A mesh consists of a single central 'introducer and vdrive' node and a large + number of 'client' nodes. If you are joining an existing mesh, the + introducer-and-vdrive node will already be running, and you'll just need to + create a client node. If you're creating a brand new mesh, you'll need to + create both an introducer-and-vdrive and a client (and then invite other + people to create their own client nodes and join your mesh). + + The introducer (-and-vdrive) node is constructed by running 'allmydata-tahoe create-introducer --basedir $HERE'. Once constructed, you can start the introducer by running 'allmydata-tahoe start --basedir $HERE' (or, if you are already in the introducer's base directory, just type 'allmydata-tahoe diff --git a/docs/architecture.txt b/docs/architecture.txt index 581ec958..e026b417 100644 --- a/docs/architecture.txt +++ b/docs/architecture.txt @@ -6,11 +6,11 @@ OVERVIEW The high-level view of this system consists of three layers: the mesh, the virtual drive, and the application that sits on top. -The lowest layer is the "mesh" or "cloud", basically a DHT (Distributed Hash -Table) which maps URIs to data. The URIs are relatively short ascii strings +The lowest layer is the "mesh", basically a DHT (Distributed Hash Table) +which maps URIs to data. The URIs are relatively short ascii strings (currently about 140 bytes), and each is used as references to an immutable arbitrary-length sequence of data bytes. This data is distributed around the -cloud in a large number of nodes, such that a statistically unlikely number +mesh in a large number of nodes, such that a statistically unlikely number of nodes would have to be unavailable for the data to become unavailable. The middle layer is the virtual drive: a tree-shaped data structure in which @@ -32,14 +32,13 @@ actual code present in the current release. Please take a look at roadmap.txt to get an idea of how much of this has been implemented so far. -THE BIG CLOUD OF PEERS +THE BIG MESH OF PEERS -Underlying the mesh/cloud is a large collection of peer nodes. These are -processes running on a wide variety of computers, all of which know about -each other in some way or another. They establish TCP connections to one -another using Foolscap, an encrypted+authenticated remote message passing -library (using TLS connections and self-authenticating identifiers called -"FURLs"). +Underlying the mesh is a large collection of peer nodes. These are processes +running on a wide variety of computers, all of which know about each other in +some way or another. They establish TCP connections to one another using +Foolscap, an encrypted+authenticated remote message passing library (using +TLS connections and self-authenticating identifiers called "FURLs"). Each peer offers certain services to the others. The primary service is the StorageServer, which offers to hold data for a limited period of time (a @@ -64,12 +63,12 @@ safety measure. In this release, peers learn about each other through the "introducer". Each peer connects to this central introducer at startup, and receives a list of all other peers from it. Each peer then connects to all other peers, creating -a full-mesh topology. Future versions will reduce the number of connections -considerably, to enable the mesh to scale to larger sizes: the design target -is one million nodes. In addition, future versions will offer relay and -NAT-traversal services to allow nodes without full internet connectivity to -participate. In the current release, only one node may be behind a NAT box -and still permit the cloud to achieve full-mesh connectivity. +a fully-connected topology. Future versions will reduce the number of +connections considerably, to enable the mesh to scale to larger sizes: the +design target is one million nodes. In addition, future versions will offer +relay and NAT-traversal services to allow nodes without full internet +connectivity to participate. In the current release, only one node may be +behind a NAT box and still permit the mesh to achieve full connectivity. FILE ENCODING @@ -166,7 +165,7 @@ which ones? The "peer selection" algorithm is used to make this choice. In the current version, the verifierid is used to consistently-permute the set of all peers (by sorting the peers by HASH(verifierid+peerid)). Each file gets a different permutation, which (on average) will evenly distribute -shares among the cloud and avoid hotspots. +shares among the mesh and avoid hotspots. This permutation places the peers around a 2^256-sized ring, like the rim of a big clock. The 100-or-so shares are then placed around the same ring (at 0, diff --git a/docs/codemap.txt b/docs/codemap.txt index a26f51a6..b61a3681 100644 --- a/docs/codemap.txt +++ b/docs/codemap.txt @@ -23,11 +23,11 @@ Within src/allmydata/ : node.py: the base Node, which handles connection establishment and application startup - client.py, queen.py: two specialized subclasses of Node, for users + client.py, introducer_and_vdrive.py: two specialized subclasses of Node, for users and the central introducer/vdrive handler, respectively introducer.py: node introduction handlers, client is used by client.py, - server is used by queen.py + server is used by introducer_and_vdrive.py storageserver.py: provides storage services to other nodes @@ -40,7 +40,7 @@ Within src/allmydata/ : filetable.py, vdrive.py: implements the current one-global-vdrive layer, part runs on client nodes, part runs on the - central vdrive handler (aka the 'queen') + central vdrive handler webish.py, web/*.xhtml: provides the web frontend, using a Nevow server @@ -57,11 +57,11 @@ Within src/allmydata/ : test/*.py: unit tests -Both the client and the central-queen node runs as a tree of +Both the client and the central introducer-and-vdrive node runs as a tree of (twisted.application.service) Service instances. The Foolscap "Tub" is one of -these. Client nodes have an Uploader service and a Downloader service that -turn data into URIs and back again. They also have a VDrive service which -provides access to the single global shared filesystem. +these. Client nodes have an Uploader service and a Downloader service that turn +data into URIs and back again. They also have a VDrive service which provides +access to the single global shared filesystem. The Uploader is given an "upload source" (which could be an open filehandle, a filename on local disk, or even a string), and returns a Deferred that diff --git a/docs/denver.txt b/docs/denver.txt index 9ba5795f..a4c3e1c3 100644 --- a/docs/denver.txt +++ b/docs/denver.txt @@ -144,7 +144,7 @@ Properties of this approach: each node must maintain open (keep-alived) connections to something like 2*log2(N) peers. In tahoe2, this number is 0 (well, probably 1 for the - queen). + introducer). during upload, each node must actively use 100 connections to a random set of peers to push data (just like tahoe2). diff --git a/misc/simulator.py b/misc/simulator.py index 5a53fadb..06b505a4 100644 --- a/misc/simulator.py +++ b/misc/simulator.py @@ -16,9 +16,9 @@ def randomid(): return os.urandom(20) class Node: - def __init__(self, nid, queen, simulator): + def __init__(self, nid, introducer_and_vdrive, simulator): self.nid = nid - self.queen = queen + self.introducer_and_vdrive = introducer_and_vdrive self.simulator = simulator self.shares = {} self.capacity = random.randrange(1000) @@ -27,7 +27,7 @@ class Node: def permute_peers(self, fileid): permuted = [(sha(fileid+n.nid),n) - for n in self.queen.get_all_nodes()] + for n in self.introducer_and_vdrive.get_all_nodes()] permuted.sort() return permuted @@ -50,7 +50,7 @@ class Node: node.delete_share(fileid) return False self.files.append((fileid, numshares)) - self.queen.please_preserve(fileid, size, tried, last_givento) + self.introducer_and_vdrive.please_preserve(fileid, size, tried, last_givento) return (True, tried) def accept_share(self, fileid, sharesize): @@ -111,10 +111,10 @@ class Node: which = random.choice(self.files) self.files.remove(which) fileid,numshares = which - self.queen.delete(fileid) + self.introducer_and_vdrive.delete(fileid) return True -class Queen: +class IntroducerAndVdrive: def __init__(self, simulator): self.living_files = {} self.utilization = 0 # total size of all active files @@ -169,7 +169,7 @@ class Simulator: self.rrd = RRD("/tmp/utilization.rrd", ds=[ds], rra=[rra], start=self.time) self.rrd.create() - self.queen = q = Queen(self) + self.introducer_and_vdrive = q = IntroducerAndVdrive(self) self.all_nodes = [Node(randomid(), q, self) for i in range(self.NUM_NODES)] q.all_nodes = self.all_nodes @@ -267,7 +267,7 @@ class Simulator: avg_tried = "NONE" else: avg_tried = sum(self.published_files) / len(self.published_files) - print time, etype, self.added_data, self.failed_files, self.lost_data_bytes, avg_tried, len(self.queen.living_files), self.queen.utilization + print time, etype, self.added_data, self.failed_files, self.lost_data_bytes, avg_tried, len(self.introducer_and_vdrive.living_files), self.introducer_and_vdrive.utilization global s s = None diff --git a/relnotes.txt b/relnotes.txt index 2198f4c8..79f88775 100644 --- a/relnotes.txt +++ b/relnotes.txt @@ -40,7 +40,7 @@ there, with download links, and forms to upload new files. Other ways to access the filesystem are planned, as well as other structures than the single globally-shared namespace implemented by this release: please -see the roadmap.txt for some rough details. +see the roadmap.txt [5] for some rough details. HACKING AND COMMUNITY diff --git a/roadmap.txt b/roadmap.txt index ca54ff5c..81f55133 100644 --- a/roadmap.txt +++ b/roadmap.txt @@ -2,8 +2,8 @@ ['*' means complete] Connection Management: -*v1: foolscap, no relay, live=connected-to-queen, broadcast updates, full mesh - v2: live != connected-to-queen, connect on demand +*v1: foolscap, no relay, live=connected-to-introducer, broadcast updates, fully connected topology + v2: live != connected-to-introducer, connect on demand v3: relay? File Encoding: @@ -39,10 +39,10 @@ Download Peer Selection: v3: denver airport? Filetable Maintenance: -*v1: queen-based tree of MutableDirectoryNodes, persisted to queen's disk +*v1: vdrive-based tree of MutableDirectoryNodes, persisted to vdrive's disk no accounts v2: move tree to client side, serialize to a file, upload, - queen.set_filetable_uri (still no accounts, just one global tree) + vdrive.set_filetable_uri (still no accounts, just one global tree) v3: break world up into accounts, separate mutable spaces. Maybe implement SSKs v4: filetree @@ -68,18 +68,13 @@ UI: v4: FUSE Operations/Deployment/Doc/Free Software/Community: - - tweak licence as per zfec's recent tweak - - move this file into the wiki - - Windows port and testing - - Trac instance - - extirpate all references to "queen" and "metatracker" - - set up public trac, buildbot reports, mailing list, download page + - move this file into the wiki ? back pocket ideas: when nodes are unable to reach storage servers, make a note of it, inform - queen eventually. queen then puts server under observation or otherwise - looks for differences between their self-reported availability and the - experiences of others + verifier/checker eventually. verifier/checker then puts server under + observation or otherwise looks for differences between their self-reported + availability and the experiences of others store filetable URI in the first 10 peers that appear after your own nodeid each entry has a sequence number, maybe a timestamp diff --git a/src/allmydata/filetree/interfaces.py b/src/allmydata/filetree/interfaces.py index 4ce08659..a40712b3 100644 --- a/src/allmydata/filetree/interfaces.py +++ b/src/allmydata/filetree/interfaces.py @@ -106,8 +106,8 @@ class ISubTree(Interface): Each subtree's populate_from_node() method is expected to use the downloader to obtain a file with the subtree's serialized contents - (probably by pulling data from some source, like the mesh, the queen, - an HTTP server, or somewhere on the local filesystem), then + (probably by pulling data from some source, like the mesh, the vdrive + server, an HTTP server, or somewhere on the local filesystem), then unserialize them and populate the subtree's state. Return a Deferred that will fire (with self) when this subtree is @@ -250,8 +250,8 @@ class ISubTreeMaker(Interface): I accept an INode-providing specification of a subtree, and return a Deferred that fires with an ISubTree-providing instance. I will perform network IO and download the serialized data that the INode - references, if necessary, or ask the queen (or other provider) for a - pointer, or read it from local disk. + references, if necessary, or ask the vdrive server (or other provider) + for a pointer, or read it from local disk. """ diff --git a/src/allmydata/filetree/nodemaker.py b/src/allmydata/filetree/nodemaker.py index d38458d8..05a1364a 100644 --- a/src/allmydata/filetree/nodemaker.py +++ b/src/allmydata/filetree/nodemaker.py @@ -13,9 +13,9 @@ all_node_types = [ file.CHKFileNode, file.SSKFileNode, redirect.LocalFileRedirectionNode, - redirect.QueenRedirectionNode, + redirect.VdriveRedirectionNode, redirect.HTTPRedirectionNode, - redirect.QueenOrLocalFileRedirectionNode, + redirect.VdriveOrLocalFileRedirectionNode, ] class NodeMaker(object): diff --git a/src/allmydata/filetree/redirect.py b/src/allmydata/filetree/redirect.py index 0a6fd8af..90bbcc2b 100644 --- a/src/allmydata/filetree/redirect.py +++ b/src/allmydata/filetree/redirect.py @@ -102,24 +102,24 @@ class LocalFileRedirection(_BaseRedirection): self._update() return None -class QueenRedirectionNode(LocalFileRedirectionNode): - prefix = "QueenRedirection" +class VdriveRedirectionNode(LocalFileRedirectionNode): + prefix = "VdriveRedirection" -class QueenRedirection(_BaseRedirection): - node_class = QueenRedirectionNode +class VdriveRedirection(_BaseRedirection): + node_class = VdriveRedirectionNode def new(self, handle): self.handle = handle return self def populate_from_node(self, node, parent_is_mutable, node_maker, downloader): - # this specifies a handle for which the Queen maintains a serialized + # this specifies a handle for which the Vdrive maintains a serialized # subtree specification. - assert isinstance(node, QueenRedirectionNode) + assert isinstance(node, VdriveRedirectionNode) self.handle = node.handle - # TODO: queen? - d = self._queen.callRemote("lookup_handle", self.handle) + # TODO: vdrive? + d = self._vdrive.callRemote("lookup_handle", self.handle) d.addCallback(self._populate_from_data, node_maker) return d @@ -127,30 +127,30 @@ class QueenRedirection(_BaseRedirection): return True # TODO: maybe, maybe not def create_node_now(self): - return QueenRedirectionNode().new(self.handle) + return VdriveRedirectionNode().new(self.handle) def update_now(self, uploader): f = StringIO() self.serialize_subtree_to_file(f) - d = self._queen.callRemote("set_handle", self.handle, f.getvalue()) + d = self._vdrive.callRemote("set_handle", self.handle, f.getvalue()) def _done(res): return self.create_node_now() d.addCallback(_done) return d def update(self, workqueue): - f, filename = workqueue.create_tempfile(".toqueen") + f, filename = workqueue.create_tempfile(".tovdrive") self.serialize_subtree_to_file(f) f.close() - workqueue.add_queen_update_handle(self.handle, filename) + workqueue.add_vdrive_update_handle(self.handle, filename) workqueue.add_delete_tempfile(filename) return None -class QueenOrLocalFileRedirectionNode(LocalFileRedirectionNode): - prefix = "QueenOrLocalFileRedirection" +class VdriveOrLocalFileRedirectionNode(LocalFileRedirectionNode): + prefix = "VdriveOrLocalFileRedirection" -class QueenOrLocalFileRedirection(_BaseRedirection): - node_class = QueenOrLocalFileRedirectionNode +class VdriveOrLocalFileRedirection(_BaseRedirection): + node_class = VdriveOrLocalFileRedirectionNode def new(self, handle, child_node): self.handle = handle @@ -161,9 +161,9 @@ class QueenOrLocalFileRedirection(_BaseRedirection): def populate_from_node(self, node, parent_is_mutable, node_maker, downloader): # there is a local file which contains a bencoded serialized - # subtree specification. The queen also has a copy. Whomever has + # subtree specification. The vdrive also has a copy. Whomever has # the higher version number wins. - assert isinstance(node, QueenOrLocalFileRedirectionNode) + assert isinstance(node, VdriveOrLocalFileRedirectionNode) self.filename = self.handle = node.handle f = open(self.filename, "rb") @@ -171,19 +171,19 @@ class QueenOrLocalFileRedirection(_BaseRedirection): local_version_and_data = f.read() f.close() - # TODO: queen? - # TODO: pubsub so we can cache the queen's results - d = self._queen.callRemote("lookup_handle", self.handle) + # TODO: vdrive? + # TODO: pubsub so we can cache the vdrive's results + d = self._vdrive.callRemote("lookup_handle", self.handle) d.addCallback(self._choose_winner, local_version_and_data) d.addCallback(self._populate_from_data, node_maker) return d - def _choose_winner(self, queen_version_and_data, local_version_and_data): - queen_version, queen_data = bencode.bdecode(queen_version_and_data) + def _choose_winner(self, vdrive_version_and_data, local_version_and_data): + vdrive_version, vdrive_data = bencode.bdecode(vdrive_version_and_data) local_version, local_data = bencode.bdecode(local_version_and_data) - if queen_version > local_version: - data = queen_data - self.version = queen_version + if vdrive_version > local_version: + data = vdrive_data + self.version = vdrive_version else: data = local_data self.version = local_version @@ -194,7 +194,7 @@ class QueenOrLocalFileRedirection(_BaseRedirection): return True def create_node_now(self): - return QueenOrLocalFileRedirectionNode().new(self.handle) + return VdriveOrLocalFileRedirectionNode().new(self.handle) def _update(self): self.version += 1 @@ -209,7 +209,7 @@ class QueenOrLocalFileRedirection(_BaseRedirection): f.write(version_and_data) f.close() - d = self._queen.callRemote("set_handle", self.handle, version_and_data) + d = self._vdrive.callRemote("set_handle", self.handle, version_and_data) def _done(res): return self.create_node_now() d.addCallback(_done) @@ -222,10 +222,10 @@ class QueenOrLocalFileRedirection(_BaseRedirection): f.write(version_and_data) f.close() - f, filename = workqueue.create_tempfile(".toqueen") + f, filename = workqueue.create_tempfile(".tovdrive") self.serialize_subtree_to_file(f) f.close() - workqueue.add_queen_update_handle(self.handle, filename) + workqueue.add_vdrive_update_handle(self.handle, filename) workqueue.add_delete_tempfile(filename) return None diff --git a/src/allmydata/filetree/vdrive.py b/src/allmydata/filetree/vdrive.py index 10305fb1..883a1750 100644 --- a/src/allmydata/filetree/vdrive.py +++ b/src/allmydata/filetree/vdrive.py @@ -18,19 +18,18 @@ all_openable_subtree_types = [ directory.CHKDirectorySubTree, directory.SSKDirectorySubTree, redirect.LocalFileRedirection, - redirect.QueenRedirection, - redirect.QueenOrLocalFileRedirection, + redirect.VdriveRedirection, + redirect.VdriveOrLocalFileRedirection, redirect.HTTPRedirection, ] class SubTreeMaker(object): implements(ISubTreeMaker) - def __init__(self, queen, downloader): + def __init__(self, downloader): # this is created with everything it might need to download and - # create subtrees. That means a Downloader and a reference to the - # queen. - self._queen = queen + # create subtrees. That means a Downloader and in the future (?) a + # reference to the vdrive. assert IDownloader(downloader) self._downloader = downloader self._node_maker = NodeMaker() @@ -85,10 +84,8 @@ class VirtualDrive(object): workqueue.set_uploader(uploader) self._downloader = downloader self._uploader = uploader - # TODO: queen? - self.queen = None self.root_node = root_node - self.subtree_maker = SubTreeMaker(self.queen, downloader) + self.subtree_maker = SubTreeMaker(downloader) # these methods are used to walk through our subtrees @@ -276,7 +273,7 @@ class VirtualDrive(object): # really say whether they're mutable or not. But we're pretty sure # that the new subtree is supposed to be mutable, because we asserted # that earlier (although I suppose perhaps someone could change a - # QueenRedirection or an SSK file while we're offline in the middle + # VdriveRedirection or an SSK file while we're offline in the middle # of our workqueue..). Tell the new subtree that their parent is # mutable so we can be sure it will believe that it itself is # mutable. diff --git a/src/allmydata/interfaces.py b/src/allmydata/interfaces.py index 346019ab..2b28371a 100644 --- a/src/allmydata/interfaces.py +++ b/src/allmydata/interfaces.py @@ -531,10 +531,10 @@ class IWorkQueue(Interface): by the parent. """ - def add_queen_update_handle(handle, source_filename): - """Arrange for a central queen to be notified that the given handle + def add_vdrive_update_handle(handle, source_filename): + """Arrange for a vdrive server to be notified that the given handle has been updated with the contents of the given tempfile. This will - send a set_handle() message to the queen.""" + send a set_handle() message to the vdrive.""" def add_retain_ssk(read_capability): """Arrange for the given SSK to be kept alive.""" @@ -582,7 +582,7 @@ class IWorkQueue(Interface): subtree with the contents. If 'subtree_node' refers to a redirection subtree like - LocalFileRedirection or QueenRedirection, then 'localpath' is + LocalFileRedirection or VdriveRedirection, then 'localpath' is ignored, because redirection subtrees don't consume path components and have no internal directory structure (they just have the one redirection target). Redirection subtrees generally retain a constant diff --git a/src/allmydata/introducer_and_vdrive.py b/src/allmydata/introducer_and_vdrive.py new file mode 100644 index 00000000..cfea26e2 --- /dev/null +++ b/src/allmydata/introducer_and_vdrive.py @@ -0,0 +1,32 @@ + +import os.path +from allmydata import node +from allmydata.filetable import GlobalVirtualDrive +from allmydata.introducer import Introducer + + +class IntroducerAndVdrive(node.Node): + CERTFILE = "introducer.pem" + PORTNUMFILE = "introducer.port" + NODETYPE = "introducer" + + def __init__(self, basedir="."): + node.Node.__init__(self, basedir) + self.urls = {} + + def tub_ready(self): + r = self.add_service(Introducer()) + self.urls["introducer"] = self.tub.registerReference(r, "introducer") + self.log(" introducer is at %s" % self.urls["introducer"]) + f = open(os.path.join(self.basedir, "introducer.furl"), "w") + f.write(self.urls["introducer"] + "\n") + f.close() + + gvd = self.add_service(GlobalVirtualDrive(self.basedir)) + self.urls["vdrive"] = self.tub.registerReference(gvd.get_root(), + "vdrive") + self.log(" vdrive is at %s" % self.urls["vdrive"]) + f = open(os.path.join(self.basedir, "vdrive.furl"), "w") + f.write(self.urls["vdrive"] + "\n") + f.close() + diff --git a/src/allmydata/manhole.py b/src/allmydata/manhole.py index c0858dc9..0015eed9 100644 --- a/src/allmydata/manhole.py +++ b/src/allmydata/manhole.py @@ -141,7 +141,7 @@ class _BaseManhole(service.MultiService): def makeNamespace(): # close over 'self' so we can get access to .parent later from allmydata import debugshell - debugshell.app = self.parent # make client/queen accesible via 'app' + debugshell.app = self.parent # make node accessible via 'app' namespace = {} for sym in dir(debugshell): if sym.startswith('__') and sym.endswith('__'): diff --git a/src/allmydata/node.py b/src/allmydata/node.py index fa41a5c3..d9f49672 100644 --- a/src/allmydata/node.py +++ b/src/allmydata/node.py @@ -12,8 +12,8 @@ import zfec import foolscap class Node(service.MultiService): - # this implements common functionality of both Client nodes and the Queen - # node. + # this implements common functionality of both Client nodes, Introducer + # nodes, and Vdrive nodes NODETYPE = "unknown NODETYPE" PORTNUMFILE = None CERTFILE = None diff --git a/src/allmydata/queen.py b/src/allmydata/queen.py deleted file mode 100644 index 91e0cf93..00000000 --- a/src/allmydata/queen.py +++ /dev/null @@ -1,32 +0,0 @@ - -import os.path -from allmydata import node -from allmydata.filetable import GlobalVirtualDrive -from allmydata.introducer import Introducer - - -class Queen(node.Node): - CERTFILE = "introducer.pem" - PORTNUMFILE = "introducer.port" - NODETYPE = "introducer" - - def __init__(self, basedir="."): - node.Node.__init__(self, basedir) - self.urls = {} - - def tub_ready(self): - r = self.add_service(Introducer()) - self.urls["introducer"] = self.tub.registerReference(r, "introducer") - self.log(" introducer is at %s" % self.urls["introducer"]) - f = open(os.path.join(self.basedir, "introducer.furl"), "w") - f.write(self.urls["introducer"] + "\n") - f.close() - - gvd = self.add_service(GlobalVirtualDrive(self.basedir)) - self.urls["vdrive"] = self.tub.registerReference(gvd.get_root(), - "vdrive") - self.log(" vdrive is at %s" % self.urls["vdrive"]) - f = open(os.path.join(self.basedir, "vdrive.furl"), "w") - f.write(self.urls["vdrive"] + "\n") - f.close() - diff --git a/src/allmydata/scripts/runner.py b/src/allmydata/scripts/runner.py index d6265697..94acfdb9 100644 --- a/src/allmydata/scripts/runner.py +++ b/src/allmydata/scripts/runner.py @@ -110,10 +110,10 @@ c.setServiceParent(application) introducer_tac = """ # -*- python -*- -from allmydata import queen +from allmydata import introducer_and_vdrive from twisted.application import service -c = queen.Queen() +c = introducer_and_vdrive.IntroducerAndVdrive() application = service.Application("allmydata_introducer") c.setServiceParent(application) diff --git a/src/allmydata/test/check_memory.py b/src/allmydata/test/check_memory.py index 4745c4a2..a9f0de16 100644 --- a/src/allmydata/test/check_memory.py +++ b/src/allmydata/test/check_memory.py @@ -4,7 +4,7 @@ import os, shutil from twisted.internet import defer, reactor, protocol, error from twisted.application import service -from allmydata import client, queen +from allmydata import client, introducer_and_vdrive from allmydata.scripts import runner from foolscap.eventual import eventually, flushEventualQueue from twisted.python import log @@ -32,7 +32,7 @@ class SystemFramework: def start(self): print "STARTING" - d = self.make_queen() + d = self.make_introducer_and_vdrive() def _more(res): self.make_nodes() self.start_client() @@ -56,23 +56,23 @@ class SystemFramework: s.setServiceParent(self.sparent) return s - def make_queen(self): - queendir = os.path.join(self.basedir, "queen") - os.mkdir(queendir) - self.queen = self.add_service(queen.Queen(basedir=queendir)) - d = self.queen.when_tub_ready() + def make_introducer_and_vdrive(self): + introducer_and_vdrive_dir = os.path.join(self.basedir, "introducer_and_vdrive") + os.mkdir(introducer_and_vdrive_dir) + self.introducer_and_vdrive = self.add_service(introducer_and_vdrive.IntroducerAndVdrive(basedir=introducer_and_vdrive_dir)) + d = self.introducer_and_vdrive.when_tub_ready() return d def make_nodes(self): - q = self.queen - self.queen_pburl = q.urls["introducer"] + q = self.introducer_and_vdrive + self.introducer_furl = q.urls["introducer"] vdrive_furl = q.urls["vdrive"] self.nodes = [] for i in range(self.numnodes): nodedir = os.path.join(self.basedir, "node%d" % i) os.mkdir(nodedir) f = open(os.path.join(nodedir, "introducer.furl"), "w") - f.write(self.queen_pburl) + f.write(self.introducer_furl) f.close() f = open(os.path.join(nodedir, "vdrive.furl"), "w") f.write(vdrive_furl) @@ -80,7 +80,7 @@ class SystemFramework: c = self.add_service(client.Client(basedir=nodedir)) self.nodes.append(c) # the peers will start running, eventually they will connect to each - # other and the queen + # other and the introducer_and_vdrive def touch_keepalive(self): f = open(self.keepalive_file, "w") @@ -98,7 +98,7 @@ class SystemFramework: runner.create_client(config) log.msg("DONE MAKING CLIENT") f = open(os.path.join(clientdir, "introducer.furl"), "w") - f.write(self.queen_pburl + "\n") + f.write(self.introducer_furl + "\n") f.close() self.keepalive_file = os.path.join(clientdir, "suicide_prevention_hotline") self.touch_keepalive() diff --git a/src/allmydata/test/test_filetree_new.py b/src/allmydata/test/test_filetree_new.py index e1e0932a..79114ce5 100644 --- a/src/allmydata/test/test_filetree_new.py +++ b/src/allmydata/test/test_filetree_new.py @@ -425,7 +425,7 @@ class VDrive(unittest.TestCase): self.failUnlessEqual(c1a, c2a) def testDirectory(self): - stm = vdrive.SubTreeMaker(None, FakeMesh()) + stm = vdrive.SubTreeMaker(FakeMesh()) # create an empty directory (stored locally) subtree = directory.LocalFileSubTree() diff --git a/src/allmydata/test/test_introducer_and_vdrive.py b/src/allmydata/test/test_introducer_and_vdrive.py new file mode 100644 index 00000000..31338a79 --- /dev/null +++ b/src/allmydata/test/test_introducer_and_vdrive.py @@ -0,0 +1,15 @@ + +from twisted.trial import unittest +from foolscap.eventual import flushEventualQueue + +from allmydata import introducer_and_vdrive +from allmydata.util import testutil + +class Basic(testutil.SignalMixin, unittest.TestCase): + def test_loadable(self): + q = introducer_and_vdrive.IntroducerAndVdrive() + d = q.startService() + d.addCallback(lambda res: q.stopService()) + d.addCallback(flushEventualQueue) + return d + diff --git a/src/allmydata/test/test_queen.py b/src/allmydata/test/test_queen.py deleted file mode 100644 index 2f79bb81..00000000 --- a/src/allmydata/test/test_queen.py +++ /dev/null @@ -1,15 +0,0 @@ - -from twisted.trial import unittest -from foolscap.eventual import flushEventualQueue - -from allmydata import queen -from allmydata.util import testutil - -class Basic(testutil.SignalMixin, unittest.TestCase): - def test_loadable(self): - q = queen.Queen() - d = q.startService() - d.addCallback(lambda res: q.stopService()) - d.addCallback(flushEventualQueue) - return d - diff --git a/src/allmydata/test/test_system.py b/src/allmydata/test/test_system.py index 00e0dc60..67f75ccc 100644 --- a/src/allmydata/test/test_system.py +++ b/src/allmydata/test/test_system.py @@ -3,7 +3,7 @@ import os from twisted.trial import unittest from twisted.internet import defer, reactor from twisted.application import service -from allmydata import client, queen, uri, download +from allmydata import client, introducer_and_vdrive, uri, download from allmydata.util import idlib, fileutil, testutil from foolscap.eventual import flushEventualQueue from twisted.python import log @@ -38,17 +38,17 @@ class SystemTest(testutil.SignalMixin, unittest.TestCase): def set_up_nodes(self, NUMCLIENTS=5): self.numclients = NUMCLIENTS - queendir = self.getdir("queen") - if not os.path.isdir(queendir): - fileutil.make_dirs(queendir) - self.queen = self.add_service(queen.Queen(basedir=queendir)) - d = self.queen.when_tub_ready() + introducer_and_vdrive_dir = self.getdir("introducer_and_vdrive") + if not os.path.isdir(introducer_and_vdrive_dir): + fileutil.make_dirs(introducer_and_vdrive_dir) + self.introducer_and_vdrive = self.add_service(introducer_and_vdrive.IntroducerAndVdrive(basedir=introducer_and_vdrive_dir)) + d = self.introducer_and_vdrive.when_tub_ready() d.addCallback(self._set_up_nodes_2) return d def _set_up_nodes_2(self, res): - q = self.queen - self.queen_furl = q.urls["introducer"] + q = self.introducer_and_vdrive + self.introducer_furl = q.urls["introducer"] self.vdrive_furl = q.urls["vdrive"] self.clients = [] for i in range(self.numclients): @@ -57,7 +57,7 @@ class SystemTest(testutil.SignalMixin, unittest.TestCase): fileutil.make_dirs(basedir) if i == 0: open(os.path.join(basedir, "webport"), "w").write("tcp:0:interface=127.0.0.1") - open(os.path.join(basedir, "introducer.furl"), "w").write(self.queen_furl) + open(os.path.join(basedir, "introducer.furl"), "w").write(self.introducer_furl) open(os.path.join(basedir, "vdrive.furl"), "w").write(self.vdrive_furl) c = self.add_service(client.Client(basedir=basedir)) self.clients.append(c) @@ -77,7 +77,7 @@ class SystemTest(testutil.SignalMixin, unittest.TestCase): basedir = self.getdir("client%d" % client_num) if not os.path.isdir(basedir): fileutil.make_dirs(basedir) - open(os.path.join(basedir, "introducer.furl"), "w").write(self.queen_furl) + open(os.path.join(basedir, "introducer.furl"), "w").write(self.introducer_furl) open(os.path.join(basedir, "vdrive.furl"), "w").write(self.vdrive_furl) c = client.Client(basedir=basedir)