From: Brian Warner <warner@allmydata.com>
Date: Wed, 2 May 2007 22:20:40 +0000 (-0700)
Subject: import foolscap-0.1.3
X-Git-Tag: allmydata-tahoe-0.2.0~3
X-Git-Url: https://git.rkrishnan.org/pf/components?a=commitdiff_plain;h=ee9b0d1de60d50a055710e1257842c58eb72f908;p=tahoe-lafs%2Ftahoe-lafs.git
import foolscap-0.1.3
---
diff --git a/src/foolscap/ChangeLog b/src/foolscap/ChangeLog
index af1c82eb..06af6258 100644
--- a/src/foolscap/ChangeLog
+++ b/src/foolscap/ChangeLog
@@ -1,5 +1,15 @@
2007-05-02 Brian Warner <warner@lothar.com>
+ * foolscap/__init__.py: release Foolscap-0.1.3
+ * misc/{sid|sarge|dapper|edgy|feisty}/debian/changelog: same
+
+2007-05-02 Brian Warner <warner@lothar.com>
+
+ * MANIFEST.in: include some recently-added files to the source
+ tarball
+
+ * NEWS: update for the upcoming release
+
* foolscap/reconnector.py (Reconnector._failed): simplify
log/no-log logic
diff --git a/src/foolscap/MANIFEST.in b/src/foolscap/MANIFEST.in
index 017bcf38..026d37fa 100644
--- a/src/foolscap/MANIFEST.in
+++ b/src/foolscap/MANIFEST.in
@@ -1,5 +1,5 @@
include ChangeLog MANIFEST.in NEWS
-include doc/*.txt doc/*.xhtml
+include doc/*.txt doc/*.xhtml doc/*.css doc/template.tpl
include doc/listings/*.py
include doc/specifications/*.xhtml
include Makefile
@@ -8,3 +8,5 @@ include misc/edgy/debian/*
include misc/feisty/debian/*
include misc/sid/debian/*
include misc/sarge/debian/*
+include misc/testutils/*
+include misc/testutils/twisted/plugins/*
diff --git a/src/foolscap/NEWS b/src/foolscap/NEWS
index b193ff1e..8ef246fb 100644
--- a/src/foolscap/NEWS
+++ b/src/foolscap/NEWS
@@ -1,5 +1,202 @@
User visible changes in Foolscap (aka newpb/pb2). -*- outline -*-
+* Release 0.1.3 (02 May 2007)
+
+** Incompatibility Warning
+
+The 'keepalive' feature described below adds a new pair of banana tokens,
+PING and PONG, which introduces a compatibility break between 0.1.2 and 0.1.3
+. Older versions would throw an error upon receipt of a PING token, so the
+version-negotiation mechanism is used to prevent banana-v2 (0.1.2) peers from
+connecting to banana-v3 (0.1.3+) peers. Our negotiation mechanism would make
+it possible to detect the older (v2) peer and refrain from using PINGs, but
+that has not been done for this release.
+
+** Tubs must be running before use
+
+Tubs are twisted.application.service.Service instances, and as such have a
+clear distinction between "running" and "not running" states. Tubs are
+started by calling startService(), or by attaching them to a running service,
+or by starting the service that they are already attached to. The design rule
+in operation here is that Tubs are not allowed to perform network IO until
+they are running.
+
+This rule was not enforced completely in 0.1.2, and calls to
+getReference()/connectTo() that occurred before the Tub was started would
+proceed normally (initiating a TCP connection, etc). Starting with 0.1.3,
+this rule *is* enforced. For now, that means that you must start the Tub
+before calling either of these methods, or you'll get an exception. In a
+future release, that may be changed to allow these early calls, and queue or
+otherwise defer the network IO until the Tub is eventually started. (the
+biggest issue is how to warn users who forget to start the Tub, since in the
+face of such a bug the getReference will simply never complete).
+
+** Keepalives
+
+Tubs now keep track of how long a connection has been idle, and will send a
+few bytes (a PING of the other end) if no other traffic has been seen for
+roughly 4 to 8 minutes. This serves two purposes. The first is to convince an
+intervening NAT box that the connection is still in use, to prevent it from
+discarding the connection's table entry, since that would block any further
+traffic. The second is to accelerate the detection of such blocked
+connections, specifically to reduce the size of a window of buggy behavior in
+Foolscap's duplicate-connection detection/suppression code.
+
+This problem arises when client A (behind a low-end NAT box) connects to
+server B, perhaps using connectTo(). The first connection works fine, and is
+used for a while. Then, for whatever reason, A and B are silent for a long
+time (perhaps as short as 20 minutes, depending upon the NAT box). During
+this silence, A's NAT box thinks the connection is no longer in use and drops
+the address-translation table entry. Now suppose that A suddenly decides to
+talk to B. If the NAT box creates a new entry (with a new outbound port
+number), the packets that arrive on B will be rejected, since they do not
+match any existing TCP connections. A sees these rejected packets, breaks the
+TCP connection, and the Reconnector initiates a new connection. Meanwhile, B
+has no idea that anything has gone wrong. When the second connection reaches
+B, it thinks this is a duplicate connection from A, and that it already has a
+perfectly functional (albeit quiet) connection for that TubID, so it rejects
+the connection during the negotiation phase. A sees this rejection and
+schedules a new attempt, which ends in the same result. This has the
+potential to prevent hosts behind NAT boxes from ever reconnecting to the
+other end, at least until the the program at the far end is restarted, or it
+happens to try to send some traffic of its own.
+
+The same problem can occur if a laptop is abruptly shut down, or unplugged
+from the network, then moved to a different network. Similar problems have
+been seen with virtual machine instances that were suspended and moved to a
+different network.
+
+The longer-term fix for this is a deep change to the way duplicate
+connections (and cross-connect race conditions) are handled. The keepalives,
+however, mean that both sides are continually checking to see that the
+connection is still usable, enabling TCP to break the connection once the
+keepalives go unacknowledged for a certain amount of time. The default
+keepalive timer is 4 minutes, and due to the way it is implemented this means
+that no more than 8 minutes will pass without some traffic being sent. TCP
+tends to time out connections after perhaps 15 minutes of unacknowledged
+traffic, which means that the window of unconnectability is probably reduced
+from infinity down to about 25 minutes.
+
+The keepalive-sending timer defaults to 4 minutes, and can be changed by
+calling tub.setOption("keepaliveTimeout", seconds).
+
+In addition, an explicit disconnect timer can be enabled, which tells
+Foolscap to drop the connection unless traffic has been seen within some
+minimum span of time. This timer can be set by calling
+tub.setOption("disconnectTimeout", seconds). Obviously it should be set to a
+higher value than the keepaliveTimeout. This will close connections faster
+than TCP will. Both TCP disconnects and the ones triggered by this
+disconnectTimeout run the risk of false negatives, of course, in the face of
+unreliable networks.
+
+** New constraints
+
+When a tuple appears in a method constraint specification, it now maps to an
+actual TupleOf constraint. Previously they mapped to a ChoiceOf constraint.
+In practice, TupleOf appears to be much more useful, and thus better
+deserving of the shortcut.
+
+For example, a method defined as follows:
+
+ def get_employee(idnumber=int):
+ return (str, int, int) # (name, room_number, age)
+
+can only return a three-element tuple, in which the first element is a string
+(specifically it conforms to a default StringConstraint), and the second two
+elements are ints (which conform to a default IntegerConstraint, which means
+it fits in a 32-bit signed twos-complement value).
+
+To specify a constraint that can accept alternatives, use ChoiceOf:
+
+ def get_record(key=str):
+ """Return the record (a string) if it is present, or None if
+ it is not present."""
+ return ChoiceOf(str, None)
+
+UnicodeConstraint has been added, with minLength=, maxLength=, and regexp=
+arguments.
+
+The previous StringConstraint has been renamed to ByteStringConstraint (for
+accuracy), and it is defined to *only* accept string objects (not unicode
+objects). 'StringConstraint' itself remains equivalent to
+ByteStringConstraint for now, but in the future it may be redefined to be a
+constraint that accepts both bytestrings and unicode objects. To accomplish
+the bytestring-or-unicode constraint now, you might try
+schema.AnyStringConstraint, but it has not been fully tested, and might not
+work at all.
+
+** Bugfixes
+
+Errors during negotiation were sometimes delivered in the wrong format,
+resulting in a "token prefix is limited to 64 bytes" error message. Several
+error messages (including that one) have been improved to give developers a
+better chance of determining where the actual problem lies.
+
+RemoteReference.notifyOnDisconnect was buggy when called on a reference that
+was already broken: it failed to fire the callback. Now it fires the callback
+soon (using an eventual-send). This should remove a race condition from
+connectTo+notifyOnDisconnect sequences and allow them to operate reliably.
+notifyOnDisconnect() is now tolerant of attempts to remove something twice,
+which should make it easier to use safely.
+
+Remote methods which raise string exceptions should no longer cause Foolscap
+to explode. These sorts of exceptions are deprecated, of course, and you
+shouldn't use them, but at least they won't break Foolscap.
+
+The Reconnector class (accessed by tub.connectTo) was not correctly
+reconnecting in certain cases (which appeared to be particularly common on
+windows). This should be fixed now.
+
+CopyableSlicer did not work inside containers when streaming was enabled.
+Thanks to iacovou-AT-gmail.com for spotting this one.
+
+** Bugs not fixed
+
+Some bugs were identified and characterized but *not* fixed in this release
+
+*** RemoteInterfaces aren't defaulting to fully-qualified classnames
+
+When defining a RemoteInterface, you can specify its name with
+__remote_name__, or you can allow it to use the default name. Unfortunately,
+the default name is only the *local* name of the class, not the
+fully-qualified name, which means that if you have an RIFoo in two different
+.py files, they will wind up with the same name (which will cause an error on
+import, since all RemoteInterfaces known to a Foolscap-using program must
+have unique names).
+
+It turns out that it is rather difficult to determine the fully-qualified
+name of the RemoteInterface class early enough to be helpful. The workaround
+is to always add a __remote_name__ to your RemoteInterface classes. The
+recommendation is to use a globally-unique string, like a URI that includes
+your organization's DNS name.
+
+*** Constraints aren't constraining inbound tokens well enough
+
+Constraints (and the RemoteInterfaces they live inside) serve three purposes.
+The primary one is as documentation, describing how remotely-accessible
+objects behave. The second purpose is to enforce that documentation, by
+inspecting arguments (and return values) before invoking the method, as a
+form of precondition checking. The third is to mitigate denial-of-service
+attacks, in which an attacker sends so much data (or carefully crafted data)
+that the receiving program runs out of memory or stack space.
+
+It looks like several constraints are not correctly paying attention to the
+tokens as they arrive over the wire, such that the third purpose is not being
+achieved. Hopefully this will be fixed in a later release. Application code
+can be unaware of this change, since the constraints are still being applied
+to inbound arguments before they are passed to the method. Continue to use
+RemoteInterfaces as usual, just be aware that you are not yet protected
+against certain DoS attacks.
+
+** Use os.urandom instead of falling back to pycrypto
+
+Once upon a time, when Foolscap was compatible with python2.3 (which lacks
+os.urandom), we would try to use PyCrypto's random-number-generation routines
+when creating unguessable object identifiers (aka "SwissNumbers"). Now that
+we require python2.4 or later, this fallback has been removed, eliminating
+the last reference to pycrypto within the Foolscap source tree.
+
+
* Release 0.1.2 (04 Apr 2007)
** Bugfixes
diff --git a/src/foolscap/doc/using-pb.xhtml b/src/foolscap/doc/using-pb.xhtml
index f7e8c457..97005d01 100644
--- a/src/foolscap/doc/using-pb.xhtml
+++ b/src/foolscap/doc/using-pb.xhtml
@@ -750,7 +750,7 @@ classname (like <code>package.module.RIFoo</code>), so that two
RemoteInterfaces with the same name in different modules can co-exist. In the
current release, these two RemoteInterfaces will collide (and provoke an
import-time error message complaining about the duplicate name). As a result,
-if you have such classes (e.g. </code>foo.RIBar</code> and
+if you have such classes (e.g. <code>foo.RIBar</code> and
<code>baz.RIBar</code>), you <b>must</b> use <code>__remote_name__</code> to
distinguish them (by naming one of them something other than
<code>RIBar</code> to avoid this error.
diff --git a/src/foolscap/foolscap/__init__.py b/src/foolscap/foolscap/__init__.py
index 27f4006f..e2243f42 100644
--- a/src/foolscap/foolscap/__init__.py
+++ b/src/foolscap/foolscap/__init__.py
@@ -1,6 +1,6 @@
"""Foolscap"""
-__version__ = "0.1.2+"
+__version__ = "0.1.3"
# here are the primary entry points
from foolscap.pb import Tub, UnauthenticatedTub, getRemoteURL_TCP
diff --git a/src/foolscap/foolscap/pb.py b/src/foolscap/foolscap/pb.py
index e8ed1675..9f7cfb9d 100644
--- a/src/foolscap/foolscap/pb.py
+++ b/src/foolscap/foolscap/pb.py
@@ -598,6 +598,7 @@ class Tub(service.MultiService):
b = self._createLoopbackBroker(tubref)
# _createLoopbackBroker will call brokerAttached, which will add
# it to self.brokers
+ # TODO: stash this in self.brokers, so we don't create multiples
return defer.succeed(b)
d = defer.Deferred()
diff --git a/src/foolscap/misc/dapper/debian/changelog b/src/foolscap/misc/dapper/debian/changelog
index 0a18ccad..f3331be3 100644
--- a/src/foolscap/misc/dapper/debian/changelog
+++ b/src/foolscap/misc/dapper/debian/changelog
@@ -1,3 +1,9 @@
+foolscap (0.1.3) unstable; urgency=low
+
+ * new release
+
+ -- Brian Warner <warner@lothar.com> Wed, 2 May 2007 14:55:49 -0700
+
foolscap (0.1.2+) unstable; urgency=low
* bump revision while between releases
diff --git a/src/foolscap/misc/edgy/debian/changelog b/src/foolscap/misc/edgy/debian/changelog
index acdc7392..93eb7f3a 100644
--- a/src/foolscap/misc/edgy/debian/changelog
+++ b/src/foolscap/misc/edgy/debian/changelog
@@ -1,3 +1,9 @@
+foolscap (0.1.3) unstable; urgency=low
+
+ * new release
+
+ -- Brian Warner <warner@lothar.com> Wed, 2 May 2007 14:55:49 -0700
+
foolscap (0.1.2+) unstable; urgency=low
* bump revision while between releases
diff --git a/src/foolscap/misc/feisty/debian/changelog b/src/foolscap/misc/feisty/debian/changelog
index acdc7392..93eb7f3a 100644
--- a/src/foolscap/misc/feisty/debian/changelog
+++ b/src/foolscap/misc/feisty/debian/changelog
@@ -1,3 +1,9 @@
+foolscap (0.1.3) unstable; urgency=low
+
+ * new release
+
+ -- Brian Warner <warner@lothar.com> Wed, 2 May 2007 14:55:49 -0700
+
foolscap (0.1.2+) unstable; urgency=low
* bump revision while between releases
diff --git a/src/foolscap/misc/sarge/debian/changelog b/src/foolscap/misc/sarge/debian/changelog
index 0a18ccad..f3331be3 100644
--- a/src/foolscap/misc/sarge/debian/changelog
+++ b/src/foolscap/misc/sarge/debian/changelog
@@ -1,3 +1,9 @@
+foolscap (0.1.3) unstable; urgency=low
+
+ * new release
+
+ -- Brian Warner <warner@lothar.com> Wed, 2 May 2007 14:55:49 -0700
+
foolscap (0.1.2+) unstable; urgency=low
* bump revision while between releases
diff --git a/src/foolscap/misc/sid/debian/changelog b/src/foolscap/misc/sid/debian/changelog
index acdc7392..93eb7f3a 100644
--- a/src/foolscap/misc/sid/debian/changelog
+++ b/src/foolscap/misc/sid/debian/changelog
@@ -1,3 +1,9 @@
+foolscap (0.1.3) unstable; urgency=low
+
+ * new release
+
+ -- Brian Warner <warner@lothar.com> Wed, 2 May 2007 14:55:49 -0700
+
foolscap (0.1.2+) unstable; urgency=low
* bump revision while between releases