User visible changes in Foolscap (aka newpb/pb2). -*- outline -*-
+* Release 0.1.3 (02 May 2007)
+
+** Incompatibility Warning
+
+The 'keepalive' feature described below adds a new pair of banana tokens,
+PING and PONG, which introduces a compatibility break between 0.1.2 and 0.1.3
+. Older versions would throw an error upon receipt of a PING token, so the
+version-negotiation mechanism is used to prevent banana-v2 (0.1.2) peers from
+connecting to banana-v3 (0.1.3+) peers. Our negotiation mechanism would make
+it possible to detect the older (v2) peer and refrain from using PINGs, but
+that has not been done for this release.
+
+** Tubs must be running before use
+
+Tubs are twisted.application.service.Service instances, and as such have a
+clear distinction between "running" and "not running" states. Tubs are
+started by calling startService(), or by attaching them to a running service,
+or by starting the service that they are already attached to. The design rule
+in operation here is that Tubs are not allowed to perform network IO until
+they are running.
+
+This rule was not enforced completely in 0.1.2, and calls to
+getReference()/connectTo() that occurred before the Tub was started would
+proceed normally (initiating a TCP connection, etc). Starting with 0.1.3,
+this rule *is* enforced. For now, that means that you must start the Tub
+before calling either of these methods, or you'll get an exception. In a
+future release, that may be changed to allow these early calls, and queue or
+otherwise defer the network IO until the Tub is eventually started. (the
+biggest issue is how to warn users who forget to start the Tub, since in the
+face of such a bug the getReference will simply never complete).
+
+** Keepalives
+
+Tubs now keep track of how long a connection has been idle, and will send a
+few bytes (a PING of the other end) if no other traffic has been seen for
+roughly 4 to 8 minutes. This serves two purposes. The first is to convince an
+intervening NAT box that the connection is still in use, to prevent it from
+discarding the connection's table entry, since that would block any further
+traffic. The second is to accelerate the detection of such blocked
+connections, specifically to reduce the size of a window of buggy behavior in
+Foolscap's duplicate-connection detection/suppression code.
+
+This problem arises when client A (behind a low-end NAT box) connects to
+server B, perhaps using connectTo(). The first connection works fine, and is
+used for a while. Then, for whatever reason, A and B are silent for a long
+time (perhaps as short as 20 minutes, depending upon the NAT box). During
+this silence, A's NAT box thinks the connection is no longer in use and drops
+the address-translation table entry. Now suppose that A suddenly decides to
+talk to B. If the NAT box creates a new entry (with a new outbound port
+number), the packets that arrive on B will be rejected, since they do not
+match any existing TCP connections. A sees these rejected packets, breaks the
+TCP connection, and the Reconnector initiates a new connection. Meanwhile, B
+has no idea that anything has gone wrong. When the second connection reaches
+B, it thinks this is a duplicate connection from A, and that it already has a
+perfectly functional (albeit quiet) connection for that TubID, so it rejects
+the connection during the negotiation phase. A sees this rejection and
+schedules a new attempt, which ends in the same result. This has the
+potential to prevent hosts behind NAT boxes from ever reconnecting to the
+other end, at least until the the program at the far end is restarted, or it
+happens to try to send some traffic of its own.
+
+The same problem can occur if a laptop is abruptly shut down, or unplugged
+from the network, then moved to a different network. Similar problems have
+been seen with virtual machine instances that were suspended and moved to a
+different network.
+
+The longer-term fix for this is a deep change to the way duplicate
+connections (and cross-connect race conditions) are handled. The keepalives,
+however, mean that both sides are continually checking to see that the
+connection is still usable, enabling TCP to break the connection once the
+keepalives go unacknowledged for a certain amount of time. The default
+keepalive timer is 4 minutes, and due to the way it is implemented this means
+that no more than 8 minutes will pass without some traffic being sent. TCP
+tends to time out connections after perhaps 15 minutes of unacknowledged
+traffic, which means that the window of unconnectability is probably reduced
+from infinity down to about 25 minutes.
+
+The keepalive-sending timer defaults to 4 minutes, and can be changed by
+calling tub.setOption("keepaliveTimeout", seconds).
+
+In addition, an explicit disconnect timer can be enabled, which tells
+Foolscap to drop the connection unless traffic has been seen within some
+minimum span of time. This timer can be set by calling
+tub.setOption("disconnectTimeout", seconds). Obviously it should be set to a
+higher value than the keepaliveTimeout. This will close connections faster
+than TCP will. Both TCP disconnects and the ones triggered by this
+disconnectTimeout run the risk of false negatives, of course, in the face of
+unreliable networks.
+
+** New constraints
+
+When a tuple appears in a method constraint specification, it now maps to an
+actual TupleOf constraint. Previously they mapped to a ChoiceOf constraint.
+In practice, TupleOf appears to be much more useful, and thus better
+deserving of the shortcut.
+
+For example, a method defined as follows:
+
+ def get_employee(idnumber=int):
+ return (str, int, int) # (name, room_number, age)
+
+can only return a three-element tuple, in which the first element is a string
+(specifically it conforms to a default StringConstraint), and the second two
+elements are ints (which conform to a default IntegerConstraint, which means
+it fits in a 32-bit signed twos-complement value).
+
+To specify a constraint that can accept alternatives, use ChoiceOf:
+
+ def get_record(key=str):
+ """Return the record (a string) if it is present, or None if
+ it is not present."""
+ return ChoiceOf(str, None)
+
+UnicodeConstraint has been added, with minLength=, maxLength=, and regexp=
+arguments.
+
+The previous StringConstraint has been renamed to ByteStringConstraint (for
+accuracy), and it is defined to *only* accept string objects (not unicode
+objects). 'StringConstraint' itself remains equivalent to
+ByteStringConstraint for now, but in the future it may be redefined to be a
+constraint that accepts both bytestrings and unicode objects. To accomplish
+the bytestring-or-unicode constraint now, you might try
+schema.AnyStringConstraint, but it has not been fully tested, and might not
+work at all.
+
+** Bugfixes
+
+Errors during negotiation were sometimes delivered in the wrong format,
+resulting in a "token prefix is limited to 64 bytes" error message. Several
+error messages (including that one) have been improved to give developers a
+better chance of determining where the actual problem lies.
+
+RemoteReference.notifyOnDisconnect was buggy when called on a reference that
+was already broken: it failed to fire the callback. Now it fires the callback
+soon (using an eventual-send). This should remove a race condition from
+connectTo+notifyOnDisconnect sequences and allow them to operate reliably.
+notifyOnDisconnect() is now tolerant of attempts to remove something twice,
+which should make it easier to use safely.
+
+Remote methods which raise string exceptions should no longer cause Foolscap
+to explode. These sorts of exceptions are deprecated, of course, and you
+shouldn't use them, but at least they won't break Foolscap.
+
+The Reconnector class (accessed by tub.connectTo) was not correctly
+reconnecting in certain cases (which appeared to be particularly common on
+windows). This should be fixed now.
+
+CopyableSlicer did not work inside containers when streaming was enabled.
+Thanks to iacovou-AT-gmail.com for spotting this one.
+
+** Bugs not fixed
+
+Some bugs were identified and characterized but *not* fixed in this release
+
+*** RemoteInterfaces aren't defaulting to fully-qualified classnames
+
+When defining a RemoteInterface, you can specify its name with
+__remote_name__, or you can allow it to use the default name. Unfortunately,
+the default name is only the *local* name of the class, not the
+fully-qualified name, which means that if you have an RIFoo in two different
+.py files, they will wind up with the same name (which will cause an error on
+import, since all RemoteInterfaces known to a Foolscap-using program must
+have unique names).
+
+It turns out that it is rather difficult to determine the fully-qualified
+name of the RemoteInterface class early enough to be helpful. The workaround
+is to always add a __remote_name__ to your RemoteInterface classes. The
+recommendation is to use a globally-unique string, like a URI that includes
+your organization's DNS name.
+
+*** Constraints aren't constraining inbound tokens well enough
+
+Constraints (and the RemoteInterfaces they live inside) serve three purposes.
+The primary one is as documentation, describing how remotely-accessible
+objects behave. The second purpose is to enforce that documentation, by
+inspecting arguments (and return values) before invoking the method, as a
+form of precondition checking. The third is to mitigate denial-of-service
+attacks, in which an attacker sends so much data (or carefully crafted data)
+that the receiving program runs out of memory or stack space.
+
+It looks like several constraints are not correctly paying attention to the
+tokens as they arrive over the wire, such that the third purpose is not being
+achieved. Hopefully this will be fixed in a later release. Application code
+can be unaware of this change, since the constraints are still being applied
+to inbound arguments before they are passed to the method. Continue to use
+RemoteInterfaces as usual, just be aware that you are not yet protected
+against certain DoS attacks.
+
+** Use os.urandom instead of falling back to pycrypto
+
+Once upon a time, when Foolscap was compatible with python2.3 (which lacks
+os.urandom), we would try to use PyCrypto's random-number-generation routines
+when creating unguessable object identifiers (aka "SwissNumbers"). Now that
+we require python2.4 or later, this fallback has been removed, eliminating
+the last reference to pycrypto within the Foolscap source tree.
+
+
* Release 0.1.2 (04 Apr 2007)
** Bugfixes
projects / tahoe-lafs / tahoe-lafs.git / commitdiff