test/common.py: fix race condition waiting for the helper connection

The wait_for_connections() method, which is used at the start of
test_system to make sure that all the clients are connected to all the
servers, did not also wait for clients to be connected to their Helpers.
Every once in a while, the helper connection would take a bit longer,
and then
test_system.SystemTest.test_filesystem._test_web._got_welcome_helper
would fail, because we'd check for a helper connection before it was
ready.

The fix is to modify wait_for_connections's polling predicate to look
for helper connections (if configured) as well as the regular
introducer- and server- connections.

Tested by temporarily adding a large (30s) delay to the connectTo() call
in Uploader.startService, simulating a long helper
connection-establishment delay. This makes the test fail consistently.
Then I fixed wait_for_connections(), and the test passed (slowly). Then
I removed the delay.

Closes #1467

diff --git a/src/allmydata/test/common.py b/src/allmydata/test/common.py
index deb9d139cb659865dd179b8a1a102256fa83fdb7..e9103a2a1365925366622412011f395faf1d29d9 100644 (file)
--- a/src/allmydata/test/common.py
+++ b/src/allmydata/test/common.py
@@ -675,12 +675,12 @@ class SystemTestMixin(pollmixin.PollMixin, testutil.StallMixin):
             sb = c.get_storage_broker()
             if len(sb.get_connected_servers()) != self.numclients:
                 return False
+            up = c.getServiceNamed("uploader")
+            if up._helper_furl and not up._helper:
+                return False
         return True
 
     def wait_for_connections(self, ignored=None):
-        # TODO: replace this with something that takes a list of peerids and
-        # fires when they've all been heard from, instead of using a count
-        # and a threshold
         return self.poll(self._check_connections, timeout=200)