This measured how long the Helper took to do a filecheck before asking
for ciphertext. The "Contacting Helper" report includes both
existence_check and the client-helper RTT.
For non-overlapping uploads, it was being returned correctly. But when
multiple upload requests overlapped, and the file was not already in the
grid, the filecheck would only run once, and its existence_check time
would be reported for all uploaders (even if they didn't have to wait
for that time). Cleaning that up proved too difficult: the only correct
place to report this time is from the initial remote_upload_chk() call,
but the return value of that is too constrained to accomodate it in the
needs-upload case.
So I'm removing it altogether. Eventually I plan to add a proper
events/times field and record more data, including this check, in a form
that can be drawn on a nice zoomable timeline view.
Old clients talking to a new Helper (which doesn't supply the value)
will tolerate the loss (they'll just display an empty field on the web
view).
def remote_upload_chk(self, storage_index):
self.count("chk_upload_helper.upload_requests")
r = upload.UploadResults()
def remote_upload_chk(self, storage_index):
self.count("chk_upload_helper.upload_requests")
r = upload.UploadResults()
si_s = si_b2a(storage_index)
lp = self.log(format="helper: upload_chk query for SI %(si)s", si=si_s)
incoming_file = os.path.join(self._chk_incoming, si_s)
si_s = si_b2a(storage_index)
lp = self.log(format="helper: upload_chk query for SI %(si)s", si=si_s)
incoming_file = os.path.join(self._chk_incoming, si_s)
d = self._check_for_chk_already_in_grid(storage_index, r, lp)
def _checked(already_present):
d = self._check_for_chk_already_in_grid(storage_index, r, lp)
def _checked(already_present):
- elapsed = time.time() - started
- r.timings['existence_check'] = elapsed
if already_present:
# the necessary results are placed in the UploadResults
self.count("chk_upload_helper.upload_already_present")
if already_present:
# the necessary results are placed in the UploadResults
self.count("chk_upload_helper.upload_already_present")
storage_index : time to compute the storage index
peer_selection : time to decide which peers will be used
contacting_helper : initial helper query to upload/no-upload decision
storage_index : time to compute the storage index
peer_selection : time to decide which peers will be used
contacting_helper : initial helper query to upload/no-upload decision
- existence_check : helper pre-upload existence check
helper_total : initial helper query to helper finished pushing
cumulative_fetch : helper waiting for ciphertext requests
total_fetch : helper start to last ciphertext response
helper_total : initial helper query to helper finished pushing
cumulative_fetch : helper waiting for ciphertext requests
total_fetch : helper start to last ciphertext response
def data_time_contacting_helper(self, ctx, data):
return self._get_time("contacting_helper")
def data_time_contacting_helper(self, ctx, data):
return self._get_time("contacting_helper")
- def data_time_existence_check(self, ctx, data):
- return self._get_time("existence_check")
-
def data_time_cumulative_fetch(self, ctx, data):
return self._get_time("cumulative_fetch")
def data_time_cumulative_fetch(self, ctx, data):
return self._get_time("cumulative_fetch")
<li>Storage Index: <span n:render="time" n:data="time_storage_index" />
(<span n:render="rate" n:data="rate_storage_index" />)</li>
<li>[Contacting Helper]: <span n:render="time" n:data="time_contacting_helper" /></li>
<li>Storage Index: <span n:render="time" n:data="time_storage_index" />
(<span n:render="rate" n:data="rate_storage_index" />)</li>
<li>[Contacting Helper]: <span n:render="time" n:data="time_contacting_helper" /></li>
- <ul>
- <li>[Helper Already-In-Grid Check]: <span n:render="time" n:data="time_existence_check" /></li>
- </ul>
<li>[Upload Ciphertext To Helper]: <span n:render="time" n:data="time_cumulative_fetch" />
(<span n:render="rate" n:data="rate_ciphertext_fetch" />)</li>
<li>[Upload Ciphertext To Helper]: <span n:render="time" n:data="time_cumulative_fetch" />
(<span n:render="rate" n:data="rate_ciphertext_fetch" />)</li>
<li>Storage Index: <span n:render="time" n:data="time_storage_index" />
(<span n:render="rate" n:data="rate_storage_index" />)</li>
<li>[Contacting Helper]: <span n:render="time" n:data="time_contacting_helper" /></li>
<li>Storage Index: <span n:render="time" n:data="time_storage_index" />
(<span n:render="rate" n:data="rate_storage_index" />)</li>
<li>[Contacting Helper]: <span n:render="time" n:data="time_contacting_helper" /></li>
- <ul>
- <li>[Helper Already-In-Grid Check]: <span n:render="time" n:data="time_existence_check" /></li>
- </ul>
<li>[Upload Ciphertext To Helper]: <span n:render="time" n:data="time_cumulative_fetch" />
(<span n:render="rate" n:data="rate_ciphertext_fetch" />)</li>
<li>[Upload Ciphertext To Helper]: <span n:render="time" n:data="time_cumulative_fetch" />
(<span n:render="rate" n:data="rate_ciphertext_fetch" />)</li>