+++ /dev/null
-#LyX 1.6.1 created this file. For more info see http://www.lyx.org/
-\lyxformat 345
-\begin_document
-\begin_header
-\textclass amsart
-\use_default_options true
-\begin_modules
-theorems-ams
-theorems-ams-extended
-\end_modules
-\language english
-\inputencoding auto
-\font_roman default
-\font_sans default
-\font_typewriter default
-\font_default_family default
-\font_sc false
-\font_osf false
-\font_sf_scale 100
-\font_tt_scale 100
-
-\graphics default
-\float_placement h
-\paperfontsize default
-\spacing single
-\use_hyperref false
-\papersize default
-\use_geometry false
-\use_amsmath 1
-\use_esint 1
-\cite_engine basic
-\use_bibtopic false
-\paperorientation portrait
-\secnumdepth 3
-\tocdepth 3
-\paragraph_separation indent
-\defskip medskip
-\quotes_language english
-\papercolumns 1
-\papersides 1
-\paperpagestyle default
-\tracking_changes false
-\output_changes false
-\author ""
-\author ""
-\end_header
-
-\begin_body
-
-\begin_layout Title
-Tahoe Distributed Filesharing System Loss Model
-\end_layout
-
-\begin_layout Author
-Shawn Willden
-\end_layout
-
-\begin_layout Date
-01/14/2009
-\end_layout
-
-\begin_layout Address
-South Weber, Utah
-\end_layout
-
-\begin_layout Email
-shawn@willden.org
-\end_layout
-
-\begin_layout Abstract
-The abstract goes here
-\end_layout
-
-\begin_layout Section
-Problem Statement
-\end_layout
-
-\begin_layout Standard
-The allmydata Tahoe distributed file system uses Reed-Solomon erasure coding
- to split files into
-\begin_inset Formula $N$
-\end_inset
-
- shares, each of which is then delivered to a randomly-selected peer in
- a distributed network.
- The file can later be reassembled from any
-\begin_inset Formula $k\leq N$
-\end_inset
-
- of the shares, if they are available.
-\end_layout
-
-\begin_layout Standard
-Over time shares are lost for a variety of reasons.
- Storage servers may crash, be destroyed or simply be removed from the network.
- To mitigate such losses, Tahoe network clients employ a repair agent which
- scans the peers once per time period
-\begin_inset Formula $A$
-\end_inset
-
- and determines how many of the shares remain.
- If less than
-\begin_inset Formula $L$
-\end_inset
-
- (
-\begin_inset Formula $k\leq L\leq N$
-\end_inset
-
-) shares remain, then the repairer reconstructs the file shares and redistribute
-s the missing ones, bringing the availability back up to full.
-\end_layout
-
-\begin_layout Standard
-The question we're trying to answer is "What's the probability that we'll
- be able to reassemble the file at some later time
-\begin_inset Formula $T$
-\end_inset
-
-?".
- We'd also like to be able to determine what values we should choose for
-
-\begin_inset Formula $k$
-\end_inset
-
-,
-\begin_inset Formula $N$
-\end_inset
-
-,
-\begin_inset Formula $A$
-\end_inset
-
-, and
-\begin_inset Formula $L$
-\end_inset
-
- in order to ensure
-\begin_inset Formula $Pr[loss]\leq t$
-\end_inset
-
- for some threshold probability
-\begin_inset Formula $t$
-\end_inset
-
-.
- This is an optimization problem because although we could obtain very low
-
-\begin_inset Formula $Pr[loss]$
-\end_inset
-
- by choosing small
-\begin_inset Formula $k,$
-\end_inset
-
- large
-\begin_inset Formula $N$
-\end_inset
-
-, small
-\begin_inset Formula $A$
-\end_inset
-
-, and setting
-\begin_inset Formula $L=N$
-\end_inset
-
-, these choices have costs.
- The peer storage and bandwidth consumed by the share distribution process
- are approximately
-\begin_inset Formula $\nicefrac{N}{k}$
-\end_inset
-
- times the size of the original file, so we would like to reduce this ratio
- as far as possible consistent with
-\begin_inset Formula $Pr[loss]\leq t$
-\end_inset
-
-.
- Likewise, frequent and aggressive repair process can be used to ensure
- that the number of shares available at any time is very close to
-\begin_inset Formula $N,$
-\end_inset
-
- but at a cost in bandwidth as the repair agent downloads
-\begin_inset Formula $k$
-\end_inset
-
- shares to reconstruct the file and uploads new shares to replace those
- that are lost.
-\end_layout
-
-\begin_layout Section
-Reliability
-\end_layout
-
-\begin_layout Standard
-The probability that the file becomes unrecoverable is dependent upon the
- probability that the peers to whom we send shares are able to return those
- copies on demand.
- Shares that are returned in corrupted form can be detected and discarded,
- so there is no need to distinguish between corruption and loss.
-\end_layout
-
-\begin_layout Standard
-There are a large number of factors that affect share availability.
- Availability can be temporarily interrupted by peer unavailability, due
- to network outages, power failures or administrative shutdown, among other
- reasons.
- Availability can be permanently lost due to failure or corruption of storage
- media, catastrophic damage to the peer system, administrative error, withdrawal
- from the network, malicious corruption, etc.
-\end_layout
-
-\begin_layout Standard
-The existence of intermittent failure modes motivates the introduction of
- a distinction between
-\noun on
-availability
-\noun default
- and
-\noun on
-reliability
-\noun default
-.
- Reliability is the probability that a share is retrievable assuming intermitten
-t failures can be waited out, so reliability considers only permanent failures.
- Availability considers all failures, and is focused on the probability
- of retrieval within some defined time frame.
-\end_layout
-
-\begin_layout Standard
-Another consideration is that some failures affect multiple shares.
- If multiple shares of a file are stored on a single hard drive, for example,
- failure of that drive may lose them all.
- Catastrophic damage to a data center may destroy all shares on all peers
- in that data center.
-\end_layout
-
-\begin_layout Standard
-While the types of failures that may occur are pretty consistent across
- even very different peers, their probabilities differ dramatically.
- A professionally-administered blade server with redundant storage, power
- and Internet located in a carefully-monitored data center with automatic
- fire suppression systems is much less likely to become either temporarily
- or permanently unavailable than the typical virus and malware-ridden home
- computer on a single cable modem connection.
- A variety of situations in between exist as well, such as the case of the
- author's home file server, which is administered by an IT professional
- and uses RAID level 6 redundant storage, but runs on old, cobbled-together
- equipment, and has a consumer-grade Internet connection.
-\end_layout
-
-\begin_layout Standard
-To begin with, let's use a simple definition of reliability:
-\end_layout
-
-\begin_layout Definition
-
-\noun on
-Reliability
-\noun default
- is the probability
-\begin_inset Formula $p_{i}$
-\end_inset
-
- that a share
-\begin_inset Formula $s_{i}$
-\end_inset
-
- will surve to (be retrievable at) time
-\begin_inset Formula $T=A$
-\end_inset
-
-, ignoring intermittent failures.
- That is, the probability that the share will be retrievable at the end
- of the current repair cycle, and therefore usable by the repairer to regenerate
- any lost shares.
-\end_layout
-
-\begin_layout Definition
-Reliability is clearly dependent on
-\begin_inset Formula $A$
-\end_inset
-
-.
- Short repair cycles offer less time for shares to
-\begin_inset Quotes eld
-\end_inset
-
-decay
-\begin_inset Quotes erd
-\end_inset
-
- into unavailability.
-\end_layout
-
-\begin_layout Subsection
-Fixed Reliability
-\begin_inset CommandInset label
-LatexCommand label
-name "sub:Fixed-Reliability"
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-In the simplest case, the peers holding the file shares all have the same
- reliability
-\begin_inset Formula $p$
-\end_inset
-
-, and are all independent from one another.
- Let
-\begin_inset Formula $K$
-\end_inset
-
- be a random variable that represents the number of shares that survive
-
-\begin_inset Formula $A$
-\end_inset
-
-.
- Each share's survival can be viewed as an indepedent Bernoulli trial with
- a succes probability of
-\begin_inset Formula $p$
-\end_inset
-
-, which means that
-\begin_inset Formula $K$
-\end_inset
-
- follows the binomial distribution with paramaters
-\begin_inset Formula $N$
-\end_inset
-
- and
-\begin_inset Formula $p$
-\end_inset
-
-.
- That is,
-\begin_inset Formula $K\sim B(N,p)$
-\end_inset
-
-.
-\end_layout
-
-\begin_layout Theorem
-Binomial Distribution Theorem
-\end_layout
-
-\begin_layout Theorem
-Consider
-\begin_inset Formula $n$
-\end_inset
-
- independent Bernoulli trials
-\begin_inset Foot
-status collapsed
-
-\begin_layout Plain Layout
-A Bernoulli trial is simply a test of some sort that results in one of two
- outcomes, one of which is designated success and the other failure.
- The classic example of a Bernoulli trial is a coin toss.
-\end_layout
-
-\end_inset
-
- that succeed with probability
-\begin_inset Formula $p$
-\end_inset
-
-, and let
-\begin_inset Formula $K$
-\end_inset
-
- be a random variable that represents the number of successes.
- We say that
-\begin_inset Formula $K$
-\end_inset
-
- follows the Binomial Distribution with parameters n and p, denoted
-\begin_inset Formula $K\sim B(n,p)$
-\end_inset
-
-.
- The probability that
-\begin_inset Formula $K$
-\end_inset
-
- takes a particular value
-\begin_inset Formula $m$
-\end_inset
-
- (the probability that there are exactly
-\begin_inset Formula $m$
-\end_inset
-
- successful trials, and therefore
-\begin_inset Formula $n-m$
-\end_inset
-
- failures) is called the probability mass function and is given by:
-\begin_inset Formula \begin{equation}
-Pr[K=m]=f(m;n,p)=\binom{n}{p}p^{m}(1-p)^{n-m}\label{eq:binomial-pmf}\end{equation}
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Proof
-Consider the specific case of exactly
-\begin_inset Formula $m$
-\end_inset
-
- successes followed by
-\begin_inset Formula $n-m$
-\end_inset
-
- failures, because each success has probability
-\begin_inset Formula $p$
-\end_inset
-
-, each failure has probability
-\begin_inset Formula $1-p$
-\end_inset
-
-, and the trials are independent, the probability of this exact case occurring
- is
-\begin_inset Formula $p^{m}\left(1-p\right)^{\left(n-m\right)}$
-\end_inset
-
-, the product of the probabilities of the outcome of each trial.
-\end_layout
-
-\begin_layout Proof
-Now consider any reordering of these
-\begin_inset Formula $m$
-\end_inset
-
- successes and
-\begin_inset Formula $n$
-\end_inset
-
- failures.
- Any such reordering occurs with the same probability
-\begin_inset Formula $p^{m}\left(1-p\right)^{\left(n-m\right)}$
-\end_inset
-
-, but with the terms of the product reordered.
- Since multiplication is commutative, each such reordering has the same
- probability.
- There are n-choose-m such orderings, and each ordering is an independent
- event, so the probability that any ordering of
-\begin_inset Formula $m$
-\end_inset
-
- successes and
-\begin_inset Formula $n-m$
-\end_inset
-
- failures occurs is given by
-\begin_inset Formula \[
-\binom{n}{m}p^{m}\left(1-p\right)^{\left(n-m\right)}\]
-
-\end_inset
-
-which is the right-hand-side of equation
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "eq:binomial-pmf"
-
-\end_inset
-
-.
-\end_layout
-
-\begin_layout Standard
-A file survives if at least
-\begin_inset Formula $k$
-\end_inset
-
- of the
-\begin_inset Formula $N$
-\end_inset
-
- shares survive.
- Equation
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "eq:binomial-pmf"
-
-\end_inset
-
- gives the probability that exactly
-\begin_inset Formula $i$
-\end_inset
-
- shares survive, for any
-\begin_inset Formula $1\leq i\leq n$
-\end_inset
-
-, so the probability that fewer than
-\begin_inset Formula $k$
-\end_inset
-
- survive is the sum of the probabilities that
-\begin_inset Formula $0,1,2,\ldots,k-1$
-\end_inset
-
- shares survive.
- That is:
-\end_layout
-
-\begin_layout Standard
-\begin_inset Formula \begin{equation}
-Pr[file\, lost]=\sum_{i=0}^{k-1}\binom{n}{i}p^{i}(1-p)^{n-i}\label{eq:simple-failure}\end{equation}
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Subsection
-Independent Reliability
-\begin_inset CommandInset label
-LatexCommand label
-name "sub:Independent-Reliability"
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-Equation
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "eq:simple-failure"
-
-\end_inset
-
- assumes that each share has the same probability of survival, but as explained
- above, this is not necessarily true.
- A more accurate model allows each share
-\begin_inset Formula $s_{i}$
-\end_inset
-
- an independent probability of survival
-\begin_inset Formula $p_{i}$
-\end_inset
-
-.
- Each share's survival can still be treated as an independent Bernoulli
- trial, but with success probability
-\begin_inset Formula $p_{i}$
-\end_inset
-
-.
- Under this assumption,
-\begin_inset Formula $K$
-\end_inset
-
- follows a generalized binomial distribution with parameters
-\begin_inset Formula $N$
-\end_inset
-
- and
-\begin_inset Formula $p_{i}$
-\end_inset
-
- where
-\begin_inset Formula $1\leq i\leq N$
-\end_inset
-
-.
-\end_layout
-
-\begin_layout Standard
-The PMF for this generalized
-\begin_inset Formula $K$
-\end_inset
-
- does not have a simple closed-form representation.
- However, the PMFs for random variables representing individual share survival
- do.
- Let
-\begin_inset Formula $S_{i}$
-\end_inset
-
- be a random variable such that:
-\end_layout
-
-\begin_layout Standard
-\begin_inset Formula \[
-S_{i}=\begin{cases}
-1 & \textnormal{if }s_{i}\textnormal{ survives}\\
-0 & \textnormal{if }s_{i}\textnormal{ fails}\end{cases}\]
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-The PMF for
-\begin_inset Formula $S_{i}$
-\end_inset
-
- is very simple:
-\begin_inset Formula \[
-Pr[S_{i}=j]=\begin{cases}
-1-p_{i} & j=0\\
-p_{i} & j=1\end{cases}\]
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-Note that since each
-\begin_inset Formula $S_{i}$
-\end_inset
-
- represents the count of shares
-\begin_inset Formula $s_{i}$
-\end_inset
-
- that survives (either 0 or 1), if we add up all of the individual survivor
- counts, we get the group survivor count.
- That is:
-\begin_inset Formula \[
-\sum_{i=1}^{N}S_{i}=K\]
-
-\end_inset
-
-Effectively,
-\begin_inset Formula $K$
-\end_inset
-
- has just been separated into the series of Bernoulli trials that make it
- up.
-\end_layout
-
-\begin_layout Theorem
-Discrete Convolution Theorem
-\end_layout
-
-\begin_layout Theorem
-Let
-\begin_inset Formula $X$
-\end_inset
-
- and
-\begin_inset Formula $Y$
-\end_inset
-
- be discrete random variables with probability mass functions given by
-\begin_inset Formula $Pr\left[X=x\right]=f(x)$
-\end_inset
-
- and
-\begin_inset Formula $Pr\left[Y=y\right]=g(y).$
-\end_inset
-
- Let
-\begin_inset Formula $Z$
-\end_inset
-
- be the discrete random random variable obtained by summing
-\begin_inset Formula $X$
-\end_inset
-
- and
-\begin_inset Formula $Y$
-\end_inset
-
-.
-\end_layout
-
-\begin_layout Theorem
-The probability mass function of
-\begin_inset Formula $Z$
-\end_inset
-
- is given by
-\begin_inset Formula \[
-Pr[Z=z]=h(z)=\left(f\star g\right)(z)\]
-
-\end_inset
-
-where
-\begin_inset Formula $\star$
-\end_inset
-
- denotes the discrete convolution operation:
-\begin_inset Formula \[
-\left(f\star g\right)\left(n\right)=\sum_{m=-\infty}^{\infty}f\left(m\right)g\left(m-n\right)\]
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Proof
-The proof is beyond the scope of this paper.
-\begin_inset Foot
-status collapsed
-
-\begin_layout Plain Layout
-\begin_inset Quotes eld
-\end_inset
-
-Beyond the scope of this paper
-\begin_inset Quotes erd
-\end_inset
-
- usually means
-\begin_inset Quotes eld
-\end_inset
-
-Too long and nasty to bore you with
-\begin_inset Quotes erd
-\end_inset
-
-.
- In this case it means
-\begin_inset Quotes eld
-\end_inset
-
-The author hasn't the foggiest idea why this is true, or how to prove it,
- but reliable authorities say it's real, and in practice it works a treat.
-\begin_inset Quotes erd
-\end_inset
-
-
-\end_layout
-
-\end_inset
-
- If you don't believe it's true, look it up on Wikipedia, which is never
- wrong.
-\end_layout
-
-\begin_layout Standard
-Applying the discrete convolution theorem, if
-\begin_inset Formula $Pr[K=i]=f(i)$
-\end_inset
-
- and
-\begin_inset Formula $Pr[S_{i}=j]=g_{i}(j)$
-\end_inset
-
-, then
-\begin_inset Formula $f=g_{1}\star g_{2}\star g_{3}\star\ldots\star g_{N}$
-\end_inset
-
-.
- Since convolution is associative, this can also be written as
-\begin_inset Formula $ $
-\end_inset
-
-
-\begin_inset Formula \begin{equation}
-f=(\ldots((g_{1}\star g_{2})\star g_{3})\star\ldots)\star g_{N})\label{eq:convolution}\end{equation}
-
-\end_inset
-
-Therefore,
-\begin_inset Formula $f$
-\end_inset
-
- can be computed as a sequence of convolution operations on the simple PMFs
- of the random variables
-\begin_inset Formula $S_{i}$
-\end_inset
-
-.
- In fact, for large
-\begin_inset Formula $N$
-\end_inset
-
-, equation
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "eq:convolution"
-
-\end_inset
-
- turns out to be a more effective means of computing the PMF of
-\begin_inset Formula $K$
-\end_inset
-
- even in the case of the standard binomial distribution, primarily because
- the binomial calculation in equation
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "eq:binomial-pmf"
-
-\end_inset
-
- produces very large values that overflow unless arbitrary precision numeric
- representations are used.
-\end_layout
-
-\begin_layout Standard
-Note also that it is not necessary to have very simple PMFs like those of
- the
-\begin_inset Formula $S_{i}$
-\end_inset
-
-.
- Any share or set of shares that has a known PMF can be combined with any
- other set with a known PMF by convolution, as long as the two share sets
- are independent.
- Since PMFs are easily represented as simple lists of probabilities, where
- the
-\begin_inset Formula $i$
-\end_inset
-
-th element in the list corresponds to
-\begin_inset Formula $Pr[K=i]$
-\end_inset
-
-, these functions are easily managed in software, and computing the convolution
- is both simple and efficient.
-\end_layout
-
-\begin_layout Subsection
-Multiple Failure Modes
-\begin_inset CommandInset label
-LatexCommand label
-name "sub:Multiple-Failure-Modes"
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-In modeling share survival probabilities, it's useful to be able to analyze
- separately each of the various failure modes.
- If reliable statistics for disk failure can be obtained, then a probability
- mass function for that form of failure can be generated.
- Similarly, statistics on other hardware failures, administrative errors,
- network losses, etc., can all be estimated independently.
- If those estimates can then be combined into a single PMF for a share,
- then we can use it to predict failures for that share.
-\end_layout
-
-\begin_layout Standard
-Combining independent failure modes for a single share is straightforward.
- If
-\begin_inset Formula $p_{i,j}$
-\end_inset
-
- is the probability of survival of the
-\begin_inset Formula $j$
-\end_inset
-
-th failure mode of share
-\begin_inset Formula $i$
-\end_inset
-
-,
-\begin_inset Formula $1\leq j\leq m$
-\end_inset
-
-, then
-\begin_inset Formula \[
-Pr[S_{i}=k]=f_{i}(k)=\begin{cases}
-\prod_{j=1}^{m}p_{i,j} & k=1\\
-1-\prod_{j=1}^{m}p_{i,j} & k=0\end{cases}\]
-
-\end_inset
-
-is the survival PMF.
-\end_layout
-
-\begin_layout Subsection
-Multi-share failures
-\begin_inset CommandInset label
-LatexCommand label
-name "sub:Multi-share-failures"
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-If there are failure modes that affect multiple computers, we can also construct
- the PMF that predicts their survival.
- The key observation is that the PMF has non-zero probabilities only for
-
-\begin_inset Formula $0$
-\end_inset
-
- survivors and
-\begin_inset Formula $n$
-\end_inset
-
- survivors, where
-\begin_inset Formula $n$
-\end_inset
-
- is the number of shares in the set.
- If
-\begin_inset Formula $p$
-\end_inset
-
- is the probability of survival, the PMF of
-\begin_inset Formula $K$
-\end_inset
-
-, a random variable representing the number of surviors is
-\begin_inset Formula \[
-Pr[K=i]=f(i)=\begin{cases}
-p & i=n\\
-0 & 0<i<n\\
-1-p & i=0\end{cases}\]
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-Group failures due to multiple independent causes can be combined as in
- section
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "sub:Multiple-Failure-Modes"
-
-\end_inset
-
-, as long as they apply to the whole group.
-\end_layout
-
-\begin_layout Example
-Putting the Pieces Together
-\end_layout
-
-\begin_layout Standard
-Sections
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "sub:Fixed-Reliability"
-
-\end_inset
-
- through
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "sub:Multi-share-failures"
-
-\end_inset
-
- provide ways of calculating the survival probability mass functions for
- a variety of share failure structures and modes.
- As an example of how these pieces can be used, consider a network with
- the following peers:
-\end_layout
-
-\begin_layout Itemize
-Four servers located in a data center in Nebraska.
- The machines have multiply-redundant Internet connections, with a failure
- probability of 0.0001.
- They store their shares on RAID arrays with failure probability of 0.0002.
- The administrative staff makes data-destroying errors with probability
- 0.003.
-\end_layout
-
-\begin_layout Itemize
-Four servers located in a data center on the island of Hawaii.
- These servers have identical failure probabilities as the servers in Nebraska,
- except that the data center is near the edge of the crater on Mount Kilauea
- (nobody said examples had to be realistic).
- There is a 0.04 chance that the volcano will erupt and bury the data center
- in molten lava, destroying it entirely.
-\end_layout
-
-\begin_layout Itemize
-Four PCs located in random homes, connected to the Internet via assorted
- cable modems and DSL.
- Their network connections fail with probability 0.009.
- Their disks fail with probability 0.001.
- Their users destroy data with probability 0.05.
-\end_layout
-
-\begin_layout Standard
-If one share is placed on each of these 20 computers, what's the probability
- mass function of share survival? To more compactly describe PMFs, we'll
- denote them as probability vectors of the form
-\begin_inset Formula $\left[\alpha_{o},\alpha_{1},\alpha_{2},\ldots\alpha_{n}\right]$
-\end_inset
-
- where
-\begin_inset Formula $\alpha_{i}$
-\end_inset
-
- is the probability that exactly
-\begin_inset Formula $i$
-\end_inset
-
- shares survive.
-\end_layout
-
-\begin_layout Standard
-The servers in the two data centers have individual survival probabilities
- of RAID failure (.0002) and administrative error (.003) giving
-\begin_inset Formula \[
-(1-.0002)\cdot(1-.003)=.9998\cdot.997=.9968\]
-
-\end_inset
-
-Using
-\begin_inset Formula $p=.9968,n=4$
-\end_inset
-
- in equation
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "eq:binomial-pmf"
-
-\end_inset
-
- gives the survival PMF
-\begin_inset Formula \[
-\left[1.049\times10^{-10},1.307\times10^{-7},6.105\times10^{-5},0.01271,0.9872\right]\]
-
-\end_inset
-
-which applies to each group of four servers.
- However, each data center also has a .0001 chance of data connection loss,
- which affects all four servers at once, and Hawaii has the additional .04
- probability of severe lava burn.
- If the network fails at a location, all the machines go offline together.
- The probability that 0 machines survive is the probability that they all
- fail for individual reasons (
-\begin_inset Formula $1.049\cdot10^{-10}$
-\end_inset
-
-) times the probability they all fail because of a network outage (
-\begin_inset Formula $.0001$
-\end_inset
-
-) less the probability they fail for both reasons:
-\begin_inset Formula \[
-\left(1.049\times10^{-10}\right)+\left(0.0001\right)-\left[\left(1.049\times10^{-10}\right)\cdot\left(0.0001\right)\right]=0.0001\]
-
-\end_inset
-
-That's the
-\begin_inset Formula $0$
-\end_inset
-
-th element of the combined PMF.
- The combined probability of survival of
-\begin_inset Formula $0<i\leq4$
-\end_inset
-
- servers is simpler: it's the probility they survive individual failure,
- from the individual failure PMF above, times the probability they survive
- network failure (.9999).
- So the combined survival PMF, which we'll denote as
-\begin_inset Formula $n(i)$
-\end_inset
-
- of the Nebraska servers is
-\begin_inset Formula \[
-n(i)=\left[0.0001,1.306\times10^{-7},6.104\times10^{-5},0.01268,0.9872\right]\]
-
-\end_inset
-
-which has the interesting property that complete failure is 1000 times more
- likely than survival of one server.
- This is because the probability of a network outage is so much greater
- than simultaneous
-\begin_inset Foot
-status collapsed
-
-\begin_layout Plain Layout
-Of course, the failures need not be truly simultaneous, they just have happen
- in the same interval between repair runs.
-\end_layout
-
-\end_inset
-
- independent failure of three servers.
-\end_layout
-
-\begin_layout Standard
-The same process for the Hawaii servers, but with group survival probability
- of
-\begin_inset Formula $(1-.0001)(1-.02)=.9799$
-\end_inset
-
- gives the survival PMF
-\begin_inset Formula \[
-h(i)=\left[0.0201,1.280\times10^{-7},5.982\times10^{-5},0.01242,0.9674\right]\]
-
-\end_inset
-
-which has the unusual property that it's more likely that all of the servers
- will be lost than that only one will survive.
- This is because in order for exactly one to survive, it's necessary for
- three to have the
-\end_layout
-
-\begin_layout Standard
-Applying the convolution operator to
-\begin_inset Formula $n(i)$
-\end_inset
-
- and
-\begin_inset Formula $h(i)$
-\end_inset
-
-, the survival PMF of all eight servers is:
-\end_layout
-
-\begin_layout Standard
-\begin_inset Formula \[
-\left(n\star h\right)\left(i\right)=\begin{cases}
-2.010\times10^{-6} & i=0\\
-2.639\times10^{-9} & i=1\\
-1.233\times10^{-6} & i=2\\
-2.560\times10^{-4} & i=3\\
-0.01994 & i=4\\
-1.769\times10^{-6} & i=5\\
-2.756\times10^{-4} & i=6\\
-0.02452 & i=7\\
-0.9559 & i=8\end{cases}\]
-
-\end_inset
-
-Note the interesting fact that losing four shares is 10,000 times more likely
- than losing three.
- This is because both data centers have a whole-center failure modes, and
- the Hawaii center's lava burn probability is so high.
-\end_layout
-
-\begin_layout Standard
-For the home PCs, their individual probability of survival is
-\begin_inset Formula \[
-(1-.009)\cdot(1-.001)\cdot(1-.05)=.991\cdot.999\cdot.95=.9405\]
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-We can then apply equation
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "eq:binomial-pmf"
-
-\end_inset
-
- with
-\begin_inset Formula $N=4$
-\end_inset
-
- and
-\begin_inset Formula $p=.9405$
-\end_inset
-
- to compute the PMF
-\begin_inset Formula $f(i),0\leq i\leq4$
-\end_inset
-
- for the PCs and finally compute
-\begin_inset Formula $s(i)=\left(f\star\left(n\star h\right)\right)\left(i\right)$
-\end_inset
-
-, the PMF of the whole share set.
- Summing the values of
-\begin_inset Formula $s(i)$
-\end_inset
-
- for
-\begin_inset Formula $0\leq i\leq k-1$
-\end_inset
-
- gives the probability that less than
-\begin_inset Formula $k$
-\end_inset
-
- shares survive and the file is unrecoverable.
- For this example, those sums are shown in table
-\begin_inset CommandInset ref
-LatexCommand vref
-reference "tab:Example-PMF"
-
-\end_inset
-
-.
-\begin_inset Float table
-wide false
-sideways false
-status collapsed
-
-\begin_layout Plain Layout
-\align center
-\begin_inset Tabular
-<lyxtabular version="3" rows="13" columns="4">
-<features>
-<column alignment="center" valignment="top" width="0">
-<column alignment="center" valignment="top" width="0">
-<column alignment="center" valignment="top" width="0">
-<column alignment="center" valignment="top" width="0">
-<row>
-<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $k$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $Pr[K=k]$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $Pr[file\, loss]=Pr[K<k]$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $N/k$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-1
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $1.60\times10^{-9}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $2.53\times10^{-11}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-12
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-2
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $3.80\times10^{-8}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $1.63\times10^{-9}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-6
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-3
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $4.04\times10^{-7}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $3.70\times10^{-8}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-4
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-4
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $2.06\times10^{-6}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $4.44\times10^{-7}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-3
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-5
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $2.10\times10^{-5}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $2.50\times10^{-6}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-2.4
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-6
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.000428$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $2.35\times10^{-5}$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-2
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-7
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.00417$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.000452$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-1.7
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-8
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.0157$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.00462$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-1.5
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-9
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.00127$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.0203$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-1.3
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-10
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.0230$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.0216$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-1.2
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-11
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.208$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.0446$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-1.1
-\end_layout
-
-\end_inset
-</cell>
-</row>
-<row>
-<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-12
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.747$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-\begin_inset Formula $0.253$
-\end_inset
-
-
-\end_layout
-
-\end_inset
-</cell>
-<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
-\begin_inset Text
-
-\begin_layout Plain Layout
-1
-\end_layout
-
-\end_inset
-</cell>
-</row>
-</lyxtabular>
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Plain Layout
-\begin_inset Caption
-
-\begin_layout Plain Layout
-\align left
-\begin_inset CommandInset label
-LatexCommand label
-name "tab:Example-PMF"
-
-\end_inset
-
-Example PMF
-\end_layout
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Plain Layout
-
-\end_layout
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-The table demonstrates the importance of the selection of
-\begin_inset Formula $k$
-\end_inset
-
-, and the tradeoff against file size expansion.
- Note that the survival of exactly 9 servers is significantly less likely
- than the survival of 8 or 10 servers.
- This is, again, an artifact of the group failure modes.
- Because of this, there is no reason to choose
-\begin_inset Formula $k=9$
-\end_inset
-
- over
-\begin_inset Formula $k=10$
-\end_inset
-
-.
- Normally, reducing the number of shares needed for reassembly improve the
- file's chances of survival, but in this case it provides a miniscule gain
- in reliability at the cost of a 10% increase in bandwidth and storage consumed.
-\end_layout
-
-\begin_layout Subsection
-Share Duplication
-\end_layout
-
-\begin_layout Standard
-Before moving on to consider issues other than single-interval file loss,
- let's analyze one more possibility, that of
-\begin_inset Quotes eld
-\end_inset
-
-cheap
-\begin_inset Quotes erd
-\end_inset
-
- file repair via share duplication.
-\end_layout
-
-\begin_layout Standard
-Initially, files are split using erasure coding, which creates
-\begin_inset Formula $N$
-\end_inset
-
- unique shares, any
-\begin_inset Formula $k$
-\end_inset
-
- of which can be used to to reconstruct the file.
- When shares are lost, proper repair downloads some
-\begin_inset Formula $k$
-\end_inset
-
- shares, reconstructs the original file and then uses the erasure coding
- algorithm to reconstruct the lost shares, then redeploys them to peers
- in the network.
- This is a somewhat expensive process.
-\end_layout
-
-\begin_layout Standard
-A cheaper repair option is simply to direct some peer that has share
-\begin_inset Formula $s_{i}$
-\end_inset
-
- to send a copy to another peer, thus increasing by one the number of shares
- in the network.
- This is not as good as actually replacing the lost share, though.
- Suppose that more shares were lost, leaving only
-\begin_inset Formula $ $
-\end_inset
-
-
-\begin_inset Formula $k$
-\end_inset
-
- shares remaining.
- If two of those shares are identical, because one was duplicated in this
- fashion, then only
-\begin_inset Formula $k-1$
-\end_inset
-
- shares truly remain, and the file can no longer be reconstructed.
-\end_layout
-
-\begin_layout Standard
-However, such cheap repair is not completely pointless; it does increase
- file survivability.
- The question is: By how much?
-\end_layout
-
-\begin_layout Standard
-Effectively, share duplication simply increases the probability that
-\begin_inset Formula $s_{i}$
-\end_inset
-
- will survive, by providing two locations from which to retrieve it.
- We can view the two copies of the single share as one, but with a higher
- probability of survival than would be provided by either of the two peers.
- In particular, if
-\begin_inset Formula $p_{1}$
-\end_inset
-
- and
-\begin_inset Formula $p_{2}$
-\end_inset
-
- are the probabilities that the two peers will survive, respectively, then
-\begin_inset Formula \[
-Pr[s_{i}\, survives]=p_{1}+p_{2}-p_{1}p_{2}\]
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-More generally, if a single share is deployed on
-\begin_inset Formula $n$
-\end_inset
-
- peers, each with a PMF
-\begin_inset Formula $f_{i}(j),0\leq j\leq1,1\leq i\leq n$
-\end_inset
-
-, the share survival count is a random variable
-\begin_inset Formula $S$
-\end_inset
-
- and the probability of share loss is
-\begin_inset Formula \[
-Pr[S=0]=(f_{1}\star f_{2}\star\ldots\star f_{n})(0)\]
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-From that, we can construct a share PMF in the obvious way, which can then
- be convolved with the other share PMFs to produce the share set PMF.
-\end_layout
-
-\begin_layout Example
-Suppose a file has
-\begin_inset Formula $N=10,k=3$
-\end_inset
-
- and that all servers have survival probability
-\begin_inset Formula $p=.9$
-\end_inset
-
-.
- Given a full complement of shares,
-\begin_inset Formula $Pr[\textrm{file\, loss}]=3.74\times10^{-7}$
-\end_inset
-
-.
- Suppose that four shares are lost, which increases
-\begin_inset Formula $Pr[\textrm{file\, loss}]$
-\end_inset
-
- to
-\begin_inset Formula $.00127$
-\end_inset
-
-, a value
-\begin_inset Formula $3400$
-\end_inset
-
- times greater.
- Rather than doing a proper reconstruction, we could direct four peers still
- holding shares to send a copy of their share to new peer, which changes
- the composition of the shares from one of six, unique
-\begin_inset Quotes eld
-\end_inset
-
-standard
-\begin_inset Quotes erd
-\end_inset
-
- shares, to one of two standard shares, each with survival probability
-\begin_inset Formula $.9$
-\end_inset
-
- and four
-\begin_inset Quotes eld
-\end_inset
-
-doubled
-\begin_inset Quotes erd
-\end_inset
-
- shares, each with survival probability
-\begin_inset Formula $2p-p^{2}\approx.99$
-\end_inset
-
-.
-\end_layout
-
-\begin_layout Example
-Combining the two single-peer share PMFs with the four double-share PMFs
- gives a new file survival probability of
-\begin_inset Formula $6.64\times10^{-6}$
-\end_inset
-
-.
- Not as good as a full repair, but still quite respectable.
- Also, if storage were not a concern, all six shares could be duplicated,
- for a
-\begin_inset Formula $Pr[file\, loss]=1.48\times10^{-7}$
-\end_inset
-
-, which is actually three time better than the nominal case.
-\end_layout
-
-\begin_layout Example
-The reason such cheap repairs may be attractive in many cases is that distribute
-d bandwidth is cheaper than bandwidth through a single peer.
- This is particularly true if that single peer has a very slow connection,
- which is common for home computers -- especially in the outbound direction.
-\end_layout
-
-\begin_layout Section
-Long-Term Reliability
-\end_layout
-
-\begin_layout Standard
-Thus far, we've focused entirely on the probability that a file survives
- the interval
-\begin_inset Formula $A$
-\end_inset
-
- between repair times.
- The probability that a file survives long-term, though, is also important.
- As long as the probability of failure during a repair period is non-zero,
- a given file will eventually be lost.
- We want to know what the probability of surviving for time
-\begin_inset Formula $T$
-\end_inset
-
- is, and how the parameters
-\begin_inset Formula $A$
-\end_inset
-
- (time between repairs) and
-\begin_inset Formula $L$
-\end_inset
-
- (share low watermark) affect survival time.
-\end_layout
-
-\begin_layout Standard
-To model file survival time, let
-\begin_inset Formula $T$
-\end_inset
-
- be a random variable denoting the time at which a given file becomes unrecovera
-ble, and
-\begin_inset Formula $R(t)=Pr[T>t]$
-\end_inset
-
- be a function that gives the probability that the file survives to time
-
-\begin_inset Formula $t$
-\end_inset
-
-.
-
-\begin_inset Formula $R(t)$
-\end_inset
-
- is the cumulative distribution function of
-\begin_inset Formula $T$
-\end_inset
-
-.
-\end_layout
-
-\begin_layout Standard
-Most survival functions are continuous, but
-\begin_inset Formula $R(t)$
-\end_inset
-
- is inherently discrete, and stochastic.
- The time steps are the repair intervals, each of length
-\begin_inset Formula $A$
-\end_inset
-
-, so
-\begin_inset Formula $T$
-\end_inset
-
--values are multiples of
-\begin_inset Formula $A$
-\end_inset
-
-.
- During each interval, the file's shares degrade according to the probability
- mass function of
-\begin_inset Formula $K$
-\end_inset
-
-.
-\end_layout
-
-\begin_layout Subsection
-Aggressive Repairs
-\end_layout
-
-\begin_layout Standard
-Let's first consider the case of an aggressive repairer.
- Every interval, this repairer checks the file for share losses and restores
- them.
- Thus, at the beginning of each interval, the file always has
-\begin_inset Formula $N$
-\end_inset
-
- shares, distributed on servers with various individual and group failure
- probalities, which will survive or fail per the output of random variable
-
-\begin_inset Formula $K$
-\end_inset
-
-.
-\end_layout
-
-\begin_layout Standard
-For any interval, then, the probability that the file will survive is
-\begin_inset Formula $f\left(k\right)=Pr[K\geq k]$
-\end_inset
-
-.
- Since each interval success or failure is independent, and assuming the
- share reliabilities remain constant over time,
-\begin_inset Formula \begin{equation}
-R\left(t\right)=f(k)^{t}\end{equation}
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-This simple survival function makes it simple to select parameters
-\begin_inset Formula $N$
-\end_inset
-
- and
-\begin_inset Formula $K$
-\end_inset
-
- such that
-\begin_inset Formula $R(t)\geq r$
-\end_inset
-
-, where
-\begin_inset Formula $r$
-\end_inset
-
- is a user-specified parameter indicating the desired probability of survival
- to time
-\begin_inset Formula $t$
-\end_inset
-
-.
- Specifically, we can solve for
-\begin_inset Formula $f\left(k\right)$
-\end_inset
-
- in
-\begin_inset Formula $r\leq f\left(k\right)^{t}$
-\end_inset
-
-, giving:
-\begin_inset Formula \begin{equation}
-f\left(k\right)\geq\sqrt[t]{r}\end{equation}
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-So, given a PMF
-\begin_inset Formula $f\left(k\right)$
-\end_inset
-
-, to assure the survival of a file to time
-\begin_inset Formula $t$
-\end_inset
-
- with probability at least
-\begin_inset Formula $r$
-\end_inset
-
-, choose
-\begin_inset Formula $k:f\left(k\right)\geq\sqrt[t]{r}$
-\end_inset
-
-.
- For example, if
-\begin_inset Formula $A$
-\end_inset
-
- is one month, and
-\begin_inset Formula $r=1-\nicefrac{1}{1000000}$
-\end_inset
-
- and
-\begin_inset Formula $t=120$
-\end_inset
-
-, or 10 years, we calculate
-\begin_inset Formula $f\left(k\right)\geq\sqrt[120]{.999999}\cong0.999999992$
-\end_inset
-
-.
- Per the PMF of table
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "tab:Example-PMF"
-
-\end_inset
-
-, this means
-\begin_inset Formula $k=2$
-\end_inset
-
-, achieves the goal, at the cose of a six-fold expansion in stored file
- size.
- If the lesser goal of no more than
-\begin_inset Formula $\nicefrac{1}{1000}$
-\end_inset
-
- probability of loss is taken, then since
-\begin_inset Formula $\sqrt[120]{.9999}=.999992$
-\end_inset
-
-,
-\begin_inset Formula $k=5$
-\end_inset
-
- achieves the goal with an expansion factor of
-\begin_inset Formula $2.4$
-\end_inset
-
-.
-\end_layout
-
-\begin_layout Subsection
-Repair Cost
-\end_layout
-
-\begin_layout Standard
-The simplicity and predictability of aggressive repair is attractive, but
- there is a downside: Repairs cost processing power and bandwidth.
- The processing power is proportional to the size of the file, since the
- whole file must be reconstructed and then re-processed using the Reed-Solomon
- algorithm, while the bandwidth cost is proportional to the number of missing
- shares that must be replaced,
-\begin_inset Formula $N-K$
-\end_inset
-
-.
-\end_layout
-
-\begin_layout Standard
-Let
-\begin_inset Formula $c\left(s,d,k\right)$
-\end_inset
-
- be a cost function that combines the processing cost of regenerating a
- file of size
-\begin_inset Formula $s$
-\end_inset
-
- and the bandwidth cost of downloading a file of size
-\begin_inset Formula $s$
-\end_inset
-
- and uploading
-\begin_inset Formula $d$
-\end_inset
-
- shares each of size
-\begin_inset Formula $\nicefrac{s}{k}$
-\end_inset
-
-.
- Also, let
-\begin_inset Formula $D$
-\end_inset
-
- denote the random variable
-\begin_inset Formula $N-K$
-\end_inset
-
-, which is the number of shares that must be redistributed to bring the
- file share set back up to
-\begin_inset Formula $N$
-\end_inset
-
- after degrading during an interval.
- The probability mass function of
-\begin_inset Formula $D$
-\end_inset
-
- is
-\begin_inset Formula \[
-Pr[D=d]=f(d)=\begin{cases}
-Pr\left[K=N\right]+Pr[K<k] & d=0\\
-Pr\left[K=N-d\right] & 0<d\leq N-k\\
-0 & N-k<d\leq N\end{cases}\]
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-The expected cost of repairs in a given interval, then, is simply
-\begin_inset Formula $c\left(s,E\left[D\right],k\right)$
-\end_inset
-
- where E is the expected value function -- in this case:
-\begin_inset Formula \begin{align*}
-E[D] & =\sum_{d=0}^{N}d\cdot Pr\left[D=d\right]\\
- & =0\cdot Pr\left[D=0\right]+\sum_{d=1}^{N-k}\left\{ d\cdot Pr\left[K=N-d\right]\right\} +\sum_{d=N-k+1}^{N}\left\{ d\cdot0\right\} \\
- & =\sum_{d=1}^{N-k}d\cdot Pr\left[K=N-d\right]\end{align*}
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-Since each interval starts with a full complement of shares, the expected
- repair cost for each interval is the same, and the cost for file that survives
- for
-\begin_inset Formula $t$
-\end_inset
-
- intervals is
-\begin_inset Formula $t\cdot c\left(s,E\left[D\right]\right)$
-\end_inset
-
-.
- To calculate the lifetime repair cost, we just take the limit over all
- intervals as
-\begin_inset Formula $t\rightarrow\infty$
-\end_inset
-
-, discounting each cost by the probability that the file has already failed.
- So, the lifetime expected repair cost is
-\begin_inset Formula \begin{align*}
-\sum_{t=1}^{\infty}R\left(t-1\right)c\left(s,E\left[D\right],k\right) & =c\left(s,E\left[D\right],k\right)\sum_{t=1}^{\infty}R\left(t-1\right)\\
- & =c\left(s,E\left[D\right],k\right)\sum_{t=1}^{\infty}f\left(k\right)^{t-1}\\
- & =c\left(s,E\left[D\right],k\right)\cdot\frac{1}{1-f\left(k\right)}\\
- & =\frac{c\left(s,E\left[D\right],k\right)}{1-f\left(k\right)}\end{align*}
-
-\end_inset
-
-
-\end_layout
-
-\begin_layout Standard
-It may also be useful to discount future cost, since CPU and bandwidth are
- both going to get cheaper over time.
- To accomodate this, we throw in an addition per-period discount rate
-\begin_inset Formula $r$
-\end_inset
-
-.
- In accordance with common discount rate usage, the discount multiplier
- at time
-\begin_inset Formula $t$
-\end_inset
-
- is
-\begin_inset Formula $\left(1-r\right)^{t}$
-\end_inset
-
-.
- This gives:
-\begin_inset Formula \begin{align*}
-\sum_{t=1}^{\infty}\left(1-r\right){}^{t}R\left(t-1\right)c\left(s,E\left[D\right],k\right) & =c\left(s,E\left[D\right],k\right)\sum_{t=1}^{\infty}\left(1-r\right)^{t}f\left(k\right)^{t-1}\\
- & =c\left(s,E\left[D\right],k\right)\sum_{t=1}^{\infty}\left(1-r\right)^{t}f\left(k\right)^{t-1}\\
- & =c\left(s,E\left[D\right],k\right)\left(1-r\right)\sum_{t=1}^{\infty}\left(1-r\right)^{t-1}f\left(k\right)^{t-1}\\
- & =\frac{c\left(s,E\left[D\right],k\right)\left(1-r\right)}{1-\left(1-r\right)f\left(k\right)}\end{align*}
-
-\end_inset
-
-If
-\begin_inset Formula $r=0$
-\end_inset
-
- this collapses to the previous result, as one would expect.
-\end_layout
-
-\begin_layout Section
-Time-Sensitive Retrieval
-\end_layout
-
-\begin_layout Standard
-The above work has almost entirely ignored the distinction between availability
- and reliability.
- In reality, temporary and permanent failures need to be modeled separately,
- and
-\end_layout
-
-\end_body
-\end_document
--- /dev/null
+#LyX 1.6.1 created this file. For more info see http://www.lyx.org/
+\lyxformat 345
+\begin_document
+\begin_header
+\textclass amsart
+\use_default_options true
+\begin_modules
+theorems-ams
+theorems-ams-extended
+\end_modules
+\language english
+\inputencoding auto
+\font_roman default
+\font_sans default
+\font_typewriter default
+\font_default_family default
+\font_sc false
+\font_osf false
+\font_sf_scale 100
+\font_tt_scale 100
+
+\graphics default
+\float_placement h
+\paperfontsize default
+\spacing single
+\use_hyperref false
+\papersize default
+\use_geometry false
+\use_amsmath 1
+\use_esint 1
+\cite_engine basic
+\use_bibtopic false
+\paperorientation portrait
+\secnumdepth 3
+\tocdepth 3
+\paragraph_separation indent
+\defskip medskip
+\quotes_language english
+\papercolumns 1
+\papersides 1
+\paperpagestyle default
+\tracking_changes false
+\output_changes false
+\author ""
+\author ""
+\end_header
+
+\begin_body
+
+\begin_layout Title
+Tahoe Distributed Filesharing System Loss Model
+\end_layout
+
+\begin_layout Author
+Shawn Willden
+\end_layout
+
+\begin_layout Date
+01/14/2009
+\end_layout
+
+\begin_layout Address
+South Weber, Utah
+\end_layout
+
+\begin_layout Email
+shawn@willden.org
+\end_layout
+
+\begin_layout Abstract
+The abstract goes here
+\end_layout
+
+\begin_layout Section
+Problem Statement
+\end_layout
+
+\begin_layout Standard
+The allmydata Tahoe distributed file system uses Reed-Solomon erasure coding
+ to split files into
+\begin_inset Formula $N$
+\end_inset
+
+ shares, each of which is then delivered to a randomly-selected peer in
+ a distributed network.
+ The file can later be reassembled from any
+\begin_inset Formula $k\leq N$
+\end_inset
+
+ of the shares, if they are available.
+\end_layout
+
+\begin_layout Standard
+Over time shares are lost for a variety of reasons.
+ Storage servers may crash, be destroyed or simply be removed from the network.
+ To mitigate such losses, Tahoe network clients employ a repair agent which
+ scans the peers once per time period
+\begin_inset Formula $A$
+\end_inset
+
+ and determines how many of the shares remain.
+ If less than
+\begin_inset Formula $L$
+\end_inset
+
+ (
+\begin_inset Formula $k\leq L\leq N$
+\end_inset
+
+) shares remain, then the repairer reconstructs the file shares and redistribute
+s the missing ones, bringing the availability back up to full.
+\end_layout
+
+\begin_layout Standard
+The question we're trying to answer is "What's the probability that we'll
+ be able to reassemble the file at some later time
+\begin_inset Formula $T$
+\end_inset
+
+?".
+ We'd also like to be able to determine what values we should choose for
+
+\begin_inset Formula $k$
+\end_inset
+
+,
+\begin_inset Formula $N$
+\end_inset
+
+,
+\begin_inset Formula $A$
+\end_inset
+
+, and
+\begin_inset Formula $L$
+\end_inset
+
+ in order to ensure
+\begin_inset Formula $Pr[loss]\leq t$
+\end_inset
+
+ for some threshold probability
+\begin_inset Formula $t$
+\end_inset
+
+.
+ This is an optimization problem because although we could obtain very low
+
+\begin_inset Formula $Pr[loss]$
+\end_inset
+
+ by choosing small
+\begin_inset Formula $k,$
+\end_inset
+
+ large
+\begin_inset Formula $N$
+\end_inset
+
+, small
+\begin_inset Formula $A$
+\end_inset
+
+, and setting
+\begin_inset Formula $L=N$
+\end_inset
+
+, these choices have costs.
+ The peer storage and bandwidth consumed by the share distribution process
+ are approximately
+\begin_inset Formula $\nicefrac{N}{k}$
+\end_inset
+
+ times the size of the original file, so we would like to reduce this ratio
+ as far as possible consistent with
+\begin_inset Formula $Pr[loss]\leq t$
+\end_inset
+
+.
+ Likewise, frequent and aggressive repair process can be used to ensure
+ that the number of shares available at any time is very close to
+\begin_inset Formula $N,$
+\end_inset
+
+ but at a cost in bandwidth as the repair agent downloads
+\begin_inset Formula $k$
+\end_inset
+
+ shares to reconstruct the file and uploads new shares to replace those
+ that are lost.
+\end_layout
+
+\begin_layout Section
+Reliability
+\end_layout
+
+\begin_layout Standard
+The probability that the file becomes unrecoverable is dependent upon the
+ probability that the peers to whom we send shares are able to return those
+ copies on demand.
+ Shares that are returned in corrupted form can be detected and discarded,
+ so there is no need to distinguish between corruption and loss.
+\end_layout
+
+\begin_layout Standard
+There are a large number of factors that affect share availability.
+ Availability can be temporarily interrupted by peer unavailability, due
+ to network outages, power failures or administrative shutdown, among other
+ reasons.
+ Availability can be permanently lost due to failure or corruption of storage
+ media, catastrophic damage to the peer system, administrative error, withdrawal
+ from the network, malicious corruption, etc.
+\end_layout
+
+\begin_layout Standard
+The existence of intermittent failure modes motivates the introduction of
+ a distinction between
+\noun on
+availability
+\noun default
+ and
+\noun on
+reliability
+\noun default
+.
+ Reliability is the probability that a share is retrievable assuming intermitten
+t failures can be waited out, so reliability considers only permanent failures.
+ Availability considers all failures, and is focused on the probability
+ of retrieval within some defined time frame.
+\end_layout
+
+\begin_layout Standard
+Another consideration is that some failures affect multiple shares.
+ If multiple shares of a file are stored on a single hard drive, for example,
+ failure of that drive may lose them all.
+ Catastrophic damage to a data center may destroy all shares on all peers
+ in that data center.
+\end_layout
+
+\begin_layout Standard
+While the types of failures that may occur are pretty consistent across
+ even very different peers, their probabilities differ dramatically.
+ A professionally-administered blade server with redundant storage, power
+ and Internet located in a carefully-monitored data center with automatic
+ fire suppression systems is much less likely to become either temporarily
+ or permanently unavailable than the typical virus and malware-ridden home
+ computer on a single cable modem connection.
+ A variety of situations in between exist as well, such as the case of the
+ author's home file server, which is administered by an IT professional
+ and uses RAID level 6 redundant storage, but runs on old, cobbled-together
+ equipment, and has a consumer-grade Internet connection.
+\end_layout
+
+\begin_layout Standard
+To begin with, let's use a simple definition of reliability:
+\end_layout
+
+\begin_layout Definition
+
+\noun on
+Reliability
+\noun default
+ is the probability
+\begin_inset Formula $p_{i}$
+\end_inset
+
+ that a share
+\begin_inset Formula $s_{i}$
+\end_inset
+
+ will surve to (be retrievable at) time
+\begin_inset Formula $T=A$
+\end_inset
+
+, ignoring intermittent failures.
+ That is, the probability that the share will be retrievable at the end
+ of the current repair cycle, and therefore usable by the repairer to regenerate
+ any lost shares.
+\end_layout
+
+\begin_layout Definition
+Reliability is clearly dependent on
+\begin_inset Formula $A$
+\end_inset
+
+.
+ Short repair cycles offer less time for shares to
+\begin_inset Quotes eld
+\end_inset
+
+decay
+\begin_inset Quotes erd
+\end_inset
+
+ into unavailability.
+\end_layout
+
+\begin_layout Subsection
+Fixed Reliability
+\begin_inset CommandInset label
+LatexCommand label
+name "sub:Fixed-Reliability"
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+In the simplest case, the peers holding the file shares all have the same
+ reliability
+\begin_inset Formula $p$
+\end_inset
+
+, and are all independent from one another.
+ Let
+\begin_inset Formula $K$
+\end_inset
+
+ be a random variable that represents the number of shares that survive
+
+\begin_inset Formula $A$
+\end_inset
+
+.
+ Each share's survival can be viewed as an indepedent Bernoulli trial with
+ a succes probability of
+\begin_inset Formula $p$
+\end_inset
+
+, which means that
+\begin_inset Formula $K$
+\end_inset
+
+ follows the binomial distribution with paramaters
+\begin_inset Formula $N$
+\end_inset
+
+ and
+\begin_inset Formula $p$
+\end_inset
+
+.
+ That is,
+\begin_inset Formula $K\sim B(N,p)$
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Theorem
+Binomial Distribution Theorem
+\end_layout
+
+\begin_layout Theorem
+Consider
+\begin_inset Formula $n$
+\end_inset
+
+ independent Bernoulli trials
+\begin_inset Foot
+status collapsed
+
+\begin_layout Plain Layout
+A Bernoulli trial is simply a test of some sort that results in one of two
+ outcomes, one of which is designated success and the other failure.
+ The classic example of a Bernoulli trial is a coin toss.
+\end_layout
+
+\end_inset
+
+ that succeed with probability
+\begin_inset Formula $p$
+\end_inset
+
+, and let
+\begin_inset Formula $K$
+\end_inset
+
+ be a random variable that represents the number of successes.
+ We say that
+\begin_inset Formula $K$
+\end_inset
+
+ follows the Binomial Distribution with parameters n and p, denoted
+\begin_inset Formula $K\sim B(n,p)$
+\end_inset
+
+.
+ The probability that
+\begin_inset Formula $K$
+\end_inset
+
+ takes a particular value
+\begin_inset Formula $m$
+\end_inset
+
+ (the probability that there are exactly
+\begin_inset Formula $m$
+\end_inset
+
+ successful trials, and therefore
+\begin_inset Formula $n-m$
+\end_inset
+
+ failures) is called the probability mass function and is given by:
+\begin_inset Formula \begin{equation}
+Pr[K=m]=f(m;n,p)=\binom{n}{p}p^{m}(1-p)^{n-m}\label{eq:binomial-pmf}\end{equation}
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Proof
+Consider the specific case of exactly
+\begin_inset Formula $m$
+\end_inset
+
+ successes followed by
+\begin_inset Formula $n-m$
+\end_inset
+
+ failures, because each success has probability
+\begin_inset Formula $p$
+\end_inset
+
+, each failure has probability
+\begin_inset Formula $1-p$
+\end_inset
+
+, and the trials are independent, the probability of this exact case occurring
+ is
+\begin_inset Formula $p^{m}\left(1-p\right)^{\left(n-m\right)}$
+\end_inset
+
+, the product of the probabilities of the outcome of each trial.
+\end_layout
+
+\begin_layout Proof
+Now consider any reordering of these
+\begin_inset Formula $m$
+\end_inset
+
+ successes and
+\begin_inset Formula $n$
+\end_inset
+
+ failures.
+ Any such reordering occurs with the same probability
+\begin_inset Formula $p^{m}\left(1-p\right)^{\left(n-m\right)}$
+\end_inset
+
+, but with the terms of the product reordered.
+ Since multiplication is commutative, each such reordering has the same
+ probability.
+ There are n-choose-m such orderings, and each ordering is an independent
+ event, so the probability that any ordering of
+\begin_inset Formula $m$
+\end_inset
+
+ successes and
+\begin_inset Formula $n-m$
+\end_inset
+
+ failures occurs is given by
+\begin_inset Formula \[
+\binom{n}{m}p^{m}\left(1-p\right)^{\left(n-m\right)}\]
+
+\end_inset
+
+which is the right-hand-side of equation
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "eq:binomial-pmf"
+
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Standard
+A file survives if at least
+\begin_inset Formula $k$
+\end_inset
+
+ of the
+\begin_inset Formula $N$
+\end_inset
+
+ shares survive.
+ Equation
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "eq:binomial-pmf"
+
+\end_inset
+
+ gives the probability that exactly
+\begin_inset Formula $i$
+\end_inset
+
+ shares survive, for any
+\begin_inset Formula $1\leq i\leq n$
+\end_inset
+
+, so the probability that fewer than
+\begin_inset Formula $k$
+\end_inset
+
+ survive is the sum of the probabilities that
+\begin_inset Formula $0,1,2,\ldots,k-1$
+\end_inset
+
+ shares survive.
+ That is:
+\end_layout
+
+\begin_layout Standard
+\begin_inset Formula \begin{equation}
+Pr[file\, lost]=\sum_{i=0}^{k-1}\binom{n}{i}p^{i}(1-p)^{n-i}\label{eq:simple-failure}\end{equation}
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Subsection
+Independent Reliability
+\begin_inset CommandInset label
+LatexCommand label
+name "sub:Independent-Reliability"
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+Equation
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "eq:simple-failure"
+
+\end_inset
+
+ assumes that each share has the same probability of survival, but as explained
+ above, this is not necessarily true.
+ A more accurate model allows each share
+\begin_inset Formula $s_{i}$
+\end_inset
+
+ an independent probability of survival
+\begin_inset Formula $p_{i}$
+\end_inset
+
+.
+ Each share's survival can still be treated as an independent Bernoulli
+ trial, but with success probability
+\begin_inset Formula $p_{i}$
+\end_inset
+
+.
+ Under this assumption,
+\begin_inset Formula $K$
+\end_inset
+
+ follows a generalized binomial distribution with parameters
+\begin_inset Formula $N$
+\end_inset
+
+ and
+\begin_inset Formula $p_{i}$
+\end_inset
+
+ where
+\begin_inset Formula $1\leq i\leq N$
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Standard
+The PMF for this generalized
+\begin_inset Formula $K$
+\end_inset
+
+ does not have a simple closed-form representation.
+ However, the PMFs for random variables representing individual share survival
+ do.
+ Let
+\begin_inset Formula $S_{i}$
+\end_inset
+
+ be a random variable such that:
+\end_layout
+
+\begin_layout Standard
+\begin_inset Formula \[
+S_{i}=\begin{cases}
+1 & \textnormal{if }s_{i}\textnormal{ survives}\\
+0 & \textnormal{if }s_{i}\textnormal{ fails}\end{cases}\]
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+The PMF for
+\begin_inset Formula $S_{i}$
+\end_inset
+
+ is very simple:
+\begin_inset Formula \[
+Pr[S_{i}=j]=\begin{cases}
+1-p_{i} & j=0\\
+p_{i} & j=1\end{cases}\]
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+Note that since each
+\begin_inset Formula $S_{i}$
+\end_inset
+
+ represents the count of shares
+\begin_inset Formula $s_{i}$
+\end_inset
+
+ that survives (either 0 or 1), if we add up all of the individual survivor
+ counts, we get the group survivor count.
+ That is:
+\begin_inset Formula \[
+\sum_{i=1}^{N}S_{i}=K\]
+
+\end_inset
+
+Effectively,
+\begin_inset Formula $K$
+\end_inset
+
+ has just been separated into the series of Bernoulli trials that make it
+ up.
+\end_layout
+
+\begin_layout Theorem
+Discrete Convolution Theorem
+\end_layout
+
+\begin_layout Theorem
+Let
+\begin_inset Formula $X$
+\end_inset
+
+ and
+\begin_inset Formula $Y$
+\end_inset
+
+ be discrete random variables with probability mass functions given by
+\begin_inset Formula $Pr\left[X=x\right]=f(x)$
+\end_inset
+
+ and
+\begin_inset Formula $Pr\left[Y=y\right]=g(y).$
+\end_inset
+
+ Let
+\begin_inset Formula $Z$
+\end_inset
+
+ be the discrete random random variable obtained by summing
+\begin_inset Formula $X$
+\end_inset
+
+ and
+\begin_inset Formula $Y$
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Theorem
+The probability mass function of
+\begin_inset Formula $Z$
+\end_inset
+
+ is given by
+\begin_inset Formula \[
+Pr[Z=z]=h(z)=\left(f\star g\right)(z)\]
+
+\end_inset
+
+where
+\begin_inset Formula $\star$
+\end_inset
+
+ denotes the discrete convolution operation:
+\begin_inset Formula \[
+\left(f\star g\right)\left(n\right)=\sum_{m=-\infty}^{\infty}f\left(m\right)g\left(m-n\right)\]
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Proof
+The proof is beyond the scope of this paper.
+\begin_inset Foot
+status collapsed
+
+\begin_layout Plain Layout
+\begin_inset Quotes eld
+\end_inset
+
+Beyond the scope of this paper
+\begin_inset Quotes erd
+\end_inset
+
+ usually means
+\begin_inset Quotes eld
+\end_inset
+
+Too long and nasty to bore you with
+\begin_inset Quotes erd
+\end_inset
+
+.
+ In this case it means
+\begin_inset Quotes eld
+\end_inset
+
+The author hasn't the foggiest idea why this is true, or how to prove it,
+ but reliable authorities say it's real, and in practice it works a treat.
+\begin_inset Quotes erd
+\end_inset
+
+
+\end_layout
+
+\end_inset
+
+ If you don't believe it's true, look it up on Wikipedia, which is never
+ wrong.
+\end_layout
+
+\begin_layout Standard
+Applying the discrete convolution theorem, if
+\begin_inset Formula $Pr[K=i]=f(i)$
+\end_inset
+
+ and
+\begin_inset Formula $Pr[S_{i}=j]=g_{i}(j)$
+\end_inset
+
+, then
+\begin_inset Formula $f=g_{1}\star g_{2}\star g_{3}\star\ldots\star g_{N}$
+\end_inset
+
+.
+ Since convolution is associative, this can also be written as
+\begin_inset Formula $ $
+\end_inset
+
+
+\begin_inset Formula \begin{equation}
+f=(\ldots((g_{1}\star g_{2})\star g_{3})\star\ldots)\star g_{N})\label{eq:convolution}\end{equation}
+
+\end_inset
+
+Therefore,
+\begin_inset Formula $f$
+\end_inset
+
+ can be computed as a sequence of convolution operations on the simple PMFs
+ of the random variables
+\begin_inset Formula $S_{i}$
+\end_inset
+
+.
+ In fact, for large
+\begin_inset Formula $N$
+\end_inset
+
+, equation
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "eq:convolution"
+
+\end_inset
+
+ turns out to be a more effective means of computing the PMF of
+\begin_inset Formula $K$
+\end_inset
+
+ even in the case of the standard binomial distribution, primarily because
+ the binomial calculation in equation
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "eq:binomial-pmf"
+
+\end_inset
+
+ produces very large values that overflow unless arbitrary precision numeric
+ representations are used.
+\end_layout
+
+\begin_layout Standard
+Note also that it is not necessary to have very simple PMFs like those of
+ the
+\begin_inset Formula $S_{i}$
+\end_inset
+
+.
+ Any share or set of shares that has a known PMF can be combined with any
+ other set with a known PMF by convolution, as long as the two share sets
+ are independent.
+ Since PMFs are easily represented as simple lists of probabilities, where
+ the
+\begin_inset Formula $i$
+\end_inset
+
+th element in the list corresponds to
+\begin_inset Formula $Pr[K=i]$
+\end_inset
+
+, these functions are easily managed in software, and computing the convolution
+ is both simple and efficient.
+\end_layout
+
+\begin_layout Subsection
+Multiple Failure Modes
+\begin_inset CommandInset label
+LatexCommand label
+name "sub:Multiple-Failure-Modes"
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+In modeling share survival probabilities, it's useful to be able to analyze
+ separately each of the various failure modes.
+ If reliable statistics for disk failure can be obtained, then a probability
+ mass function for that form of failure can be generated.
+ Similarly, statistics on other hardware failures, administrative errors,
+ network losses, etc., can all be estimated independently.
+ If those estimates can then be combined into a single PMF for a share,
+ then we can use it to predict failures for that share.
+\end_layout
+
+\begin_layout Standard
+Combining independent failure modes for a single share is straightforward.
+ If
+\begin_inset Formula $p_{i,j}$
+\end_inset
+
+ is the probability of survival of the
+\begin_inset Formula $j$
+\end_inset
+
+th failure mode of share
+\begin_inset Formula $i$
+\end_inset
+
+,
+\begin_inset Formula $1\leq j\leq m$
+\end_inset
+
+, then
+\begin_inset Formula \[
+Pr[S_{i}=k]=f_{i}(k)=\begin{cases}
+\prod_{j=1}^{m}p_{i,j} & k=1\\
+1-\prod_{j=1}^{m}p_{i,j} & k=0\end{cases}\]
+
+\end_inset
+
+is the survival PMF.
+\end_layout
+
+\begin_layout Subsection
+Multi-share failures
+\begin_inset CommandInset label
+LatexCommand label
+name "sub:Multi-share-failures"
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+If there are failure modes that affect multiple computers, we can also construct
+ the PMF that predicts their survival.
+ The key observation is that the PMF has non-zero probabilities only for
+
+\begin_inset Formula $0$
+\end_inset
+
+ survivors and
+\begin_inset Formula $n$
+\end_inset
+
+ survivors, where
+\begin_inset Formula $n$
+\end_inset
+
+ is the number of shares in the set.
+ If
+\begin_inset Formula $p$
+\end_inset
+
+ is the probability of survival, the PMF of
+\begin_inset Formula $K$
+\end_inset
+
+, a random variable representing the number of surviors is
+\begin_inset Formula \[
+Pr[K=i]=f(i)=\begin{cases}
+p & i=n\\
+0 & 0<i<n\\
+1-p & i=0\end{cases}\]
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+Group failures due to multiple independent causes can be combined as in
+ section
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "sub:Multiple-Failure-Modes"
+
+\end_inset
+
+, as long as they apply to the whole group.
+\end_layout
+
+\begin_layout Example
+Putting the Pieces Together
+\end_layout
+
+\begin_layout Standard
+Sections
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "sub:Fixed-Reliability"
+
+\end_inset
+
+ through
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "sub:Multi-share-failures"
+
+\end_inset
+
+ provide ways of calculating the survival probability mass functions for
+ a variety of share failure structures and modes.
+ As an example of how these pieces can be used, consider a network with
+ the following peers:
+\end_layout
+
+\begin_layout Itemize
+Four servers located in a data center in Nebraska.
+ The machines have multiply-redundant Internet connections, with a failure
+ probability of 0.0001.
+ They store their shares on RAID arrays with failure probability of 0.0002.
+ The administrative staff makes data-destroying errors with probability
+ 0.003.
+\end_layout
+
+\begin_layout Itemize
+Four servers located in a data center on the island of Hawaii.
+ These servers have identical failure probabilities as the servers in Nebraska,
+ except that the data center is near the edge of the crater on Mount Kilauea
+ (nobody said examples had to be realistic).
+ There is a 0.04 chance that the volcano will erupt and bury the data center
+ in molten lava, destroying it entirely.
+\end_layout
+
+\begin_layout Itemize
+Four PCs located in random homes, connected to the Internet via assorted
+ cable modems and DSL.
+ Their network connections fail with probability 0.009.
+ Their disks fail with probability 0.001.
+ Their users destroy data with probability 0.05.
+\end_layout
+
+\begin_layout Standard
+If one share is placed on each of these 20 computers, what's the probability
+ mass function of share survival? To more compactly describe PMFs, we'll
+ denote them as probability vectors of the form
+\begin_inset Formula $\left[\alpha_{o},\alpha_{1},\alpha_{2},\ldots\alpha_{n}\right]$
+\end_inset
+
+ where
+\begin_inset Formula $\alpha_{i}$
+\end_inset
+
+ is the probability that exactly
+\begin_inset Formula $i$
+\end_inset
+
+ shares survive.
+\end_layout
+
+\begin_layout Standard
+The servers in the two data centers have individual survival probabilities
+ of RAID failure (.0002) and administrative error (.003) giving
+\begin_inset Formula \[
+(1-.0002)\cdot(1-.003)=.9998\cdot.997=.9968\]
+
+\end_inset
+
+Using
+\begin_inset Formula $p=.9968,n=4$
+\end_inset
+
+ in equation
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "eq:binomial-pmf"
+
+\end_inset
+
+ gives the survival PMF
+\begin_inset Formula \[
+\left[1.049\times10^{-10},1.307\times10^{-7},6.105\times10^{-5},0.01271,0.9872\right]\]
+
+\end_inset
+
+which applies to each group of four servers.
+ However, each data center also has a .0001 chance of data connection loss,
+ which affects all four servers at once, and Hawaii has the additional .04
+ probability of severe lava burn.
+ If the network fails at a location, all the machines go offline together.
+ The probability that 0 machines survive is the probability that they all
+ fail for individual reasons (
+\begin_inset Formula $1.049\cdot10^{-10}$
+\end_inset
+
+) times the probability they all fail because of a network outage (
+\begin_inset Formula $.0001$
+\end_inset
+
+) less the probability they fail for both reasons:
+\begin_inset Formula \[
+\left(1.049\times10^{-10}\right)+\left(0.0001\right)-\left[\left(1.049\times10^{-10}\right)\cdot\left(0.0001\right)\right]=0.0001\]
+
+\end_inset
+
+That's the
+\begin_inset Formula $0$
+\end_inset
+
+th element of the combined PMF.
+ The combined probability of survival of
+\begin_inset Formula $0<i\leq4$
+\end_inset
+
+ servers is simpler: it's the probility they survive individual failure,
+ from the individual failure PMF above, times the probability they survive
+ network failure (.9999).
+ So the combined survival PMF, which we'll denote as
+\begin_inset Formula $n(i)$
+\end_inset
+
+ of the Nebraska servers is
+\begin_inset Formula \[
+n(i)=\left[0.0001,1.306\times10^{-7},6.104\times10^{-5},0.01268,0.9872\right]\]
+
+\end_inset
+
+which has the interesting property that complete failure is 1000 times more
+ likely than survival of one server.
+ This is because the probability of a network outage is so much greater
+ than simultaneous
+\begin_inset Foot
+status collapsed
+
+\begin_layout Plain Layout
+Of course, the failures need not be truly simultaneous, they just have happen
+ in the same interval between repair runs.
+\end_layout
+
+\end_inset
+
+ independent failure of three servers.
+\end_layout
+
+\begin_layout Standard
+The same process for the Hawaii servers, but with group survival probability
+ of
+\begin_inset Formula $(1-.0001)(1-.02)=.9799$
+\end_inset
+
+ gives the survival PMF
+\begin_inset Formula \[
+h(i)=\left[0.0201,1.280\times10^{-7},5.982\times10^{-5},0.01242,0.9674\right]\]
+
+\end_inset
+
+which has the unusual property that it's more likely that all of the servers
+ will be lost than that only one will survive.
+ This is because in order for exactly one to survive, it's necessary for
+ three to have the
+\end_layout
+
+\begin_layout Standard
+Applying the convolution operator to
+\begin_inset Formula $n(i)$
+\end_inset
+
+ and
+\begin_inset Formula $h(i)$
+\end_inset
+
+, the survival PMF of all eight servers is:
+\end_layout
+
+\begin_layout Standard
+\begin_inset Formula \[
+\left(n\star h\right)\left(i\right)=\begin{cases}
+2.010\times10^{-6} & i=0\\
+2.639\times10^{-9} & i=1\\
+1.233\times10^{-6} & i=2\\
+2.560\times10^{-4} & i=3\\
+0.01994 & i=4\\
+1.769\times10^{-6} & i=5\\
+2.756\times10^{-4} & i=6\\
+0.02452 & i=7\\
+0.9559 & i=8\end{cases}\]
+
+\end_inset
+
+Note the interesting fact that losing four shares is 10,000 times more likely
+ than losing three.
+ This is because both data centers have a whole-center failure modes, and
+ the Hawaii center's lava burn probability is so high.
+\end_layout
+
+\begin_layout Standard
+For the home PCs, their individual probability of survival is
+\begin_inset Formula \[
+(1-.009)\cdot(1-.001)\cdot(1-.05)=.991\cdot.999\cdot.95=.9405\]
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+We can then apply equation
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "eq:binomial-pmf"
+
+\end_inset
+
+ with
+\begin_inset Formula $N=4$
+\end_inset
+
+ and
+\begin_inset Formula $p=.9405$
+\end_inset
+
+ to compute the PMF
+\begin_inset Formula $f(i),0\leq i\leq4$
+\end_inset
+
+ for the PCs and finally compute
+\begin_inset Formula $s(i)=\left(f\star\left(n\star h\right)\right)\left(i\right)$
+\end_inset
+
+, the PMF of the whole share set.
+ Summing the values of
+\begin_inset Formula $s(i)$
+\end_inset
+
+ for
+\begin_inset Formula $0\leq i\leq k-1$
+\end_inset
+
+ gives the probability that less than
+\begin_inset Formula $k$
+\end_inset
+
+ shares survive and the file is unrecoverable.
+ For this example, those sums are shown in table
+\begin_inset CommandInset ref
+LatexCommand vref
+reference "tab:Example-PMF"
+
+\end_inset
+
+.
+\begin_inset Float table
+wide false
+sideways false
+status collapsed
+
+\begin_layout Plain Layout
+\align center
+\begin_inset Tabular
+<lyxtabular version="3" rows="13" columns="4">
+<features>
+<column alignment="center" valignment="top" width="0">
+<column alignment="center" valignment="top" width="0">
+<column alignment="center" valignment="top" width="0">
+<column alignment="center" valignment="top" width="0">
+<row>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $k$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $Pr[K=k]$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $Pr[file\, loss]=Pr[K<k]$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $N/k$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+1
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $1.60\times10^{-9}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $2.53\times10^{-11}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+12
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+2
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $3.80\times10^{-8}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $1.63\times10^{-9}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+6
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+3
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $4.04\times10^{-7}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $3.70\times10^{-8}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+4
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+4
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $2.06\times10^{-6}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $4.44\times10^{-7}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+3
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+5
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $2.10\times10^{-5}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $2.50\times10^{-6}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+2.4
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+6
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.000428$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $2.35\times10^{-5}$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+2
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+7
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.00417$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.000452$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+1.7
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+8
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.0157$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.00462$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+1.5
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+9
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.00127$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.0203$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+1.3
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+10
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.0230$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.0216$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+1.2
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+11
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.208$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.0446$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+1.1
+\end_layout
+
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+12
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.747$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+\begin_inset Formula $0.253$
+\end_inset
+
+
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+1
+\end_layout
+
+\end_inset
+</cell>
+</row>
+</lyxtabular>
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Plain Layout
+\begin_inset Caption
+
+\begin_layout Plain Layout
+\align left
+\begin_inset CommandInset label
+LatexCommand label
+name "tab:Example-PMF"
+
+\end_inset
+
+Example PMF
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+The table demonstrates the importance of the selection of
+\begin_inset Formula $k$
+\end_inset
+
+, and the tradeoff against file size expansion.
+ Note that the survival of exactly 9 servers is significantly less likely
+ than the survival of 8 or 10 servers.
+ This is, again, an artifact of the group failure modes.
+ Because of this, there is no reason to choose
+\begin_inset Formula $k=9$
+\end_inset
+
+ over
+\begin_inset Formula $k=10$
+\end_inset
+
+.
+ Normally, reducing the number of shares needed for reassembly improve the
+ file's chances of survival, but in this case it provides a miniscule gain
+ in reliability at the cost of a 10% increase in bandwidth and storage consumed.
+\end_layout
+
+\begin_layout Subsection
+Share Duplication
+\end_layout
+
+\begin_layout Standard
+Before moving on to consider issues other than single-interval file loss,
+ let's analyze one more possibility, that of
+\begin_inset Quotes eld
+\end_inset
+
+cheap
+\begin_inset Quotes erd
+\end_inset
+
+ file repair via share duplication.
+\end_layout
+
+\begin_layout Standard
+Initially, files are split using erasure coding, which creates
+\begin_inset Formula $N$
+\end_inset
+
+ unique shares, any
+\begin_inset Formula $k$
+\end_inset
+
+ of which can be used to to reconstruct the file.
+ When shares are lost, proper repair downloads some
+\begin_inset Formula $k$
+\end_inset
+
+ shares, reconstructs the original file and then uses the erasure coding
+ algorithm to reconstruct the lost shares, then redeploys them to peers
+ in the network.
+ This is a somewhat expensive process.
+\end_layout
+
+\begin_layout Standard
+A cheaper repair option is simply to direct some peer that has share
+\begin_inset Formula $s_{i}$
+\end_inset
+
+ to send a copy to another peer, thus increasing by one the number of shares
+ in the network.
+ This is not as good as actually replacing the lost share, though.
+ Suppose that more shares were lost, leaving only
+\begin_inset Formula $ $
+\end_inset
+
+
+\begin_inset Formula $k$
+\end_inset
+
+ shares remaining.
+ If two of those shares are identical, because one was duplicated in this
+ fashion, then only
+\begin_inset Formula $k-1$
+\end_inset
+
+ shares truly remain, and the file can no longer be reconstructed.
+\end_layout
+
+\begin_layout Standard
+However, such cheap repair is not completely pointless; it does increase
+ file survivability.
+ The question is: By how much?
+\end_layout
+
+\begin_layout Standard
+Effectively, share duplication simply increases the probability that
+\begin_inset Formula $s_{i}$
+\end_inset
+
+ will survive, by providing two locations from which to retrieve it.
+ We can view the two copies of the single share as one, but with a higher
+ probability of survival than would be provided by either of the two peers.
+ In particular, if
+\begin_inset Formula $p_{1}$
+\end_inset
+
+ and
+\begin_inset Formula $p_{2}$
+\end_inset
+
+ are the probabilities that the two peers will survive, respectively, then
+\begin_inset Formula \[
+Pr[s_{i}\, survives]=p_{1}+p_{2}-p_{1}p_{2}\]
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+More generally, if a single share is deployed on
+\begin_inset Formula $n$
+\end_inset
+
+ peers, each with a PMF
+\begin_inset Formula $f_{i}(j),0\leq j\leq1,1\leq i\leq n$
+\end_inset
+
+, the share survival count is a random variable
+\begin_inset Formula $S$
+\end_inset
+
+ and the probability of share loss is
+\begin_inset Formula \[
+Pr[S=0]=(f_{1}\star f_{2}\star\ldots\star f_{n})(0)\]
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+From that, we can construct a share PMF in the obvious way, which can then
+ be convolved with the other share PMFs to produce the share set PMF.
+\end_layout
+
+\begin_layout Example
+Suppose a file has
+\begin_inset Formula $N=10,k=3$
+\end_inset
+
+ and that all servers have survival probability
+\begin_inset Formula $p=.9$
+\end_inset
+
+.
+ Given a full complement of shares,
+\begin_inset Formula $Pr[\textrm{file\, loss}]=3.74\times10^{-7}$
+\end_inset
+
+.
+ Suppose that four shares are lost, which increases
+\begin_inset Formula $Pr[\textrm{file\, loss}]$
+\end_inset
+
+ to
+\begin_inset Formula $.00127$
+\end_inset
+
+, a value
+\begin_inset Formula $3400$
+\end_inset
+
+ times greater.
+ Rather than doing a proper reconstruction, we could direct four peers still
+ holding shares to send a copy of their share to new peer, which changes
+ the composition of the shares from one of six, unique
+\begin_inset Quotes eld
+\end_inset
+
+standard
+\begin_inset Quotes erd
+\end_inset
+
+ shares, to one of two standard shares, each with survival probability
+\begin_inset Formula $.9$
+\end_inset
+
+ and four
+\begin_inset Quotes eld
+\end_inset
+
+doubled
+\begin_inset Quotes erd
+\end_inset
+
+ shares, each with survival probability
+\begin_inset Formula $2p-p^{2}\approx.99$
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Example
+Combining the two single-peer share PMFs with the four double-share PMFs
+ gives a new file survival probability of
+\begin_inset Formula $6.64\times10^{-6}$
+\end_inset
+
+.
+ Not as good as a full repair, but still quite respectable.
+ Also, if storage were not a concern, all six shares could be duplicated,
+ for a
+\begin_inset Formula $Pr[file\, loss]=1.48\times10^{-7}$
+\end_inset
+
+, which is actually three time better than the nominal case.
+\end_layout
+
+\begin_layout Example
+The reason such cheap repairs may be attractive in many cases is that distribute
+d bandwidth is cheaper than bandwidth through a single peer.
+ This is particularly true if that single peer has a very slow connection,
+ which is common for home computers -- especially in the outbound direction.
+\end_layout
+
+\begin_layout Section
+Long-Term Reliability
+\end_layout
+
+\begin_layout Standard
+Thus far, we've focused entirely on the probability that a file survives
+ the interval
+\begin_inset Formula $A$
+\end_inset
+
+ between repair times.
+ The probability that a file survives long-term, though, is also important.
+ As long as the probability of failure during a repair period is non-zero,
+ a given file will eventually be lost.
+ We want to know what the probability of surviving for time
+\begin_inset Formula $T$
+\end_inset
+
+ is, and how the parameters
+\begin_inset Formula $A$
+\end_inset
+
+ (time between repairs) and
+\begin_inset Formula $L$
+\end_inset
+
+ (share low watermark) affect survival time.
+\end_layout
+
+\begin_layout Standard
+To model file survival time, let
+\begin_inset Formula $T$
+\end_inset
+
+ be a random variable denoting the time at which a given file becomes unrecovera
+ble, and
+\begin_inset Formula $R(t)=Pr[T>t]$
+\end_inset
+
+ be a function that gives the probability that the file survives to time
+
+\begin_inset Formula $t$
+\end_inset
+
+.
+
+\begin_inset Formula $R(t)$
+\end_inset
+
+ is the cumulative distribution function of
+\begin_inset Formula $T$
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Standard
+Most survival functions are continuous, but
+\begin_inset Formula $R(t)$
+\end_inset
+
+ is inherently discrete, and stochastic.
+ The time steps are the repair intervals, each of length
+\begin_inset Formula $A$
+\end_inset
+
+, so
+\begin_inset Formula $T$
+\end_inset
+
+-values are multiples of
+\begin_inset Formula $A$
+\end_inset
+
+.
+ During each interval, the file's shares degrade according to the probability
+ mass function of
+\begin_inset Formula $K$
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Subsection
+Aggressive Repairs
+\end_layout
+
+\begin_layout Standard
+Let's first consider the case of an aggressive repairer.
+ Every interval, this repairer checks the file for share losses and restores
+ them.
+ Thus, at the beginning of each interval, the file always has
+\begin_inset Formula $N$
+\end_inset
+
+ shares, distributed on servers with various individual and group failure
+ probalities, which will survive or fail per the output of random variable
+
+\begin_inset Formula $K$
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Standard
+For any interval, then, the probability that the file will survive is
+\begin_inset Formula $f\left(k\right)=Pr[K\geq k]$
+\end_inset
+
+.
+ Since each interval success or failure is independent, and assuming the
+ share reliabilities remain constant over time,
+\begin_inset Formula \begin{equation}
+R\left(t\right)=f(k)^{t}\end{equation}
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+This simple survival function makes it simple to select parameters
+\begin_inset Formula $N$
+\end_inset
+
+ and
+\begin_inset Formula $K$
+\end_inset
+
+ such that
+\begin_inset Formula $R(t)\geq r$
+\end_inset
+
+, where
+\begin_inset Formula $r$
+\end_inset
+
+ is a user-specified parameter indicating the desired probability of survival
+ to time
+\begin_inset Formula $t$
+\end_inset
+
+.
+ Specifically, we can solve for
+\begin_inset Formula $f\left(k\right)$
+\end_inset
+
+ in
+\begin_inset Formula $r\leq f\left(k\right)^{t}$
+\end_inset
+
+, giving:
+\begin_inset Formula \begin{equation}
+f\left(k\right)\geq\sqrt[t]{r}\end{equation}
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+So, given a PMF
+\begin_inset Formula $f\left(k\right)$
+\end_inset
+
+, to assure the survival of a file to time
+\begin_inset Formula $t$
+\end_inset
+
+ with probability at least
+\begin_inset Formula $r$
+\end_inset
+
+, choose
+\begin_inset Formula $k:f\left(k\right)\geq\sqrt[t]{r}$
+\end_inset
+
+.
+ For example, if
+\begin_inset Formula $A$
+\end_inset
+
+ is one month, and
+\begin_inset Formula $r=1-\nicefrac{1}{1000000}$
+\end_inset
+
+ and
+\begin_inset Formula $t=120$
+\end_inset
+
+, or 10 years, we calculate
+\begin_inset Formula $f\left(k\right)\geq\sqrt[120]{.999999}\cong0.999999992$
+\end_inset
+
+.
+ Per the PMF of table
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "tab:Example-PMF"
+
+\end_inset
+
+, this means
+\begin_inset Formula $k=2$
+\end_inset
+
+, achieves the goal, at the cose of a six-fold expansion in stored file
+ size.
+ If the lesser goal of no more than
+\begin_inset Formula $\nicefrac{1}{1000}$
+\end_inset
+
+ probability of loss is taken, then since
+\begin_inset Formula $\sqrt[120]{.9999}=.999992$
+\end_inset
+
+,
+\begin_inset Formula $k=5$
+\end_inset
+
+ achieves the goal with an expansion factor of
+\begin_inset Formula $2.4$
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Subsection
+Repair Cost
+\end_layout
+
+\begin_layout Standard
+The simplicity and predictability of aggressive repair is attractive, but
+ there is a downside: Repairs cost processing power and bandwidth.
+ The processing power is proportional to the size of the file, since the
+ whole file must be reconstructed and then re-processed using the Reed-Solomon
+ algorithm, while the bandwidth cost is proportional to the number of missing
+ shares that must be replaced,
+\begin_inset Formula $N-K$
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Standard
+Let
+\begin_inset Formula $c\left(s,d,k\right)$
+\end_inset
+
+ be a cost function that combines the processing cost of regenerating a
+ file of size
+\begin_inset Formula $s$
+\end_inset
+
+ and the bandwidth cost of downloading a file of size
+\begin_inset Formula $s$
+\end_inset
+
+ and uploading
+\begin_inset Formula $d$
+\end_inset
+
+ shares each of size
+\begin_inset Formula $\nicefrac{s}{k}$
+\end_inset
+
+.
+ Also, let
+\begin_inset Formula $D$
+\end_inset
+
+ denote the random variable
+\begin_inset Formula $N-K$
+\end_inset
+
+, which is the number of shares that must be redistributed to bring the
+ file share set back up to
+\begin_inset Formula $N$
+\end_inset
+
+ after degrading during an interval.
+ The probability mass function of
+\begin_inset Formula $D$
+\end_inset
+
+ is
+\begin_inset Formula \[
+Pr[D=d]=f(d)=\begin{cases}
+Pr\left[K=N\right]+Pr[K<k] & d=0\\
+Pr\left[K=N-d\right] & 0<d\leq N-k\\
+0 & N-k<d\leq N\end{cases}\]
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+The expected cost of repairs in a given interval, then, is simply
+\begin_inset Formula $c\left(s,E\left[D\right],k\right)$
+\end_inset
+
+ where E is the expected value function -- in this case:
+\begin_inset Formula \begin{align*}
+E[D] & =\sum_{d=0}^{N}d\cdot Pr\left[D=d\right]\\
+ & =0\cdot Pr\left[D=0\right]+\sum_{d=1}^{N-k}\left\{ d\cdot Pr\left[K=N-d\right]\right\} +\sum_{d=N-k+1}^{N}\left\{ d\cdot0\right\} \\
+ & =\sum_{d=1}^{N-k}d\cdot Pr\left[K=N-d\right]\end{align*}
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+Since each interval starts with a full complement of shares, the expected
+ repair cost for each interval is the same, and the cost for file that survives
+ for
+\begin_inset Formula $t$
+\end_inset
+
+ intervals is
+\begin_inset Formula $t\cdot c\left(s,E\left[D\right]\right)$
+\end_inset
+
+.
+ To calculate the lifetime repair cost, we just take the limit over all
+ intervals as
+\begin_inset Formula $t\rightarrow\infty$
+\end_inset
+
+, discounting each cost by the probability that the file has already failed.
+ So, the lifetime expected repair cost is
+\begin_inset Formula \begin{align*}
+\sum_{t=1}^{\infty}R\left(t-1\right)c\left(s,E\left[D\right],k\right) & =c\left(s,E\left[D\right],k\right)\sum_{t=1}^{\infty}R\left(t-1\right)\\
+ & =c\left(s,E\left[D\right],k\right)\sum_{t=1}^{\infty}f\left(k\right)^{t-1}\\
+ & =c\left(s,E\left[D\right],k\right)\cdot\frac{1}{1-f\left(k\right)}\\
+ & =\frac{c\left(s,E\left[D\right],k\right)}{1-f\left(k\right)}\end{align*}
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+It may also be useful to discount future cost, since CPU and bandwidth are
+ both going to get cheaper over time.
+ To accomodate this, we throw in an addition per-period discount rate
+\begin_inset Formula $r$
+\end_inset
+
+.
+ In accordance with common discount rate usage, the discount multiplier
+ at time
+\begin_inset Formula $t$
+\end_inset
+
+ is
+\begin_inset Formula $\left(1-r\right)^{t}$
+\end_inset
+
+.
+ This gives:
+\begin_inset Formula \begin{align*}
+\sum_{t=1}^{\infty}\left(1-r\right){}^{t}R\left(t-1\right)c\left(s,E\left[D\right],k\right) & =c\left(s,E\left[D\right],k\right)\sum_{t=1}^{\infty}\left(1-r\right)^{t}f\left(k\right)^{t-1}\\
+ & =c\left(s,E\left[D\right],k\right)\sum_{t=1}^{\infty}\left(1-r\right)^{t}f\left(k\right)^{t-1}\\
+ & =c\left(s,E\left[D\right],k\right)\left(1-r\right)\sum_{t=1}^{\infty}\left(1-r\right)^{t-1}f\left(k\right)^{t-1}\\
+ & =\frac{c\left(s,E\left[D\right],k\right)\left(1-r\right)}{1-\left(1-r\right)f\left(k\right)}\end{align*}
+
+\end_inset
+
+If
+\begin_inset Formula $r=0$
+\end_inset
+
+ this collapses to the previous result, as one would expect.
+\end_layout
+
+\begin_layout Section
+Time-Sensitive Retrieval
+\end_layout
+
+\begin_layout Standard
+The above work has almost entirely ignored the distinction between availability
+ and reliability.
+ In reality, temporary and permanent failures need to be modeled separately,
+ and
+\end_layout
+
+\end_body
+\end_document
projects / tahoe-lafs / tahoe-lafs.git / commitdiff