From: Zooko O'Whielacronx Date: Sun, 15 Apr 2007 00:47:48 +0000 (-0700) Subject: pyfec: add bragging about how fec is way faster than "par2" X-Git-Url: https://git.rkrishnan.org/pf/content/simplejson/frontends/index.php?a=commitdiff_plain;h=502c979307183460a721bf708bcbe891c7cdaab9;p=tahoe-lafs%2Ftahoe-lafs.git pyfec: add bragging about how fec is way faster than "par2" --- diff --git a/pyfec/README.txt b/pyfec/README.txt index b8f70e1d..bb96d7c5 100644 --- a/pyfec/README.txt +++ b/pyfec/README.txt @@ -78,12 +78,12 @@ input to the encoding step. The bin/ directory contains two Unix-style, command-line tools "fec" and "unfec". Execute "fec --help" or "unfec --help" for usage instructions. -Note: a Unix-style tool like "fec" does only one thing -- in this case erasure -coding -- and leaves other tasks to other tools. Other Unix-style tools that -go well with "fec" include "GNU tar" for packaging up multiple files and -directories into one bundle, "rzip" for compression, and "GNU Privacy Guard" -for encryption. It is best to do things in that order: first package, then -compress, then encrypt, then erasure code. +Note: a Unix-style tool like "fec" does only one thing -- in this case +erasure coding -- and leaves other tasks to other tools. Other Unix-style +tools that go well with "fec" include "GNU tar" for packaging up multiple +files and directories into one bundle, "rzip" for compression, and "GNU +Privacy Guard" for encryption. It is important to do things in order: first +package, then compress, then encrypt, then erasure code. * API @@ -157,18 +157,30 @@ objects (e.g. Python strings) to hold the data that you pass to pyfec. * Utilities -The filefec.py module which has a utility function for efficiently reading a -file and encoding it piece by piece. +The filefec.py module has a utility function for efficiently reading a file +and encoding it piece by piece. This module is used by the "fec" and "unfec" +command-line tools from the bin/ directory. * Dependencies -A C compiler is required. For the Python API, we have tested it with Python -v2.4 and v2.5. +A C compiler is required. To use the Python API or the command-line tools a +Python interpreter is also required. We have tested it with Python v2.4 and +v2.5. * Performance Measurements +On my Athlon 64 2.4 GHz workstation (running Linux), the "fec" command-line +tool encoded a 160 MB file with m=100, k=94 (about 6% redundancy) in 3.9 +seconds, where the "par2" tool encoded the file with about 6% redundancy in +27 seconds. "fec" encoded the same file with m=12, k=6 (100% redundancy) in +4.1 seconds, where par2 encoded it with about 100% redundancy in 7 minutes +and 56 seconds. + +The underlying C library in benchmark mode encoded from a file at about +4.9 million bytes per second and decoded at about 5.8 million bytes per second. + On Peter's fancy Intel Mac laptop (2.16 GHz Core Duo), it encoded from a file at about 6.2 million bytes per second. @@ -178,10 +190,6 @@ at about 6.8 million bytes per second. On my old PowerPC G4 867 MHz Mac laptop, it encoded from a file at about 1.3 million bytes per second. -On my Athlon 64 2.4 GHz workstation (running Linux), it encoded from a file at -about 4.9 million bytes per second and decoded at about 5.8 million bytes per -second. - * Acknowledgements