- $ time python fastq-unique.py < /usr/local/src/methylcode/data/dme/en.wt.1.fastq > unique.fastq
- 36754782 records in file
- checking 2525155 potential duplicates in a python set
- 4699 false-positive duplicates in the bloom filter
- real 11m31.077s
- user 11m1.937s
- sys 0m23.093s