$ time python crawl.py 100 1

real    0m26.989s
user    0m1.419s
sys 0m0.375s

$ time python crawl.py 100 4

real    0m7.232s
user    0m1.540s
sys 0m0.359s

$ time python crawl.py 100 8

real    0m4.783s
user    0m1.576s
sys 0m0.434s