Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- $ crawl -v --max_depth=2 --whitelist="^.*sourceforge\.net.*$" http://sourceforge.net/projects/xine/files/xine-lib/
- Crawling http://sourceforge.net/projects/xine/files/xine-lib/
- Traceback (most recent call last):
- File "/usr/bin/crawl", line 9, in <module>
- load_entry_point('spyda==0.0.3.dev', 'console_scripts', 'crawl')()
- File "/usr/lib/python2.7/site-packages/spyda/crawler.py", line 277, in main
- result = crawl(url, **opts.__dict__)
- File "/usr/lib/python2.7/site-packages/spyda/crawler.py", line 114, in crawl
- whitelist.extend(allowed_urls)
- TypeError: 'NoneType' object is not iterable
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement