Advertisement
Guest User

Scrapy shell error

a guest
Nov 27th, 2016
442
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Python 2.34 KB | None | 0 0
  1. 2016-11-27 13:35:47 [scrapy] INFO: Scrapy 1.2.1 started (bot: rekrute)
  2. 2016-11-27 13:35:47 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'rekrute.spiders', 'ROBOTSTXT_OBEY': 'False', 'DUPEFILTER_CLASS': 'scrapy.dupefilters.BaseDupeFilter', 'SPIDER_MODULES': ['rekrute.spiders'], 'BOT_NAME': 'rekrute', 'LOGSTATS_INTERVAL': 0}
  3. 2016-11-27 13:35:47 [scrapy] INFO: Enabled extensions:
  4. ['scrapy.extensions.telnet.TelnetConsole',
  5.  'scrapy.extensions.corestats.CoreStats']
  6. Traceback (most recent call last):
  7.   File "/usr/local/bin/scrapy", line 11, in <module>
  8.     sys.exit(execute())
  9.   File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 142, in execute
  10.     _run_print_help(parser, _run_command, cmd, args, opts)
  11.   File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 88, in _run_print_help
  12.     func(*a, **kw)
  13.   File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 149, in _run_command
  14.     cmd.run(args, opts)
  15.   File "/usr/local/lib/python2.7/dist-packages/scrapy/commands/shell.py", line 65, in run
  16.     crawler.engine = crawler._create_engine()
  17.   File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 97, in _create_engine
  18.     return ExecutionEngine(self, lambda _: self.stop())
  19.   File "/usr/local/lib/python2.7/dist-packages/scrapy/core/engine.py", line 68, in __init__
  20.     self.downloader = downloader_cls(crawler)
  21.   File "/usr/local/lib/python2.7/dist-packages/scrapy/core/downloader/__init__.py", line 88, in __init__
  22.     self.middleware = DownloaderMiddlewareManager.from_crawler(crawler)
  23.   File "/usr/local/lib/python2.7/dist-packages/scrapy/middleware.py", line 58, in from_crawler
  24.     return cls.from_settings(crawler.settings, crawler)
  25.   File "/usr/local/lib/python2.7/dist-packages/scrapy/middleware.py", line 36, in from_settings
  26.     mw = mwcls.from_crawler(crawler)
  27.   File "/usr/local/lib/python2.7/dist-packages/scrapy/downloadermiddlewares/robotstxt.py", line 33, in from_crawler
  28.     return cls(crawler)
  29.   File "/usr/local/lib/python2.7/dist-packages/scrapy/downloadermiddlewares/robotstxt.py", line 24, in __init__
  30.     if not crawler.settings.getbool('ROBOTSTXT_OBEY'):
  31.   File "/usr/local/lib/python2.7/dist-packages/scrapy/settings/__init__.py", line 129, in getbool
  32.     return bool(int(self.get(name, default)))
  33. ValueError: invalid literal for int() with base 10: 'False'
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement