Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Erebos:scraper andreas$ scrapy shell -s USER_AGENT="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" http://www.firmenabc.at/result.aspx?what=&where=Graz
- [1] 26354
- Erebos:scraper andreas$ 2017-02-26 14:47:34 [scrapy.utils.log] INFO: Scrapy 1.3.2 started (bot: scrapybot)
- 2017-02-26 14:47:34 [scrapy.utils.log] INFO: Overridden settings: {'DUPEFILTER_CLASS': 'scrapy.dupefilters.BaseDupeFilter', 'LOGSTATS_INTERVAL': 0, 'USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36'}
- 2017-02-26 14:47:34 [scrapy.middleware] INFO: Enabled extensions:
- ['scrapy.extensions.corestats.CoreStats',
- 'scrapy.extensions.telnet.TelnetConsole']
- 2017-02-26 14:47:34 [scrapy.middleware] INFO: Enabled downloader middlewares:
- ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
- 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
- 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
- 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
- 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
- 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
- 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
- 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
- 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
- 'scrapy.downloadermiddlewares.stats.DownloaderStats']
- 2017-02-26 14:47:34 [scrapy.middleware] INFO: Enabled spider middlewares:
- ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
- 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
- 'scrapy.spidermiddlewares.referer.RefererMiddleware',
- 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
- 'scrapy.spidermiddlewares.depth.DepthMiddleware']
- 2017-02-26 14:47:34 [scrapy.middleware] INFO: Enabled item pipelines:
- []
- 2017-02-26 14:47:34 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023
- 2017-02-26 14:47:34 [scrapy.core.engine] INFO: Spider opened
- 2017-02-26 14:47:34 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET http://www.firmenabc.at/result.aspx?what=> (failed 1 times): [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion.>]
- 2017-02-26 14:47:35 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET http://www.firmenabc.at/result.aspx?what=> (failed 2 times): [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion.>]
- 2017-02-26 14:47:35 [scrapy.downloadermiddlewares.retry] DEBUG: Gave up retrying <GET http://www.firmenabc.at/result.aspx?what=> (failed 3 times): [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion.>]
- Traceback (most recent call last):
- File "/usr/local/bin/scrapy", line 11, in <module>
- sys.exit(execute())
- File "/usr/local/lib/python3.6/site-packages/scrapy/cmdline.py", line 142, in execute
- _run_print_help(parser, _run_command, cmd, args, opts)
- File "/usr/local/lib/python3.6/site-packages/scrapy/cmdline.py", line 88, in _run_print_help
- func(*a, **kw)
- File "/usr/local/lib/python3.6/site-packages/scrapy/cmdline.py", line 149, in _run_command
- cmd.run(args, opts)
- File "/usr/local/lib/python3.6/site-packages/scrapy/commands/shell.py", line 73, in run
- shell.start(url=url, redirect=not opts.no_redirect)
- File "/usr/local/lib/python3.6/site-packages/scrapy/shell.py", line 48, in start
- self.fetch(url, spider, redirect=redirect)
- File "/usr/local/lib/python3.6/site-packages/scrapy/shell.py", line 115, in fetch
- reactor, self._schedule, request, spider)
- File "/usr/local/lib/python3.6/site-packages/twisted/internet/threads.py", line 122, in blockingCallFromThread
- result.raiseException()
- File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 372, in raiseException
- raise self.value.with_traceback(self.tb)
- twisted.web._newclient.ResponseNeverReceived: [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion.>]
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement