Advertisement
Guest User

Scrapy

a guest
Nov 2nd, 2016
369
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 1.92 KB | None | 0 0
  1. >>> print HEADER
  2. {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36', 'Host': 'book.zi5.me', 'Accept-Language': 'zh-CN,zh;q=0.
  3. 8,en-US;q=0.6,en;q=0.4,ja;q=0.2,zh-TW;q=0.2', 'Accept-Encoding': 'gzip, deflate, sdch', 'Referer': 'http://book.zi5.me/', 'Cache-Control': 'max-age=0', 'Cookie': 'pgv_pvi=5922583552; PHPSES
  4. SID=b831ea263b8d8c0ad3d0c04c82b325d4; pgv_si=s5073116160', 'Upgrade-Insecure-Requests': '1', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Proxy-C
  5. onnection': 'keep-alive'}
  6. >>> req=scrapy.Request(url,headers=HEADER)
  7. >>> fetch(req)
  8. 2016-11-02 15:52:36 [scrapy] INFO: Spider opened
  9. 2016-11-02 15:52:37 [scrapy] DEBUG: Retrying <GET http://book.zi5.me> (failed 1 times): [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side
  10. was lost in a non-clean fashion.>]
  11. 2016-11-02 15:52:37 [scrapy] DEBUG: Retrying <GET http://book.zi5.me> (failed 2 times): [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side
  12. was lost in a non-clean fashion.>]
  13. 2016-11-02 15:52:37 [scrapy] DEBUG: Gave up retrying <GET http://book.zi5.me> (failed 3 times): [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the oth
  14. er side was lost in a non-clean fashion.>]
  15. Traceback (most recent call last):
  16. File "<console>", line 1, in <module>
  17. File "c:\python27\lib\site-packages\scrapy\shell.py", line 112, in fetch
  18. reactor, self._schedule, request, spider)
  19. File "c:\python27\lib\site-packages\twisted\internet\threads.py", line 122, in blockingCallFromThread
  20. result.raiseException()
  21. File "<string>", line 2, in raiseException
  22. ResponseNeverReceived: [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion.>]
  23. >>>
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement