Advertisement
Guest User

Untitled

a guest
May 4th, 2015
417
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 2.91 KB | None | 0 0
  1. C:\allenheath>scrapy crawl products -o items.csv
  2. 2015-05-04 11:12:46-0700 [scrapy] INFO: Scrapy 0.24.6 started (bot: allenheath)
  3. 2015-05-04 11:12:46-0700 [scrapy] INFO: Optional features available: ssl, http11
  4.  
  5. 2015-05-04 11:12:46-0700 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE'
  6. : 'allenheath.spiders', 'FEED_FORMAT': 'csv', 'SPIDER_MODULES': ['allenheath.spi
  7. ders'], 'FEED_URI': 'items.csv', 'BOT_NAME': 'allenheath'}
  8. 2015-05-04 11:12:46-0700 [scrapy] INFO: Enabled extensions: FeedExporter, LogSta
  9. ts, TelnetConsole, CloseSpider, WebService, CoreStats, SpiderState
  10. 2015-05-04 11:12:46-0700 [scrapy] INFO: Enabled downloader middlewares: HttpAuth
  11. Middleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, Def
  12. aultHeadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, Redirec
  13. tMiddleware, CookiesMiddleware, ChunkedTransferMiddleware, DownloaderStats
  14. 2015-05-04 11:12:46-0700 [scrapy] INFO: Enabled spider middlewares: HttpErrorMid
  15. dleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddlew
  16. are
  17. 2015-05-04 11:12:46-0700 [scrapy] INFO: Enabled item pipelines: itemcodeCleaner,
  18. ImagesPipeline
  19. 2015-05-04 11:12:46-0700 [products] INFO: Spider opened
  20. 2015-05-04 11:12:46-0700 [products] INFO: Crawled 0 pages (at 0 pages/min), scra
  21. ped 0 items (at 0 items/min)
  22. 2015-05-04 11:12:46-0700 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6
  23. 023
  24. 2015-05-04 11:12:46-0700 [scrapy] DEBUG: Web service listening on 127.0.0.1:6080
  25.  
  26. 2015-05-04 11:12:46-0700 [products] DEBUG: Crawled (200) <GET http://www.allen-h
  27. eath.com/products/> (referer: None)
  28. 2015-05-04 11:12:46-0700 [products] DEBUG: Scraped from <200 http://www.allen-he
  29. ath.com/products/>
  30. {'desc': [],
  31. 'image_urls': [],
  32. 'imageorig': [],
  33. 'images': [],
  34. 'itemcode': [],
  35. 'model': [],
  36. 'series': [],
  37. 'shortdesc': []}
  38. 2015-05-04 11:12:46-0700 [products] INFO: Closing spider (finished)
  39. 2015-05-04 11:12:46-0700 [products] INFO: Stored csv feed (1 items) in: items.cs
  40. v
  41. 2015-05-04 11:12:46-0700 [products] INFO: Dumping Scrapy stats:
  42. {'downloader/request_bytes': 227,
  43. 'downloader/request_count': 1,
  44. 'downloader/request_method_count/GET': 1,
  45. 'downloader/response_bytes': 8164,
  46. 'downloader/response_count': 1,
  47. 'downloader/response_status_count/200': 1,
  48. 'finish_reason': 'finished',
  49. 'finish_time': datetime.datetime(2015, 5, 4, 18, 12, 46, 926000),
  50. 'item_scraped_count': 1,
  51. 'log_count/DEBUG': 4,
  52. 'log_count/INFO': 8,
  53. 'response_received_count': 1,
  54. 'scheduler/dequeued': 1,
  55. 'scheduler/dequeued/memory': 1,
  56. 'scheduler/enqueued': 1,
  57. 'scheduler/enqueued/memory': 1,
  58. 'start_time': datetime.datetime(2015, 5, 4, 18, 12, 46, 454000)}
  59. 2015-05-04 11:12:46-0700 [products] INFO: Spider closed (finished)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement