Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- C:\allenheath>scrapy crawl products -o items.csv
- 2015-05-04 11:12:46-0700 [scrapy] INFO: Scrapy 0.24.6 started (bot: allenheath)
- 2015-05-04 11:12:46-0700 [scrapy] INFO: Optional features available: ssl, http11
- 2015-05-04 11:12:46-0700 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE'
- : 'allenheath.spiders', 'FEED_FORMAT': 'csv', 'SPIDER_MODULES': ['allenheath.spi
- ders'], 'FEED_URI': 'items.csv', 'BOT_NAME': 'allenheath'}
- 2015-05-04 11:12:46-0700 [scrapy] INFO: Enabled extensions: FeedExporter, LogSta
- ts, TelnetConsole, CloseSpider, WebService, CoreStats, SpiderState
- 2015-05-04 11:12:46-0700 [scrapy] INFO: Enabled downloader middlewares: HttpAuth
- Middleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, Def
- aultHeadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, Redirec
- tMiddleware, CookiesMiddleware, ChunkedTransferMiddleware, DownloaderStats
- 2015-05-04 11:12:46-0700 [scrapy] INFO: Enabled spider middlewares: HttpErrorMid
- dleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddlew
- are
- 2015-05-04 11:12:46-0700 [scrapy] INFO: Enabled item pipelines: itemcodeCleaner,
- ImagesPipeline
- 2015-05-04 11:12:46-0700 [products] INFO: Spider opened
- 2015-05-04 11:12:46-0700 [products] INFO: Crawled 0 pages (at 0 pages/min), scra
- ped 0 items (at 0 items/min)
- 2015-05-04 11:12:46-0700 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6
- 023
- 2015-05-04 11:12:46-0700 [scrapy] DEBUG: Web service listening on 127.0.0.1:6080
- 2015-05-04 11:12:46-0700 [products] DEBUG: Crawled (200) <GET http://www.allen-h
- eath.com/products/> (referer: None)
- 2015-05-04 11:12:46-0700 [products] DEBUG: Scraped from <200 http://www.allen-he
- ath.com/products/>
- {'desc': [],
- 'image_urls': [],
- 'imageorig': [],
- 'images': [],
- 'itemcode': [],
- 'model': [],
- 'series': [],
- 'shortdesc': []}
- 2015-05-04 11:12:46-0700 [products] INFO: Closing spider (finished)
- 2015-05-04 11:12:46-0700 [products] INFO: Stored csv feed (1 items) in: items.cs
- v
- 2015-05-04 11:12:46-0700 [products] INFO: Dumping Scrapy stats:
- {'downloader/request_bytes': 227,
- 'downloader/request_count': 1,
- 'downloader/request_method_count/GET': 1,
- 'downloader/response_bytes': 8164,
- 'downloader/response_count': 1,
- 'downloader/response_status_count/200': 1,
- 'finish_reason': 'finished',
- 'finish_time': datetime.datetime(2015, 5, 4, 18, 12, 46, 926000),
- 'item_scraped_count': 1,
- 'log_count/DEBUG': 4,
- 'log_count/INFO': 8,
- 'response_received_count': 1,
- 'scheduler/dequeued': 1,
- 'scheduler/dequeued/memory': 1,
- 'scheduler/enqueued': 1,
- 'scheduler/enqueued/memory': 1,
- 'start_time': datetime.datetime(2015, 5, 4, 18, 12, 46, 454000)}
- 2015-05-04 11:12:46-0700 [products] INFO: Spider closed (finished)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement