API tools faq
paste
Guest User

Untitled

a guest
Jul 14th, 2020
106
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Bash 15.39 KB | None | 0 0
raw download clone embed print report
  1. 2020-07-14 22:24:06 [scrapy.utils.log] INFO: Scrapy 2.2.0 started (bot: edadeal)
  2. 2020-07-14 22:24:06 [scrapy.utils.log] INFO: Versions: lxml 4.5.2.0, libxml2 2.9.10, cssselect 1.1.0, parsel 1.6.0, w3lib 1.22.0, Twisted 20.3.0, Python 3.6.10 (default, Dec 19 2019, 23:04:32) - [GCC 5.4.0 20160609], pyOpenSSL 19.1.0 (OpenSSL 1.1.1g  21 Apr 2020), cryptography 2.9.2, Platform Linux-4.4.0-185-generic-x86_64-with-Ubuntu-16.04-xenial
  3. 2020-07-14 22:24:06 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.epollreactor.EPollReactor
  4. 2020-07-14 22:24:06 [scrapy.crawler] INFO: Overridden settings:
  5. {'BOT_NAME': 'edadeal',
  6.  'DUPEFILTER_CLASS': 'scrapy_splash.SplashAwareDupeFilter',
  7.  'NEWSPIDER_MODULE': 'edadeal.spiders',
  8.  'SPIDER_MODULES': ['edadeal.spiders']}
  9. 2020-07-14 22:24:06 [scrapy.extensions.telnet] INFO: Telnet Password: f4ca2ba9b82d7513
  10. 2020-07-14 22:24:06 [scrapy.middleware] INFO: Enabled extensions:
  11. ['scrapy.extensions.corestats.CoreStats',
  12.  'scrapy.extensions.telnet.TelnetConsole',
  13.  'scrapy.extensions.memusage.MemoryUsage',
  14.  'scrapy.extensions.logstats.LogStats']
  15. 2020-07-14 22:24:06 [scrapy.middleware] INFO: Enabled downloader middlewares:
  16. ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
  17.  'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
  18.  'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
  19.  'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
  20.  'scrapy.downloadermiddlewares.retry.RetryMiddleware',
  21.  'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
  22.  'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
  23.  'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
  24.  'scrapy_splash.SplashCookiesMiddleware',
  25.  'scrapy_splash.SplashMiddleware',
  26.  'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
  27.  'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
  28.  'scrapy.downloadermiddlewares.stats.DownloaderStats']
  29. 2020-07-14 22:24:06 [scrapy.middleware] INFO: Enabled spider middlewares:
  30. ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
  31.  'scrapy_splash.SplashDeduplicateArgsMiddleware',
  32.  'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
  33.  'scrapy.spidermiddlewares.referer.RefererMiddleware',
  34.  'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
  35.  'scrapy.spidermiddlewares.depth.DepthMiddleware']
  36. 2020-07-14 22:24:06 [scrapy.middleware] INFO: Enabled item pipelines:
  37. []
  38. 2020-07-14 22:24:06 [scrapy.core.engine] INFO: Spider opened
  39. 2020-07-14 22:24:06 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  40. 2020-07-14 22:24:06 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  41. 2020-07-14 22:24:06 [py.warnings] WARNING: /home/alex/root_folder/scrapy/2/venv/lib/python3.6/site-packages/scrapy_splash/request.py:41: ScrapyDeprecationWarning: Call to deprecated function to_native_str. Use to_unicode instead.
  42.   url = to_native_str(url)
  43.  
  44. 2020-07-14 22:24:26 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://edadeal.ru/moskva/retailers/5ka via http://localhost:8050/render.html> (referer: None)
  45. 2020-07-14 22:24:26 [scrapy.core.scraper] DEBUG: Scraped from <200 https://edadeal.ru/moskva/retailers/5ka>
  46. {'data': '<!DOCTYPE html><html><head> <meta charset="utf-8"> <title>Каталог магазина "Пятёрочка" - Акции с 9 июля по 6 августа</title> <meta name="description" content="Скидки и акции в магазине &quot;Пятёрочка&quot; на сегодня в Москве. Актуальный каталог скидок c 9&nbsp;июля по 6&nbsp;августа на сайте и в мобильном приложении. Узнайте, где покупать дешевле и экономьте до 50% уже сегодня c Едадил."> <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes"> <meta name="fragment" content="!">  <meta name="google-site-verification" content="5isjz1gObpPn7PREXwtlGeFu6h7rNcYIEvFU6hLeeNI"> <script src="https://yastatic.net/pcode-native/loaders/loader.js"></script><script src="https://an.yandex.ru/system/context.js"></script><script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==" async="" src="https://mc.yandex.ru/metrika/watch.js"></script><script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==" async="" src="https://www.tns-counter.ru/tcounter.js"></script><script type="application/ld+json" nonce="heVv4uj5YaOLYQDG2Q1k9A==">{ "@context": "http://schema.org", "@type": "WebSite", "name": "Едадил", "alternateName": "Edadeal", "url": "https://edadeal.ru" }</script> <meta name="theme-color" content="#27ae61"> <link rel="icon" type="image/png" sizes="32x32" href="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/static/img/favicon-32x32.png"><link rel="icon" type="image/png" sizes="96x96" href="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/static/img/favicon-96x96.png"><link rel="icon" type="image/png" sizes="16x16" href="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/static/img/favicon-16x16.png"> <link rel="icon" href="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/static/img/favicon.ico">  <meta property="og:url" content="https://edadeal.ru/moskva/retailers/5ka"><meta property="og:watson" content="https://leonardo.edadeal.io/dyn/re/retailers/images/icons/sq/85cd6908f8a34aefbe66dd1948cc3d39.png?res=l"><meta property="og:type" content="website"><meta property="og:site_name" content="Едадил"><meta property="og:title" content="Каталог магазина &quot;Пятёрочка&quot; - Акции с 9 июля по 6 августа"><meta property="og:image" content="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/static/img/share_pic.png"><meta property="og:image:width" content="1200"><meta property="og:image:height" content="628"> <meta property="twitter:card" content="summary_large_image"><meta property="twitter:site" content="@edadeal_tw"><meta property="twitter:creator" content="@edadeal_tw"><meta property="twitter:title" content="Каталог магазина &quot;Пятёрочка&quot; - Акции с 9 июля по 6 августа"><meta property="twitter:description" content="Скидки и акции в магазине &quot;Пятёрочка&quot; на сегодня в Москве. Актуальный каталог скидок c 9&nbsp;июля по 6&nbsp;августа на сайте и в мобильном приложении. Узнайте, где покупать дешевле и экономьте до 50% уже сегодня c Едадил."><meta property="twitter:image" content="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/static/img/share_pic.png"> <link rel="canonical" href="https://edadeal.ru/moskva/retailers/5ka"> <script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==">!function(i,t){if(i.Ya=i.Ya||{},Ya.Rum)throw new Error("Rum: interface is already defined");var n=i.performance,e=n&&n.timing&&n.timing.navigationStart||Ya.startPageLoad||+new Date,s=i.requestAnimationFrame;Ya.Rum={enabled:!!n,vsStart:document.visibilityState,vsChanged:!1,_defTimes:[],_defRes:[],_deltaMarks:{},_settings:{},_vars:{},init:function(i,t){this._settings=i,this._vars=t},getTime:n&&n.now?function(){return n.now()}:Date.now?function(){return Date.now()-e}:function(){return new Date-e},time:function(i){this._deltaMarks[i]=[this.getTime()]},timeEnd:function(i,t){var n=this._deltaMarks[i];n&&0!==n.length&&n.push(this.getTime(),t)},sendTimeMark:function(i,t,n,e){void 0===t&&(t=this.getTime()),this._defTimes.push([i,t,e]),this.mark(i,t)},sendResTiming:function(i,t){this._defRes.push([i,t])},sendRaf:function(i){var t=this.getSetting("forceFirstPaintTimeSending");if(s&&(t||!this.isVisibilityChanged())){var n=this,e="2616."+i;s(function(){!t&&n.isVisibilityChanged()||(n.sendTimeMark(e+".205"),s(function(){!t&&n.isVisibilityChanged()||n.sendTimeMark(e+".1928")}))})}},isVisibilityChanged:function(){return this.vsStart&&("visible"!==this.vsStart||this.vsChanged)},mark:n&&n.mark?function(i,t){n.mark(i+(t?": "+t:""))}:function(){},getSetting:function(i){var t=this._settings[i];return null===t?null:t||""}},document.addEventListener&&document.addEventListener("visibilitychange",function i(){Ya.Rum.vsChanged=!0,document.removeEventListener("visibilitychange",i)})}(window);</script><script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==">!function(){"use strict";if(window.PerformanceLongTaskTiming){var e=Ya.Rum._tti={events:[],observer:new PerformanceObserver(function(n){e.events=e.events.concat(n.getEntries()),e.events.length>100&&e.events.shift()})};e.observer.observe({entryTypes:["longtask"]})}}();</script> <script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==" src="https://yastatic.net/pcode/adfox/loader.js" crossorigin="anonymous"></script><script type="text/javascript" async="" src="https://yastatic.net/pcode-bundles/0.1770/banners.js" charset="utf-8" crossorigin="anonymous"></script><link rel="preconnect" href="https://an.yandex.ru"><link rel="preconnect" href="https://ads.adfox.ru"><script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==">var GLOBAL_NONCE = "heVv4uj5YaOLYQDG2Q1k9A==";</script> <script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==">!function(t,e,n){(t[n]=t[n]||[]).push(function(){try{tnsCounter=new TNS.TnsCounter({account:"yandex_ru",tmsec:"edadeal_total"})}catch(t){}});var c=e.createElement("script");c.type="text/javascript","undefined"!=typeof GLOBAL_NONCE&&c.setAttribute("nonce",GLOBAL_NONCE),c.async=!0,c.src=("https:"==e.location.protocol?"https:":"http:")+"//www.tns-counter.ru/tcounter.js";var o=e.getElementsByTagName("script")[0];o.parentNode.insertBefore(c,o)}(window,document,"tnscounter_callback");</script> <script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==">!function(e,t,a){(t[a]=t[a]||[]).push(function(){try{t.yaCounter=new Ya.Metrika({id:34675050,clickmap:!0,trackLinks:!0,accurateTrackBounce:!0,webvisor:!0,trackHash:!0})}catch(e){}});var n=e.getElementsByTagName("script")[0],c=e.createElement("script"),r=function(){n.parentNode.insertBefore(c,n)};c.type="text/javascript","undefined"!=typeof GLOBAL_NONCE&&c.setAttribute("nonce",GLOBAL_NONCE),c.async=!0,c.src="https://mc.yandex.ru/metrika/watch.js","[object Opera]"==t.opera?e.addEventListener("DOMContentLoaded",r,!1):r()}(document,window,"yandex_metrika_callbacks");</script> <!--Global site tag (gtag.js) - AdWords: 942218954 --><script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==">function gtag(){dataLayer.push(arguments)}window.dataLayer=window.dataLayer||[],gtag("js",new Date),gtag("config","AW-942218954");</script> <script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==">var $M=document.documentElement.clientWidth<960,BUILD_ID="ae0c5b9d70691",GLOBAL_NONCE="heVv4uj5YaOLYQDG2Q1k9A==",MANIFEST={"p-retailer":["p-offers"],$:["libs.js","base"]},ENVIRONMENT="production",VERSION="4.1.48",BLENDER=!1,CLIENT_BASE_URL="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/client",STATIC_PATH="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/static",RTB_BLOCKS=$M?{offer:"R-I-338786-6",offers:"R-I-338786-1",footer:"R-I-338786-2"}:{cashbacks_cards:"R-I-338751-8",offers_horizontal:"R-I-338751-7",pagination:"R-I-338751-1",offer:"R-I-338751-9",sidebar_middle:"R-I-338751-5",cashbacks_horizontal:"R-I-338751-7",offers_cards:"R-I-338751-8",mainpage:"R-I-338751-6",cashback:"R-I-338751-4",sku:"R-I-338751-3",sidebar:"R-I-338751-2"},RTB_PARAMS={},ANALYTICS={logInConsole:!1},COLLECTION_LOCAL_CACHE_ATTRS={nonce:"heVv4uj5YaOLYQDG2Q1k9A=="},API_WEB="https://api.edadeal.ru/web/",API_CB="https://cb.edadeal.ru/v2/",API_USR="https://usr.edadeal.ru/auth/v1/",API_BARCODE="https://barcode.edadeal.ru/barcode/v3/",API_CHERCHER="https://search.edadeal.io/api/v2",API_BASE_URL="https://api.edadeal.ru/web/",CB_URL="https://cb.edadeal.ru/v2/",AUTH_URL="https://usr.edadeal.ru/auth/v1/";$M&&(VERSION=\'v3.0.51\',document.write(\'<script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==" crossorigin="" src="https://yastatic.net/s3/edadeal-public-static/web/moby/v3.0.51/client/globals.js"><\\/script>\')); </script> <script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==">!function(s,e,p,r,i,t,n,c,h,l,o,u){for(e.$&&r.push.apply(r,e.$),e["p-retailer"]&&r.push.apply(r,e["p-retailer"]),r.push("p-retailer");s!==r.length;++s)-1!==(u=r[s]).indexOf(".")?-1!==u.indexOf(".css")?i.push(u):t.push(u):(i.push(u+".css"),t.push(u+".js"));c+=p,n+=p,document.write(c+i.join(\'">\'+c)+\'">\'+n+t.join(\'"><\\/script>\'+n)+\'"><\\/script>\')}(0,MANIFEST,CLIENT_BASE_URL+"/",[],[],[],\'<script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==" crossorigin="" src="\',\'<link rel="stylesheet" nonce="heVv4uj5YaOLYQDG2Q1k9A==" href="\'); </script><link rel="stylesheet" nonce="heVv4uj5YaOLYQDG2Q1k9A==" href="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/client/base.css"><link rel="stylesheet" nonce="heVv4uj5YaOLYQDG2Q1k9A==" href="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/client/p-offers.css"><link rel="stylesheet" nonce="heVv4uj5YaOLYQDG2Q1k9A==" href="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/client/p-retailer.css"><script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==" crossorigin="" src="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/client/libs.js"></script><script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==" crossorigin="" src="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/client/base.js"></script><script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==" crossorigin="" src="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/client/p-offers.js"></script><script type="text/javascript" nonce="heVv4uj5YaOLYQDG2Q1k9A==" crossorigin="" src="https://yastatic.net/s3/edadeal-public-static/web/app/ae0c5b9d70691/client/p-retailer.js"></script>  <style type="text/css" class="mb-style-tag">.aNLP4g6{width:100%;height:inherit;min-height:inherit;max-height:inherit}</style><script charset="utf-8" src="https://an.yandex.ru/partner-code-bundles/11724/656914953a16a63f529d.js"></script><script id="https://an.yandex.ru/partner-code-bundles/11724/context_static.js" async="" importance="high" src="https://an.yandex.ru/partner-code-bundles/11724/context_static.js" crossorigin="anonymous"></script></head><body>  <div id="root"></div> </body></html>'}
  47. 2020-07-14 22:24:26 [scrapy.core.engine] INFO: Closing spider (finished)
  48. 2020-07-14 22:24:26 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  49. {'downloader/request_bytes': 526,
  50. 'downloader/request_count': 1,
  51. 'downloader/request_method_count/POST': 1,
  52. 'downloader/response_bytes': 11498,
  53. 'downloader/response_count': 1,
  54. 'downloader/response_status_count/200': 1,
  55. 'elapsed_time_seconds': 19.971284,
  56. 'finish_reason': 'finished',
  57. 'finish_time': datetime.datetime(2020, 7, 14, 19, 24, 26, 553337),
  58. 'item_scraped_count': 1,
  59. 'log_count/DEBUG': 2,
  60. 'log_count/INFO': 10,
  61. 'log_count/WARNING': 1,
  62. 'memusage/max': 53882880,
  63. 'memusage/startup': 53882880,
  64. 'response_received_count': 1,
  65. 'scheduler/dequeued': 2,
  66. 'scheduler/dequeued/memory': 2,
  67. 'scheduler/enqueued': 2,
  68. 'scheduler/enqueued/memory': 2,
  69. 'splash/render.html/request_count': 1,
  70. 'splash/render.html/response_count/200': 1,
  71. 'start_time': datetime.datetime(2020, 7, 14, 19, 24, 6, 582053)}
  72. 2020-07-14 22:24:26 [scrapy.core.engine] INFO: Spider closed (finished)
  73. AAAAAAAAAAAAAAAAA 0
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!