Advertisement
Guest User

Untitled

a guest
Aug 23rd, 2017
73
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 1.13 KB | None | 0 0
  1. def parse_page1(self, response):
  2. item = MyItem()
  3. item['main_url'] = response.url
  4. request = Request("http://www.example.com/some_page.html",
  5. callback=self.parse_page2)
  6. request.meta['item'] = item
  7. return request
  8.  
  9. def parse_page2(self, response):
  10. item = response.meta['item']
  11. item['other_url'] = response.url
  12. return item
  13.  
  14. scrapy.Request("http://www.example.com.html",callback=parse_page1)`
  15.  
  16. scrapy.Request("http://www.example.com/some_page.html",callback=parse_page2)
  17.  
  18. item['main_url'] = response.url # send "http://www.example.com.html" to item
  19. request = scrapy.Request("http://www.example.com/some_page.html",
  20. callback=self.parse_page2)
  21. request.meta['item'] = item # store item in request.meta
  22.  
  23. item = response.meta['item'] #response.meta is equal to request.meta,so the item in request.meta of page1 is send to request.meta of page2,and here item['main_url'] ="http://www.example.com.html".
  24.  
  25. item['other_url'] = response.url # response.url ="http://www.example.com/some_page.html"
  26.  
  27. return item #finally,we get the item recordind urls of page1 and page2.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement