Advertisement
so_hard

Untitled

Nov 21st, 2020
190
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Python 0.46 KB | None | 0 0
  1. class CustomRetryMiddleware(RetryMiddleware):
  2.     def process_response(self, request, response, spider):
  3.         if request.meta.get('dont_retry', False):
  4.             return response
  5.         if 'showcaptcha?' in response.url:
  6.             reason = 'capcha'
  7.             original_url = request.meta['original_url']
  8.             new_request = request.replace(url=original_url)
  9.             return self._retry(new_request, reason, spider) or response
  10.         return response
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement