Advertisement
kastielspb

Proxies getter

May 3rd, 2019
309
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Python 0.45 KB | None | 0 0
  1. import requests
  2.  
  3. from bs4 import BeautifulSoup
  4.  
  5. __all__ = (
  6.     'get_proxies',
  7. )
  8.  
  9. def get_proxies():
  10.     url = 'https://www.sslproxies.org/'
  11.     response = requests.get(url)
  12.     soup = BeautifulSoup(response.content, "lxml")
  13.  
  14.     proxies = set()
  15.     trs = soup.select('tr', {'role': 'row'})
  16.  
  17.     for tr in trs[1:21]:
  18.         tds = tr.select('td')
  19.         proxy = f'{tds[0].text}:{tds[1].text}'
  20.         proxies.add(proxy)
  21.     return proxies
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement