furas

Python - scraping modules and tools

Mar 29th, 2022 (edited)
233
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. # Scraping: Python tools and modules for scraping (updated)
  2. # https://blog.furas.pl/python-scraping-tools-and-modules-for-scraping-update-2022-03-28-gb.html
  3.  
  4. '''
  5. Get HTML from server
  6. - urllib.request
  7. - requests
  8.  
  9. Search data in HTML
  10. - BeautifulSoup
  11. - lxml
  12. - parsel
  13. - cssselector
  14. - pyquery
  15.  
  16. Scraping framework(s)
  17. - Scrapy
  18. - MechanicalSoup
  19. - RobotFramework
  20. - RoboBrowser
  21. - Mechanize
  22.  
  23. Work with JavaScript
  24. - Selenium
  25. - pyppeteer
  26. - Playwright
  27.  
  28. Other tools for help
  29. - httpbin.org
  30. - ToScrape.com
  31. - curl.trillworks.com
  32. - DevTools in Chrome and Firefox
  33. - Postman
  34. - Insomnia
  35. - charlesproxy.com
  36. - mitmproxy.org
  37. '''
RAW Paste Data Copied