Advertisement
askew0

Untitled

May 23rd, 2017
1,330
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 3.41 KB | None | 0 0
  1. "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
  2. "Mozilla/5.0 (compatible; MJ12bot/v1.4.7; http://mj12bot.com/)"
  3. "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  4. "Mozilla/5.0 (compatible; Siteliner/1.0; +http://www.siteliner.com/bot)"
  5. "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)"
  6. "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
  7. "Mozilla/5.0 (compatible; SputnikBot/2.3; +http://corp.sputnik.ru/webmaster)"
  8. "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.0; trendictionbot0.5.0; trendiction search; http://www.trendiction.de/bot; please let us know of any problems; web at trendiction.com) Gecko/20071127 Firefox/3.0.0.11"
  9. "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
  10. "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexMobileBot/3.0; +http://yandex.com/bots)"
  11. "Mediatoolkitbot (complaints@mediatoolkit.com)"
  12. "Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)"
  13. "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)"
  14. "Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)"
  15. "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  16. "Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Fast/2.0; +http://go.mail.ru/help/robots)"
  17. "Domain Re-Animator Bot (http://domainreanimator.com) - support@domainreanimator.com"
  18. "Gigabot/1.0"
  19. "Vegi bot (we follow your robots.txt settings before crawling, you can slow down the bot by change the Crawl-Delay parameter in the settings.if you have an enquiry, please email to: abuse-report@terrykyleseoagency.com)"
  20. "Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Img/2.0; +http://go.mail.ru/help/robots)"
  21. "Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots)"
  22. "Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)"
  23. "Mozilla/5.0 (compatible; archive.org_bot +http://www.archive.org/details/archive.org_bot)"
  24. "Mozilla/5.0 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)"
  25. "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; Girafabot; girafabot at girafa dot com; http://www.girafa.com)"
  26. "Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
  27. "Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv 11.0) like Gecko (compatible; Zombiebot/2.1; +http://www.zombiedomain.net/robot/)"
  28. "Mozilla/5.0 (compatible; seocharger-robot/1.0; +http://seocharger.com/robot)"
  29. "CCBot/2.0 (http://commoncrawl.org/faq/)"
  30. "Linguee Bot (http://www.linguee.com/bot; bot@linguee.com)"
  31. "Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)"
  32. "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0"
  33. "Wotbox/2.01 (+http://www.wotbox.com/bot/)"
  34. "magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net)"
  35. "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement