Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
- "Mozilla/5.0 (compatible; MJ12bot/v1.4.7; http://mj12bot.com/)"
- "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
- "Mozilla/5.0 (compatible; Siteliner/1.0; +http://www.siteliner.com/bot)"
- "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)"
- "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
- "Mozilla/5.0 (compatible; SputnikBot/2.3; +http://corp.sputnik.ru/webmaster)"
- "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.0; trendictionbot0.5.0; trendiction search; http://www.trendiction.de/bot; please let us know of any problems; web at trendiction.com) Gecko/20071127 Firefox/3.0.0.11"
- "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
- "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexMobileBot/3.0; +http://yandex.com/bots)"
- "Mediatoolkitbot (complaints@mediatoolkit.com)"
- "Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)"
- "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)"
- "Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)"
- "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
- "Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Fast/2.0; +http://go.mail.ru/help/robots)"
- "Domain Re-Animator Bot (http://domainreanimator.com) - support@domainreanimator.com"
- "Gigabot/1.0"
- "Vegi bot (we follow your robots.txt settings before crawling, you can slow down the bot by change the Crawl-Delay parameter in the settings.if you have an enquiry, please email to: abuse-report@terrykyleseoagency.com)"
- "Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/Img/2.0; +http://go.mail.ru/help/robots)"
- "Mozilla/5.0 (compatible; YandexImages/3.0; +http://yandex.com/bots)"
- "Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)"
- "Mozilla/5.0 (compatible; archive.org_bot +http://www.archive.org/details/archive.org_bot)"
- "Mozilla/5.0 (compatible; SMTBot/1.0; +http://www.similartech.com/smtbot)"
- "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; Girafabot; girafabot at girafa dot com; http://www.girafa.com)"
- "Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
- "Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv 11.0) like Gecko (compatible; Zombiebot/2.1; +http://www.zombiedomain.net/robot/)"
- "Mozilla/5.0 (compatible; seocharger-robot/1.0; +http://seocharger.com/robot)"
- "CCBot/2.0 (http://commoncrawl.org/faq/)"
- "Linguee Bot (http://www.linguee.com/bot; bot@linguee.com)"
- "Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)"
- "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0"
- "Wotbox/2.01 (+http://www.wotbox.com/bot/)"
- "magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net)"
- "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement