Advertisement
Guest User

Untitled

a guest
Jul 21st, 2015
313
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 10.50 KB | None | 0 0
  1. User-agent: Rogerbot
  2. User-agent: Exabot
  3. User-agent: MJ12bot
  4. User-agent: Dotbot
  5. User-agent: Gigabot
  6. User-agent: AhrefsBot
  7. User-agent: BlackWidow
  8. User-agent: Bot\ mailto:craftbot@yahoo.com
  9. User-agent: ChinaClaw
  10. User-agent: Custo
  11. User-agent: DISCo
  12. User-agent: Download\ Demon
  13. User-agent: eCatch
  14. User-agent: EirGrabber
  15. User-agent: EmailSiphon
  16. User-agent: EmailWolf
  17. User-agent: Express\ WebPictures
  18. User-agent: ExtractorPro
  19. User-agent: EyeNetIE
  20. User-agent: FlashGet
  21. User-agent: GetRight
  22. User-agent: GetWeb!
  23. User-agent: Go!Zilla
  24. User-agent: Go-Ahead-Got-It
  25. User-agent: GrabNet
  26. User-agent: Grafula
  27. User-agent: HMView
  28. User-agent: HTTrack
  29. User-agent: Image\ Stripper
  30. User-agent: Image\ Sucker
  31. User-agent: Indy\ Library
  32. User-agent: InterGET
  33. User-agent: Internet\ Ninja
  34. User-agent: JetCar
  35. User-agent: JOC\ Web\ Spider
  36. User-agent: larbin
  37. User-agent: LeechFTP
  38. User-agent: Mass\ Downloader
  39. User-agent: MIDown\ tool
  40. User-agent: Mister\ PiX
  41. User-agent: Navroad
  42. User-agent: NearSite
  43. User-agent: NetAnts
  44. User-agent: NetSpider
  45. User-agent: Net\ Vampire
  46. User-agent: NetZIP
  47. User-agent: Octopus
  48. User-agent: Offline\ Explorer
  49. User-agent: Offline\ Navigator
  50. User-agent: PageGrabber
  51. User-agent: Papa\ Foto
  52. User-agent: pavuk
  53. User-agent: pcBrowser
  54. User-agent: RealDownload
  55. User-agent: ReGet
  56. User-agent: SiteSnagger
  57. User-agent: SmartDownload
  58. User-agent: SuperBot
  59. User-agent: SuperHTTP
  60. User-agent: Surfbot
  61. User-agent: tAkeOut
  62. User-agent: Teleport\ Pro
  63. User-agent: VoidEYE
  64. User-agent: Web\ Image\ Collector
  65. User-agent: Web\ Sucker
  66. User-agent: WebAuto
  67. User-agent: WebCopier
  68. User-agent: WebFetch
  69. User-agent: WebGo\ IS
  70. User-agent: WebLeacher
  71. User-agent: WebReaper
  72. User-agent: WebSauger
  73. User-agent: Website\ eXtractor
  74. User-agent: Website\ Quester
  75. User-agent: WebStripper
  76. User-agent: WebWhacker
  77. User-agent: WebZIP
  78. User-agent: Wget
  79. User-agent: Widow
  80. User-agent: WWWOFFLE
  81. User-agent: Xaldon\ WebSpider
  82. User-agent: Zeus
  83. User-agent: libwww-perl
  84. User-agent: libwwwperl
  85. User-agent: attach
  86. User-agent: ASPSeek
  87. User-agent: appie
  88. User-agent: AbachoBOT
  89. User-agent: autoemailspider
  90. User-agent: anarchie
  91. User-agent: antibot
  92. User-agent: asterias
  93. User-agent: B2w
  94. User-agent: BackWeb
  95. User-agent: BackDoorBot
  96. User-agent: Bandit
  97. User-agent: BatchFTP
  98. User-agent: Black\ Hole
  99. User-agent: Baidu
  100. User-agent: BlowFish
  101. User-agent: BuiltBotTough
  102. User-agent: Bot\ mailto
  103. User-agent: BotALot
  104. User-agent: Buddy
  105. User-agent: Bullseye
  106. User-agent: bumblebee
  107. User-agent: BunnySlippers
  108. User-agent: ClariaBot
  109. User-agent: curl
  110. User-agent: clsHTTP
  111. User-agent: ChinaClaw
  112. User-agent: CheeseBot
  113. User-agent: CherryPicker
  114. User-agent: Crescent
  115. User-agent: CherryPickerSE
  116. User-agent: CherryPickerElite
  117. User-agent: Collector
  118. User-agent: COAST\ WebMaster
  119. User-agent: cosmos
  120. User-agent: CopyRightCheck
  121. User-agent: ColdFusion
  122. User-agent: Copier
  123. User-agent: Crescent
  124. User-agent: DA
  125. User-agent: DTS\ Agent
  126. User-agent: DISCo\ Pump
  127. User-agent: DittoSpyder
  128. User-agent: Diamond
  129. User-agent: Download\ Demon
  130. User-agent: Download\ Wonder
  131. User-agent: Downloader
  132. User-agent: dloader
  133. User-agent: Drip
  134. User-agent: eCatch
  135. User-agent: EirGrabber
  136. User-agent: Express\ WebPictures
  137. User-agent: Extreme\ Picture\ Finder
  138. User-agent: EmailCollector
  139. User-agent: EmailSiphon
  140. User-agent: EmailWolf
  141. User-agent: EasyDL
  142. User-agent: EirGrabber
  143. User-agent: EroCrawler
  144. User-agent: ExtractorPro
  145. User-agent: EyeNetIE
  146. User-agent: FAST\ WebCrawler
  147. User-agent: FileHound
  148. User-agent: Fetch\ API\ Request
  149. User-agent: FlashGet
  150. User-agent: FlickBot
  151. User-agent: FrontPage
  152. User-agent: FreeFind.com
  153. User-agent: GetRight
  154. User-agent: GetSmart
  155. User-agent: Generic
  156. User-agent: Go!Zilla
  157. User-agent: Go-Ahead-Got-It
  158. User-agent: gotit
  159. User-agent: Grabber
  160. User-agent: GrabNet
  161. User-agent: Grafula
  162. User-agent: Gulliver
  163. User-agent: Harvest
  164. User-agent: HMView
  165. User-agent: Heretrix
  166. User-agent: HitboxDoctor
  167. User-agent: HTTPapp
  168. User-agent: HTTrack
  169. User-agent: HTTPTrack
  170. User-agent: HTTPviewer
  171. User-agent: httplib
  172. User-agent: httpfetcher
  173. User-agent: httpscraper
  174. User-agent: hloader
  175. User-agent: humanlinks
  176. User-agent: ia_archiver
  177. User-agent: InterGET
  178. User-agent: Internet\ Ninja
  179. User-agent: InfoNaviRobot
  180. User-agent: InternetSeer.com
  181. User-agent: Iria
  182. User-agent: IRLbot
  183. User-agent: JetCar
  184. User-agent: JOC
  185. User-agent: JOC\ Web\ Spider
  186. User-agent: JoBo
  187. User-agent: Java
  188. User-agent: JustView
  189. User-agent: Jonzilla
  190. User-agent: JennyBot
  191. User-agent: Kenjin\ Spider
  192. User-agent: Keyword\ Density
  193. User-agent: larbin
  194. User-agent: LeechFTP
  195. User-agent: Lachesis
  196. User-agent: LexiBot
  197. User-agent: libWeb
  198. User-agent: Libby_
  199. User-agent: LinkScan
  200. User-agent: LinkWalker
  201. User-agent: LinkextractorPro
  202. User-agent: lftp
  203. User-agent: likse
  204. User-agent: Link
  205. User-agent: lwp-trivial
  206. User-agent: lwp\ request
  207. User-agent: Magnet
  208. User-agent: Mag-Net
  209. User-agent: Mass\ Downloader
  210. User-agent: MIIxpc
  211. User-agent: Microsoft\ URL\ Control
  212. User-agent: MSFrontPage
  213. User-agent: MSIECrawler
  214. User-agent: MicrosoftURL
  215. User-agent: Missigua
  216. User-agent: Mewsoft\ Search\ Engine
  217. User-agent: moget
  218. User-agent: Mata\ Hari
  219. User-agent: Memo
  220. User-agent: Metacarta
  221. User-agent: Mercator
  222. User-agent: MIDown\ tool
  223. User-agent: MFC_Tear_Sample
  224. User-agent: Mirror
  225. User-agent: MIIxpc
  226. User-agent: Mister\ PiX
  227. User-agent: NationalDirectory\ WebSpider
  228. User-agent: NICErsPRO
  229. User-agent: Nikto
  230. User-agent: Navroad
  231. User-agent: NearSite
  232. User-agent: NetAnts
  233. User-agent: NetSpider
  234. User-agent: NICErsPRO
  235. User-agent: NetResearchServer
  236. User-agent: NetMechanic
  237. User-agent: Net\ Vampire
  238. User-agent: Net\ Probe
  239. User-agent: NetZip
  240. User-agent: nexuscache
  241. User-agent: Ninja
  242. User-agent: NPBot
  243. User-agent: our\ agent
  244. User-agent: onestop
  245. User-agent: oBot
  246. User-agent: Octopus
  247. User-agent: Offline\ Explorer
  248. User-agent: Openfind
  249. User-agent: Openfind\ data\ gatherer
  250. User-agent: OrangeBot
  251. User-agent: PageGrabber
  252. User-agent: Papa\ Foto
  253. User-agent: PHP\ version
  254. User-agent: PHP
  255. User-agent: PHPot
  256. User-agent: Perl
  257. User-agent: pcBrowser
  258. User-agent: pavuk
  259. User-agent: Pockey
  260. User-agent: Ping
  261. User-agent: PingALink\ Monitoring\ Services
  262. User-agent: ProWebWalker
  263. User-agent: ProPowerBot
  264. User-agent: Pump
  265. User-agent: Pompos
  266. User-agent: psbot
  267. User-agent: Python\ urllib
  268. User-agent: Python-urllib
  269. User-agent: QueryN
  270. User-agent: RealDownload
  271. User-agent: Reaper
  272. User-agent: Recorder
  273. User-agent: RepoMonkey
  274. User-agent: psycheclone
  275. User-agent: RMA
  276. User-agent: Rico
  277. User-agent: Robozilla
  278. User-agent: ReGet
  279. User-agent: Siphon
  280. User-agent: SiteSnagger
  281. User-agent: sitecheck.internetseer.com
  282. User-agent: SmartDownload
  283. User-agent: Snake
  284. User-agent: spanner
  285. User-agent: Stealer
  286. User-agent: SpaceBison
  287. User-agent: SpankBot
  288. User-agent: Spinne
  289. User-agent: Stripper
  290. User-agent: slysearch
  291. User-agent: Sucker
  292. User-agent: Snoopy
  293. User-agent: ScoutAbout
  294. User-agent: Scooter
  295. User-agent: SuperBot
  296. User-agent: SuperHTTP
  297. User-agent: Snapbot
  298. User-agent: Surfbot
  299. User-agent: suzuran
  300. User-agent: Szukacz
  301. User-agent: Sqworm
  302. User-agent: tAkeOut
  303. User-agent: Teleport\ Pro
  304. User-agent: Telesoft
  305. User-agent: TurnitinBot
  306. User-agent: turingos
  307. User-agent: toCrawl
  308. User-agent: TightTwatBot
  309. User-agent: True_Robot
  310. User-agent: The\ Intraformant
  311. User-agent: TheNomad
  312. User-agent: Titan
  313. User-agent: UrlDispatcher
  314. User-agent: URLy\ Warning
  315. User-agent: Vayala
  316. User-agent: Vagabondo
  317. User-agent: Vintage
  318. User-agent: Vacuum
  319. User-agent: VCI
  320. User-agent: VoidEYE
  321. User-agent: W3C_Validator
  322. User-agent: Webdownloader
  323. User-agent: Web\ Downloader
  324. User-agent: Webhook
  325. User-agent: Webmole
  326. User-agent: Webminer
  327. User-agent: Webmirror
  328. User-agent: Websucker
  329. User-agent: Websites
  330. User-agent: Web\ Image\ Collector
  331. User-agent: Web\ Sucker
  332. User-agent: WebAuto
  333. User-agent: WebCopier
  334. User-agent: WebFetch
  335. User-agent: WebReaper
  336. User-agent: WebSauger
  337. User-agent: Website
  338. User-agent: Webster
  339. User-agent: WebStripper
  340. User-agent: WebCopier
  341. User-agent: WebViewer
  342. User-agent: WebWhacker
  343. User-agent: WebEnhancer
  344. User-agent: Wells
  345. User-agent: WebZIP
  346. User-agent: Wget
  347. User-agent: Whacker
  348. User-agent: Widow
  349. User-agent: Xaldon
  350. User-agent: Wildsoft\ Surfer
  351. User-agent: WinHttpRequest
  352. User-agent: WinHttp
  353. User-agent: Webster\ Pro
  354. User-agent: Web\ Image\ Collector
  355. User-agent: WebZip
  356. User-agent: WebAuto
  357. User-agent: Website\ Quester
  358. User-agent: WWWOFFLE
  359. User-agent: WWW-Collector-E
  360. User-agent: Xaldon\ WebSpider
  361. User-agent: Xenu
  362. User-agent: Xara
  363. User-agent: Y!TunnelPro
  364. User-agent: YahooYSMcm
  365. User-agent: Zade
  366. User-agent: ZBot
  367. User-agent: Zeus
  368. User-agent: Rogerbot
  369. User-agent: Exabot
  370. User-agent: MJ12bot
  371. User-agent: Dotbot
  372. User-agent: Gigabot
  373. User-agent: AhrefsBot
  374. User-agent: BlackWidow
  375. User-agent: Bot\ mailto:craftbot@yahoo.com
  376. User-agent: ChinaClaw
  377. User-agent: Custo
  378. User-agent: DISCo
  379. User-agent: Download\ Demon
  380. User-agent: eCatch
  381. User-agent: EirGrabber
  382. User-agent: EmailSiphon
  383. User-agent: EmailWolf
  384. User-agent: Express\ WebPictures
  385. User-agent: ExtractorPro
  386. User-agent: EyeNetIE
  387. User-agent: FlashGet
  388. User-agent: GetRight
  389. User-agent: GetWeb!
  390. User-agent: Go!Zilla
  391. User-agent: Go-Ahead-Got-It
  392. User-agent: GrabNet
  393. User-agent: Grafula
  394. User-agent: HMView
  395. User-agent: HTTrack
  396. User-agent: Image\ Stripper
  397. User-agent: Image\ Sucker
  398. User-agent: Indy\ Library
  399. User-agent: InterGET
  400. User-agent: Internet\ Ninja
  401. User-agent: JetCar
  402. User-agent: JOC\ Web\ Spider
  403. User-agent: larbin
  404. User-agent: LeechFTP
  405. User-agent: Mass\ Downloader
  406. User-agent: MIDown\ tool
  407. User-agent: Mister\ PiX
  408. User-agent: Navroad
  409. User-agent: NearSite
  410. User-agent: NetAnts
  411. User-agent: NetSpider
  412. User-agent: Net\ Vampire
  413. User-agent: NetZIP
  414. User-agent: Octopus
  415. User-agent: Offline\ Explorer
  416. User-agent: Offline\ Navigator
  417. User-agent: PageGrabber
  418. User-agent: Papa\ Foto
  419. User-agent: pavuk
  420. User-agent: pcBrowser
  421. User-agent: RealDownload
  422. User-agent: ReGet
  423. User-agent: SiteSnagger
  424. User-agent: SmartDownload
  425. User-agent: SuperBot
  426. User-agent: SuperHTTP
  427. User-agent: Surfbot
  428. User-agent: tAkeOut
  429. User-agent: Teleport\ Pro
  430. User-agent: VoidEYE
  431. User-agent: Web\ Image\ Collector
  432. User-agent: Web\ Sucker
  433. User-agent: WebAuto
  434. User-agent: WebCopier
  435. User-agent: WebFetch
  436. User-agent: WebGo\ IS
  437. User-agent: WebLeacher
  438. User-agent: WebReaper
  439. User-agent: WebSauger
  440. User-agent: Website\ eXtractor
  441. User-agent: Website\ Quester
  442. User-agent: WebStripper
  443. User-agent: WebWhacker
  444. User-agent: WebZIP
  445. User-agent: Wget
  446. User-agent: Widow
  447. User-agent: WWWOFFLE
  448. User-agent: Xaldon\ WebSpider
  449. Disallow: /
  450.  
  451. User-agent: *
  452. Disallow: /wp-admin/
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement