SHARE
TWEET

2018 Bad Bots .htaccess List (Updates Log)

hackrepair Feb 14th, 2016 (edited) 2,394 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. 2018.08.15
  2. - Removed user agent: Jakarta\ Commons-HttpClient
  3. - Removed user agent: Indy\ Library
  4.  
  5. 2018.02.15
  6. - Added user agent: 360SE
  7. - Added user agent: AlexaBot
  8. - Added user agent: aria2
  9. - Added user agent: bixocrawler
  10. - Added user agent: GroupHigh
  11. - Added user agent: LBBROWSER
  12. - Added user agent: Owlin
  13. - Added user agent: whisper
  14.  
  15. - Removed user agent: Water Conserve
  16. - Removed user agent: Water Conserve Portal
  17. - Removed user agent: Web CEO Online
  18. - Removed user agent: Web Image Collector
  19.  
  20. 2018.01.05
  21. - Removed user agent: DA
  22.  
  23. 2017.08.15
  24. - Added user agent: Vegi bot
  25. - Removed user agent: RMA
  26.  
  27. 2017.05.15
  28. - Removed user agent: SeznamBot
  29.  
  30. 2017.04.04
  31. - Removed user agent: AhrefsBot
  32. - Removed user agent: gsa
  33.  
  34. 2016.11.20
  35. - Removed user agent: comodo
  36.  
  37. 2016.09.20
  38. - Added user agent: Pcore-HTTP
  39. - Removed user agent: NG
  40. - Removed user agent: PROXY
  41. - Removed user agent: OsO
  42.  
  43. 2016.07.15
  44. - Added user agents:
  45. linkdexbot
  46. SafetyNet
  47.  
  48. 2016.03.15
  49. - Removed bot "heritrix" from bad bot list
  50.  
  51. 2016.02.20
  52. - Added notes and link to header, Wish to "deny" countries from accessing your site
  53. - Added notes and link to header, Alternate web.config settings for IIS, PHP or WordPress
  54.  
  55. 2016.02.15
  56. - Retired the old "Bad Bots .htaccess, List from HackRepair.com" list (changing format as list has grown too large)
  57. - Removed IP blocks lists used in older list (No longer seemingly effective)
  58. - Added "Block Common Malicious Bot Queries" section
  59. - Added "Browser access to log files" section
  60. - Fully replaced "Abuse User Agents Blocking" list and formatting
  61.  
  62.  
  63. +++
  64.  
  65. 2016 Bad Bots .htaccess List from HackRepair.com: http://pastebin.com/BPRv4TDd
  66.  
  67. Frequently Asked Questions:
  68.  
  69. 1.
  70. Why is your list compressed?
  71. The latest uncompressed list is just too long and may not work on some shared hosts.
  72. Bad bots list is compressed and broken into 4 or more sections (to work better with most hosts).
  73.  
  74. 2.
  75. How do your test your user agents?
  76. I used Chrome browser User Agent Switcher extension by Chris Pederick
  77.  
  78. Testing from command line:
  79. User agents: curl -IL -A "1Noonbot" http://yoursite
  80. HTTP referrer: curl -e semalt.com -IL http://yoursite
  81.  
  82. 3.
  83. Who wrote this?
  84. I am Jim Walker, The Hack Repair Guy, https://HackRepair.com
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
Top