hackrepair

Bad Bot .htaccess, List (Updates Log)

Feb 21st, 2014
1,700
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. 2015.11.03
  2. - Updated and made more readable the "Abuse HTTP Referrer Blocking" section
  3. cPanel users may wish to remove the "Abuse HTTP Referrer Blocking" section,
  4. as it may cause cPanel "Hotlink Protection" settings issues.
  5.  
  6. 2015.08.25
  7. - Added Referrer: FunWebProducts
  8.  
  9. 2015.07.25
  10. - Removed darodar (ghost referrer does not visit site)
  11. - Added Referrer: anticrawler
  12. - Added Referrer: best-seo-offer
  13. - Added Referrer: best-seo-solution
  14. - Added Referrer: buttons-for-your-website
  15. - Added Referrer: ranksonic
  16. - Added Referrer: medispainstitute
  17. - Added Referrer: offers.bycontext
  18. - Added Referrer: 100dollars-seo
  19. - Added Referrer: sitevaluation
  20. - Added Referrer: dailyrank
  21. - Added Referrer: 4webmasters.org
  22. - Added Referrer: trafficmonetize.org
  23. - Added Referrer: webmonetizer.net
  24.  
  25. 2015.07.21
  26. - Error in rewrite rule. Corrected, from:
  27. # Abuse HTTP Referrer Blocking
  28. RewriteEngine on
  29. RewriteCond %{HTTP_REFERER} ^([^.]+.)*?semalt.com [NC, OR]
  30. RewriteCond %{HTTP_REFERER} ^([^.]+.)*?kambasoft.com [NC, OR]
  31. RewriteCond %{HTTP_REFERER} ^([^.]+.)*?savetubevideo.com [NC, OR]
  32. RewriteRule ^.* - [F,L]
  33. # Abuse HTTP Referrer Blocking end
  34. corrected to:
  35. # Abuse HTTP Referrer Blocking
  36. RewriteCond %{HTTP_REFERER} (?:semalt|kambasoft|savetubevideo|7makemoneyonline|buttons-for-website|darodar|ilovevitaly)\. [NC]
  37. RewriteRule ^.* - [F,L]
  38. # Abuse HTTP Referrer Blocking end
  39. And added additional bots below:
  40. - Added Referrer: 7makemoneyonline
  41. - Added Referrer: buttons-for-website
  42. - Added Referrer: darodar
  43. - Added Referrer: ilovevitaly
  44.  
  45. 2015.07.10
  46. - Removed SeznamBot
  47. Bot from Czech Republic appears to be following the rules now.
  48. https://twitter.com/hledani_seznam/status/667318724647518208
  49.  
  50. 2015.06.15
  51. - Added Aboundex
  52. Deny from 173.192.34.95
  53. - Added Referrer: 360Spider
  54. - Added Referrer: EasouSpider
  55. - Added Referrer: LinksCrawler
  56. - Added Referrer: SearchmetricsBot
  57. - Added Referrer: SeznamBot
  58. - Added Referrer: Sogou
  59. - Added Referrer: XoviBot
  60. - Added Referrer: YisouSpider
  61. So many new bots! I even for a centon considered adding "Bing" (due to very serious toll they take on
  62. many a poor web server). Oh well, can't fight the man...
  63.  
  64. 2014.09.11
  65. - Added Referrer: semalt.com
  66. - Added Referrer: savetubevideo.com
  67. - Added Referrer: kambasoft.com
  68. May add-to or remove this completely in future. Folks who "live in" Google Anayltics day-to-day (like me)
  69. consider referrer spam as abuse. I think Matt Cutts sums up the "what is referrer spam" pretty well in
  70. his video, http://goo.gl/5oL8Ag
  71.  
  72. 2014.07.10
  73. - Added UserAgent: kanagawa
  74. - Added UserAgent: g00g1e
  75. - Added UserAgent: MJ12bot
  76. - Added UserAgent: AhrefsBot
  77. - Removed "all" IP's within the #Block comment spammers, bad bots and some proxies section.
  78. RewriteCond %{REMOTE_HOST} 12.226.240.248 [OR]
  79. RewriteCond %{REMOTE_HOST} 24.111.102.26 [OR]
  80. RewriteCond %{REMOTE_HOST} 24.117.121.113 [OR]
  81. RewriteCond %{REMOTE_HOST} 65.30.216.140 [OR]
  82. RewriteCond %{REMOTE_HOST} 67.87.64.23 [OR]
  83. RewriteCond %{REMOTE_HOST} 68.12.149.198 [OR]
  84. RewriteCond %{REMOTE_HOST} 69.139.167.203 [OR]
  85. RewriteCond %{REMOTE_HOST} 74.95.182.57 [OR]
  86. RewriteCond %{REMOTE_HOST} 91.121.3.29 [OR]
  87. RewriteCond %{REMOTE_HOST} 203.94.229.227 [OR]
  88. RewriteCond %{REMOTE_HOST} 208.96.122.142 [OR]
  89. RewriteCond %{REMOTE_HOST} 210.0.141.247 [OR]
  90. RewriteCond %{REMOTE_HOST} 210.197.97.67 [OR]
  91. RewriteCond %{REMOTE_HOST} 212.179.127.188 [OR]
  92. RewriteCond %{REMOTE_HOST} 216.246.60.183 [OR]
  93. RewriteCond %{REMOTE_HOST} 220.156.189.233 [OR]
  94. RewriteCond %{REMOTE_HOST} 222.36.12.42 [OR]
  95. No longer seeing this IP list as effective. May add other IP addresses in future so leaving section empty for now.
  96. - Added latest Pastebin address to final line (many folks calling me on this is only reason why).
  97. # End HackRepair.com Blacklist, http://pastebin.com/u/hackrepair
  98.  
  99. 2014.04.30
  100. - Added UserAgent: WPScan
  101. Sadly, escalating abuse of the otherwise excellent script from wpscan.org has led me add "WPScan" to our blocklist.
  102.  
  103. 2014.04.02
  104. - Removed ^Link
  105. which in retrospect was too broad a block term.
  106. Caused web page preview within LinkedIn's post to display only URL instead of web page preview. LinkIn's bot is "LinkedInBot"
  107.  
  108. 2014.03.26
  109. - Added UserAgent: BOT for JCE
  110.  
  111. 2014.03.24
  112. - Removed text "(already blocked 38.*)" on line: # cyveillance (already blocked 38.*)
  113.  
  114. 2014.02.21
  115. - Removed Yandex IP Address List
  116. - Removed McAfee Scanner IP Address List
  117. - Removed UserAgent: User-Agent [NC,OR]
  118. - Added UserAgent: ^$ (is a blank user agent)
  119. - Added UserAgent: Acunetix
  120. - Added UserAgent: checkprivacy
  121. - Added UserAgent: clshttp
  122. - Added UserAgent: extract
  123. - Added UserAgent: FHscan
  124. - Added UserAgent: grab
  125. - Added UserAgent: harvest
  126. - Added UserAgent: libwww
  127. - Added UserAgent: miner
  128. - Added UserAgent: Yandex
  129.  
  130.  
  131. FAQ:
  132.  
  133. 1.
  134. Why Jim is your list not compressed?
  135. Less human readability leads more folks to ask more questions. :)
  136. I have yet to see this more readable .htaccess list cause a site to load more slowly.
  137.  
  138. 2.
  139. How do your test your user agents?
  140. I use FireFox browser User Agent Switcher Addon
  141.  
  142. 3.
  143. Who are you?
  144. I am HackRepair.com
RAW Paste Data