Advertisement
Guest User

robots-agent-code

a guest
Jun 6th, 2013
43
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.56 KB | None | 0 0
  1. # Google Image
  2. User-agent: Googlebot-Image
  3. Disallow:
  4. Allow: /*
  5.  
  6. # Google AdSense
  7. User-agent: Mediapartners-Google
  8. Disallow:
  9.  
  10. # digg mirror
  11. User-agent: duggmirror
  12. Disallow: /
  13.  
  14. # global
  15. User-agent: *
  16. Disallow: /cgi-bin/
  17. Disallow: /wp-admin/
  18. Disallow: /wp-includes/
  19. Disallow: /wp-content/plugins/
  20. Disallow: /wp-content/cache/
  21. Disallow: /wp-content/themes/
  22. Disallow: /trackback/
  23. Disallow: /feed/
  24. Disallow: /comments/
  25. Disallow: /category/*/*
  26. Disallow: */trackback/
  27. Disallow: */feed/
  28. Disallow: */comments/
  29. Disallow: /*?
  30. Allow: /wp-content/uploads/
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement