API tools faq
paste
Advertisement
Guest User

robots.txt

a guest
Feb 23rd, 2017
113
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 1.20 KB | None | 0 0
raw download clone embed print report
  1. User-agent: *
  2. Disallow: /cgi-bin
  3. Disallow: /wp-admin
  4. Disallow: /wp-includes
  5. Disallow: /wp-content/plugins
  6. Disallow: /wp-content/cache
  7. Disallow: /wp-content/themes
  8. Disallow: /trackback
  9. Disallow: */trackback
  10. Disallow: */*/trackback
  11. Disallow: /feed
  12. Disallow: */feed
  13. Disallow: /category/*/*
  14. Disallow: /comments
  15. Disallow: */comment-*
  16. Disallow: */trackback
  17.  
  18. User-agent: Yandex
  19. Disallow: /cgi-bin
  20. Disallow: /wp-admin
  21. Disallow: /wp-includes
  22. Disallow: /wp-content/plugins
  23. Disallow: /wp-content/cache
  24. Disallow: /wp-content/themes
  25. Disallow: /trackback
  26. Disallow: */trackback
  27. Disallow: */*/trackback
  28. Disallow: /feed
  29. Disallow: */feed
  30. Disallow: /category/*/*
  31. Disallow: /comments
  32. Disallow: */comment-*
  33. Disallow: */trackback
  34.  
  35. Host: сайт.ру (без www, единственное, сюда можно приписать https, если сайт работает на нём. но на ВП такого не бывает)
  36. Sitemap: http://сайт.ру/sitemap.xml (без www, в редких случаях менять http на https, когда сайт работает на нём)
  37. User-agent: Googlebot-Image
  38. Allow: /wp-content/uploads/
  39.  
  40. User-agent: YandexImages
  41. Allow: /wp-content/uploads/
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!