API tools faq
paste
Advertisement
Guest User

robo old

a guest
Jul 3rd, 2015
167
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.82 KB | None | 0 0
raw download clone embed print report
  1. User-agent: Yandex
  2. Disallow: /*?page_no
  3. Disallow: /*?order_by=name
  4. Disallow: /*?is_mall
  5. Disallow: /site_media/
  6. Disallow: /search/
  7. Disallow: /apps/
  8. Disallow: /basket/
  9. Disallow: /discont/
  10. Disallow: /read_basked/
  11. Disallow: /account/
  12. Disallow: /category/0/
  13.  
  14. User-agent: Googlebot
  15. Disallow: /*?page_no
  16. Disallow: /*?order_by=name
  17. Disallow: /*?is_mall
  18. Disallow: /site_media/
  19. Disallow: /search/
  20. Disallow: /apps/
  21. Disallow: /basket/
  22. Disallow: /discont/
  23. Disallow: /read_basked/
  24. Disallow: /account/
  25. Disallow: /category/0/
  26.  
  27. User-agent: *
  28. Disallow: /*?page_no
  29. Disallow: /*?order_by=name
  30. Disallow: /*?is_mall
  31. Disallow: /site_media/
  32. Disallow: /search/
  33. Disallow: /apps/
  34. Disallow: /basket/
  35. Disallow: /discont/
  36. Disallow: /read_basked/
  37. Disallow: /account/
  38. Disallow: /category/0/
  39. Disallow: /product/*
  40. Disallow: /product/
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!