API tools faq
paste
GreenZoneOfficiel

#Google robots. \

GreenZoneOfficiel
Sep 15th, 2013
96
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 7.25 KB | None | 0 0
raw download clone embed print report
  1. #Google robots.
  2.  
  3.  
  4. User-agent: *
  5. Disallow: /search
  6. Disallow: /sdch
  7. Disallow: /groups
  8. Disallow: /images
  9. Disallow: /catalogs
  10. Allow: /catalogs/about
  11. Allow: /catalogs/p?
  12. Disallow: /catalogues
  13. Disallow: /news
  14. Allow: /news/directory
  15. Disallow: /nwshp
  16. Disallow: /setnewsprefs?
  17. Disallow: /index.html?
  18. Disallow: /?
  19. Allow: /?hl=
  20. Disallow: /?hl=*&
  21. Disallow: /addurl/image?
  22. Disallow: /pagead/
  23. Disallow: /relpage/
  24. Disallow: /relcontent
  25. Disallow: /imgres
  26. Disallow: /imglanding
  27. Disallow: /sbd
  28. Disallow: /keyword/
  29. Disallow: /u/
  30. Disallow: /univ/
  31. Disallow: /cobrand
  32. Disallow: /custom
  33. Disallow: /advanced_group_search
  34. Disallow: /googlesite
  35. Disallow: /preferences
  36. Disallow: /setprefs
  37. Disallow: /swr
  38. Disallow: /url
  39. Disallow: /default
  40. Disallow: /m?
  41. Disallow: /m/
  42. Disallow: /wml?
  43. Disallow: /wml/?
  44. Disallow: /wml/search?
  45. Disallow: /xhtml?
  46. Disallow: /xhtml/?
  47. Disallow: /xhtml/search?
  48. Disallow: /xml?
  49. Disallow: /imode?
  50. Disallow: /imode/?
  51. Disallow: /imode/search?
  52. Disallow: /jsky?
  53. Disallow: /jsky/?
  54. Disallow: /jsky/search?
  55. Disallow: /pda?
  56. Disallow: /pda/?
  57. Disallow: /pda/search?
  58. Disallow: /sprint_xhtml
  59. Disallow: /sprint_wml
  60. Disallow: /pqa
  61. Disallow: /palm
  62. Disallow: /gwt/
  63. Disallow: /purchases
  64. Disallow: /hws
  65. Disallow: /bsd?
  66. Disallow: /linux?
  67. Disallow: /mac?
  68. Disallow: /microsoft?
  69. Disallow: /unclesam?
  70. Disallow: /answers/search?q=
  71. Disallow: /local?
  72. Disallow: /local_url
  73. Disallow: /shihui?
  74. Disallow: /shihui/
  75. Disallow: /froogle?
  76. Disallow: /products?
  77. Disallow: /products/
  78. Disallow: /froogle_
  79. Disallow: /product_
  80. Disallow: /products_
  81. Disallow: /products;
  82. Disallow: /print
  83. Disallow: /books/
  84. Disallow: /bkshp?*q=*
  85. Disallow: /books?*q=*
  86. Disallow: /books?*output=*
  87. Disallow: /books?*pg=*
  88. Disallow: /books?*jtp=*
  89. Disallow: /books?*jscmd=*
  90. Disallow: /books?*buy=*
  91. Disallow: /books?*zoom=*
  92. Allow: /books?*q=related:*
  93. Allow: /books?*q=editions:*
  94. Allow: /books?*q=subject:*
  95. Allow: /books/about
  96. Allow: /booksrightsholders
  97. Allow: /books?*zoom=1*
  98. Allow: /books?*zoom=5*
  99. Disallow: /ebooks/
  100. Disallow: /ebooks?*q=*
  101. Disallow: /ebooks?*output=*
  102. Disallow: /ebooks?*pg=*
  103. Disallow: /ebooks?*jscmd=*
  104. Disallow: /ebooks?*buy=*
  105. Disallow: /ebooks?*zoom=*
  106. Allow: /ebooks?*q=related:*
  107. Allow: /ebooks?*q=editions:*
  108. Allow: /ebooks?*q=subject:*
  109. Allow: /ebooks?*zoom=1*
  110. Allow: /ebooks?*zoom=5*
  111. Disallow: /patents?
  112. Disallow: /patents/download/
  113. Disallow: /patents/pdf/
  114. Disallow: /patents/related/
  115. Disallow: /scholar
  116. Disallow: /citations?
  117. Allow: /citations?user=
  118. Allow: /citations?view_op=new_profile
  119. Allow: /citations?view_op=top_venues
  120. Disallow: /complete
  121. Disallow: /s?
  122. Disallow: /sponsoredlinks
  123. Disallow: /videosearch?
  124. Disallow: /videopreview?
  125. Disallow: /videoprograminfo?
  126. Allow: /maps?hq=http://maps.google.com/help/maps/directions/biking/mapleft.kml&ie=UTF8&ll=37.687624,-122.319717&spn=0.346132,0.727158&z=11&lci=bike&dirflg=b&f=d
  127. Allow: /maps/api/js?
  128. Disallow: /maps?
  129. Disallow: /mapstt?
  130. Disallow: /mapslt?
  131. Disallow: /maps/stk/
  132. Disallow: /maps/br?
  133. Disallow: /mapabcpoi?
  134. Disallow: /maphp?
  135. Disallow: /mapprint?
  136. Disallow: /maps/api/js/
  137. Disallow: /maps/api/staticmap?
  138. Disallow: /mld?
  139. Disallow: /staticmap?
  140. Disallow: /places/
  141. Allow: /places/$
  142. Disallow: /maps/preview
  143. Disallow: /maps/place
  144. Disallow: /help/maps/streetview/partners/welcome/
  145. Disallow: /help/maps/indoormaps/partners/
  146. Disallow: /lochp?
  147. Disallow: /center
  148. Disallow: /ie?
  149. Disallow: /sms/demo?
  150. Disallow: /katrina?
  151. Disallow: /blogsearch?
  152. Disallow: /blogsearch/
  153. Disallow: /blogsearch_feeds
  154. Disallow: /advanced_blog_search
  155. Disallow: /uds/
  156. Disallow: /chart?
  157. Disallow: /transit?
  158. Disallow: /mbd?
  159. Disallow: /extern_js/
  160. Disallow: /xjs/
  161. Disallow: /calendar/feeds/
  162. Disallow: /calendar/ical/
  163. Disallow: /cl2/feeds/
  164. Disallow: /cl2/ical/
  165. Disallow: /coop/directory
  166. Disallow: /coop/manage
  167. Disallow: /trends?
  168. Disallow: /trends/music?
  169. Disallow: /trends/hottrends?
  170. Disallow: /trends/viz?
  171. Disallow: /notebook/search?
  172. Disallow: /musica
  173. Disallow: /musicad
  174. Disallow: /musicas
  175. Disallow: /musicl
  176. Disallow: /musics
  177. Disallow: /musicsearch
  178. Disallow: /musicsp
  179. Disallow: /musiclp
  180. Disallow: /browsersync
  181. Disallow: /call
  182. Disallow: /archivesearch?
  183. Disallow: /archivesearch/url
  184. Disallow: /archivesearch/advanced_search
  185. Disallow: /base/reportbadoffer
  186. Disallow: /urchin_test/
  187. Disallow: /movies?
  188. Disallow: /codesearch?
  189. Disallow: /codesearch/feeds/search?
  190. Disallow: /wapsearch?
  191. Disallow: /safebrowsing
  192. Allow: /safebrowsing/diagnostic
  193. Allow: /safebrowsing/report_badware/
  194. Allow: /safebrowsing/report_error/
  195. Allow: /safebrowsing/report_phish/
  196. Disallow: /reviews/search?
  197. Disallow: /orkut/albums
  198. Allow: /jsapi
  199. Disallow: /views?
  200. Disallow: /c/
  201. Disallow: /cbk
  202. Allow: /cbk?output=tile&cb_client=maps_sv
  203. Disallow: /recharge/dashboard/car
  204. Disallow: /recharge/dashboard/static/
  205. Disallow: /translate_a/
  206. Disallow: /translate_c
  207. Disallow: /translate_f
  208. Disallow: /translate_static/
  209. Disallow: /translate_suggestion
  210. Disallow: /profiles/me
  211. Allow: /profiles
  212. Disallow: /s2/profiles/me
  213. Allow: /s2/profiles
  214. Allow: /s2/photos
  215. Allow: /s2/static
  216. Disallow: /s2
  217. Allow: /s2/search/social
  218. Disallow: /transconsole/portal/
  219. Disallow: /gcc/
  220. Disallow: /aclk
  221. Disallow: /cse?
  222. Disallow: /cse/home
  223. Disallow: /cse/panel
  224. Disallow: /cse/manage
  225. Disallow: /tbproxy/
  226. Disallow: /imesync/
  227. Disallow: /shenghuo/search?
  228. Disallow: /support/forum/search?
  229. Disallow: /reviews/polls/
  230. Disallow: /hosted/images/
  231. Disallow: /ppob/?
  232. Disallow: /ppob?
  233. Disallow: /ig/add?
  234. Disallow: /adwordsresellers
  235. Disallow: /accounts/o8
  236. Allow: /accounts/o8/id
  237. Disallow: /topicsearch?q=
  238. Disallow: /xfx7/
  239. Disallow: /squared/api
  240. Disallow: /squared/search
  241. Disallow: /squared/table
  242. Disallow: /toolkit/
  243. Allow: /toolkit/*.html
  244. Disallow: /globalmarketfinder/
  245. Allow: /globalmarketfinder/*.html
  246. Disallow: /qnasearch?
  247. Disallow: /app/updates
  248. Disallow: /sidewiki/entry/
  249. Disallow: /quality_form?
  250. Disallow: /labs/popgadget/search
  251. Disallow: /buzz/post
  252. Disallow: /compressiontest/
  253. Disallow: /analytics/reporting/
  254. Disallow: /analytics/admin/
  255. Disallow: /analytics/web/
  256. Disallow: /analytics/feeds/
  257. Disallow: /analytics/settings/
  258. Disallow: /alerts/
  259. Disallow: /ads/search
  260. Disallow: /phone/compare/?
  261. Allow: /alerts/manage
  262. Allow: /alerts/remove
  263. Disallow: /travel/clk
  264. Disallow: /hotelfinder/rpc
  265. Disallow: /hotels/rpc
  266. Disallow: /flights/rpc
  267. Disallow: /commercesearch/services/
  268. Disallow: /evaluation/
  269. Disallow: /chrome/browser/mobile/tour
  270. Disallow: /compare/*/apply*
  271. Disallow: /forms/perks/
  272. Disallow: /baraza/*/search
  273. Disallow: /baraza/*/report
  274. Disallow: /shopping/suppliers/search
  275. Disallow: /ct/
  276. Disallow: /edu/cs4hs/
  277. Disallow: /trustedstores/s/
  278. Disallow: /trustedstores/tm2
  279. Disallow: /trustedstores/verify
  280. Disallow: /adwords/proposal
  281. Sitemap: http://www.gstatic.com/culturalinstitute/sitemaps/www_google_com_culturalinstitute/sitemap-index.xml
  282. Sitemap: http://www.google.com/hostednews/sitemap_index.xml
  283. Sitemap: http://www.google.com/sitemaps_webmasters.xml
  284. Sitemap: http://www.gstatic.com/sitemaps/websearch_hreflang/sitemap_index.xml
  285. Sitemap: http://www.google.com/ventures/sitemap_ventures.xml
  286. Sitemap: http://www.gstatic.com/dictionary/static/sitemaps/sitemap_index.xml
  287. Sitemap: http://www.gstatic.com/earth/gallery/sitemaps/sitemap.xml
  288. Sitemap: http://www.gstatic.com/s2/sitemaps/profiles-sitemap.xml
  289. Sitemap: http://www.gstatic.com/trends/websites/sitemaps/sitemapindex.xml
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!