# robots.txt file for http://*.hoyre.no # please contact webmaster@hoyre.no with questions or concerns # "Any empty [Disallow] value, indicates that all URLs can be retrieved. # At least one Disallow field needs to be present in a record." # Define for Mediapartners @ google User-agent: Mediapartners-Google* Crawl-delay: 1 Disallow: /search Disallow: //search # Define for Google User-agent: Googlebot Crawl-delay: 1 Disallow: /search Disallow: //search # Define for 80legs User-agent: 008 Crawl-delay: 4 Disallow: /search Disallow: //search # Define for Dotbot User-agent: dotbot Crawl-delay: 4 Disallow: /search Disallow: //search # Define for Yahoo! Slurp User-agent: Slurp Disallow: /search Disallow: //search Crawl-delay: 2 # Define for Fast User-agent: FAST Disallow: /search Disallow: //search Crawl-delay: 3 # Define for MSN User-agent: msnbot Disallow: /search Disallow: //search Crawl-delay: 3 # Define for WebTrends User-agent: WebTrends Disallow: /search Disallow: //search Crawl-delay: 3 # Define for all User-agent: * Disallow: /search Disallow: //search Crawl-delay: 4