# Google crawler (see http://www.google.com/support/webmasters/) User-Agent: Googlebot Disallow: /private/ Disallow: /admin/ Disallow: /a4j/ # Yahoo crawler (see http://help.yahoo.com/l/us/yahoo/search/webcrawler/) User-agent: Slurp Disallow: /private/ Disallow: /admin/ Disallow: /a4j/ # All the rest User-agent: * Disallow: /private/ Disallow: /admin/ Disallow: /a4j/ Crawl-delay: 120