# robots.txt to ask robots not to index non-unique or sensitive information. User-agent: * Disallow: /doc/ Disallow: /cgi-bin/