User-agent: * Disallow: #Info on format of robots.txt file at #http://info.webcrawler.com/mak/projects/robots/faq.html#log