# sample robots.txt file for limiting robot visits # More info at: # http://info.webcrawler.com/mak/projects/robots/robots.html # mail webmaster@webcrawler.com for constructive criticism User-agent: * Disallow: /logs Disallow: /public Disallow: /images Disallow: /stats Disallow: /cath.html