# /robots.txt file for http://webcrawler.com/ # mail webmaster@webcrawler.com for constructive criticism # For help see http://www.robotstxt.org/wc/exclusion-admin.html User-agent: * Disallow: /shutdown/choice_process.php Disallow: /shutdown/choose.php