# Tells Scanning Robots Where They Are And Are Not Welcome # User-agent: can also specify by name; "*" is for everyone # Disallow: if this matches first part of requested path, forget it #User-agent: MSFT #Disallow: / #User-agent: msnbot.msn.com #Disallow: / User-agent: cuill.com Disallow: / User-agent: TWICELER Disallow: / User-agent: akam.net Disallow: / User-agent: * Disallow: /stats/ Disallow: /_db_backups/ Disallow: /_script/ Disallow: /images/ Disallow: .htaccess Disallow: .htpasswd Disallow: search_results.php Disallow: /_members/