User-agent: * Disallow: Sitemap: http://www.villainesque.com/sitemap.xml # IRLbot crawls too much and servers no purpose # http://irl.cs.tamu.edu/crawler/ User-agent: IRLbot Disallow: / # oBot crawls too much and servers no purpose User-agent: oBot Disallow: /