# Robot Exclusion File -- robots.txt User-agent: * Disallow: /__en/ Disallow: /___dt/ Disallow: /flags/ User-agent: SurveyBot Disallow: / User-agent: WebsiteWiki Disallow: / User-agent: ShopWiki Disallow: / User-agent: Jakarta Commons-HttpClient Disallow: / User-Agent: AboutUsBot Disallow: / User-agent: Semager Disallow: /