User-agent: ABACHOBot Disallow: / User-agent: Accoona-AI-Agent Disallow: / User-agent: AnyApexBot Disallow: / User-agent: Arachmo Disallow: / User-agent: B-l-i-t-z-B-O-T Disallow: / User-agent: Baiduspider Disallow: / User-agent: BecomeBot Disallow: / User-agent: Bimbot Disallow: / User-agent: BlitzBOT Disallow: / User-agent: boitho.com-dc Disallow: / User-agent: boitho.com-robot Disallow: / User-agent: btbot Disallow: / User-agent: Cerberian Drtrs Disallow: / User-agent: Charlotte Disallow: / User-agent: ConveraCrawler Disallow: / User-agent: cosmos Disallow: / User-agent: DataparkSearch Disallow: / User-agent: DiamondBot Disallow: / User-agent: Discobot Disallow: / User-agent: EmeraldShield.com WebBot Disallow: / User-agent: envolk[ITS]spider Disallow: / User-agent: EsperanzaBot Disallow: / User-agent: Exabot Disallow: / User-agent: FAST Enterprise Crawler Disallow: / User-agent: FAST-WebCrawler Disallow: / User-agent: FDSE robot Disallow: / User-agent: FindLinks Disallow: / User-agent: FurlBot Disallow: / User-agent: FyberSpider Disallow: / User-agent: g2crawler Disallow: / User-agent: Gaisbot Disallow: / User-agent: GalaxyBot Disallow: / User-agent: genieBot Disallow: / User-agent: Gigabot Disallow: / User-agent: Girafabot Disallow: / User-agent: hl_ftien_spider Disallow: / User-agent: htdig Disallow: / User-agent: ia_archiver Disallow: / User-agent: ichiro Disallow: / User-agent: IRLbot Disallow: / User-agent: IssueCrawler Disallow: / User-agent: Jyxobot Disallow: / User-agent: LapozzBot Disallow: / User-agent: Larbin Disallow: / User-agent: LinkWalker Disallow: / User-agent: lmspider Disallow: / User-agent: lwp-trivial Disallow: / User-agent: mabontland Disallow: / User-agent: magpie-crawler Disallow: / User-agent: Mediapartners-Google Disallow: / User-agent: MJ12bot Disallow: / User-agent: Mnogosearch Disallow: / User-agent: mogimogi Disallow: / User-agent: MojeekBot Disallow: / User-agent: Morning Paper Disallow: / User-agent: MSRBot Disallow: / User-agent: MVAClient Disallow: / User-agent: NetResearchServer Disallow: / User-agent: NG-Search Disallow: / User-agent: nicebot Disallow: / User-agent: noxtrumbot Disallow: / User-agent: Nusearch Spider Disallow: / User-agent: NutchCVS Disallow: / User-agent: obot Disallow: / User-agent: oegp Disallow: / User-agent: OmniExplorer_Bot Disallow: / User-agent: OOZBOT Disallow: / User-agent: Orbiter Disallow: / User-agent: PageBitesHyperBot Disallow: / User-agent: polybot Disallow: / User-agent: Pompos Disallow: / User-agent: Psbot Disallow: / User-agent: PycURL Disallow: / User-agent: RAMPyBot Disallow: / User-agent: RufusBot Disallow: / User-agent: SandCrawler Disallow: / User-agent: SBIder Disallow: / User-agent: Scrubby Disallow: / User-agent: SearchSight Disallow: / User-agent: Seekbot Disallow: / User-agent: semanticdiscovery Disallow: / User-agent: Sensis Web Crawler Disallow: / User-agent: SEOChat::Bot Disallow: / User-agent: Shim-Crawler Disallow: / User-agent: ShopWiki Disallow: / User-agent: Shoula robot Disallow: / User-agent: silk Disallow: / User-agent: Snappy Disallow: / User-agent: sogou spider Disallow: / User-agent: Speedy Spider Disallow: / User-agent: Sqworm Disallow: / User-agent: StackRambler Disallow: / User-agent: SurveyBot Disallow: / User-agent: SynooBot Disallow: / User-agent: Teoma Disallow: / User-agent: TerrawizBot Disallow: / User-agent: TheSuBot Disallow: / User-agent: Thumbnail.CZ robot Disallow: / User-agent: TinEye Disallow: / User-agent: TurnitinBot Disallow: / User-agent: updated Disallow: / User-agent: Vagabondo Disallow: / User-agent: VoilaBot Disallow: / User-agent: Mozilla Disallow: / User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (support.voilabot@orange-ftgroup.com) Disallow: / User-agent: Vortex Disallow: / User-agent: voyager Disallow: / User-agent: VYU2 Disallow: / User-agent: webcollage Disallow: / User-agent: Websquash.com Disallow: / User-agent: wf84 Disallow: / User-agent: WoFindeIch Robot Disallow: / User-agent: Xaldon_WebSpider Disallow: / User-agent: yacy Disallow: / User-agent: Yahoo! Slurp China Disallow: / User-agent: YahooSeeker-Testing Disallow: / User-agent: yoogliFetchAgent Disallow: / User-agent: Zao Disallow: / User-agent: Zealbot Disallow: / User-agent: zspider Disallow: / User-agent: ZyBorg Disallow: / User-agent: AbiLogicBot Disallow: / User-agent: Link Valet Disallow: / User-agent: Link Validity Check Disallow: / User-agent: LinksManager.com_bot Disallow: / User-agent: Mojoo Robot Disallow: / User-agent: Notifixious Disallow: / User-agent: online link validator Disallow: / User-agent: Ploetz + Zeller Disallow: / User-agent: Reciprocal Link System PRO Disallow: / User-agent: REL Link Checker Lite Disallow: / User-agent: SiteBar Disallow: / User-agent: Vivante Link Checker Disallow: / User-agent: W3C-checklink Disallow: / User-agent: Xenu Link Sleuth Disallow: / User-agent: Yandex/1.01.001 (compatible; Win16; I) Disallow: / User-agent: Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) Disallow: / User-agent: * Crawl-delay: 20