Sitemap: http://www.3fbc.com/sitemap.xml User-agent: Twiceler Disallow: / User-agent: ia_archiver Disallow: / User-agent: psbot Disallow: / User-agent: yahoo-blogs/v3.9 Disallow: / #fake ms bots User-agent: Microsoft URL Control Disallow: / User-agent: microsoft.url Disallow: / # blocks other common spam bots User-agent: WebCopier v.2.2 Disallow: / User-agent: Custom Bot/Robot #20 Disallow: / User-agent: RPT-HTTPClient Disallow: / User-agent: OutfoxBot Disallow: / User-agent: bright.net caching robot Disallow: / User-agent: iaea Disallow: / User-agent: Test.Com Disallow: / User-agent: Test Disallow: / User-agent: Indy Disallow: / User-agent: BecomeBot Disallow: / User-agent: Program Shareware Disallow: / User-agent: libwww-perl Disallow: / User-agent: Larbin Disallow: / User-agent: WinHttp.WinHttpRequest Disallow: / User-agent: Jakarta Commons-HttpClient Disallow: / User-agent: Advanced Email Extractor Disallow: / User-agent: Netcraft Web Server Survey Disallow: / User-agent: Python-urllib Disallow: / User-agent: DTS Agent Disallow: / User-agent: COMBINE Disallow: / User-agent: IP*Works! HTTP/S Component Disallow: / User-agent: ISC Systems iRc Search Disallow: / User-agent: LinksManager.com_bot Disallow: / User-agent: lwp-trivial Disallow: / User-agent: semanticdiscovery Disallow: / User-agent: Fetch API Request Disallow: / User-agent: Faxobot Disallow: / User-agent: INDEXU Spider Link Checker Disallow: / User-agent: e-collector Disallow: / User-agent: linkwalker Disallow: / User-agent: LinksManager Disallow: / User-agent: T-H-U-N-D-E-R-S-T-O-N-E Disallow: / User-agent: Microsoft Data Access Disallow: / User-agent: NetResearchServer Disallow: / User-agent: Asterias Crawler Disallow: / User-agent: Zeus Disallow: / User-agent: Stupid email harvester Disallow: / User-agent: Shim-Crawler Disallow: / User-agent: GaisBot Disallow: / User-agent: TurnitinBot Disallow: / User-agent: SurveyBot Disallow: / User-agent: Java1.x.x Disallow: / User-agent: AnswerBus Disallow: / User-agent: VoilaBot Disallow: / User-agent: IRLbot Disallow: / User-agent: Xenu Link Sleuth Disallow: / User-agent: CherryPicker Disallow: / User-agent: Crescent Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: ExtractorPro Disallow: / User-agent: NICErsPRO Disallow: / User-agent: Website\ eXtractor Disallow: / User-agent: Baiduspider+ Disallow: / User-agent: DepSpid Disallow: / User-agent: Charlotte Disallow: / User-agent: Exabot/2.0 Disallow: / User-agent: Kyluka Disallow: / User-agent: MJ12bot Disallow: / User-agent: multicrawler Disallow: / User-agent: Nutch Disallow: / User-agent: Speedy Spider Disallow: / User-agent: TheRarestParser Disallow: / User-agent: NEWT Disallow: / User-agent: * Disallow: /iba Disallow: /links Crawl-Delay: 10