|
| 1 | +Ace Explorer |
| 2 | +Apple iPhone v1.1.4 CoreMedia v1.0.0.4A102 |
| 3 | +Apple-PubSub/65.1.1 |
| 4 | +ArabyBot (compatible; Mozilla/5.0; GoogleBot; FAST Crawler 6.4; http://www.araby.com;) |
| 5 | +curl/7.10.x (i386-redhat-linux-gnu) libcurl/7.10.x OpenSSL/0.9.7a ipv6 zlib/1.1.4 |
| 6 | +curl/7.7.x (i386--freebsd4.3) libcurl 7.7.x (SSL 0.9.6) (ipv6 enabled) |
| 7 | +curl/7.8 (i686-pc-linux-gnu) libcurl 7.8 (OpenSSL 0.9.6) |
| 8 | +curl/7.9.x (win32) libcurl 7.9.x |
| 9 | +del.icio.us-thumbnails/1.0 Mozilla/5.0 (compatible; Konqueror/3.4; FreeBSD) KHTML/3.4.2 (like Gecko) |
| 10 | + |
| 11 | + |
| 12 | +libcurl-agent/1.0 |
| 13 | +Microsoft_Internet_Explorer_5.00.438 ( [email protected]) |
| 14 | +Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Deepnet Explorer) |
| 15 | +Mozilla/5.0 |
| 16 | +Mozilla/5.0 (+http://www.eurekster.com/mammoth) Mammoth/0.1 |
| 17 | +Mozilla/5.0 (+http://www.sli-systems.com/) Mammoth/0.1 |
| 18 | +Mozilla/5.0 (Clustered-Search-Bot/1.0; [email protected]; http://www.clush.com/) |
| 19 | +Mozilla/5.0 (compatible) GM RSS Panel X |
| 20 | +Mozilla/5.0 (compatible; +http://www.evri.com/evrinid) |
| 21 | +Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/spider.html;) Gecko/2008032620 |
| 22 | +Mozilla/5.0 (compatible; Abonti/0.8 - http://www.abonti.com) |
| 23 | +Mozilla/5.0 (compatible; aiHitBot/1.0; +http://www.aihit.com/) |
| 24 | +Mozilla/5.0 (compatible; AnsearchBot/1.x; +http://www.ansearch.com.au/) |
| 25 | +Mozilla/5.0 (compatible; archive.org_bot/1.10.0 +http://www.loc.gov/minerva/crawl.html) |
| 26 | +Mozilla/5.0 (compatible; archive.org_bot/1.13.1x http://crawler.archive.org) |
| 27 | +Mozilla/5.0 (compatible; archive.org_bot/1.5.0-200506132127 http://crawler.archive.org) Hurricane Katrina |
| 28 | +Mozilla/5.0 (compatible; Ask Jeeves/Teoma; http://about.ask.com/en/docs/about/webmasters.shtml) |
| 29 | +Mozilla/5.0 (compatible; BanBots/2.0b; Fetch; +http://www.banbots.com) |
| 30 | +Mozilla/5.0 (compatible; BecomeBot/1.23; http://www.become.com/webmasters.html) |
| 31 | +Mozilla/5.0 (compatible; BecomeBot/1.xx; MSIE 6.0 compatible; http://www.become.com/webmasters.html) |
| 32 | +Mozilla/5.0 (compatible; BecomeBot/2.0beta; http://www.become.com/webmasters.html) |
| 33 | +Mozilla/5.0 (compatible; BecomeBot/2.x; MSIE 6.0 compatible; http://www.become.com/site_owners.html) |
| 34 | +Mozilla/5.0 (compatible; BecomeJPBot/2.3; MSIE 6.0 compatible; +http://www.become.co.jp/site_owners.html) |
| 35 | +Mozilla/5.0 (compatible; BlogRefsBot/0.1; http://www.blogrefs.com/about/bloggers) |
| 36 | +Mozilla/5.0 (compatible; Bot; +http://pressemitteilung.ws/spamfilter |
| 37 | +Mozilla/5.0 (compatible; BuzzRankingBot/1.0; +http://www.buzzrankingbot.com/) |
| 38 | +Mozilla/5.0 (compatible; Charlotte/1.0b; [email protected]) |
| 39 | +Mozilla/5.0 (compatible; Charlotte/1.0b; http://www.searchme.com/support/) |
| 40 | +Mozilla/5.0 (compatible; Crawling jpeg; http://www.yama.info.waseda.ac.jp) |
| 41 | +Mozilla/5.0 (compatible; Custo 3 (Netwu.com); Windows NT 5.1) |
| 42 | +Mozilla/5.0 (compatible; de/1.13.2 +http://www.de.com) |
| 43 | +Mozilla/5.0 (compatible; Diffbot/0.1; +http://www.diffbot.com) |
| 44 | +Mozilla/5.0 (compatible; DNS-Digger-Explorer/1.0; +http://www.dnsdigger.com) |
| 45 | +Mozilla/5.0 (compatible; DNS-Digger/1.0; +http://www.dnsdigger.com) |
| 46 | +Mozilla/5.0 (compatible; EARTHCOM.info/2.01; http://www.earthcom.info) |
| 47 | +Mozilla/5.0 (compatible; EARTHCOM/2.2; +http://enter4u.eu) |
| 48 | +Mozilla/5.0 (compatible; egothor/8.0g; +http://ego.ms.mff.cuni.cz/) |
| 49 | +Mozilla/5.0 (compatible; Exabot Test/3.0; +http://www.exabot.com/go/robot) |
| 50 | +Mozilla/5.0 (compatible; FatBot 2.0; http://www.thefind.com/main/CrawlerFAQs.fhtml) |
| 51 | +Mozilla/5.0 (compatible; Galbot/1.0; +http://www.galbot.com/bot.html) |
| 52 | +Mozilla/5.0 (compatible; Google Desktop) Paros/3.2.12 |
| 53 | +Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html) |
| 54 | +Mozilla/5.0 (compatible; heritrix/1.10.2 +http://i.stanford.edu/) |
| 55 | +Mozilla/5.0 (compatible; heritrix/1.12.1 +http://newstin.com/) |
| 56 | +Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.page-store.com) |
| 57 | +Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.page-store.com) [email: [email protected]] |
| 58 | +Mozilla/5.0 (compatible; heritrix/1.4.0 +http://www.chepi.net) |
| 59 | +Mozilla/5.0 (compatible; heritrix/1.4t http://www.truveo.com/) |
| 60 | +Mozilla/5.0 (compatible; heritrix/1.5.0 http://www.l3s.de/~kohlschuetter/projects/crawling/) |
| 61 | +Mozilla/5.0 (compatible; heritrix/1.5.0-200506231921 http://pandora.nla.gov.au/crawl.html) |
| 62 | +Mozilla/5.0 (compatible; heritrix/1.6.0 http://www.worio.com/) |
| 63 | +Mozilla/5.0 (compatible; heritrix/1.7.0 +http://www.greaterera.com/) |
| 64 | +Mozilla/5.0 (compatible; Heritrix/1.8.0 http://www.hanzoarchives.com) |
| 65 | +Mozilla/5.0 (compatible; heritrix/1.x.x +http://www.accelobot.com) |
| 66 | +Mozilla/5.0 (compatible; heritrix/2.0.0-RC1 +http://www.aol.com) |
| 67 | +Mozilla/5.0 (compatible; Hermit Search. Com; +http://www.hermitsearch.com) |
| 68 | +Mozilla/5.0 (compatible; http://www.IsMySiteUp.Net/bot/ ) |
| 69 | +Mozilla/5.0 (compatible; http://www.UptimeAuditor.com/bot/ ) |
| 70 | +Mozilla/5.0 (compatible; HyperixScoop/1.3; +http://www.hyperix.com) |
| 71 | +Mozilla/5.0 (compatible; iaskspider/1.0; MSIE 6.0) |
| 72 | +Mozilla/5.0 (compatible; IDBot/1.0; +http://www.id-search.org/bot.html) |
| 73 | +Mozilla/5.0 (compatible; InterseekWeb/3.x) |
| 74 | +Mozilla/5.0 (compatible; Jim +http://www.hanzoarchives.com) |
| 75 | +Mozilla/5.0 (compatible; Konqueror/2.0.1; X11); Supports MD5-Digest; Supports gzip encoding |
| 76 | +Mozilla/5.0 (compatible; Konqueror/2.1.1; X11) |
| 77 | +Mozilla/5.0 (compatible; Konqueror/2.2.2) |
| 78 | +Mozilla/5.0 (compatible; Konqueror/2.2.2; Linux 2.4.14-xfs; X11; i686) |
| 79 | +Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Exabot-Thumbnails) |
| 80 | +Mozilla/5.0 (compatible; LemSpider 0.1) |
| 81 | +Mozilla/5.0 (compatible; LinksManager.com_bot http://linksmanager.com/linkchecker.html) |
| 82 | +Mozilla/5.0 (compatible; LinkStash Bookmark Manager; http://www.xrayz.co.uk/) |
| 83 | +Mozilla/5.0 (compatible; MojeekBot/2.0; http://www.mojeek.com/bot.html) |
| 84 | +Mozilla/5.0 (compatible; MOSBookmarks/v2.6-Plus; Link Checker) |
| 85 | +Mozilla/5.0 (compatible; MSIE 6.0; Podtech Network; [email protected]) |
| 86 | +Mozilla/5.0 (compatible; OnetSzukaj/5.0; http://szukaj.onet.pl) |
| 87 | +Mozilla/5.0 (compatible; PagestackerBot; http://www.pagestacker.com) |
| 88 | +Mozilla/5.0 (compatible; PalmeraBot; http://www.links24h.com/help/palmera) Version 0.001 |
| 89 | +Mozilla/5.0 (compatible; PEAR HTTP_Request class; http://feed.moo.jp/) |
| 90 | +Mozilla/5.0 (compatible; Phonifier; +http://www.phonifier.com) |
| 91 | +Mozilla/5.0 (compatible; pmoz.info ODP link checker; +http://pmoz.info/doc/botinfo.htm) |
| 92 | +Mozilla/5.0 (compatible; pogodak.ba/3.x) |
| 93 | +Mozilla/5.0 (compatible; Pogodak.hr/3.1) |
| 94 | +Mozilla/5.0 (compatible; Proximic crawler; +http://www.proximic.com/en/about-us/contact-us.html) |
| 95 | +Mozilla/5.0 (compatible; PWeBot/3.1; http://www.programacionweb.net/robot.php) |
| 96 | +Mozilla/5.0 (compatible; Quantcastbot/1.0; www.quantcast.com) |
| 97 | +Mozilla/5.0 (compatible; robtexbot/1.0; http://www.robtex.com/ ) |
| 98 | +Mozilla/5.0 (compatible; ScoutJet; +http://www.scoutjet.com/) |
| 99 | +Mozilla/5.0 (compatible; Scrubby/2.2; http://www.scrubtheweb.com/) |
| 100 | +Mozilla/5.0 (compatible; ShunixBot/1.x.x +http://www.shunix.com/robot.htm) |
| 101 | +Mozilla/5.0 (compatible; ShunixBot/1.x; http://www.shunix.com/bot.htm) |
| 102 | +Mozilla/5.0 (compatible; SkreemRBot +http://skreemr.com) |
| 103 | +Mozilla/5.0 (compatible; SnapPreviewBot; en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 |
| 104 | +Mozilla/5.0 (compatible; SpurlBot/0.2) |
| 105 | +Mozilla/5.0 (compatible; SummizeBot +http://www.summize.com) |
| 106 | +Mozilla/5.0 (compatible; SYCLIKControl/LinkChecker;) |
| 107 | +Mozilla/5.0 (compatible; Synoobot/0.9; http://www.synoo.com/search/bot.html) |
| 108 | +Mozilla/5.0 (compatible; Theophrastus/x.x; http://users.cs.cf.ac.uk/N.A.Smith/theophrastus.php) |
| 109 | +Mozilla/5.0 (compatible; TridentSpider/3.1) |
| 110 | +Mozilla/5.0 (compatible; Vagabondo/2.1; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/) |
| 111 | +Mozilla/5.0 (compatible; Webduniabot/1.0; +http://search.webdunia.com/bot.aspx) |
| 112 | +Mozilla/5.0 (compatible; Windows NT 5.0; phpwebbrainBot/0.1 - http://www.monsterli.ch/phpwebbrain/) |
| 113 | +Mozilla/5.0 (compatible; worio bot heritrix/1.10.0 +http://worio.com) |
| 114 | +Mozilla/5.0 (compatible; WoW Lemmings Kathune/2.0;http://www.wowlemmings.com/kathune.html) |
| 115 | +Mozilla/5.0 (compatible; XTbot/1.0v; +http://www.externaltest.com) |
| 116 | +Mozilla/5.0 (compatible; Yahoo! DE Slurp; http://help.yahoo.com/help/us/ysearch/slurp) |
| 117 | +Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html) |
| 118 | +Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) |
| 119 | +Mozilla/5.0 (compatible; YesupBot/1.0; +http://www.yesup.net/bot.html) |
| 120 | +Mozilla/5.0 (compatible; Yoono; http://www.yoono.com/) |
| 121 | +Mozilla/5.0 (compatible; YoudaoBot/1.0; http://www.youdao.com/help/webmaster/spider/; ) |
| 122 | +Mozilla/5.0 (compatible; Zenbot/1.3; +http://zen.co.za/webmasters/) |
| 123 | +Mozilla/5.0 (compatible; zermelo +http://www.powerset.com) [email: [email protected], [email protected]] |
| 124 | +Mozilla/5.0 (compatible;archive.org_bot/1.7.1; collectionId=316; Archive-It; +http://www.archive-it.org) |
| 125 | +Mozilla/5.0 (compatible;archive.org_bot/heritrix-1.9.0-200608171144 +http://pandora.nla.gov.au/crawl.html) |
| 126 | +Mozilla/5.0 (compatible;FindITAnswersbot/1.0;+http://search.it-influentials.com/bot.htm) |
| 127 | +Mozilla/5.0 (compatible;MAINSEEK_BOT) |
| 128 | +Mozilla/5.0 (Gecko/20070310 Mozshot/0.0.20070628; http://mozshot.nemui.org/) |
| 129 | +Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5 |
| 130 | +Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.0.1) Gecko/20021219 Chimera/0.6 |
| 131 | +Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.0.1) Gecko/20030306 Camino/0.7 |
| 132 | +Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US) AppleWebKit/xx (KHTML like Gecko) OmniWeb/v5xx.xx |
| 133 | +Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/xxx.x (KHTML like Gecko) Safari/12x.x |
| 134 | +Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 |
| 135 | + |
| 136 | +Mozilla/5.0 (Sage) |
| 137 | +Mozilla/5.0 (Slurp/cat; [email protected]; http://www.inktomi.com/slurp.html) |
| 138 | +Mozilla/5.0 (Slurp/si; [email protected]; http://www.inktomi.com/slurp.html) |
| 139 | +Mozilla/5.0 (SunOS 5.8 sun4u; U) Opera 5.0 [en] |
| 140 | +Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html) |
| 141 | +Mozilla/5.0 (Version: xxxx Type:xx) |
| 142 | + |
| 143 | +Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 |
| 144 | +Mozilla/5.0 (Windows; U; Win98; en-US; rv:x.xx) Gecko/20030423 Firebird Browser/0.6 |
| 145 | +Mozilla/5.0 (Windows; U; Win9x; en; Stable) Gecko/20020911 Beonex/0.8.1-stable |
| 146 | +Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.7) NimbleCrawler 1.11 obeys UserAgent NimbleCrawler For problems contact: crawler_at_dataalchemy.com |
| 147 | +Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.2.153.1 Safari/525.19 |
| 148 | +Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.5) Gecko/20060731 Firefox/1.5.0.5 Flock/0.7.4.1 |
| 149 | +Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.4/Megaupload x.0 |
| 150 | +Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008092215 Firefox/3.0.1 Orca/1.1 beta 3 |
| 151 | +Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:x.x.x) Gecko/20041107 Firefox/x.x |
| 152 | +Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:x.xx) Gecko/20030504 Mozilla Firebird/0.6 |
| 153 | +Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:x.xxx) Gecko/20041027 Mnenhy/0.6.0.104 |
| 154 | +Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 ( [email protected]) |
| 155 | +Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 |
| 156 | +Mozilla/5.0 (Windows; U;XMPP Tiscali Communicator v.10.0.1; Windows NT 5.1; it; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 |
| 157 | +Mozilla/5.0 (Windows;) NimbleCrawler 1.12 obeys UserAgent NimbleCrawler For problems contact: crawler@health |
| 158 | +Mozilla/5.0 (Windows;) NimbleCrawler 1.12 obeys UserAgent NimbleCrawler For problems contact: [email protected] |
| 159 | +Mozilla/5.0 (X11; Linux i686; U;rv: 1.7.13) Gecko/20070322 Kazehakase/0.4.4.1 |
| 160 | +Mozilla/5.0 (X11; U; Linux 2.4.2-2 i586; en-US; m18) Gecko/20010131 Netscape6/6.01 |
| 161 | +Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:1.8.0.2) Gecko/20060309 SeaMonkey/1.0 |
| 162 | +Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.7.6) Gecko/20050405 Epiphany/1.6.1 (Ubuntu) (Ubuntu package 1.0.2) |
| 163 | +Mozilla/5.0 (X11; U; Linux i686; en-US; Nautilus/1.0Final) Gecko/20020408 |
| 164 | +Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.3) Gecko/20010801 |
| 165 | +Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1; aggregator:Spinn3r (Spinn3r 3.1); http://spinn3r.com/robot) Gecko/20021130 |
| 166 | +Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2b) Gecko/20021007 Phoenix/0.3 |
| 167 | +Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040413 Epiphany/1.2.1 |
| 168 | +Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 SnapPreviewBot |
| 169 | +Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061129 BonEcho/2.0 |
| 170 | +Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.1) Gecko/20061205 Iceweasel/2.0.0.1 (Debian-2.0.0.1+dfsg-2) |
| 171 | +Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9a8) Gecko/2007100619 GranParadiso/3.0a8 |
| 172 | +Mozilla/5.0 Galeon/1.0.2 (X11; Linux i686; U;) Gecko/20011224 |
| 173 | +Mozilla/5.0 gURLChecker/0.x.x (Linux) |
| 174 | +Mozilla/5.0 URL-Spider |
| 175 | +Mozilla/5.0 usww.com-Spider-for-w8.net |
| 176 | + |
| 177 | +Mozilla/5.0 whoiam [http://www.axxus.de/] |
| 178 | +Mozilla/5.0 [en] (compatible; Gulper Web Bot 0.2.4 www.ecsl.cs.sunysb.edu/~maxim/cgi-bin/Link/GulperBot) |
| 179 | +Offline Explorer 1.* |
| 180 | +OmniExplorer_Bot/1.0x (+http://www.omni-explorer.com) Internet CategorizerOmniExplorer http://www.omni-explorer.com/ car & shopping search (64.62.175.xxx) |
| 181 | +OmniExplorer_Bot/1.0x (+http://www.omni-explorer.com) Job Crawler |
| 182 | +OmniExplorer_Bot/1.1x (+http://www.omni-explorer.com) Torrent Crawler |
| 183 | +OmniExplorer_Bot/x.xx (+http://www.omni-explorer.com) WorldIndexer |
| 184 | +Website Explorer/0.9.x.x |
| 185 | +zermelo Mozilla/5.0 compatible; heritrix/1.12.1 (+http://www.powerset.com) [email: [email protected],email: [email protected]] |
| 186 | +Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (FM Scene 4.6.1) |
| 187 | +Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) (Prevx 3.0.5) |
0 commit comments