| 183 different robots | Hits | Bandwidth | Last visit |
|---|
| MSNBot | 5737057+88135 | 197.92 GB | 31 Dec 2009 - 20:59 |
| Googlebot | 4653608+3228 | 263.09 GB | 31 Dec 2009 - 20:59 |
| Yahoo Slurp | 3157882+14379 | 247.46 GB | 31 Dec 2009 - 20:59 |
| Unknown robot (identified by 'robot') | 1923516+4559 | 134.34 GB | 31 Dec 2009 - 19:37 |
| Unknown robot (identified by 'crawl') | 1126746+20664 | 75.69 GB | 31 Dec 2009 - 20:37 |
| Yandex bot | 1047471+1764 | 60.99 GB | 31 Dec 2009 - 20:59 |
| Voila | 980782+4324 | 70.99 GB | 31 Dec 2009 - 20:58 |
| HTTrack off-line browser | 616785+428 | 35.66 GB | 31 Dec 2009 - 11:59 |
| WebVac | 395939+43 | 56.04 GB | 17 Dec 2009 - 07:47 |
| twiceler | 320095+2598 | 32.77 GB | 31 Dec 2009 - 20:59 |
| Ultraseek | 258890+15382 | 35.04 GB | 31 Dec 2009 - 02:53 |
| bot[+:,\.\;\/\\-] | 245286+6990 | 26.77 GB | 31 Dec 2009 - 20:56 |
| MSNBot-media | 222318+26055 | 12.82 GB | 31 Dec 2009 - 20:50 |
| java | 226583+668 | 20.73 GB | 31 Dec 2009 - 20:55 |
| Ask | 191737+4380 | 17.99 GB | 31 Dec 2009 - 20:42 |
| Ocelli | 163670+97 | 8.61 GB | 07 Dec 2009 - 12:53 |
| Nutch | 150945+6285 | 27.22 GB | 31 Dec 2009 - 20:41 |
| Unknown robot (identified by empty user agent string) | 107063+13255 | 14.90 GB | 31 Dec 2009 - 20:41 |
| fast\senterprise\scrawler | 116401+210 | 16.66 GB | 14 May 2009 - 08:48 |
| Unknown robot (identified by hit on 'robots.txt') | 0+108151 | 36.71 MB | 31 Dec 2009 - 20:07 |
| Exabot | 102654+1971 | 8.92 GB | 31 Dec 2009 - 20:55 |
| BaiDuSpider | 72876+3278 | 10.60 GB | 31 Dec 2009 - 20:39 |
| \wbot[\/\-] | 66983+918 | 37.17 GB | 14 May 2009 - 07:19 |
| DoCoMo | 66812+291 | 7.40 GB | 31 Dec 2009 - 06:51 |
| Unknown robot (identified by 'spider') | 52492+5911 | 7.54 GB | 31 Dec 2009 - 20:04 |
| Speedy Spider | 38089+4785 | 2.12 GB | 31 Dec 2009 - 20:58 |
| psbot | 39999+2164 | 576.27 MB | 23 Dec 2009 - 11:45 |
| Turn It In | 41432+206 | 3.44 GB | 29 Dec 2009 - 14:36 |
| Custo | 34602+3082 | 4.93 GB | 31 Dec 2009 - 18:35 |
| W3C Link Checker | 20074+17585 | 958.91 MB | 31 Dec 2009 - 20:43 |
| larbin | 34534+2486 | 2.40 GB | 31 Dec 2009 - 16:36 |
| MJ12bot | 30928+430 | 4.82 GB | 31 Dec 2009 - 07:18 |
| ichiro | 30701+10 | 3.55 GB | 07 Nov 2009 - 13:14 |
| yacy | 29223+1131 | 7.97 GB | 30 Dec 2009 - 13:17 |
| Gaisbot | 27137+345 | 3.12 GB | 11 Dec 2009 - 22:34 |
| Heritrix | 25098+1997 | 1.70 GB | 31 Dec 2009 - 17:50 |
| Alexa (IA Archiver) | 22823+1834 | 2.09 GB | 31 Dec 2009 - 15:45 |
| [+:,\.\;\/\\-]bot | 23655+913 | 1.83 GB | 31 Dec 2009 - 19:00 |
| Python-urllib | 13163+8701 | 1.71 GB | 31 Dec 2009 - 20:58 |
| Vagabondo | 17374+609 | 1.41 GB | 28 Dec 2009 - 01:00 |
| Mail.ru bot | 15307+623 | 772.33 MB | 29 Dec 2009 - 03:03 |
| Yahoo-MMCrawler | 15401+13 | 81.11 MB | 21 Dec 2009 - 13:31 |
| yahoo!\sslurp\schina | 14042+1290 | 943.05 MB | 14 May 2009 - 12:41 |
| GigaBot | 11833+2727 | 3.12 GB | 02 Dec 2009 - 14:58 |
| InfoBot | 13228+73 | 3.13 GB | 31 Dec 2009 - 20:58 |
| BSpider | 10619+1555 | 1.31 GB | 31 Dec 2009 - 20:45 |
| WebCopy | 11792 | 3.12 MB | 16 Oct 2009 - 11:38 |
| StackRambler | 9810+139 | 137.48 MB | 29 Dec 2009 - 17:24 |
| OutfoxBot/YodaoBot | 6353+3567 | 247.24 MB | 31 Dec 2009 - 18:26 |
| Webdup | 9425 | 99.06 MB | 27 Dec 2009 - 07:45 |
| CFNetwork | 8920+1 | 4.19 GB | 30 Dec 2009 - 21:47 |
| Voyager | 8677+173 | 4.69 GB | 31 Dec 2009 - 20:52 |
| tencenttraveler | 5483+1 | 797.19 MB | 14 May 2009 - 07:40 |
| Jakarta commons-httpclient | 5033+256 | 956.85 MB | 30 Dec 2009 - 19:21 |
| MSIECrawler | 3945+611 | 430.13 MB | 26 Dec 2009 - 10:30 |
| Grub.org | 4070+105 | 61.23 MB | 19 Nov 2009 - 14:44 |
| Sunrise | 4131 | 15.67 MB | 03 Nov 2009 - 13:25 |
| Sogou Spider | 3388+313 | 48.82 MB | 31 Dec 2009 - 18:50 |
| SuperBot | 3680 | 526.30 MB | 22 Dec 2009 - 07:32 |
| Seekbot | 3253+15 | 1.95 MB | 16 Jul 2009 - 10:37 |
| legs | 2032+1203 | 485.60 MB | 31 Dec 2009 - 18:42 |
| Spiderman | 3024+122 | 1.12 GB | 28 Dec 2009 - 09:41 |
| The web archive (IA Archiver) | 0+3122 | 1.07 MB | 30 Dec 2009 - 22:47 |
| Powermarks | 2435 | 24.93 MB | 31 Dec 2009 - 14:50 |
| Checkbot | 2193 | 81.25 MB | 30 Dec 2009 - 05:42 |
| BecomeBot | 2143+31 | 25.42 MB | 04 Feb 2009 - 02:16 |
| WordPress | 2010 | 426.59 MB | 29 Dec 2009 - 21:55 |
| LinkChecker | 1350+255 | 17.92 MB | 28 Dec 2009 - 12:02 |
| Google AdSense | 1323+2 | 148.53 MB | 31 Dec 2009 - 18:44 |
| xenu\slink\ssleuth | 1055 | 277.55 KB | 14 May 2009 - 12:42 |
| WebReaper | 953+7 | 226.08 MB | 09 Dec 2009 - 05:28 |
| LinkScan | 950 | 1.18 GB | 31 Dec 2009 - 02:43 |
| Netluchs | 827+57 | 118.97 MB | 08 Dec 2009 - 00:40 |
| up\.browser | 813 | 23.34 MB | 12 May 2009 - 00:12 |
| SeznamBot | 563+216 | 24.24 MB | 31 Dec 2009 - 08:13 |
| ASPseek | 747 | 6.19 MB | 31 Dec 2009 - 18:05 |
| lwp-trivial | 681+2 | 14.31 MB | 31 Dec 2009 - 02:57 |
| T-H-U-N-D-E-R-S-T-O-N-E | 586 | 10.01 MB | 04 Dec 2009 - 05:14 |
| microsoft\surl\scontrol | 525 | 23.92 MB | 14 May 2009 - 05:05 |
| ms\ssearch\s4\.0\srobot | 207+239 | 5.31 MB | 30 Apr 2009 - 16:42 |
| LinkWalker | 220+220 | 77.13 KB | 31 Dec 2009 - 04:54 |
| WWWC Ver 0.2.5 | 389 | 433.38 KB | 26 Dec 2009 - 08:14 |
| BBot | 382+6 | 211.61 MB | 30 Dec 2009 - 14:34 |
| NutchCVS | 359+25 | 16.34 MB | 11 Dec 2009 - 07:21 |
| xenu's\slink\ssleuth | 367 | 440.47 KB | 03 May 2009 - 16:21 |
| Holmes | 78+246 | 966.81 KB | 31 Dec 2009 - 06:02 |
| WebCollage | 318 | 12.72 MB | 31 Dec 2009 - 18:16 |
| Walhello appie | 223+94 | 5.46 MB | 11 Sep 2009 - 23:10 |
| MOMspider | 293+13 | 115.91 MB | 13 Dec 2009 - 22:57 |
| Fish search | 292 | 33.75 MB | 31 Dec 2009 - 09:20 |
| oracle\sultra\ssearch | 287 | 73.23 MB | 17 Apr 2009 - 22:51 |
| IBM Almaden Research Center WebFountain™ | 166+88 | 882.77 MB | 24 Oct 2009 - 05:24 |
| QihooBot | 231 | 10.37 MB | 28 Dec 2009 - 05:00 |
| Metaspinner | 221+9 | 5.01 MB | 18 Jul 2009 - 19:16 |
| lmspider | 163+51 | 141.27 MB | 04 Jul 2009 - 20:54 |
| vse | 198+4 | 7.73 MB | 24 Dec 2009 - 18:02 |
| HTMLParser | 160 | 9.39 MB | 23 Dec 2009 - 12:01 |
| Netcraft | 153 | 240.36 KB | 15 Dec 2009 - 17:06 |
| MojeekBot | 119+30 | 1.55 MB | 30 Oct 2009 - 20:37 |
| Emacs-w3 Search Engine | 146 | 6.45 MB | 31 Dec 2009 - 09:28 |
| YahooSeeker Yahoo! Blog crawler | 53+93 | 1.20 MB | 27 Dec 2009 - 23:26 |
| Robozilla | 144 | 3.31 MB | 04 Oct 2009 - 11:15 |
| UCSD Crawl | 132 | 531.12 KB | 30 Dec 2009 - 15:29 |
| NG 2.x (Exalead) | 77+44 | 70.20 MB | 03 Dec 2009 - 18:57 |
| Web Core / Roots | 106+12 | 2.85 MB | 17 Dec 2009 - 15:03 |
| Cursor | 103 | 1.89 MB | 13 Dec 2009 - 15:55 |
| archive.org bot | 69+29 | 7.62 MB | 26 Dec 2009 - 08:34 |
| W3C Validator | 91 | 707.87 KB | 05 Dec 2009 - 01:20 |
| LinkBot | 89 | 0 | 13 Dec 2009 - 15:33 |
| Combine System | 53+30 | 568.04 KB | 21 Oct 2009 - 10:41 |
| Steeler | 77+5 | 32.55 MB | 23 Nov 2009 - 23:04 |
| Girafabot | 3+68 | 36.52 KB | 22 Dec 2009 - 21:53 |
| Y!J Yahoo Japan | 67+4 | 9.93 MB | 29 Dec 2009 - 21:52 |
| FavOrg | 70 | 7.30 MB | 30 Dec 2009 - 21:14 |
| arks | 59+5 | 812.42 KB | 24 Dec 2009 - 06:55 |
| Digger | 39+22 | 327.96 KB | 19 Dec 2009 - 02:02 |
| SurveyBot | 14+44 | 62.88 KB | 27 Dec 2009 - 13:08 |
| GoForIt.com | 51+4 | 1.15 MB | 03 Jul 2009 - 10:02 |
| Susie | 52 | 1.26 MB | 25 Dec 2009 - 22:19 |
| gpu\sp2p\scrawler | 45+4 | 404.02 KB | 09 May 2009 - 09:41 |
| Motor | 39+3 | 311.04 KB | 27 Oct 2009 - 22:40 |
| omniexplorer\_bot | 19+15 | 66.59 MB | 11 Apr 2009 - 06:17 |
| Dumbot | 32 | 3.27 MB | 30 Oct 2009 - 21:11 |
| Findlinks | 22+8 | 222.55 KB | 28 Dec 2009 - 03:52 |
| VORTEX | 30 | 101.28 KB | 20 Jul 2009 - 01:05 |
| TITAN | 23 | 347.02 KB | 15 Nov 2009 - 09:57 |
| ASpider (Associative Spider) | 22 | 676.38 KB | 31 Dec 2009 - 04:28 |
| Feedburner | 22 | 0 | 13 Dec 2009 - 11:14 |
| Feedfetcher-Google | 21 | 86.06 KB | 09 Dec 2009 - 12:23 |
| GouGou | 20 | 67.79 KB | 29 Oct 2009 - 18:32 |
| lwp-request | 19 | 462.79 KB | 25 Nov 2009 - 10:52 |
| web\sdownloader | 19 | 108.06 KB | 09 Apr 2009 - 07:40 |
| Harvest | 9+9 | 2.55 MB | 19 Nov 2009 - 10:19 |
| WallPaper (alias crawlpaper) | 17 | 95.56 KB | 03 Nov 2009 - 13:54 |
| link\svalet\sonline | 15 | 0 | 21 Apr 2009 - 09:07 |
| EasyDL | 15 | 346.63 KB | 29 Oct 2009 - 22:35 |
| ht://Dig | 0+14 | 2.45 KB | 29 Jul 2009 - 18:54 |
| Sleek | 14 | 0 | 18 Dec 2009 - 15:25 |
| NetSprint | 7+7 | 26.18 KB | 30 Apr 2009 - 07:23 |
| The World Wide Web Worm | 11 | 310.35 KB | 06 Nov 2009 - 11:09 |
| BlogPulse ISSpider intelliseek.com | 5+6 | 2.44 MB | 22 Dec 2009 - 06:23 |
| Alkaline | 10 | 0 | 24 Mar 2009 - 12:45 |
| Snappy | 10 | 42.73 KB | 24 Feb 2009 - 15:46 |
| Scooter | 1+9 | 26.52 MB | 03 Nov 2009 - 16:06 |
| Perman surfer | 9+1 | 198.92 KB | 15 Apr 2009 - 00:06 |
| arianna.libero.it | 2+7 | 52.32 KB | 29 May 2009 - 13:48 |
| TailRank | 8 | 89.78 KB | 27 Oct 2009 - 13:39 |
| SBIder | 5+3 | 42.19 KB | 15 Sep 2009 - 00:43 |
| wells\ssearch | 8 | 65.37 KB | 17 Jan 2009 - 05:45 |
| Internet Shinchakubin | 7 | 31.07 KB | 08 Nov 2009 - 05:10 |
| JoBo Java Web Robot | 3+3 | 57.62 KB | 10 Dec 2009 - 01:01 |
| Bookmark-Manager | 6 | 0 | 16 Sep 2009 - 05:07 |
| Muncher | 4+2 | 644.81 KB | 02 Jun 2009 - 14:42 |
| missigua\slocator | 5 | 35.00 KB | 21 Jan 2009 - 09:09 |
| MagpieRSS | 5 | 97.71 KB | 05 Dec 2009 - 22:28 |
| mnoGoSearch search engine software | 5 | 16.90 KB | 27 Oct 2009 - 23:30 |
| MSRBOT | 2+3 | 57.62 KB | 31 Jul 2009 - 10:35 |
| eventax | 2+2 | 27.83 KB | 23 Oct 2009 - 10:40 |
| Suke | 4 | 16.77 KB | 18 Dec 2009 - 10:21 |
| Geniebot | 2+2 | 19.22 KB | 20 Feb 2009 - 19:50 |
| NewsGator Online | 3 | 0 | 23 Jul 2009 - 01:01 |
| Orbiter | 3 | 74.88 KB | 08 Jun 2009 - 13:53 |
| Zeus Webster Pro | 2+1 | 4.23 KB | 29 Apr 2009 - 08:58 |
| Pioneer | 3 | 1.41 MB | 05 Oct 2009 - 06:57 |
| JBot Java Web Robot | 2 | 21.39 KB | 12 Oct 2009 - 04:45 |
| z\-add\slink\schecker | 2 | 0 | 15 Mar 2009 - 04:38 |
| SpiderView(tm) | 2 | 6.78 KB | 25 May 2009 - 12:08 |
| Snooper | 2 | 9.94 KB | 25 May 2009 - 22:39 |
| Boris | 2 | 9.94 KB | 08 Apr 2009 - 13:42 |
| wume crawler | 1+1 | 1.30 KB | 26 Aug 2009 - 15:35 |
| Gulper Bot | 2 | 292.57 KB | 31 Jul 2009 - 06:02 |
| CipinetBot | 1+1 | 3.74 KB | 19 Mar 2009 - 17:26 |
| RoadHouse Crawling System | 1 | 36.24 KB | 01 Dec 2009 - 01:56 |
| Calif | 1 | 61.00 KB | 25 Jul 2009 - 02:42 |
| gammaSpider, FocusedCrawler | 1 | 45.51 KB | 28 Apr 2009 - 09:17 |
| Ingrid | 1 | 405.94 KB | 12 Aug 2009 - 01:15 |
| boitho.com-dc | 1 | 3.39 KB | 02 Dec 2009 - 04:17 |
| NHSE Web Forager | 1 | 79.51 KB | 27 Jan 2009 - 15:45 |
| RoJo aggregator | 1 | 376.45 KB | 07 Jul 2009 - 04:59 |
| RBSE Spider | 1 | 375.63 KB | 04 Jun 2009 - 16:27 |
| WebFilter | 1 | 11.19 KB | 05 May 2009 - 19:43 |
| Sift | 1 | 37.88 KB | 22 Jun 2009 - 11:47 |
| test\scrawler | 1 | 3.36 KB | 01 Jan 2009 - 05:51 |
| Others | 0 | 0 | |