181 different robots | Hits | Bandwidth | Last visit |
Unknown robot (identified by 'discovery') | 97,732+14044 | 26.36 GB | 30 Nov 2010 - 21:02 |
Unknown robot (identified by 'bot*') | 568,751+34996 | 118.32 GB | 30 Nov 2010 - 21:02 |
Yahoo Slurp | 3,264,660+9322 | 435.36 GB | 30 Nov 2010 - 21:02 |
Googlebot | 2,267,674+3334 | 324.19 GB | 30 Nov 2010 - 21:02 |
WebVac | 290,711+32 | 36.04 GB | 30 Nov 2010 - 21:02 |
Yandex bot | 618,808+2510 | 52.05 GB | 30 Nov 2010 - 21:02 |
Voila | 902,065+5202 | 58.59 GB | 30 Nov 2010 - 21:02 |
WGet tools | 1,422,478+641 | 258.34 GB | 30 Nov 2010 - 21:02 |
Unknown robot (identified by empty user agent string) | 190,166+15105 | 15.60 GB | 30 Nov 2010 - 21:01 |
BaiDuSpider | 59,052+5425 | 12.69 GB | 30 Nov 2010 - 21:00 |
BSpider | 23,042+2050 | 2.60 GB | 30 Nov 2010 - 20:56 |
Unknown robot (identified by 'crawl') | 10,181,500+30301 | 117.73 GB | 30 Nov 2010 - 20:56 |
Unknown robot (identified by 'spider') | 68,966+4743 | 17.86 GB | 30 Nov 2010 - 20:56 |
Unknown robot (identified by 'robot') | 2,354,648+2011 | 72.60 GB | 30 Nov 2010 - 20:52 |
Alexa (IA Archiver) | 10,790+1550 | 644.78 MB | 30 Nov 2010 - 20:52 |
yacy | 3,415+724 | 585.03 MB | 30 Nov 2010 - 20:52 |
W3C Link Checker | 15,598+15126 | 832.45 MB | 30 Nov 2010 - 20:48 |
Perl tool | 48,703+10 | 5.03 GB | 30 Nov 2010 - 20:46 |
MJ12bot | 24,896+571 | 3.83 GB | 30 Nov 2010 - 20:44 |
Ask | 107,641+2852 | 9.31 GB | 30 Nov 2010 - 20:36 |
Google AdSense | 1,478 | 155.52 MB | 30 Nov 2010 - 20:30 |
The web archive (IA Archiver) | 10+1810 | 907.24 KB | 30 Nov 2010 - 20:30 |
Python-urllib | 16,033+162 | 3.81 GB | 30 Nov 2010 - 20:29 |
WWWC Ver 0.2.5 | 2,491 | 16.72 KB | 30 Nov 2010 - 20:28 |
Speedy Spider | 88,131+14560 | 11.38 GB | 30 Nov 2010 - 20:27 |
WordPress | 9,457 | 3.69 GB | 30 Nov 2010 - 20:21 |
Nutch | 49,885+4671 | 6.24 GB | 30 Nov 2010 - 20:20 |
Java (Often spam bot) | 176,423+302 | 12.62 GB | 30 Nov 2010 - 20:18 |
Findlinks | 7,378+79 | 200.63 MB | 30 Nov 2010 - 20:17 |
Unknown robot (identified by '*bot') | 98,370+786 | 4.91 GB | 30 Nov 2010 - 20:11 |
Exabot | 76,815+1675 | 3.05 GB | 30 Nov 2010 - 19:48 |
Unknown robot (identified by hit on 'robots.txt') | 0+21378 | 7.30 MB | 30 Nov 2010 - 19:43 |
docomo | 36,276+655 | 7.71 GB | 30 Nov 2010 - 19:41 |
CFNetwork | 3,922 | 3.12 GB | 30 Nov 2010 - 19:27 |
Web Core / Roots | 144+2 | 452.10 MB | 30 Nov 2010 - 19:00 |
LinkChecker | 1,422+282 | 14.45 MB | 30 Nov 2010 - 19:00 |
Sogou Spider | 3,248+424 | 83.60 MB | 30 Nov 2010 - 18:20 |
ASPseek | 664 | 5.40 MB | 30 Nov 2010 - 18:02 |
Custo | 37,163+2726 | 3.82 GB | 30 Nov 2010 - 17:35 |
MSNBot | 3,657,101+86766 | 160.65 GB | 30 Nov 2010 - 17:31 |
Powermarks | 1,705 | 38.79 MB | 30 Nov 2010 - 17:28 |
MSNBot-media | 175,867+48560 | 6.55 GB | 30 Nov 2010 - 17:06 |
OutfoxBot/YodaoBot | 3,477+1138 | 22.25 MB | 30 Nov 2010 - 16:32 |
Jakarta commons-httpclient | 3,252+20 | 350.53 MB | 30 Nov 2010 - 16:18 |
Turn It In | 37,963+163 | 2.76 GB | 30 Nov 2010 - 16:16 |
MojeekBot | 100+22 | 2.69 MB | 30 Nov 2010 - 15:05 |
EasyDL | 1,445 | 95.72 MB | 30 Nov 2010 - 14:16 |
MSIECrawler | 2,575+206 | 74.23 MB | 30 Nov 2010 - 14:03 |
larbin | 2,633+503 | 63.83 MB | 30 Nov 2010 - 13:15 |
SeznamBot | 521+263 | 42.56 MB | 30 Nov 2010 - 12:38 |
StackRambler | 9,070+158 | 118.50 MB | 30 Nov 2010 - 10:34 |
Unknown robot (identified by 'scanner') | 352+9 | 53.72 MB | 30 Nov 2010 - 10:23 |
Voyager | 385+25 | 12.48 MB | 30 Nov 2010 - 10:01 |
Mail.ru bot | 32,748+119 | 3.24 GB | 30 Nov 2010 - 09:09 |
Holmes | 43+181 | 1.14 MB | 30 Nov 2010 - 08:55 |
GigaBot | 374+305 | 7.73 MB | 30 Nov 2010 - 08:41 |
Gaisbot | 1,775+80 | 13.76 MB | 30 Nov 2010 - 08:40 |
Allrati | 334 | 5.36 MB | 30 Nov 2010 - 05:07 |
Ultraseek | 23,113+3369 | 3.12 GB | 30 Nov 2010 - 04:49 |
LinkScan | 1,962 | 326.13 MB | 30 Nov 2010 - 02:39 |
SpiderBot | 4+1 | 13.91 KB | 29 Nov 2010 - 22:17 |
HTTrack off-line browser | 1,284,203+360 | 63.15 GB | 29 Nov 2010 - 22:00 |
Unknown robot (identified by 'checker') | 954+17 | 9.37 MB | 29 Nov 2010 - 21:19 |
Sift | 1 | 48.29 KB | 29 Nov 2010 - 18:48 |
LinkBot | 52 | 0 | 29 Nov 2010 - 17:25 |
lwp-request | 115 | 3.48 MB | 29 Nov 2010 - 15:52 |
Motor | 263+1 | 687.07 MB | 29 Nov 2010 - 14:23 |
Checkbot | 1,950 | 265.86 KB | 29 Nov 2010 - 12:25 |
FavOrg | 59 | 309.82 KB | 29 Nov 2010 - 12:08 |
Fish search | 231 | 24.04 MB | 29 Nov 2010 - 10:42 |
YahooSeeker Yahoo! Blog crawler | 1,773+106 | 107.85 MB | 29 Nov 2010 - 06:33 |
ichiro | 32,558+115 | 4.52 GB | 27 Nov 2010 - 20:23 |
Susie | 72 | 1.70 MB | 27 Nov 2010 - 11:03 |
archive.org bot | 43,348+123 | 5.23 GB | 27 Nov 2010 - 09:38 |
Netcraft | 154 | 244.05 KB | 26 Nov 2010 - 21:44 |
Grub.org | 11 | 38.65 KB | 26 Nov 2010 - 06:32 |
legs | 27,470+3520 | 3.05 GB | 26 Nov 2010 - 01:04 |
ConveraMultiMediaCrawler | 1 | 3.39 KB | 25 Nov 2010 - 18:27 |
NG 1.x (Exalead) | 199 | 11.18 MB | 25 Nov 2010 - 17:37 |
FaceBook bot | 124+2 | 5.80 MB | 25 Nov 2010 - 11:04 |
Emacs-w3 Search Engine | 210 | 26.79 MB | 25 Nov 2010 - 01:45 |
Unknown robot (identified by 'sucker') | 45,605+25 | 4.89 GB | 24 Nov 2010 - 23:47 |
W3C Validator | 66 | 830.17 KB | 24 Nov 2010 - 22:21 |
QihooBot | 294 | 28.69 MB | 24 Nov 2010 - 19:19 |
Spiderman | 1,709+309 | 175.82 MB | 24 Nov 2010 - 13:37 |
FavIconizer | 24 | 490.41 KB | 23 Nov 2010 - 21:40 |
SuperBot | 91,224 | 6.66 GB | 23 Nov 2010 - 20:45 |
Vagabondo | 11,048+375 | 910.79 MB | 23 Nov 2010 - 19:47 |
Heritrix | 18,093+183 | 806.80 MB | 23 Nov 2010 - 01:47 |
SurveyBot | 31+31 | 115.95 KB | 22 Nov 2010 - 08:05 |
WebCollage | 161 | 5.00 MB | 21 Nov 2010 - 17:30 |
JoBo Java Web Robot | 290+9 | 2.65 MB | 21 Nov 2010 - 08:26 |
BBot | 268 | 118.20 MB | 21 Nov 2010 - 08:09 |
Tarantula | 2 | 92.72 KB | 20 Nov 2010 - 01:14 |
MagpieRSS | 6+64 | 207.00 KB | 19 Nov 2010 - 23:59 |
Nagios | 410 | 1.36 MB | 19 Nov 2010 - 21:19 |
Netluchs | 1,068+90 | 41.89 MB | 19 Nov 2010 - 20:43 |
TITAN | 7 | 1.03 MB | 19 Nov 2010 - 09:04 |
Feedfetcher-Google | 190 | 7.49 MB | 19 Nov 2010 - 05:06 |
Sleek | 15 | 0 | 15 Nov 2010 - 13:42 |
Feedburner | 57 | 0 | 15 Nov 2010 - 00:40 |
MOMspider | 0+15 | 5.26 KB | 14 Nov 2010 - 22:59 |
BetaBot | 248+123 | 7.71 MB | 14 Nov 2010 - 19:31 |
Cursor | 16+1 | 865.25 KB | 14 Nov 2010 - 00:25 |
mnoGoSearch search engine software | 10+1 | 33.27 KB | 12 Nov 2010 - 11:27 |
Unknown robot (identified by 'hunter') | 64+1 | 3.77 MB | 10 Nov 2010 - 09:26 |
Ocelli | 222,034+31 | 19.47 GB | 09 Nov 2010 - 07:15 |
Digger | 31+21 | 416.68 KB | 08 Nov 2010 - 11:17 |
Y!J Yahoo Japan | 41+3 | 2.50 MB | 04 Nov 2010 - 08:41 |
arks | 445+42 | 30.10 MB | 02 Nov 2010 - 03:07 |
InternetSeer | 4 | 11.82 KB | 29 Oct 2010 - 20:51 |
BlogPulse ISSpider intelliseek.com | 1+1 | 3.74 KB | 21 Oct 2010 - 10:38 |
HTMLParser | 194 | 12.02 MB | 19 Oct 2010 - 11:07 |
psbot | 16,889+688 | 873.00 MB | 18 Oct 2010 - 12:03 |
Crawl libcrawl | 37+21 | 660.94 KB | 15 Oct 2010 - 01:07 |
WebReaper | 587+8 | 7.29 MB | 13 Oct 2010 - 07:14 |
gammaSpider, FocusedCrawler | 1 | 144.34 KB | 12 Oct 2010 - 09:32 |
UCSD Crawl | 3 | 52.86 KB | 11 Oct 2010 - 13:48 |
NHSE Web Forager | 1 | 175.39 KB | 11 Oct 2010 - 12:14 |
InfoBot | 17,796+22 | 3.89 GB | 08 Oct 2010 - 13:50 |
RBSE Spider | 4 | 6.78 KB | 07 Oct 2010 - 06:00 |
lmspider | 2+2 | 2.59 KB | 01 Oct 2010 - 15:18 |
T-H-U-N-D-E-R-S-T-O-N-E | 28+8 | 2.63 MB | 29 Sep 2010 - 04:46 |
Bookmark-Manager | 2 | 0 | 28 Sep 2010 - 03:19 |
FeedValidator | 11 | 38.92 KB | 27 Sep 2010 - 16:57 |
ASpider (Associative Spider) | 26+1 | 26.93 MB | 19 Sep 2010 - 01:55 |
lwp-trivial | 411+96 | 8.60 MB | 12 Sep 2010 - 03:38 |
TestBot | 2 | 1.81 MB | 10 Sep 2010 - 15:46 |
xirq | 1 | 40.80 KB | 07 Sep 2010 - 13:30 |
Combine System | 7+3 | 71.48 KB | 06 Sep 2010 - 20:05 |
Knowledge.com | 1 | 3.39 KB | 28 Aug 2010 - 04:08 |
Hylanda | 3 | 33.27 KB | 27 Aug 2010 - 02:17 |
NewsGator Online | 1 | 0 | 23 Aug 2010 - 08:14 |
ComputingSite Robi/1.0 | 1 | 14.01 KB | 22 Aug 2010 - 15:12 |
Sven | 1 | 175.39 KB | 21 Aug 2010 - 02:58 |
Internet Shinchakubin | 2 | 36.08 KB | 14 Aug 2010 - 07:29 |
Pioneer | 11 | 151.70 KB | 10 Aug 2010 - 12:34 |
Muncher | 2 | 21.39 KB | 07 Aug 2010 - 10:39 |
twiceler | 213,447+13717 | 17.58 GB | 30 Jul 2010 - 18:58 |
DNAbot | 16+8 | 936.00 KB | 27 Jul 2010 - 12:23 |
Webdup | 251 | 1.01 MB | 14 Jul 2010 - 06:30 |
NetSprint | 7+1 | 224.81 KB | 08 Jul 2010 - 08:03 |
UdmSearch | 1 | 42.50 KB | 07 Jul 2010 - 22:14 |
FeedCrawl by feed@aobo.com | 10 | 54.01 KB | 03 Jul 2010 - 14:00 |
WallPaper (alias crawlpaper) | 1 | 8.48 KB | 02 Jul 2010 - 13:30 |
VSE | 0+1 | 359 Bytes | 29 Jun 2010 - 11:30 |
Scooter | 6 | 1.42 KB | 22 Jun 2010 - 08:20 |
MSRBOT | 120+45 | 62.66 MB | 16 Jun 2010 - 17:27 |
marvin/infoseek | 4 | 12.72 KB | 11 Jun 2010 - 15:44 |
NutchCVS | 10+3 | 349.69 KB | 03 Jun 2010 - 21:00 |
TailRank | 2 | 445.73 KB | 03 Jun 2010 - 09:16 |
Deepindex | 1 | 3.39 KB | 31 May 2010 - 16:28 |
DWCP (Dridus' Web Cataloging Project) | 1 | 38.66 KB | 31 May 2010 - 11:00 |
XYLEME Robot | 1 | 3.39 KB | 29 May 2010 - 16:23 |
fido | 1 | 18.02 KB | 29 May 2010 - 13:37 |
Girafabot | 0+22 | 7.71 KB | 23 May 2010 - 06:06 |
EARTHCOM.info | 1 | 3.39 KB | 13 May 2010 - 22:18 |
Flexum Search Engine | 1 | 3.39 KB | 11 May 2010 - 22:24 |
Harvest | 0+1 | 359 Bytes | 09 May 2010 - 21:04 |
Sunrise | 3 | 967.55 KB | 08 May 2010 - 04:17 |
BlackWidow | 1 | 3.39 KB | 07 May 2010 - 15:54 |
aipbot | 1 | 3.39 KB | 03 May 2010 - 15:56 |
ConveraCrawler | 1 | 3.39 KB | 01 May 2010 - 13:08 |
Fast-Webcrawler | 1 | 3.39 KB | 30 Apr 2010 - 14:00 |
IBM Almaden Research Center WebFountain™ | 2+2 | 1.49 MB | 22 Apr 2010 - 19:44 |
Perman surfer | 2+1 | 206.40 KB | 19 Apr 2010 - 05:42 |
JBot Java Web Robot | 6 | 2.34 KB | 06 Apr 2010 - 07:22 |
Digout4u | 1 | 3.39 KB | 14 Mar 2010 - 18:30 |
LinkWalker | 39+39 | 18.96 KB | 12 Mar 2010 - 03:04 |
VORTEX | 2 | 6.78 KB | 28 Feb 2010 - 13:35 |
Yahoo-MMCrawler | 1,778+1 | 8.62 MB | 23 Feb 2010 - 12:32 |
Robozilla | 48 | 1.14 MB | 17 Feb 2010 - 15:37 |
I, Robot | 1 | 2.81 KB | 17 Feb 2010 - 15:01 |
NG 2.x (Exalead) | 0+1 | 359 Bytes | 27 Jan 2010 - 05:39 |
Fluid Dynamics Search Engine robot | 2+1 | 5.93 KB | 21 Jan 2010 - 07:12 |
bot[+:,\.\;\/\\-] | 2,632+148 | 110.42 MB | 08 Jan 2010 - 12:39 |
java | 4,231+9 | 383.47 MB | 08 Jan 2010 - 11:41 |
[+:,\.\;\/\\-]bot | 1,140+20 | 48.95 MB | 08 Jan 2010 - 11:17 |
vse | 2 | 29.27 KB | 05 Jan 2010 - 14:47 |
arianna.libero.it | 0+1 | 359 Bytes | 04 Jan 2010 - 13:17 |
WebCopy | 31 | 248.68 KB | 02 Jan 2010 - 02:32 |
Others | 0 | 5 Bytes | |
|