Dave's Brain

Crawlers

Crawlers in the last week

CrawlerVisits
BLEXBot/1.0769
MJ12bot/v1.4.8155
Cliqzbot/1.0134
SeznamBot/3.2120
bingbot/2.0106
DotBot/1.189
MJ12bot/v1.4.775
Googlebot/2.175
YandexBot/3.070
Yahoo! Slurp62
istellabot/t.1.1360
BUbiNG (+http://law.di.unimi.it/BUbiNG.html)59
Windows NT 10.0; WOW6450
proximic49
ia_archiver40
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)38
DeuSu/5.0.237
coccocbot-web/1.036
AhrefsBot/5.235
GrapeshotCrawler/2.026
SemrushBot/1.2~bl25
Baiduspider/2.024
istellabot/t.123
Qwantify/2.4w23
WOW6420
Windows NT 5.1; rv:6.0.219
archive.org_bot18
Clickagy Intelligence Bot v216
Linux x86_648
Daum/4.18
U8
MaxPointCrawler/Nutch-1.10 (maxpoint.crawler at maxpointinteractive dot com)7
SiteExplorer/1.1b7
Plukkie/1.66
Exabot/3.06
GetIntent Crawler (http://getintent.com/bot.html)6
Steeler/3.56
Windows NT 6.1; WOW646
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36/Nutch-1.136
linkdexbot/2.05
CCBot/2.0 (http://commoncrawl.org/faq/)5
MSIE 10.05
Dataprovider.com4
Macintosh; Intel Mac OS X 10_9_34
Win644
Findxbot/1.04
Windows NT 6.3; WOW644
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36/Nutch-1.123
yoozBot-2.23
spbot/5.0.33
Uptimebot/1.03
MSIE 9.03
Macintosh; Intel Mac OS X 10_12_33
SEOkicks-Robot2
SMTBot/1.02
Python-urllib/2.72
panscient.com2
Windows NT 6.12
Nutch/Nutch-2.3.12
AwarioSmartBot/1.0 (+https://awario.com/bots.html; bots@awario.com)2
Windows NT 5.1; rv:6.02
SafeDNSBot (https://www.safedns.com/searchbot)2
Macintosh; Intel Mac OS X 10_9_22
oBot/2.3.12
compatible; ips-agent2
linkdexbot/2.22
Intel Mac OS X 10.91
Mozilla/5.0/Firefox/42.0 - nbertaupete95(at)gmail.com1
netEstate NE Crawler (+http://www.website-datenbank.de/)1
psbot/0.1 (+http://www.picsearch.com/bot.html)1
compatible; archive.org_bot +http://www.archive.org/details/archive.org_bot1
HTML Analyzer/Nutch-1.121
SurdotlyBot/1.01
SalesIntelligent (+https://www.salesintelligent.com)1
DomainStatsBot/1.0 (http://domainstats.io/our-bot)1
Windows NT 6.01
DomainCrawler/3.0 (info@domaincrawler.com; http://www.domaincrawler.com/davekb.com)1
Macintosh; Intel Mac OS X 10_9_51
MSIE 7.01
MegaIndex.ru/2.01
amd64 Linux 3.16.0-4-amd641
amd64 Windows 2003 5.21
X11; Linux x86_641
Windows NT 5.11
crawler4j (https://github.com/yasserg/crawler4j/)1
-1
YisouSpider1
Macintosh; Intel Mac OS X 10_11_41
Wotbox/2.01 (+http://www.wotbox.com/bot/)1
nutch-1.4/Nutch-1.41
Contacts-Crawler/0.1 (+http://www.scrapinghub.com)1
rogerbot/1.1 (http://moz.com/help/guides/search-overview/crawl-diagnostics#more-help, rogerbot-crawler+pr4-crawler-15@moz.com)1
Macintosh; Intel Mac OS X 10_11_61
GarlikCrawler/1.2 (http://garlik.com/, crawler@garlik.com)1
Ubuntu1

Copyright © 2008-2017, dave - Code samples on Dave's Brain is licensed under the Creative Commons Attribution 2.5 License. However other material, including English text has all rights reserved.