CrawlTrack, the webmaster dashboard. Free php/MySQL script

CrawlTrack has already been downloaded 23.511 times. What are you waiting for install it on your site?	Bookmark With del.icio.us Stumble It!
	Open-source and free

-crawlers visits tracking
-pages viewed by crawlers
-visits send by search engines
-backlinks
-number of indexed pages
-keywords
-entry pages
-hacking attempts blockage

Home Demo Documentation Forum Plugins Thanks Links Directory

My others sites
Balade au Japon
Pictures travel in Japan, from Tokyo to Kyoto. It's the french version.

Two weeks in Japan
Pictures travel in Japan, from Tokyo to Kyoto. It's the english version.

Lo's girafes
A fabulous collection of girafes, that site is in french but as it's mainly a photo galery don't hesitate to visit it.

Webmaster tools
Crawler list
Have a look on the crawler list used by Crawltrack.

IP origin
Who is hide behind that IP? A crawler or a hacker?

Recommended Links

Crawler	User agent	Owner
192.com	192.comAgent	192.com
4anything	4anything.com LinkChecker v2.0	4anything
A-Online	A-Online Search	Aon
ABCdatos	ABCdatos BotLink/5.xx.xxx#BBL	ABCdatos
AOL	Sqworm/2.9.81-BETA (beta_release; 20011102-760; i686-pc-linux-gnu)	AOL
ASAHA	ASAHA Search Engine Turkey V.001 (http://www.asaha.com/)	ASAHA
ASPseek	ASPseek/1.2.xx	ASPseek
	ASPSeek/1.2.5
	ASPSeek/1.2.x
	ASPseek/1.2.9d
	ASPSeek/1.2.xxpre
	ASPSeek/1.2.xa
AVSearch	AVSearch-1.0(peter.turney@nrc.ca)	National Research Council Canada
AbachoBot	AbachoBOT (Mozilla compatible)	Abacho
	AbachoBOT
Aberja Checkoma	Aberja Checkoma	Aberja
Abot	abot/0.1 (abot; http://www.abot.com; abot@abot.com)	Abot.com
About	About/0.1libwww-perl/5.47	About
AboutUsBot	Mozilla/5.0 (compatible; AboutUsBot/0.9; +http://www.aboutus.org/AboutUsBot)	AboutUs
Accelobot	Mozilla/5.0 (compatible; heritrix/1.12.0 +http://www.accelobot.com)	Accelovation
	Mozilla/5.0 (compatible; heritrix/1.8.0 +http://www.accelobot.com)
Accoona	Accoona-AI-Agent/1.1.1 (crawler at accoona dot com)	Accoona
	accoona
Acoi	AcoiRobot	Acoi
Acoon Robot	Acoon Robot v1.50.001	Acoon
Acoon-Robot	Acoon-Robot v3.00 (http://www.acoon.de and http://www.acoon.com)	Acoon
Acorn	Acorn/Nutch-0.9 (Non-Profit Search Engine; acorn.isara.org; acorn at isara dot o rg)	Isara
Activtourist	Mozilla/4.0 (JemmaTheTourist;http://www.activtourist.com)	Activtourist
Aesop	AESOP_com_SpiderMan	Aesop
Agada	Mozilla/4.0 (agadine3.0) www.agada.de	Agada
	agadine/1.x.x (+http://www.agada.de)
	Mozilla/4.0 (agadine3.0)
AgentName	AgentName/0.1 libwww-perl/5.48	Linkomatic
Aibot	AIBOT/2.1 By +(www.21seek.com , A Real artificial intelligence search engine , C hina)	21seek
Aicrawler	Accoona-AI-Agent/1.1.2 (aicrawler at accoonabot dot com)	Accoona
Aipbot	aipbot/1.0 (aipbot; http://www.aipbot.com; aipbot@aipbot.com)	Aipbot
Alacra	PortalBSpider/2.0 (spider@portalb.com)	Alacra
Aladin.de	Aladin/3.324	Abacho
Aleksika Danmark	Aleksika Spider/1.0 (+http://www.aleksika.com/)	Aleksika
Alexa	ia_archiver	Alexa
AlkalineBOT	AlkalineBOT/1.4 (1.4.0326.0 RTM)	Vestris
	AlkalineBOT/1.3
Allesklar.de	Allesklar/0.1 libwww-perl/5.46	Allesklar
Almaden	http://www.almaden.ibm.com/cs/crawler	IBM
	http://www.almaden.ibm.com/cs/crawler [hc5]
Altavista	Scooter-W3-1.0	Altavista
	Scooter-W3.1.2
	scooter-venus-3.0.vns
	Scooter2_Mercator_x-x.0
	Scooter/3.3
	Scooter-3.0QI
	Scooter-3.0.VNS
	Scooter-3.0.HD
	Scooter-3.2
	Scooter-3.2.BT
	Scooter-3.2.EX
	Scooter-3.2.DIL
	Scooter-3.0.FS
	Scooter-3.0.EU
	Scooter/2.0 G.R.A.B V1.0
	Scooter/1.0
	Scooter/1.0 scooter@pa.dec.com
	Scooter/1.1 (custom)
	Scooter/2.0 G.R.A.B. X2.0
	Scooter/2.0 G.R.A.B. V1.1.0
	Scooter_trk3-3.0.3
	Scooter-3.2.JT
	Scooter-3.3dev
	Scooter-ARS-1.1
	Scooter-ARS-1.1-ih
	Scooter_bh0-3.0.3
	Scooter/3.3_SF
	Scooter/3.3.vscooter
	Scooter/3.3.QA.pczukor
	Scooter-3.2.snippet
	Scooter-3.2.SF0
	Scooter-3.2.NIV
Amfibibot	Amfibibot/0.06 (Amfibi Robot; http://www.amfibi.com; agent@amfibi.com)	Amfibi
Amidalla	libwww-perl/5.65	Amidalla
Annomille	AnnoMille spider 0.1 alpha - http://www.annomille.it	Annomille
AnsearchBot	Mozilla/5.0 (compatible; AnsearchBot/1.0; +http://www.ansearch.com.au/)	Ansearch
AnswerBus	AnswerBus (http://www.answerbus.com/)	AnswerBus
Answerchase	PROve AnswerBot 4.0	Answerchase
Antibot	antibot-V1.3.3.1/debian-linux-sarge	Antidot
Any Search Info	Mozilla/4.0 (Sleek Spider/1.2)	Search-Info
Anzwers Australia	AnzwersCrawl/2.0 (anzwerscrawl@anzwers.com.au;Engine)	Anzwers Australia
Apexoo Spider	Apexoo Spider 1.0	Apexoo
Aport	Aport	Aport
Appie	appie 1.1 (www.walhello.com)	Walhello
	appie 1.1 (www.walhello.com)
Arabulbot	Mozilla/5.0 (compatible; arabulbot/1.1; +http://www.arabul.com/bot.html)	Arabul
ArabyBot	ArabyBot (compatible; Mozilla/5.0; GoogleBot; FAST Crawler 6.4; http://www.araby .com;)	Araby
Arachnoidea	Arachnoidea (arachnoidea@euroseek.com)	Euroseek
ArchitextSpider	ArchitextSpider	Excite
Archive.org_bot	Mozilla/5.0 (compatible;archive.org_bot/1.7.1; collectionId=316; Archive-It; +ht tp://www.archive-it.org)	Archive.org
Arexera	TECOMAC-Crawler/0.x	Arexera
	X-Crawler
Arianna	www.arianna.it	Libero
Arikus_Spider	Arikus_Spider	Arikus
Asahina	Asahina-Antenna/1.x	Asahina
	Asahina-Antenna/1.x (libhina.pl/x.x ; libtime.pl/x.x)
Ask 24x Info	ask.24x.info	Ask 24x
Ask Jeeves/Teoma	Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/w ebmasters.shtml)	Ask Jeeves
	Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/w ebmasters.shtml)
	Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_cr awling.html)
	Mozilla/2.0 (compatible; Ask Jeeves/Teoma; http://about.ask.com/en/docs/about/we bmasters.shtml)
Asked	asked/Nutch-0.8 (web crawler; http://asked.jp; epicurus at gmail dot com)	Asked
Askpeter_bot	Mozilla/5.0 (compatible; askpeter_bot/3.2; +http://www.askpeter.info)	Askpeter
Asterias	asterias/2.0	Singing Fish
Asterias Crawler	Mozilla/4.0 (compatible; MSIE 6.0 compatible; Asterias Crawler v4; +http://www.s ingingfish.com/help/spider.html; webmaster@singingfish.com); SpiderThread Revis ion: 3.11	Singingfish
	Mozilla/4.0 (compatible; MSIE 6.0 compatible; Asterias Crawler v4; +http://www.s ingingfish.com/help/spider.html; webmaster@singingfish.com); SpiderThread Revisi on: 3.10
Astrafind!	Mozilla/4.0 (compatible: AstraSpider V.2.1 : astrafind.com)	Seeq
Atlocal	AtlocalBot/1.1 +(http://www.atlocal.com/local-web-site-owner.html)	@Local
Attentio	Attentio/Nutch-0.9-dev (Attentio's beta blog crawler; www.attentio.com; info@att entio.com)	Attentio
Augurnet Swiss	augurfind	Augurnet Swiss
	augurnfind V-1.x
Axada	axadine/ (Axadine Crawler; http://www.axada.de/; )	Axada
Axandra	Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; IBP; .NET CLR 1.1.4322)	Axandra
Axmo	AxmoRobot - Crawling your site for better indexing on www.axmo.com search engine .	Axmo
Ay-Up	FastBug http://www.ay-up.com	Ay-up
BE Internet Search Engine	Blaiz-Bee/2.00.8222 (BE Internet Search Engine http://www.rawgrunt.com)	Rawgrunt
Ba.be	Mozilla/4.72 [en] (BACS http://www.ba.be)	BA
BaBoom Web Portal	BaboomBot/1.x.x (+http://www.baboom.us)	Baboum
BabalooSpider	BabalooSpider/1.2 (BabalooSpider; http://www.babaloo.si; spider@babaloo.si)	Babaloo
Backlink-Check	Backlink-Check.de (+http://www.backlink-check.de/bot.html)	Backlink-Check
Baiduspider	Baiduspider+(+http://www.baidu.com/search/spider_jp.html)	Baidu.com
	Baiduspider+(+http://www.baidu.com/search/spider.htm)
Balihoo	Bloodhound/Nutch-0.9 (Testing Crawler for Research - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com)	Balihoo
	TestCrawler/Nutch-0.9 (Testing Crawler for Research ; http://balihoo.com/index.a spx; tgautier at balihoo dot com)
BanBot	BanBots/1.2 (spider@banbots.com)	Banbot
BeamMachine	BeamMachine/0.5 (dead link remover of www.beammachine.net)	BeamMachine
Beauty (Cosmoty)	beautybot/1.0 (+http://www.uchoose.de/crawler/beautybot/)	uCHOOSE
BebopBot	BebopBot/2.5.1 ( crawler http://www.apassion4jazz.net/bebopbot.html )	Apassion4jazz
BecomeBot	Mozilla/5.0 (compatible; BecomeBot/1.83; MSIE 6.0 compatible; +http://www.become .com/site_owners.html)	BecomeBot
	Mozilla/5.0 (compatible; BecomeBot/3.0; MSIE 6.0 compatible; +http://www.become. com/site_owners.html)
BecomeJPBot	Mozilla/5.0 (compatible; BecomeJPBot/2.3; MSIE 6.0 compatible; +http://www.becom e.co.jp/site_owners.html)	Become
BeijingCrawler	BeijingCrawler	Unknown
BigClique	BigCliqueBOT/1.03-dev (bigclicbot; http://www.bigclique.com; bot@bigclique.com)	BigClique
Biglotron	BIGLOTRON (Beta 2;GNU/Linux)	Biglotron
Bigsearch	Bigsearch.ca/Nutch-1.0-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.c a/; info@enhancededge.com)	Bigsearch
	Bigsearch.ca/Nutch-0.9-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.c a/; info@enhancededge.com)
BilBasen	FAST Enterprise Crawler 6 used by BilBasen ApS (michael@bilinfo.dk)	Bilinfo
BilgiBetaBot	BilgiBetaBot/0.8-dev (bilgi.com (Beta) ; http://lucene.apache.org/nutch/bot.html ; nutch-agent@lucene.apache.org)	Bilgi
BilgiBot	BilgiBot/1.0(beta) (http://www.bilgi.com/; bilgi at bilgi dot com)	Bilgi
Bisnisseek	Custom Spider www.bisnisseek.com /1.0	Bisnisseek
Bitacle Robot	Bitacle Robot (V:1.0;) (http://www.bitacle.com)	Bitacle
	Bitacle bot/1.1
Blaiz Enterprises	Blaiz-Bee/1.0 (+http://www.blaiz.net)	Blaiz Enterprises
Blaiz-Bee	Blaiz-Bee/2.00.5502 (+http://www.blaiz.net)	Blaiz
	Blaiz-Bee/2.00.5622 ( http://www.blaiz.net)
Blitzsuche	BlitzBOT@tricus.net	RP ONLINE
	BlitzBOT@tricus.net (Mozilla compatible)
	Mozilla/4.0 (compatible; B_L_I_T_Z_B_O_T)
BlogRefsBot	Mozilla/5.0 (compatible; BlogRefsBot/0.1; http://www.blogrefs.com/about/bloggers )	BlogRefs
BlogSearch	BlogSearch/1.0 +http://www.icerocket.com/	IceRocket
	BlogSearch/1.x +http://www.icerocket.com/
BlogWatcher	blogWatcher_Spider/0.1 (http://www.lr.pi.titech.ac.jp/blogWatcher/)	Okumura Group
Blogbot	Naamah 1.0a/Blogbot (http://blogbot.de/)	Blogbot
	Naamah 1.0.1/Blogbot (http://blogbot.de/)
Blogdex	BlogBot/1.x	Massachusetts Institute of Technology
Blogdimension BlogBot	Blogdimension/Alpha2 (Blogdimension BlogBot; http://www.blogdimension.com)	Blogdimension
Bloglines	Bloglines Title Fetch/1.0 (http://www.bloglines.com)	Bloglines
Bloglines-Images	Bloglines-Images/0.1 (http://www.bloglines.com)	Bloglines
BlogzIce	BlogzIce/1.0 +http://www.icerocket.com/	IceRocket
	BlogzIce/1.0 (+http://icerocket.com; rhodes@icerocket.com)
Boitho	boitho.com-robot/1.x (http://www.boitho.com/bot.html)	Boitho
	boitho.com-dc/0.xx (http://www.boitho.com/dcbot.html)
	boitho.com-dc
	boitho.com-robot/1.x
Bot	bot/1.0 (bot; http://; bot@bot.bot)	Unknown
BotSeer	Mozilla 4.0(compatible; BotSeer/1.0; +http://botseer.ist.psu.edu)	Penn State College of Information Sciences and Technology
Botmobi	Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi http://fi nd.mobi/bot.html abuse@mtld.mobi)	Find.mobi
BravoBrian bSTOP	BStop.BravoBrian.it Agent Detector	BravoBrian
	BravoBrian SpiderEngine MarcoPolo
BrightCrawler	BrightCrawler (http://www.brightcloud.com/brightcrawler.asp)	Brightcloud
Bruinbot	BruinBot (+http://webarchive.cs.ucla.edu/bruinbot.html)	University of California
Btbot	BTbot/0.x (+http://www.btbot.com/btbot.html)	Btbot
BuildCMS crawler	BuildCMS crawler (http://www.buildcms.com/crawler)	BuildCMS
BuiltWith	Mozilla/5.0 (compatible; BuiltWith/0.1; +http://builtwith.com/bot.html)	BuiltWith
BullsEye/Intelliseek	BullsEye	Intelliseek
BurstFindCrawler	BurstFindCrawler/1.1 (crawler.burstfind.com; http://crawler.burstfind.com; crawl er@burstfind.com)	Burstfind
Buscaplus	Buscaplus Robi/1.0 (http://www.buscaplus.com/robi/)	Buscaplus
CEA	larbin_2.6_basileocaml (basile.starynkevitch@cea.fr)	CEA
CMP	libwww-perl/5.41	CMP United Business Media
CUPS	PrivacyFinder Cache Bot v1.0	PrivacyBird
Camcrawler	Camcrawler (+http://www.camdiscover.com/crawler.html)	Sensation Internet Services
CanadianContent Search	RoboCrawl (www.canadiancontent.net)	CanadianContent
	RoboCrawl (http://www.canadiancontent.net)
Carleson	carleson/1.0	Cosmix
Catall Spider	Catall Spider	Catall
Catall-Spider	Catall-Spider/3.3.3(www.Catall.de)	Catall
CazoodleBot	CazoodleBot/CazoodleBot-0.1 (CazoodleBot Crawler; http://www.cazoodle.com/cazood lebot; cazoodlebot@cazoodle.com)	Cazoodle
	CazoodleBot/0.1 (CazoodleBot Crawler; http://www.cazoodle.com; mqbot@cazoodle.co m)
Ccubee	ccubee/x.0	Empyreum
Changedetection	Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; http://www.changedetect ion.com/bot.html )	Changedetection
Charlotte	Mozilla/5.0 (compatible; Charlotte/1.0b; http://www.searchme.com/support/)	Searchme
	Mozilla/5.0 (compatible; Charlotte/1.0b; charlotte@betaspider.com)
Christcentral	ChristCRAWLER 2.0	Christcentral
	Mozilla/4.0 (compatible; ChristCrawler.com, ChristCrawler@ChristCENTRAL.com)
CipinetBot	CipinetBot/1.0 (http://www.cipinet.com/bot.html)	Cipinet
	CipinetBot (http://www.cipinet.com/bot.html)
CjLogbot	Mozilla/5.0 (compatible; CjLogbot 1.0; +http://www.cjlog.com/bot)	CjLog
Claymont Search	Claymont.com	Claymont Search
CloakDetect	CloakDetect/0.9 (+http://fulltext.seznam.cz/)	Seznam
Clushbot	Clushbot/3.xx-Ajax (+http://www.clush.com/bot.html)	Clush
	Clushbot/3.xx-Peleus (+http://www.clush.com/bot.html)
	Clushbot/3.31-BinaryFury (+http://www.clush.com/bot.html)
	Clushbot/3.xx-Hector (+http://www.clush.com/bot.html)
	Clushbot/3.x-BinaryFury (+http://www.clush.com/bot.html)
	Clushbot/2.x (+http://www.clush.com/bot.html)
	Mozilla/5.0 (Clustered-Search-Bot/1.0; support@clush.com; http://www.clush.com/)
Cnet robot	Mozilla/4.6 [en] (http://www.cnet.com/)	Search.com
CoBITSProbe	CoBITSProbe	Academia Sinica
Cobion	oBot ((compatible;Win32))	Cobion
	Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; obot)
	Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; QXW03018)
Combine	Combine/2.0 http://combine.it.lth.se/	Combine
	Combine/3 http://combine.it.lth.se/
	Combine/2.0
Cometrics-bot	cometrics-bot, http://www.cometrics.de	Cometrics
Cometsystems	Crawler (cometsearch@cometsystems.com)	Cometsystems
	Crawler (cometsearch@cometsystems.com)
Comperio	FAST Enterprise Crawler 6 used by Comperio AS (sts@comperio.no)	Comperio
Compete.com	larbin_2.2.0 (crawl@compete.com)	Compete Inc
Computerorgs	htdig/3.1.6 (http://computerorgs.com)	Computerorgs.com
Comrite	Comrite/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucen e.apache.org)	Comrite
ConveraCrawler	ConveraCrawler/0.9d (+http://www.authoritativeweb.com/crawl)	Convera
	ConveraCrawler/0.9e ( http://www.authoritativeweb.com/crawl)
Converas RetrievalWare	ConveraMultiMediaCrawler/0.1 (+http://www.authoritativeweb.com/crawl)	Convera
	CrawlConvera0.1 (CrawlConvera@yahoo.com)
	ConveraCrawler/0.2
	Convera Internet Spider V6.x
CoolBot	CoolBot	SuchMaschine21
Cortina	Vision Research Lab image spider at vision.ece.ucsb.edu	Vision Research Lab
CougarSearch	CougarSearch/0.1 (+http://www.cougarsearch.com/faq.shtml)	CougarSearch
Cowbot	Cowbot-0.1 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com)	Naver
	Cowbot-0.1.x (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com)
CrawlerBoy	CrawlerBoy Pinpoint.com	Motricity
Crawling jpeg	Mozilla/5.0 (compatible; Crawling jpeg; http://www.yama.info.waseda.ac.jp)	Yamana Laboratory - Waseda University Japan
Crawllybot	Crawllybot/0.1 (Crawllybot; +http://www.crawlly.com; crawler@crawlly.com)	Crawlly
Croccrawler	CrocCrawler vx.3 [en] (http://www.croccrawler.com) (X11; I; Linux 2.0.44 i686)	Croccrawler.com
CsCrawler	Hi! I'm CsCrawler, my homepage: http://www.kde.cs.uni-kassel.de/lehre/ss2005/goo glespam/crawler.html RPT-HTTPClient/0.3-3	University of Kassel
Csci_b659/0.13	csci_b659/0.13	Indiana University School of Informatics
Cuasar	Cuasarbot/0.9b http://www.cuasar.com/spider_beta/	Cuasar
CurryGuide	CurryGuide SiteScan 1.1	CurryGuide
CyberAlerts	Mozilla/3.0 (compatible; Webinator-indexer.cyberalert.com/2.56)	CyberAlerts
Cydral	CydralSpider/1.x (Cydral Web Image Search; http://www.cydral.com)	Cydral
CydralSpider	CydralSpider/2.2 (Cydral Image Search; http://www.cydral.com)	Cydral
	CydralSpider/2.4 (Cydral Image Search; http://www.cydral.com)
DAUM RSS Robot	ELI/20070402:2.0 (DAUM RSS Robot, Daum Communications Corp.; +http://ws.daum.net /aboutkr.html)	Daum
DAUM Web Robot	Mozilla/4.0 (compatible; MSIE enviable; DAUMOA 2.0; DAUM Web Robot; Daum Communi cations Corp., Korea; +http://ws.daum.net/aboutkr.html)	Daum
	Mozilla/4.0 (compatible; MSIE is not me; DAUMOA/1.0.1; DAUM Web Robot; Daum Comm unications Corp., Korea)
	Mozilla/4.0 (compatible; MSIE enviable; DAUMOA/1.0.1; DAUM Web Robot; Daum Commu nications Corp., Korea; +http://ws.daum.net/aboutkr.html)
	Mozilla/4.0 (compatible; MSIE is not me; DAUMOA/1.0.0; DAUM Web Robot; Daum Comm unications Corp., Korea)
DNS-Digger	Mozilla/5.0 (compatible; DNS-Digger/1.0; +http://www.dnsdigger.com)	Dnsdigger
DailyOrbit	Orbiter/T-2.0 (+http://www.dailyorbit.com/bot.htm)	DailyOrbit
DataFountains	DataFountains/DMOZ Feature Vector Corpus Creator (http://ivia.ucr.edu/useragents .shtml)	University of California
	DataFountains/DMOZ Downloader
DataSpear Spider Bot	DataSpear/1.0 (Spider; http://www.dataspear.com/spider.html; spider@dataspear.co m)	DataSpear
	DataSpearSpiderBot/0.2 (DataSpear Spider Bot; http://dssb.dataspear.com/bot.html ; dssb@dataspear.com)
DataparkSearch	DataparkSearch/4.47 (+http://dataparksearch.org/bot)	DataparkSearch
	DataparkSearch/4.xx (http://www.dataparksearch.org/)
DaviesBot	DaviesBot/1.7 (www.wholeweb.net)	Wholeweb
Daypop	daypopbot/0.x	Daypop
DbDig	dbDig(http://www.prairielandconsulting.com)	Connections
De.com	Mozilla/5.0 (compatible; de/1.13.2 +http://www.de.com)	De.com
DeepIndexer	DeepIndexer.ca	Deepindex
Deepak-USC/ISI	deepak-USC/ISI	University of Southern California
Deepindex	DeepIndex (www.en.deepindex.com)	Deepindex
	Deepindex V2
	DeepIndex
Denmex Websearch	Denmex websearch (http://search.denmex.com)	Denmex Websearch
DepSpid	Mozilla/4.0 (compatible; DepSpid/5.03; +http://about.depspid.net)	DepSpid
Dev-spider2	dev-spider2.searchpsider.com/1.3b	Searchspider
DiaGem Japan	DiaGem/1.1 (http://www.skyrocket.gr.jp/diagem.html)	DiaGem Japan
Die Kraehe	-DIE-KRAEHE- META-SEARCH-ENGINE/1.1 http://www.die-kraehe.de	Die Kraehe
Diggit	Digger/1.0 JDK/1.3.0rc3	Diggit
Direct Hit	Mozilla/2.0 (compatible; EZResult -- Internet Search Engine)	Teoma
Disco-crawl	disco/Nutch-0.9 (experimental crawler; www.discoveryengine.com; disco-crawl@disc overyengine.com)	Discoveryengine
	disco/Nutch-1.0-dev (experimental crawler; www.discoveryengine.com; disco-crawl@ discoveryengine.com)
Ditto	DittoSpyder	Ditto
DoCoMo	DoCoMo/1.0/Nxxxi/c10	NTT DoCoMo
	DoCoMo/2.0 P900iV(c100;TB;W24H11)
	DoCoMo/1.0/Nxxxi/c10/TB
Dodgebot	dodgebot/experimental	Agmlab
DotBot	DotBot/1.0.1	Dotnetdotcom
	Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, crawler@dotne tdotcom.org)
Doubanbot	Doubanbot/1.0 (bot@douban.com http://www.douban.com)	Douban
Download-Tipp	Download-Tipp Linkcheck (http://download-tipp.de/)	Download-Tipp
	EyeCatcher (Download-tipp.de)/1.0
Drecombot	Drecombot/1.0 (http://career.drecom.jp/bot.html)	Drecom Japan
DtSearchSpider	dtSearchSpider	dtSearch
Dumbot	Dumbot(version 0.1 beta)	DumbFind.com
	Dumbot(version 0.1 beta - dumbfind.com)
	Dumbot(version 0.1 beta - http://www.dumbfind.com/dumbot.html)
E-SocietyRobot	e-SocietyRobot(http://www.yama.info.waseda.ac.jp/~yamana/es/)	Yamana Laboratory
E-StyleISP	eStyleSearch 4 (compatible; MSIE 6.0; Windows NT 5.0)	e-StyleISP
EApolloBot	eApolloBot/1.0 (eApollo search engine robot; http://www.eapollo.com; eapollo at global-opto dot com)	EApollo
EMPAS_ROBOT	EMPAS_ROBOT	Empas
ESISmartSpider	ESISmartSpider	smart-spider.com
Earthcom	EARTHCOM.info/1.x [www.earthcom.info]	Earthcom.info
	Mozilla/5.0 (compatible; EARTHCOM.info/2.01; http://www.earthcom.info)
	Mozilla/5.0 (compatible; EARTHCOM/2.2; +http://enter4u.eu)
	EARTHCOM.info/1.xbeta [www.earthcom.info]
EasyDL	EasyDL/3.04 http://keywen.com/Encyclopedia/Bot	Keywen
	EasyDL/3.xx http://keywen.com/Encyclopedia/Bot
	EasyDL/3.xx
Echo.com	Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) TrueRobot; 1.5	Echo.com
Echo.fr	EchO!/2.0	Echo.fr
Egothor	Mozilla/5.0 (compatible; egothor/8.0g; +http://ego.ms.mff.cuni.cz/)	Charles University in Prague
Egotobot	EgotoBot/4.8 (+http://www.egoto.com/about.htm)	Egoto.com
Elfbot	elfbot/1.0 (+http://www.uchoose.de/crawler/elfbot/)	uCHOOSE
Elsop	LinkScan/11.0beta2 Unix	LinkScan
	LinkScan/9.0g Unix
	LinkScan/x.x Unix
EmeraldShield.com Web Spider	EmeraldShield.com Web Spider (http://www.emeraldshield.com/webbot.aspx)	Emeraldshield
Enfish Tracker	Enfish Tracker	Enfish
Enoola	enoola (http://www.enoola.com)	Enoola
Enterprise Search	Enterprise_Search/1.0	Innerprise
	Enterprise_Search/1.00.xxx;MSSQL (http://www.innerprise.net/es-spider.asp)
	Search/1.0 (http://www.innerprise.net/es-spider.asp)
	Enterprise_Search/1.0.xxx
	ES.NET_Crawler/2.0 (http://search.innerprise.net/)
Entireweb	Speedy_Spider (http://www.entireweb.com)	Entireweb
	Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)
	Speedy Spider (Beta/x.x; speedy@entireweb.com)
	WorldLight
	Mozilla/4.0 (compatible; SpeedySpider; www.entireweb.com)
Envolkspider	envolk/1.7 (+http://www.envolk.com/envolkspiderinfo.php)	Envolk
	envolk[ITS]spider/1.6(+http://www.envolk.com/envolkspider.html)
	envolk/1.7 (+http://www.envolk.com/envolkspiderinfo.html)
EroCrawler	EroCrawler	EroCrawler
Eruvo-bot	eruvo-bot 4.8.1 (http://www.eruvo.com)	Eruvo
EuripBot	EuripBot/0.4 (+http://www.eurip.com) PreCheck	Eurip.com
	EuripBot/0.2 (+http://www.eurip.com) GetRobots
	EuripBot/0.4 (+http://www.eurip.com) GetFile
	EuripBot/0.5 (+http://www.eurip.com) PreCheck
Euro-spider	Euro-Spider Shopping 1.0	Euro-spider
Evaal	Evaal/0.7.1 (Evaal; http://search.evaal.com/bot.html; bot@evaal.com)	Evaal
EvaalSE	EvaalSE - bot@evaal.com	Evaal
Eventax	eventax/1.3 (eventax; http://www.eventax.de/; info@eventax.de)	Eventax
Everest-Vulcan	Everest-Vulcan Inc./0.1 (R&D project; host=e-1-24; http://everest.vulcan.com/cra wlerhelp)	Vulcan
Exabot	Exalead NG/MimeLive Client (convert/http/0.120)	Exalead
	Exabot/3.0
	Mozilla/5.0 (compatible; Konqueror/3.2; Linux) (KHTML, like Gecko)
	Mozilla/5.0 (compatible; Exabot Test/3.0; +http://www.exabot.com/go/robot)
	ExaBotTest/2.0
	Exabot/2.0
	Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)
	NG/2.0
	ExaBotTest/3.0
	Exabot-Test/1.0
Exabot-Images	NG/4.0.1229	Exalead
	Exabot-Images/1.0
	Mozilla/5.0 (compatible; Exabot-Images/3.0; +http://www.exabot.com/go/robot)
ExactSEEK	eseek-larbin_2.6.2 (crawler@exactseek.com)	ExactSEEK
	ExactSeek Crawler/0.1
	exactseek.com
	exactseek-pagereaper-2.63 (crawler@exactseek.com)
	exactseek-crawler-2.63 (crawler@exactseek.com)
ExactSeek_Spider	ExactSeek_Spider	ExactSeek
Excalibur	Excalibur Internet Spider V6.5.4	Convera
Execrawl	Execrawl/1.0 (Execrawl; http://www.execrawl.com/; bot@execrawl.com)	Execrawl
FAST-WebCrawler	FAST Enterprise Crawler/6.4.18 (crawler@fast.no)	FAST
	FAST-WebCrawler/2.1-pre7 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.2-pre4 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.2-pre5 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.2-pre3 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.2-pre2 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.2-pre1 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.2-pre8 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.2-pre9 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.1-pre14 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea rch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.1-pre13 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea rch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.1-pre12 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea rch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.1-pre11 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea rch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.1-pre6 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.1-pre5 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.1.prealpha.2000-04-07.1 (ashen@looksmart.net)
	fastlwspider/1.0
	FAST-WebCrawler/2.1-pre2 (ashen@looksmart.net)
	FAST-WebCrawler/2.1.pre.2000-04-14.1 (ashen@looksmart.net)
	FAST-WebCrawler/3.7/FirstPage (atw-crawler at fast dot no;http://fast.no/support /crawler.asp)
	FAST-WebCrawler/2.1-pre10 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsea rch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.1.pre.2000-04-18.1 (crawler@fast.no; http://www.fast.no/faq/fa qfastwebsearch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.1-pre4 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear ch/faqfastwebcrawler.html)
	FAST-WebCrawler/2.0.9 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsearch/ faqfastwebcrawler.html)
	FAST-WebCrawler/2.0.10 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsearch /faqfastwebcrawler.html)
	FAST-SoccerCrawler/2.2-pre-cvs (oyvinda@fast.no; http://www.fast.no/faq/faqfastw ebsearch/faqfastwebcrawler.html)
FDSE	Mozilla/4.0 (compatible; FDSE robot)	Abadoor
FaXobot	Faxobot/1.0	Faxo
Factbot	Factbot 1.09 (see http://www.factbites.com/webmasters.php)	Factbites
	factbot : http://www.factbites.com/robots
Fast Search	PycURL	FAST
Fastbot	fastbot crawler beta 2.0 (+http://www.fastbot.de)	Fastbot
Favo.eu crawler	favo.eu crawler/0.6 (http://www.favo.eu)	Favo
Feed24	Feed24.com	Feed24
FeedChecker	FeedChecker/0.01	University of Tokyo
Feedfetcher-Google	Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)	Google
Feedster Crawler	Feedster Crawler/3.0; Feedster, Inc.	Feedster
Felix	Felix - Mixcat Crawler (+http://mixcat.com)	MixCat
Filangy	Filangy/1.01 (Filangy; http://www.filangy.com/filangyinfo.jsp?inc=robots.jsp; fi langy-agent@filangy.com)	Filangy
FindLinks	http://wortschatz.uni-leipzig.de/findlinks/	University of Leipzig
	findlinks/1.1.1-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)
	findlinks/1.0.9 (+http://wortschatz.uni-leipzig.de/findlinks/)
	findlinks/1.1.1 (+http://wortschatz.uni-leipzig.de/findlinks/)
	findlinks/0.901 (+http://wortschatz.uni-leipzig.de/findlinks/)
	findlinks/1.1.4-beta1 ( http://wortschatz.uni-leipzig.de/findlinks/)
	findlinks/1.1.1-a2 (+http://wortschatz.uni-leipzig.de/findlinks/)
Findexa Crawler	Findexa Crawler (http://www.findexa.no/gulesider/article26548.ece)	Findexa
FineBot	FineBot	Finesearch
Firefly	Firefly/1.0 (compatible; Mozilla 4.0; MSIE 5.5)	Fireball
	Firefly/1.0
FirstGov	FirstGov.gov Search - POC:firstgov.webmasters@gsa.gov	U.S.Government
Firstsbot	firstsbot	Firstsfind
Flapbot	Flapbot/0.7.2 (Flaptor Crawler; http://www.flaptor.com; crawler at flaptor perio d com)	Flaptor
Flatlandbot	flatlandbot/flatlandbot (Flatland Industries Web Spider; http://www.flatlandindu stries.com/flatlandbot.php; jason@flatlandindustries.com)	Flatland Industries
	great-plains-web-spider/flatlandbot (Flatland Industries Web Spider; http://www. flatlandindustries.com/flatlandbot.php; jason@flatlandindustries.com)
	great-plains-web-spider/gpws (Flatland Industries Web Spider; http://www.flatlan dindustries.com/flatlandbot.php; jason@flatlandindustries.com)
FlickBot	FlickBot 2.0 RPT-HTTPClient/0.3-3	DivX.com
Fluffy the spider	Mozilla/3.0 (compatible; Fluffy the spider; http://www.searchhippo.com/; info@se archhippo.com)	Searchhippo
Folkd.com Spider	Folkd.com Spider/0.1 beta 1 (www.folkd.com)	Folkd
ForAll.pl-Crawler	ForAll.pl-Crawler/1.0	ForAll
Francis	Francis/1.0 (francis@neomo.de http://www.neomo.de/)	Neomo
FreshNotes crawler	FreshNotes crawler< report problems to crawler-at-freshnotes-dot-com	FreshNotes
	FreshNotes crawler, report problems to crawler-at-freshnotes-dot-com
Freshmeat	freshmeat.net URL validator/1.1	Freshmeat
FuchsBot	FuchsBot +http://www.fuchsbot.tld	FuchsBot
FurlBot	Mozilla/4.0 compatible FurlBot/Furl Search 2.0 (FurlBot; http://www.furl.net; wn .furlbot@looksmart.net)	Furl
FuseBulb	FuseBulb.Com	FuseBulb
FyberSpider	FyberSpider (+http://www.fybersearch.com/fyberspider.php)	FyberSearch
GAIS Robot	GAIS Robot/1.0B2	Seed
GEXTEST-00393	gsa-crawler (Enterprise; GEXTEST-00393; gsasymbiosys@gmail.com,xeonbox4@gmail.co m)	Unknown
GPU p2p crawler	Mozilla/4.0 (compatible; GPU p2p crawler http://gpu.sourceforge.net/search_engin e.php)	GPU
GSiteCrawler	GSiteCrawler/v1.20 rev. 273 (http://gsitecrawler.com/)	GSiteCrawler
Gaaz	gazz/x.x (gazz@nttrd.com)	Infobee
Gaisbot	Gaisbot/3.0+(robot06@gais.cs.ccu.edu.tw;+http://gais.cs.ccu.edu.tw/robot.php)	Gais
	Gaisbot/3.0+(robot@gais.cs.ccu.edu.tw;+http://gais.cs.ccu.edu.tw/robot.php)
	Gaisbot/3.0+(indexer@gais.cs.ccu.edu.tw;+http://gais.cs.ccu.edu.tw/robot.php)
GalaxyBot	Mozilla/4.0 (compatible; MSIE 5.0; www.galaxy.com/galaxybot.html)	Galaxy
	GalaxyBot/1.0 (http://www.galaxy.com/galaxybot.html)
	Mozilla/4.0 (compatible; www.galaxy.com)
Gamekitbot	gamekitbot/1.0 (+http://www.uchoose.de/crawler/gamekitbot/)	Uchoose
GammaSpider	GammaSpider/1.0	Gammasite
GenieKnows	Mozilla/5.0 (wgao@genieknows.com)	GenieKnows
	larbin_2.6.3 (wgao@genieknows.com)
	geniebot wgao@genieknows.com
	Mozilla/5.0 wgao@genieknows.com
GeonaBot	GeonaBot 1.x; http://www.geona.com/	Geona
Georgia Institute of Technology	larbin_2.6.2 (listonATccDOTgatechDOTedu)	Georgia Institute of Technology
Geourl	Mozilla/5.0 (compatible; geourl/2.0b16 - http://geourl.org/bot)	Geourl
GigaBaz Brainbot	MicroBaz	Gigabaz
	gigabaz/3.1x (baz@gigabaz.com; http://gigabaz.com/gigabaz/)
Gigabot	Gigabot/2.0/gigablast.com/spider.html	Gigablast
	Gigabot/2.0; http://www.gigablast.com/spider.html
	Gigabot/2.0att
	Gigabot/2.0
	Gigabot/3.0 (http://www.gigablast.com/spider.html)
Girafabot	Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0; Girafabot; girafabot at giraf a dot com; http://www.girafa.com)	Girafa
GlobalQueue	Look.com	Multi-mode
GnodSpider	GNODSPIDER (www.gnod.net)	Gnod
GoForIt	GoForIt.com	GoForIt
	GOFORITBOT ( http://www.goforit.com/about/ )
Goblin	Goblin/0.9 (http://www.goguides.org/)	GoGuides
	Goblin/0.9.x (http://www.goguides.org/goblin-info.html)
Gonzo1	gonzo1[P] +http://www.suchen.de/popups/faq.jsp	T-info
Gonzo2	gonzo2[P] mailto:crawleradmin.t-info@telekom.de	T-info
	gonzo1[P] mailto:crawleradmin.t-info@telekom.de
	gonzo2[P] +http://www.suchen.de/faq.html
Goo (Japan)	Mozilla/3.0 (Slurp.so/Goo; slurp@inktomi.com; http://www.inktomi.com/slurp.html)
Google-Adsense	Mediapartners-Google/2.1	Google
	Mediapartners-Google/2.1 ( http://www.googlebot.com/bot.html)
	Mediapartners-Google
Google-Image	Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)	Google
	Googlebot-Image/1.0
	Googlebot-Image/1.0 ( http://www.googlebot.com/bot.html)
Google-Sitemaps	Google-Sitemaps/1.0	Google
Google-WAP	Nokia-WAPToolkit/1.2 googlebot(at)googlebot.com	Google
	Google WAP Proxy/1.0
GoogleBot	Googlebot/1.0 (googlebot@googlebot.com)	Google
	Googlebot/1.0 (googlebot@googlebot.com http://googlebot.com/)
	Googlebot/2.1 ( http://www.googlebot.com/bot.html)
	Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)
	Googlebot/2.0 beta (googlebot@googlebot.com)
	Googlebot-w/2.1 (+http://googlebot.com/bot.html)
	Googlebot/2.0 (+http://googlebot.com/bot.html)
	Mozilla/4.0 (MobilePhone SCP-5500/US/1.0) NetFront/3.0 MMP/2.0 FAKE (compatible; Googlebot/2.1; http://www.google.com/bot.html)
	Googlebot/2.1 (+http://www.google.com/bot.html)
	Googlebot/Test ( http://www.googlebot.com/bot.html)
	Mozilla/4.0 (MobilePhone SCP-5500/US/1.0) NetFront/3.0 MMP/2.0 (compatible; Goog lebot/2.1; http://www.google.com/bot.html)
	Googlebot/2.1 ( http://www.google.com/bot.html)
	Googlebot/2.1 (+http://www.googlebot.com/bot.html)
	Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
	Googlebot/2.1w (+http://googlebot.com/bot.html)
	Googlebot/1.0
Googlebot-Mobile	Generic Mobile Phone (compatible; Googlebot-Mobile/2.1; +http://www.google.com/b ot.html)	Google
	Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Google bot-Mobile/2.1; +http://www.google.com/bot.html)
	KDDI-CA33 UP.Browser/6.2.0.10.4 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
Greaterera	Mozilla/5.0 (compatible; heritrix/1.7.0 +http://www.greaterera.com/)	Greaterera
GrigorBot	GrigorBot 0.8 (http://www.grigor.biz/bot.html)	Grigor
Gromit	Gromit/1.0	Australasian Legal Information Institute
Grub-client	Mozilla/4.0 (compatible; grub-client-1.4.3; Crawl your own stuff with http://gru b.org)	Grub
Gsa-crawler	gsa-crawler (Enterprise; GIX-03519; cknuetter@stubhub.com)	IBM
	gsa-crawler (Enterprise; GIX-04637; rex_li@trend.com.tw)
Gulliver	Gulliver/1.3	Northernlight
	Gulliver/1.2
GulperBot	Mozilla/5.0 [en] (compatible; Gulper Web Bot 0.2.4 www.ecsl.cs.sunysb.edu/~maxim /cgi-bin/Link/GulperBot)	University of New-York
	Gulper Web Bot 0.2.4 (www.ecsl.cs.sunysb.edu/~maxim/cgi-bin/Link/GulperBot)
Gungho-crawler	Gungho/0.08004 (http://code.google.com/p/gungho-crawler/wiki/Index)	Gungho
GurujiBot	GurujiBot/1.0 (+http://www.guruji.com/WebmasterFAQ.html)	Guruji
	GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html)
Harvest-NG	Harvest-NG/1.0.2	Harvest-NG
Hatena Antenna	Hatena Antenna/0.4 (http://a.hatena.ne.jp/help#robot)	Hatena Antenna
HatenaScreenshot	HatenaScreenshot/1.0 (checker)	Hatena
	HatenaScreenshot/1.0 (checker)
Hbtronix.spider	hbtronix.spider.2 -- http://hbtronix.de/spider.php	Hbtronix
HeinrichderMiragoRobot	HeinrichderMiragoRobot (http://www.miragorobot.com/scripts/deinfo.asp)	Mirago
Helix	Helix/1.x (+http://www.sitesearch.ca/helix/)	SiteSearch
HenriLeRobotMirago	HenriLeRobotMirago (http://www.miragorobot.com/scripts/frinfo.asp)	Mirago
HenryTheMiragoRobot	HenryTheMiragoRobot (http://www.miragorobot.com/scripts/mrinfo.asp)	Mirago
Heritrix	Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; heritrix/1.3.0 +http://www.cs .washington.edu/research/networking/websys/)	University of Washington
	archive.org_bot
	mozilla/5.0 (compatible; heritrix/1.3.0 +http://archive.crawler.org)
Heritrix L3S	Mozilla/5.0 (compatible; heritrix/1.5.0 +http://www.l3s.de/~kohlschuetter/projec ts/crawling/)	L3S Research Center
Heritrix/1.4.0	Mozilla/5.0 (compatible; heritrix/1.4.0 +http://www.chepi.net)	Chepi
Hermits Search	Mozilla/5.0 (compatible; Hermit Search. Com; +http://www.hermitsearch.com)	Hermits Search
Hiiglespider	Hiiglespider/0.1, Hiigle.com, http://hiigle.com/spider	Hiigle
Hitwise Spider	Hitwise Spider v1.0 http://www.hitwise.com	Hitwise
Holmes	holmes/3.11 (OnetSzukaj/5.0; +http://szukaj.onet.pl)	Szukaj.onet
	holmes/x.x
	holmes/3.10.1 (OnetSzukaj/5.0; +http://szukaj.onet.pl)
	holmes/3.11 (http://morfeo.centrum.cz/bot)
	holmes/3.9 (OnetSzukaj/5.0; +http://szukaj.onet.pl)
HomePageSearch	HomePageSearch(hpsearch.uni-trier.de)	HomePageSearch
Homerbot	Homerbot: www.homerweb.com	Homerweb
Honda-Search	Honda-Search/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; search@honda -search.com)	Honda-Search
Hoowwwer	HooWWWer/2.1.0 (+http://cosco.hiit.fi/search/hoowwwer/ | mailto:crawler-info<at> hiit.fi)	NGIR
	HooWWWer/2.1.3 (debugging run) (+http://cosco.hiit.fi/search/hoowwwer/ | mailto: crawler-info<at>hiit.fi)
Htdig	htdig/3.1.x (root@localhost)	ht:/dig
Htdig/3.1.6	htdig/3.1.6 (unconfigured@htdig.searchengine.maintainer)	Acad?mie de Toulouse
I1searchbot	i1searchbot/2.0 (i1search web crawler; http://www.i1search.com; crawler@i1search .com)	I1search
ICC-Crawler	ICC-Crawler(Mozilla-compatible;http://kc.nict.go.jp/icc/crawl.html;icc-crawl-con tact(at)ml(dot)nict(dot)go(dot)jp)	NICT
	ICC-Crawler(Mozilla-compatible; http://kc.nict.go.jp/icc/crawl.html; icc-crawl(a t)ml(dot)nict(dot)go(dot)jp)
	ICC-Crawler(Mozilla-compatible; http://kc.nict.go.jp/icc/crawl.html; icc-crawl-c ontact(at)ml(dot)nict(dot)go(dot)jp)
ICCrawler	ICCrawler - ICjobs (http://www.icjobs.de/bot.htm)	ICCenter
ICRA_Label_spider	ICRA_label_spider/x.0	Icra
IDBot	Mozilla/5.0 (compatible; IDBot/1.0; +http://www.id-search.org/bot.html)	Id-search
IIITBOT	IIITBOT/1.1 (Indian Language Web Search Engine; http://webkhoj.iiit.net; pvvpr a t iiit dot ac dot in)	Webkhoj
INGRID	Mozilla/3.0 (INGRID/3.0 MT; webcrawler@NOSPAMexperimental.net; http://webmaster. ilse.nl/jsp/webmaster.jsp)	Ilse
IP2MapBot	IP2MapBot/1.1 http://www.ip2map.com	Ip2Map
IPiumBot	IPiumBot laurion(dot)com	Laurions
IRLbot	IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)	Texas A&M University
	IRLbot/1.0 (+http://irl.cs.tamu.edu/crawler)
	IRLbot/2.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)
IWAgent	IWAgent/ 1.0 - www.brandprotect.com	Brandprotect
Iaskspider2	iaskspider2 (iask@staff.sina.com.cn)	Sina
Ichiro	ichiro/1.0 (ichiro@nttr.co.jp)	Goo
	ichiro/2.0 (http://help.goo.ne.jp/door/crawler.html)
	ichiro/1.0 (ichiro@nttr.co.jp)
IconSurf	IconSurf/2.0 favicon monitor (see http://iconsurf.com/robot.html)	IconSurf
	IconSurf/2.0 favicon finder (see http://iconsurf.com/robot.html)
Icsbot	icsbot-0.1	International Christian school of Seoul
Ideare	ideare - SignSite/1.x	Ideare
IlTrovatore	IlTrovatore/1.2 (IlTrovatore; http://www.iltrovatore.it/bot.html; bot@iltrovator e.it)	IlTrovatore
Ilial/Nutch	ilial/Nutch-0.9-dev	University of California
	ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company. Fo r more information please visit http://www.ilial.com/crawler; http://www.ilial.c om/crawler; crawl@ilial.com)
Ilse	Mozilla/3.0 (Vagabondo/2.0 MT; webcrawler@NOSPAMexperimental.net; http://aanmeld en.ilse.nl/?aanmeld_mode=webhints)	Ilse
	Mozilla/3.0 (INGRID/3.0 MT; webcrawler@NOSPAMexperimental.net; http://aanmelden. ilse.nl/?aanmeld_mode=webhints)
ImageWalker	ImageWalker/2.0 (www.bdbrandprotect.com)	Bdbrandprotect
IncyWincy	IncyWincy(http://www.loopimprovements.com/robot.html)	LoopImprovements
	IncyWincy/2.1(loopimprovements.com/robot.html)
	IncyWincy page crawler(webmaster@loopimprovements.com,http://www.loopimprovement s.com/robot.html)
	NetResearchServer/x.x(loopimprovements.com/robot.html)
	IncyWincy data gatherer(webmaster@loopimprovements.com,http://www.loopimprovemen ts.com/robot.html)
IncyWincy (Look)	IncyWincy(http://www.look.com)	Look
IndexTheWeb	IndexTheWeb.com Crawler7	IndexTheWeb
Indonesia Interactive	Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; Site Server 3.0 Robot) Indonesia Interactive	Indonesia Interactive
InelaBot	InelaBot/0.2 (+http://inelegant.org/bot)	Inelegant
Inet Library	Inet library	Inet Library
InfoFly	InfoFly/1.0 (http://www.versions-project.org/)	Versions-project
InfoLab robot	Mozilla/5.0 (compatible; heritrix/1.10.2 +http://i.stanford.edu/)	Stanford University
InfoSec Search Bot	RedCell/0.1 (InfoSec Search Bot (Coming Soon); http://www.telegenetic.net/bot.ht ml; lhall@telegenetic.net)	Telegenetic
Infoseek	InfoSeek Sidewinder/0.9	Go
Inria	larbin_2.2.1_de_Viennot (Laurent.Viennot@inria.fr)	Inria
Insitor Search robot	Insitor.com search and find world wide!	Insitor
Insitornaut	Insitornaut	Insitor
Internet Ninja	Internet Ninja x.0	Dream Train Internet
Internetseer	InternetSeer.com	Internetseer
Iprospect	Mozilla/3.0 (compatible; Webinator-DEV01.home.iprospect.com/2.56)	Iprospect
IpselonBot	IpselonBot/0.xx-beta (Ipselon; http://www.ipselon.com; ipselonbot@ipselon.com)	Ipselon
Iseekbot	iSEEKbot/iSEEKbot-0.9-dev (http://beta.iseek.com/iseekbot.html; bot at iseek dot com)	Iseek
Ishida Lab	larbin_2.2.2 (sugayama@lab7.kuis.kyoto-u.ac.jp)	Kyoto University
It-bot	IlTrovatore-Setaccio/1.2 (It-bot;compatible;MSIE 6.0;Mozilla/4.0; http://www.ilt rovatore.it/bot.html; bot@iltrovatore.it)	IlTrovatore
Jabot	Jabot/6.x (http://odin.ingrid.org/)	ODIN Directory
	Jabot/7.x.x (http://odin.ingrid.org/)
Jambot	Jambot/0.2.1 (Jambot; http://www.jambot.com/blog/static.php?page=webmaster-robot ; crawler@jambot.com)	Jambot
	Jambot/0.1.1 (Jambot; http://www.jambot.com/blog; crawler@jambot.com)
Jayde Crawler	Jayde Crawler. http://www.jayde.com	Jayde
Jeanie	jeanie/3.3.3(www.sidedc.net/;compatible;MSIE 6.0;Windows NT 5.51)	Sidedc
Jetbot	Jetbot/1.0	JetEye
Jobs.de-Robot	Mozilla/5.0 (compatible; jobs.de-Robot http://www.jobs.de; jobsde@jobscout24.de) ( newsexpress e-mail: newsexpress-l@neofonie.de http://www.neofonie.de/loesunge n/search/robot.html )	Neofonie
Jongaimpi	jongaimpi/2.10 (jonga; http://www.jonga.co.za; info@jonga.co.za)	Jonga
Jyxobot	Jyxobot/1	Jyxo
	Jyxobot/x
K2 Spider	k2spider	Verity
KAIST AITrc Crawler	KAIST AITrc Crawler	AITrc
KFSW-Bot	KFSW-Bot (Version: 1.01, powered by KFSW, www.kfsw.de)	KFSW
KIT_Fireball	KIT_Fireball/2.0	Dino-online
KSbot	KSbot/1.0 (KnowledgeStorm crawler; http://www.knowledgestorm.com/resources/conte nt/crawler/index.html; crawleradmin@knowledgestorm.com)	Knowledgestorm
KakleBot	KakleBot - www.kakle.com/0.1 (KakleBot - www.kakle.com; http:// www.kakle.com/bo t.html; support@kakle.com)	akle
KaloogaBot	kalooga/KaloogaBot (Kalooga; http://www.kalooga.com; info@kalooga.com)	Kalooga
Kasparek	Firefox_1.0.6 (kasparek@naparek.cz)	Czech Technical University Prague
Keegeebot	Keegeebot/2.1 (+http://www.keegee.com/keegee/bot.html)	Keegee
Kenjin Spider	Kenjin Spider	Kenjin
Kevin	Kevin http://dznet.com/kevin/	Dznet.com
	Kevin http://websitealert.net/kevin/
KicktooBot	kicktooBotV1.1 kictooBot@kictoo.com	Kicktoo
Kinja-imagebot	kinja-imagebot (http://www.kinja.com/)	Kinja
Kinjabot	kinjabot (http://www.kinja.com)	Kinja
KnowItAll	KnowItAll(knowitall@cs.washington.edu)	University of Washington
Knowledge.com	Knowledge.com/0.x	knowledge.com
Krugle	Krugle/Krugle,Nutch/0.8+ (Krugle web crawler; http://www.krugle.com/crawler/info .html; webcrawler@krugle.com)	Krugle
Kulokobot	kuloko-bot/0.x	Kuloko
	kulokobot www.kuloko.com kuloko@backweave.com
Kulturarw	kulturarw3/0.1	National Library of Sweden
Kumm	KummHttp/1.1 (compatible; KummClient; Linux rulez)	Sanoma
Kyluka crawl	Mozilla/5.0 (compatible; Kyluka crawl; http://www.kyluka.com/crawl.html; crawl@k yluka.com)	Kyluka
LECodeChecker	LECodeChecker/3.0 libgetdoc/1.0	Linkexchange
LNSpiderguy	LNSpiderguy	Lexis-Nexis
LapozzBot	LapozzBot/1.4 ( http://robot.lapozz.com)	Lapozz
	LapozzBot/1.4 (+http://robot.lapozz.hu)
	LapozzBot/1.5 (+http://robot.lapozz.hu)
Larbin_2.6.3	larbin_2.6.3 larbin2.6.3@unspecified.mail	Unknown
	larbin_2.6.3 marzia.polito@intel.com
Lawinfo-crawler	lawinfo-crawler/Nutch-0.9-dev (Crawler for lawinfo.com pages; http://www.lawinfo .com; webmaster@lawinfo.com)	Lawinfo
Lemur Consulting	larbin_2.6.2 (tom@lemurconsulting.com)	Lemur Consulting
Lexibot	LexiBot/1.00	BrightPlanet
	Mata Hari/2.00
Liafa	larbin_2.2.2_guillaume (guillaume@liafa.jussieu.fr)	Liafa
LibWeb	libWeb/clsHTTP -- hiongun@kt.co.kr	Korea Telecom
LibertyW	LibertyW (+http://www.lw01.com)	LibertyW
	LibertyW (+http://www.libertyw.eu)
LijitSpider	LijitSpider/Nutch-0.9 (Reports crawler; http://www.lijit.com/; info(a)lijit(d)co m)	Lijit
LinkWalker	LinkWalker	Seven TwentyFour
Linknzbot	linknzbot	LinkNZ
Links2Go	Mozilla/3.01 (Compatible; Links2Go Similarity Engine)	Links2Go
Links4US-Crawler	Links4US-Crawler, (+http://links4us.com/)	Links4US
LinksManager.com_bot	Mozilla/5.0 (compatible; LinksManager.com_bot +http://linksmanager.com/linkcheck er.html)	Unknown
Llaut	Llaut/1.0 (http://mnm.uib.es/~gallir/llaut/bot.html)	Universitat de les Illes Balears
Lmspider	lmspider (lmspider@scansoft.com)	Nuance
LocalBot	LocalBot/1.0 ( http://www.localbot.co.uk/)	LocalBot
Lockstep Spider	Lockstep Spider/1.0	Lockstep
Look.com	NetResearchServer(http://www.look.com)
LookdirBot	LookdirBot	Lookdir
Lovel	Lovel as 1.0 ( +http://www.everatom.com)	Everatom
Ltaa_web_crawler	larbin_2.6.3 (ltaa_web_crawler@groupes.epfl.ch)	Ecole Polytechnique F?d?rale de Lausanne
Luchs.at URL checker	luchs.at URL checker	Luchs
Lycos_Spider	Lycos_Spider_(T-Rex)	Lycos
	Mozilla/4.0 (compatible; MSIE 5.0; Windows 98;Lycos_Spider_Beta2(T-Rex) ; Lycos_ Spider_Beta2(T-Rex) )
	Mozilla/4.0 (compatible; MSIE 5.0; Windows 98;Lycos_Spider_(T-Rex) ; Lycos_Spide r_(T-Rex) )
	Lycos_Spider_(modspider)
MJ12bot	MJ12bot/v1.1.2 (http://majestic12.co.uk/bot.php?+)	Majestic 12
	MJ12bot/v1.0.7 (http://majestic12.co.uk/bot.php?+)
	MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)
	MJ12bot/v1.1.1 (http://majestic12.co.uk/bot.php?+)
	MJ12bot/v1.2.0 (http://majestic12.co.uk/bot.php?+)
	MJ12bot/vx.x.x (http://www.majestic12.co.uk/projects/dsearch/mj12bot.php)
MQBOT	MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://falcon.cs.uiuc.edu; mqbot@cs.ui uc.edu)	University of Illinois
MSN Bot	msnbot/1.1 (+http://search.msn.com/msnbot.htm)	MSN
	msnbot-media/1.0 (+http://search.msn.com/msnbot.htm)
	msnbot/0.9 (+http://search.msn.com/msnbot.htm)
	Mozilla/4.0 (compatible; MSIE 6.0; Windows NT; MS Search 4.0 Robot)
	msnbot/1.0 (+http://search.msn.com/msnbot.htm)
	msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)
	msnbot/0.3 (+http://search.msn.com/msnbot.htm)
	msnbot-Products/1.0 (+http://search.msn.com/msnbot.htm)
MSNBOT_Mobile	MSNBOT_Mobile MSMOBOT Mozilla/2.0 (compatible; MSIE 4.02; Windows CE; Default)	MSN
MSRBOT	MSRBOT (http://research.microsoft.com/research/sv/msrbot)	Microsoft
MSRBot	MSRBOT (http://research.microsoft.com/research/sv/msrbot/)	Microsoft
	MSRBOT (http://research.microsoft.com/research/sv/msrbot/
MaSagool	MaSagool/1.0 (MaSagool; http://sagool.jp/; info@sagool.jp)	Sagool
Mail.Ru	Mail.Ru/1.0	Mail.Ru
Mainseek_Bot	Mozilla/5.0 (compatible;MAINSEEK_BOT)	Mainseek
Mammoth	mammoth/1.0 (+http://www.sli-systems.com/)	SLI Systems
	Mozilla/5.0 (+http://www.sli-systems.com/) Mammoth/0.1
	Mozilla/5.0 (+http://www.eurekster.com/mammoth) Mammoth/0.1
MantraAgent	MantraAgent	LookSmart
Mariner	Mariner/5.1b [de] (Win95; I ;Kolibri gncwebbot)	Kolibri
Martini	MARTINI	LookSmart
	Martini
Marvin	Marvin v0.3	Health On the Net Fondation
Masterseek	MasterSeek	Masterseek
Maxbot	Spider/maxbot.com admin@maxbot.com	Maxbot
Maxomobot	maxomobot/dev-20051201 (maxomo; http://67.102.134.34:4047/MAXOMO/MAXOMObot.html; maxomobot@maxomo.com)	Maxomo
MediaCrawler	MediaCrawler-1.0 (Experimental)	Media Find
MediaSearch	MediaSearch/0.1	WWW.FI
Mediater Rechercher	libwww/5.3.2	Mediater
MegaSheep	MegaSheep v1.0 (www.searchuk.com internet sheep)	Search UK
Megaglobe Crawler	Mozilla/5.0 (compatible; Megaglobe Crawler/1.0; http://www.megaglobe.com)	Megaglobe
Melbot WebSpider	Melbot WebSpider & RSS News Crawler www.melbot.info (V.2.42 by A.I.C.E.)	Melbot
Mercator	Mercator-2.0	Altavista
	Mercator-Scrub-1.1
	Mercator-1.x
Merl.com	larbin_2.1.1 larbin2.1.1@somewhere.com	Mitsubishi Electrical Research Lab
MetaGer_PreChecker	MetaGer_PreChecker0.1	MetaGer
Metacarta	Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) (samualt9@bigfoot.com)	Metacarta
	Mozilla/5.0 (compatible; heritrix/1.5 +http://www.metacarta.com)
Metaeuro Web Crawler	Metaeuro Web Crawler/0.2 (MetaEuro Web Search Clustering Engine; http://www.meta euro.com; crawler at metaeuro dot com)	Metaeuro
Metager-Linkchecker	MetaGer-LinkChecker	Metager
MetagerBot	MetagerBot/0.8-dev (MetagerBot; http://metager.de; )	Metager
Metaquerier	MQbot http://metaquerier.cs.uiuc.edu/crawler	University of Illinois
	MQbot metaquerier.cs.uiuc.edu/crawler
Metaspinner	Metaspinner/0.01 (Metaspinner; http://www.meta-spinner.de/; support@meta-spinner .de/)	Metaspinner
Metatagsdir	metatagsdir/0.7 (+http://metatagsdir.com/directory/)	Metatagsdir
Microsoft Small Business Indexer	Microsoft Small Business Indexer	Microsoft
Microsoft URL Control	Microsoft URL Control - 6.01.9782	Unknown