Hi Every one ,
A crawler is a program of search engines which visit on the web pages automatically to read, store and index them; whatever on the web pages. Its another name are web spider, web robot, ants, bots, worms etc. but here I am talking about the name of major search engines spider. Below is the list -
Search Engines -- Spiders or Crawler Name
Google -- Googlebot
Yahoo -- Yahoo Slurp
MSN -- MSNbot
Ask -- Teoma
Altavista -- Scooter
Excite -- Architext spider
Inktomi.com -- Slurp
Alexa -- ia_archiver
Looksmart -- MantraAgent
Lycus -- Lycos_Spider_(T-Rex)
Cuil -- Twiceler.
A crawler is a program of search engines which visit on the web pages automatically to read, store and index them; whatever on the web pages. Its another name are web spider, web robot, ants, bots, worms etc. but here I am talking about the name of major search engines spider. Below is the list -
Search Engines -- Spiders or Crawler Name
Google -- Googlebot
Yahoo -- Yahoo Slurp
MSN -- MSNbot
Ask -- Teoma
Altavista -- Scooter
Excite -- Architext spider
Inktomi.com -- Slurp
Alexa -- ia_archiver
Looksmart -- MantraAgent
Lycus -- Lycos_Spider_(T-Rex)
Cuil -- Twiceler.