A Web crawler is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing. A Web crawler may also be called a Web spider, an ant, an automatic indexer, or (in the FOAF software context) a Web scutter. Web search engines and some other sites use Web crawling or spidering software to update their web ...
Found on
http://en.wikipedia.org/wiki/Web_crawler
this is an automatic program (sometimes called a 'robot') which explores the World Wide Web, following the links and searching for information or building a database - such programs are often used to build automated indexes for the Web, allowing users to do keyword searches for Web documents - it is also the name used by one of the most popular pub...
Found on
http://www.archivemag.co.uk/
No exact match found.
