Robot of indexation
An indexing robot is software that automatically scans the Web. It is usually designed to collect resources, in order to allow a search engine to index them. Running on the same principle, some malicious robots are used to archive resources or collect email addresses to send emails. In French, since 2013, crawler is replaceable by the word collector. There are also collectors who carefully analyze the contents in order to bring back only part of their information. Since the 1990s, automatic price comparators and then price / performance comparators for microprocessors have existed. Framed frameworks like Scrapy exist to write such robots.