Spider:
A spider is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index.
Home » What is Spider? Why is it important | GVATE LLC
A spider is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index.