If you never mind, I would like to correct your question ! There are two separate terms called Spider and Crawler. Spider is a program that downloads web pages, whereas Crawler is a program that automatically follows all of the links on each web page. And now comes Indexer which is a program that analyzes web pages downloaded by the spider and crawler. According to your question, Cache: Since the content of Web pages can change, the caching software is always checking for newer versions of the page and downloading them.