Search Engines are using crawlers to find websites, think of them as bots. They go from one link, to another and another. There are of course lots of bots, not a single one. About the ranking google has to extract the perfect content on a topic to rank sites accordingly, they are using Latent Semantic Indexing then and yet again with the help of crawlers which build their database, so they can extract this perfect content, called also ContentDNA.