Running a web search engine is a very expensive venture. When a user enters a search query they demand instant results: in tenths of a second the engine must run through trillions of documents, determine the most relevant ones and finally rank them to give the user the best results for their specific situation.
For this reason, search engines must take many shortcuts – by understanding the process of returning results to a user, you can build websites which have a better chance of being returned to a user and ranked highly.
This guide explores the four common aspects of search engines: crawling, indexing, information retrieval and ranking. Commercial search engines are infinitely more complicated than is presented here, but the basics are constant and arguably are more important than the ever-changing minutiae.