While there are established fundamental differences between most search engines and the way they work, search engines can be generalized based on three basic tasks that they all perform:
They “search” the Internetà find pieces of information on the internet based on a few keywords that we (the users) provide them.
They keep an index, a sort of record, of all the information they find, and where exactly they find it (the web address).
They even allow users to search for combinations of the words that they found and kept in their index.
If you have read the page about the history of search engines and the detailed story of how they came about, you would figure that the first few search engines had an extremely limited scope. They held an index (or record) of a few hundred odd sites and usually did not handle any more than a couple of hundred search queries in a day.
Today, obviously, things are vastly different. Evident from our basic dependence on search engines, their scope has magnified beyond imagination. Today even a search engine that’s not the most popular one, will easily index hundreds of millions of pages in a day and go on to handle tens of millions of queries in a day.
Fascinating isn’t it? The obvious next question all of us are asking, ‘how do they do this?’ How does a single search engine handle such vast loads of information, sift through all of that, and yet manage to present to us exactly what we were looking for in a matter of half a second?
