Search engines are essentially, of two broad types. We will tell you about both types.
Crawler based search engines
The automatic sorts. A great example of a crawler based site is Google. These search engines create their listing automatically. The crawl through the web, make a list of what they have found and then people search through whatever they have found.
These search engines send programs called crawlers or spiders to scour the World Wide Web, find out the addresses of all the sites, their content, all too regularly update their indices.
If you had your own website, then your website will be listed on the search engines by just being on the Web, the spiders will eventually find you!
Once the spiders do eventually stumble upon your site, they will attempt to navigate through all the pages in your website and list all of them in your index. If there are pages that are buried very deeply in the layers of the website, some spiders might not attempt to crawl all those links to discover those pages. Of course, there are enough ways to make sure that you inhibit or shut out the spiders and prevent them from indexing your pages. Also, remember that the Web is a very large place, so the spiders might take some time to find your page (but they will surely get there eventually).
So, in summary, it’s free and automatic to get your website listed with a search engine. Sure, some search engines might charge you a fee to speed up the inclusion and some might charge a fee for you to submit your site to them (manually) for inclusion.
Also, if a website is changed, it is likely to be reflected on the crawler search engine only the next time the spider visits your page.
For example, if you have a website all about “European history”. The first time the crawler of a search engine finds your site, it will index your site as pages that have information about “European history”. However, if you subsequently change your website content completely and make it about “Egyptian history”, the search engine would still, for some time, think that your pages are about “European history”, and your site will still show up in a search query for “European history”, but not in a search query for “Egyptian history”. Only, after the spider visits your page (say some days later), will it realise that you have changed your page, and it will then list your page as containing information about “Egyptian history”. Then, if your try both queries, your page will show up in the results page for Egyptian history!
Human powered directories
The next kind of a search engine is called a human powered directory, like Open directory. These kinds of search engines depend on humans for their listings.
Again, let’s presume that you create a web page and you want it to appear on one of these directories. Then you need to submit a short description to the directory about your entire site, or the editors write a review for the site (that they have reviewed). A search query will make matched only for the descriptions that they have submitted. Evidently, unlike the crawler based search engines, these search engines do not know what all the pages on the Web contain.
Whereas in crawler based search engines, changing your site will eventually see a change in the search listing, here, changing your site will have no effect on its listing. Whenever you read the article on ‘search engine optimization’, things that they mention about improving your site listing on a search engine have nothing to do with their listing in a directory. The only exception might be a god site, with excellent content, that might get reviewed (and re-reviewed) for free than a poor site.
These search engines depend on a human reviewer. Your website is included or not included on their say and so in the manner they think is appropriate.
So let’s just go over the process of getting a website listed on a human-powered directory (since it evidently isn’t automatic). You must first submit your website if you are to be included and it may take a while before you come to the top of the list for even consideration. You need to select a category for your website to be potentially under. You then submit a short description to the directory for your website, or expect the editors for one of the sites to write it a review. A search would look for matcher only in the description of the sites submitted and not in the page content itself. If you change your site, it will only be reflected in the search engine listing if it is re-submitted and reviewed.
Also, there might be a payment for inclusion and there might even be an annual charge.
Hybrid search engines or mixed results
In the early days of the web, a search engine either presented crawler-based results or human-powered listings. Today, it is very common to see both typed of results to be presented. This is a hybrid search engine.
Usually, a hybrid will favour one type of listing over another. For example, Live Search (MSN) will present human powered listings from LookSmart, but also crawler based results (provided by Inktomi).
