Search Engines and Directories on the Internet

[ Home | Web Design Portfolio | The Internet Tutor | Resume | Contact ]

Question: "Where is the best place to go to find sites with information on what I want?"

Answer: "Depends what you are trying to find."

There are two conduits to find the information, directories and search engines. Both have their strengths and weaknesses.

Directories are groupings of sites arranged in a hierarchical format. In other words, you look at broad categories and then select more narrowly defined categories in succession until you get to what you want. This is particularly good if you want all the sites that fall into a group (like "card games" instead of "poker," or "reptiles" instead of "boa constrictors").

It is important to note that all directories are "hand-selected," meaning that there are no automated "Web bots" indexing every word on every page. This is good because you are much less likely to get Web pages listed that have obscure references, but rather, ones that have the content that you want. This can also be bad. Since everything must be categorized, directories are relatively small (100,000 to 350,000) as opposed to pure search engines which may range up to a reported 50 million sites.

Yahoo! has long been the king of directories, but LookSmart is a close second. Both break down listings well and have search capabilities. I always check Yahoo! before any other resource. I usually find what I want.

If, however, I am getting too many entries for a broad category (i.e. news or sports sites) and I want to get the best quality Web sites from those, I look for Yahoo!'s "cool sites," as indicated by a pair of orange sunglasses beside the entry, or I jump to Magellan or PointCom.

On Magellan, by The McKinley Group, every site listed is reviewed and rated from one to four stars. It is easy to find top-ranked Web sites.

Originally called "PointCom's Top 5% of the Web," now owned and operated by Lycos (a search engine), the Top 5% symbol is still the most common way to designate that your page is among the best in the World. I use the PointCom site because of its thorough reviews and rankings by "content," "design" and "overall."

In contrast to directories, pure Web search engines are the only way to go when you need the nitty-gritty details or are searching for obscure topics.

Most major search engines do the same basic thing to get their listings. They send out "Web bots" or "spiders" to crawl the Internet and look for Web sites. As an example, a spider may stumble across a Web site on fishing. It may then index every word on that page. After indexing that page, it will check each link listed on that page, whether on the same site or not, and then read and index each of those pages, ad infinitum.

Here is the problem with large search engines. Let's pretend that someone created a great page about home repair and happened to casually mention needing to "cover the garden area when painting." If I went to a large search engine and typed in "garden" I might easily pull up that Web site just because the word "garden" was mentioned.

Fortunately, search engines are getting a little better and are starting to look at the number of times a word occurs on a page, the page's title and more, but can still be extremely frustrating getting hundreds of pages listed with little or nothing that you really want.

Large search engines also seem to have a higher percentage of out-of-date or missing pages than most directories, still, there is often no place else to find an obscure topic.

For the best large search engines, take your pick of AltaVista, Infoseek, Hotbot, Excite or Lycos. Each has its own merits, quirks and ways to search. Always check the "help" sections to find out how to maximize the effectiveness of your searches.

Finally, to find a near complete list of all search engines, check Beaucoup or Search.Com. Happy searching!