Search tool sizes


Bigger is not necessarily better, though it can be for some unusual or hard-to-find information, and a large index means they have a greater chance of locating it. When working with larger indexes you need to use more advanced search features.

In this section we will look at three different comparisions for the most common search engines. The comparisions are based on the size of the directory, the size of the search engine index and the percentage of Internet indexed by search engine. All of the information on this page is from Search Engine Watch


Directory Size

Directories are usually human-compiled guides to the web, where sites are organized by category. The chart below compares the size of directories at various services, along with other key data. A ? symbol indicates where information is not known or hasn't been released.

Service Type Editors Cats Links... ..per day As Of
Yahoo D 80+ ? 1.2 million+ about 750 4/99
LookSmart D 160 24,000 800,000 8,000 4/99
Infoseek SE ? 50,000 500,000 ? 12/98
Open Directory D 10,200 70,000 500,000 2,000 5/99
Snap D 60 50,000 400,000 ? 5/99
eBLAST D ? ? 125,000 ? 7/98
AskJeeves AS 30 n/a 7 million answers ? 11/98
AltaVista SE See LookSmart
Excite SE Unknown for all categories. Excite's directory also powers WebCrawler and portions of Netscape's channels.
HotBot SE See Open Directory
Lycos D See Open Directory
MSN Search SE See LookSmart
Netscape SE See Open Directory

Type: Shows whether a service is primarily a directory (D) or a search engine (SE). AskJeeves is an answer service (AS) and is more like a directory, as human beings compile its listings.

Editors: Shows how many people are involved in producing the listings. More is not necessarily better, as some services claim that technology helps them do more. For example, the Lycos directory is automatically created using its WiseWire technology. That essentially means it has no editors involved (and thus the n/a on the chart). However, a large number of editors is a good sign that a quality directory is being built and keeping up with the growth of the web.

Cats: Shows how many categories each directory has.

Links: Shows how many individual links exist in the directory. Entire sites may be listed, or individual web pages may be listed, depending on their exact content. Some sites may be listed in more than one category.

Per Day: Shows how many links are added per day.

As Of: Shows how current the information is, usually drawn from recent interviews I've conducted. In the case of Yahoo, numbers are estimates based on those from its 1998 financial filings.


Size of the Index

AV = Altavista, NL = Northern Light, INK = Inktomi(Hotbot), FAST = Fast
EX = Excite, LY = Lycos, IS = Infoseek, WC = Webcrawler

Numbers are as of May 1999


AV = Altavista, NL = Northern Light, INK = Inktomi(Hotbot), FAST = Fast
EX = Excite, LY = Lycos, IS = Infoseek, WC = Webcrawler

Numbers are as of May 1999