The NEC Research Institute of Princeton, NJ recently surveyed 11 Internet search engines and concluded that none has indexed more than 16 percent of the more than 800 million accessible Web pages. While some search engines have superior coverage, they often have a large number of invalid links-links to pages which no longer exist.
The Institute submitted 1,050 identical queries to the 11 search engines, determining each one's "hits" as a percentage of the total links recorded by all of the search engines. Northern Light led the way with 38.3 percent coverage, but it also had the second highest rate of invalid links. In contrast, HotBot with 27.1 percent coverage had very few invalid links. Alta Vista, which had 37.1 percent coverage, was the most up-to-date.
The research, which was undertaken as a market study for several unnamed clients, concluded that none of the search engines is keeping up with the explosive growth of the Web. Therefore, several engines should be used in order to take full advantage of the Web.
The study did not look at meta-search tools, those which simultaneously search several search engines. Among the widely used metasearch tools are Internet Sleuth (www.isleuth.com), MetaCrawler (www.go2net.com), Profusion (www.profusion.com), and Savvy Search (,a href="http://www.savvysearch.com">www.savvysearch.com). While using a metasearch tool has the advantage of retrieving more pages, users can be frustrated by the fact that some of the tools ignore Boolean expressions, quote marks, asterisks, and other symbols used by advanced searchers.
An excellent article on meta-search tools appeared in Computers in Libraries earlier this year. The citation is: Repman, Judi and Randal D. Carlson, "Surviving the Storm: using Metasearch Engines Effectively," Computers in Libraries, May 1999, pp. 50-55.
There is a useful Web site which offers advice in choosing a search engine: www.albany.edu/library-internet/choose. The site solicits what the user is seeking to do, then provides links to search engines and metasearch tools which are suitable.
