How they fared: search result performance compared
Evaluating search engine results isn't an exact science, and test results will change from one day to the next as each engine reindexes the Web. Keeping that in mind, here's how we tested search results from the various engines.
We ran a series of scripts through each engine: small/medium-size business, student, shopping, people search, news, financial research, and local information. We used exactly the same search terms and configurations for each engine; we then tallied the number of total search hits and noted the rank of the first relevant search result. All of our test searches were performed during the afternoon of October 27, 2004.
| |
A9 |
AltaVista |
Ask
Jeeves |
Google |
HotBot |
MSN |
Yahoo |
| SMB -- "Arizona Partnership Income Tax Return" (or any other state tax filing form) |
| Total number |
3 |
19 |
6 |
16 |
4 |
4 |
21 |
| Student -- "economics Andorra" (economics of a very small democracy in Central Europe) |
| Total number |
43,400 |
121,000 |
12,000 |
168,000 |
15,138 |
14,722 |
117,000 |
| Shopping -- "Saab 92x" (specific make and model of auto) |
| Total number |
145 |
207 |
262 |
603 |
47 |
42 |
230 |
| Personal (friends/family) -- (find a distant relative or lost friend) |
| Total number |
26 |
19 |
27 |
120 |
10 |
10 |
28 |
| News -- "Flu vaccination shortage" (list # of news items returned) |
| Total number |
16,100 |
57,000 |
8,590 |
38,800 |
9,042 |
9,234 |
52,200 |
| Financial research -- "Merck" (makers of Vioxx, among other medications) |
| Total number |
828,000 |
2,310,000 |
854,400 |
2,040,000 |
486,982 |
470,364 |
2,350,000 |
| Local information -- "Plumber + (zip code)" |
| Total number |
23 |
72 |
57 |
120 |
36 |
26 |
72 |
| |
| Total |
887,697 |
2,488,317 |
875,342 |
2,247,659 |
511,259 |
494,402 |
2,519,551 |
|
Returned most relevant search result first
Did not return most relevant search result first
 Return to intro page
Read the CNET editor's take
|
|
7.3 out of 10
6.7 out of 10
8.3 out of 10
