Tuesday, March 7, 2006

Search: Yahoo > Google: Jakob Nielsen made an interesting comment on my comparative study of search engines (on John Battele's blog). He explains that users often scan the first few hits before deciding where to click, as eyetracking studies show (the famous golden triangle). They probably use proximal cues to quickly assess the comparative quality of results. [...] Again, Yahoo is first, with 67.1% of screens containing at least one result graded 5 (only 61.4% for Google). However, if only the upper half of the screens is used, Google has a slight advantage. (Via Technologies du Langage.) The title of this story is misleading. It is based on just one measure of effectiveness, average of relevance scores of the top 10 results. Why this measure? Researchers in information retrieval typically use measures of effectiveness like mean reciprocal rank or average precision that favor algorithms that rank relevant documents more highly, unlike the measure that Veronis privileged here. It would be interesting to have those measures for the data collected by his students. 11:05:05 AM