Search

MetaCrystal aims to help users find the documents that are most likely to be relevant. It compares the results returned by multiple search engines and displays the documents based on the number of engines that found them and their rank positions in the result lists. First, this paper examines the validity of the MetaCrystal design approach by calculating the probability that a document is relevant based on the number of engines that found it and its rank positions in the result lists. Using TREC data, it is shown that MetaCrystal’s visual design is sound because it visually emphasizes documents based on their likelihood of being relevant. Second, this paper investigates if users are able to use the MetaCrystal‘s visualizations to find the documents that are most likely to be relevant. The results of a user study are reported that show that novice users are able to accomplish this task.

This paper provides an analysis of the overlap between the search results of retrieval systems that participated in TREC 3, 6, 7 and 8 to provide empirical support for some of the key assumptions guiding the design of data fusion methods. It shows that the potential relevance of a document increases exponentially as the number of search methods retrieving it increases – called the Authority Effect. It also shows that documents higher up in ranked lists and found by more systems are more likely to be relevant – called the Ranking Effect. Both effects can help explain why major data fusion methods perform well.