evaluation by highly relevant documents 2001
TRANSCRIPT
-
8/9/2019 Evaluation by Highly Relevant Documents 2001
1/9
Evaluation by Highly Relevant Documents
Ellen M. VoorheesNational Institute of Standards and Technology
Gaithersburg, Maryland [email protected]
ABSTRACT
1. INTRODUCTION
This paper is authored by an employee of the United States Government andis in the public domain.SIGIR01, September 9-12, 2001, New Orleans, Louisiana, USA..Copyright 2001 ACM 1-58113-331-6/01/0009 ... 5.00.
2. THE TREC-9 MAIN WEB TASK TESTCOLLECTION
-
8/9/2019 Evaluation by Highly Relevant Documents 2001
2/9
3. EVALUATION BY MULTIPLE RELE-
VANCE LEVELS
3.1 Using only highly relevant documents
-
8/9/2019 Evaluation by Highly Relevant Documents 2001
3/9
-
8/9/2019 Evaluation by Highly Relevant Documents 2001
4/9
-
3.2 Discounted cumulative gain
-
8/9/2019 Evaluation by Highly Relevant Documents 2001
5/9
-
-
-
-
4. BEST DOCUMENT EVALUATION
-
8/9/2019 Evaluation by Highly Relevant Documents 2001
6/9
0 20 40 60 80 100
Rank
5
10
AverageDiscountedCumulativeGain
Highly relevant gain = 1
0 20 40 60 80 100
Rank
5
10
15
AverageDiscountedCumulativeGain
Highly relevant gain = 2
0 20 40 60 80 100
Rank
5
10
15
20
25
AverageD
iscountedCumulativeGain
Highly relevant gain = 5
0 20 40 60 80 100
Rank
10
20
30
40
AverageD
iscountedCumulativeGain
Highly relevant gain = 10
0 20 40 60 80 100
Rank
100
200
300
AverageDiscountedCumulativeGain
Highly relevant gain = 100
0 20 40 60 80 100
Rank
1000
2000
3000
AverageDiscountedCumulativeGain
Highly relevant gain = 1000
-
8/9/2019 Evaluation by Highly Relevant Documents 2001
7/9
-
8/9/2019 Evaluation by Highly Relevant Documents 2001
8/9
-
5. CONCLUSION
6. REFERENCES
-
8/9/2019 Evaluation by Highly Relevant Documents 2001
9/9