trec 2016: looking forward panel
TRANSCRIPT
![Page 2: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/2.jpg)
Q: “TREC Anniversary”
![Page 3: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/3.jpg)
Top Result: 50 years of Star Trek
(Article on the Verge about Facebook Like buttons)
![Page 4: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/4.jpg)
Science Fiction Defining a TREC task or a track is like time-travel in Back
to the Future
Note to the audience: that is just 74 characters
You could even add the hashtag #TREC #TRECCelebrations and my Twitter handle @arjenpdevries
![Page 5: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/5.jpg)
![Page 6: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/6.jpg)
Better Search – “Deep Personalization” “Even more broadly than trying to get people the right
content based on their context, we as a community need to be thinking about how to support people through the entire search experience.”
Jaime Teevan on “Slow Search”
Search as a dialogue
My first journal paper: De Vries, Van der Veer and Blanken: Let’s talk about it: dialogues with multimedia databases (1998)
![Page 7: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/7.jpg)
Moving Forward Elements of the “Slow Search movement” at TREC today:
- Sessions- Tasks- Dynamic domains- Total recall- Complex Answer Retrieval (new!)
![Page 8: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/8.jpg)
Missing from TREC! Access to rich personal data including email, browsing
history, documents read and contents of the user’s home directory…
![Page 9: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/9.jpg)
![Page 10: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/10.jpg)
Trade log data!
IR-809: (2011) Feild, H., Allan, J. and Glatt, J., "CrowdLogging: Distributed, private, and anonymous search logging," Proceedings of the International Conference on Research and Development in Information Retrieval (SIGIR'11), pp. 375-384. [View bibtex]We describe an approach for distributed search log collection, storage, and mining, with the dual goals of preserving privacy and making the mined information broadly available. [..] The approach works with any search behavior artifact that can be extracted from a search log, including queries, query reformulations, and query-click pairs.
![Page 11: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/11.jpg)
Open challenges How to select the part of your log data you are willing to
trade?
How to estimate the value of this log data?
And a social challenge, not so much scientific:How to get people to participate?
![Page 12: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/12.jpg)
Branding
![Page 13: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/13.jpg)
Branding (NL)
![Page 14: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/14.jpg)
The TREC Brand A community that creates reusable test collections
![Page 15: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/15.jpg)
Extra Slides
![Page 16: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/16.jpg)
Reproducibility vs. Representativeness Increasing representativeness of a TREC task should not
come at the cost of sacrificing reproducibility
(104 characters )
Samar, T., Bellogín, A. & de Vries, A.P. Inf Retrieval J (2016) 19: 230. doi: 10.1007/s10791-015-9276-9
![Page 17: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/17.jpg)
Baltimore
![Page 18: TREC 2016: Looking Forward Panel](https://reader034.vdocuments.net/reader034/viewer/2022042908/58f23eb01a28abd6648b4573/html5/thumbnails/18.jpg)
Baltimore Title query of TREC topic 478 for the information need “Who is
the mayor of Baltimore”
“The honest conclusion of this year’s evaluation should be that we underestimated the problem of handling Web data. Surprising is the performance of the title-only queries doing better than queries including description or even narrative. It seems that the web-track topics are really different from the previous TREC topics in the ad-hoc task, for which we never weighted title terms different from description or narrative.”
(Quote from the CWI TREC-9 paper)