Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Evaluation for operational IR applications: generalizability and automation
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.
2013 (English)In: LivingLab '13 Proceedings of the 2013 workshop on Living labs for information retrieval evaluation, New York: Association for Computing Machinery (ACM), 2013, 11-12 p.Conference paper, Published paper (Refereed)
Abstract [en]

Black box information retrieval (IR) application evaluation allows practitioners to measure the quality of their IR application. Instead of evaluating specific components, e.g. solely the search engine, a complete IR application, including the user's perspective, is evaluated. The evaluation methodology is designed to be applicable to operational IR applications. The black box evaluation methodology could be packaged into an evaluation and monitoring tool, making it usable for industry stakeholders. The tool should lead practitioners through the evaluation process and maintain the test results for the manual and automatic tests. This paper shows that the methodology is generalizable, even though the diversity of IR applications is high. The challenges in automating tests are the simulation of tasks that require intellectual effort and the handling of different visualizations of the same concept.

Place, publisher, year, edition, pages
New York: Association for Computing Machinery (ACM), 2013. 11-12 p.
Keyword [en]
Information retrieval, application evaluation, user perception
National Category
Information Systems
Research subject
Man-Machine-Interaction (MMI)
Identifiers
URN: urn:nbn:se:su:diva-97726DOI: 10.1145/2513150.2513160ISBN: 978-1-4503-2420-5 (print)OAI: oai:DiVA.org:su-97726DiVA: diva2:679956
Conference
ACM International Conference on Information and Knowledge Management (CIKM 2013), October 27th - November 1st, 2013, Burlingame, CA, USA
Available from: 2013-12-17 Created: 2013-12-17 Last updated: 2013-12-19Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Hansen, Preben
By organisation
Department of Computer and Systems Sciences
Information Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 11 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf