Measuring the Impact of Recommender Systems – A Position Paper on Item Consumption in User Studies

Loepp, B., & Ziegler, J. (to appear). ImpactRS ’19: Proceedings of the 1st Workshop on Impact of Recommender Systems.

Abstract

While participants of recommender systems user studies usually cannot experience recommended items, it is common practice that researchers ask them to fill in questionnaires regarding the quality of systems and recommendations. While this has been shown to work well under certain circumstances, it sometimes seems not possible to assess user experience without enabling users to consume items, raising the question of whether the impact of recommender systems has always been measured adequately in past user studies. In this position paper, we aim at exploring this question by means of a literature review and at identifying aspects that need to be further investigated in terms of their influence on assessments in users studies, for instance, the difference between consumption of products or only of related information as well as the effect of domain, domain knowledge and other possibly confounding factors.

Resources

Related publications

Impact of Consuming Suggested Items on the Assessment of Recommendations in User Studies on Recommender Systems

Impact of Item Consumption on Assessment of Recommendations in User Studies

The Influence of Trust Cues on the Trustworthiness of Online Reviews for Recommendations

More »