Development of an Instrument for Measuring Users’ Perception of Transparency in Recommender Systems
Hellmann, M., Hernandez Bocanegra, D., & Ziegler, J. (2022). In A. Smith-Renner & O. Amir (Eds.), Workshops at the International Conference on Intelligent User Interfaces (IUI) 2022: Proceedings of the IUI 2022 Workshops: APEx-UI, HAI-GEN, HEALTHI, HUMANIZE, TExSS, SOCIALIZE (Vol. 3124, pp. 156–165). RWTH Aachen.
Abstract
Transparency is increasingly seen as a critical requirement for achieving the goal of human-centered AI systems in general and also, specifically, recommender systems (RS). However, defining and operationalizing the concept is still difficult, due to its multi-faceted nature. Currently, there are hardly any measurement instruments to adequately assess the perceived transparency of RS in user studies. Thus, we present the development of a measurement instrument that aims at capturing perceived transparency as a multidimensional construct. The results of our validation show that transparency can be distinguished with respect to input (what data does the system use?), functionality (how and why is an item recommended?), output (why and how well does an item fit one’s preferences?), and interaction (what needs to be changed for a different prediction?). The study is intended as a first iteration in the development of a reliable and fully validated measurement tool for assessing transparency in RS.
