Mental Models, Explanations, Visualizations: Promoting User-Centered Qualities in Recommender Systems
Kunkel, J. (2022). [PhD thesis, University of Duisburg-Essen].
PhD thesis by Johannes Kunkel
Abstract
Recommender systems (RSs) are powerful tools that proactively suggest a set of personalized items to users. In doing so, they aim to predict the preferences of their users, wherein they are considered to be very accurate. In addition to algorithmic precision, user-centered qualities have recently been increasingly taken into account when evaluating the success of RSs. Examples for such qualities include the transparency of an RS, the control users are able to exert over their recommendations, and the means of exploring the item space in context of recommendations. However, research on aspects focused on human-computer interaction in RSs is still at a rather early stage. The main focus of the present thesis is to study and design RSs more holistically. In this regard, the mental models that users create of RSs are explored, explanations and their impact on user-centered variables of RSs are investigated, and techniques from information visualization (InfoVis) are applied to let users scrutinize the global context of their recommendations. The results of this research and the contributions I make to the state of the art in this context are described in greater detail below. A key contribution of this thesis consists of the results of two studies that shed light on the mental models that users of RSs develop and how these models influence the users’ perception of different system qualities. A key finding of the first, qualitative study is that many mental models tend to follow a procedural structure that can be used, for instance, as a template for designing explanations to promote transparency in RSs. In the second study, which relied on a larger sample and thus allowed quantitative conclusions, this type of procedurally structured mental models was found to correlate with a high perception of system transparency and confidence in the users’ own comprehension of the inner workings of the system. Apart from that, some users seemed to humanize the RS, assigning attributes such as “social”, “organic”, and “empathic”. Such a comprehension of the system was accompanied by higher levels of trust—a finding that may be leveraged by system designers. In general, mental models that deviate greatly from the actual functioning of the system should be corrected so that they do not lead to false expectations on the part of the users and hence to a potentially rejection of recommendations. A prominent method for improving system transparency and thus the soundness of users’ mental models is to provide textual explanations along with the recommendations. These explanations usually follow a very simple scheme based on similarity—especially in productive environments. To investigate implications of such simple explanations, another experiment contained in this thesis asked users to explain recommendations in their own words and compared them to explanations automatically generated by a system. The results indicate many benefits of providing more extensive explanations for recommendations, such as increased trust and higher perceived quality of recommendations. Another finding is that many participants, as opposed to the system, provided a broader context of the decision behind their recommendation. The extent to which textual explanations can provide context for recommendations is limited,though. While a local context is relatively easy to explain textually—e.g. by linking recommendations to a user’s preferences—it is difficult, if not impossible, to provide users with a global context. Such a global context would need to explain the relationship of recommendations to all other items in the dataset from which a RS selects its candidates. Comprehending such an item space at a global scale can unlock several beneficial properties of an RS, such as preventing filter bubbles, fostering creativity, and encouraging a user’s self-development. In this thesis, I argue that to provide such a global context, RSs should go beyond explaining recommendations textually and better exploit the capabilities of computer systems compared to humans. Three of the six papers included in this cumulative dissertation explore how methods of InfoVis can be applied to RSs to provide users with a global context of recommendations and how this affects the users’ perception of these systems. One result of these studies is that even simple means of representing the item space can already successfully convey a sense of overview over the item space and provide transparency for recommendations. However, another finding is that artificial maps that distribute all items on a two-dimensional plane according to their similarity are a promising visualization style that can be used to deeply integrate means of interactively controlling recommendations into the visualization of the item space. Such maps have also been found to trigger user excitement, which can also influence the perception of recommendations. In another experiment, we found that a treemap can also be used as a control panel for a RSs. The results of this experiment further underline that treemaps can effectively alert their users to potential biases or blind spots in their preference profile. In this thesis, I discuss such implications of the InfoVis method to depict the item space of RSs. Finally, in this thesis I take an elevated perspective on the findings of the papers contained and argue that researchers should consider user-centered aspects of RSs more holistically, for instance, in terms of the deep interconnectedness of perceptual variables. In this sense, I observed that the user experience of an application can influence as how novel recommendations are perceived to be, and that the degree of overview of the item space users are able to obtain can positively affect the perceived quality of recommendations. This thesis represents thus a further argument for looking at RSs from a highly user-centered viewpoint.
