Mixed-Modality Interaction in Conversational Recommender Systems
Ma, Y., Kleemann, T., & Ziegler, J. (2021). Proceedings of the 8th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems, 2948, 21–37.
Abstract
Recent advances in natural language processing have made modern chatbots and Conversational Recommender Systems (CRS) increasingly intelligent, enabling them to handle more complex user inputs. Still, the interaction with a CRS is often tedious and error-prone. Especially when using written text as the form of conversation, the interaction is often less efficient in comparison to conventional GUI- style interaction. To keep the flexibility and mixed-initiative style of language-based conversation while leveraging the efficiency and simplicity of interacting through graphical widgets, we investigate the de- sign space of integrating GUI elements into text-based conversations. While simple response buttons have already been used in chatbots, the full range of such mixed-modality interactions has not yet been investigated in existing research. We propose two design dimensions along which integrations can be defined and analyze their applicability for preference elicitation and for critiquing the CRS’s responses at different levels. We report a user study in which we investigated user preferences and perceived usability of different techniques based on video prototypes.
