Getting at the Cognitive Complexity of Linguistic Metadata Annotation: A Pilot Study Using Eye-Tracking

Lohmann, S., Tomanek, K., Ziegler, J., and Hahn, U. (2010). Proceedings of the 32nd Annual Meeting of the Cognitive Science Society (CogSci 2010). Austin, TX: Cognitive Science Society.


We report on an experiment where the decision behavior of annotators issuing linguistic metadata is observed with an eye-tracking device. As experimental conditions we consider the role of textual context and linguistic complexity classes. Still preliminary in nature, our data suggests that semantic complexity is much harder to deal with than syntactic one, and that full-scale textual context is negligible for annotation, with the exception of semantic high-complexity cases. We claim that such observational data might lay the foundation for empirically grounded annotation cost models and the design of cognitively adequate annotation user interfaces.

Related focus areas


Related publications

A Cognitive Cost Model of Annotations Based on Eye-Tracking Data

Tomanek, K., Hahn, U., Lohmann, S., and Ziegler, J. (2010). Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010). Uppsala: ACL.

E-Quest: Ein Online-Befragungswerkzeug für Web Usability

El Jerroudi, Z., Ziegler, J., Meissner, S., and Philipsenburg, A. (2005). Mensch & Computer 2005: Kunst und Wissenschaft - Grenzüberschreitungen der interaktiven ART. München: Oldenbourg Verlag.

Usability Engineering für web-basierte Applikationen

Bullinger, h.-J., Heidmann, F., and Ziegler, J. (2002). it+ti Informationstechnik und Technische Informatik, 44(1), 5–13.