1. Emotion detection from touch interactions during text entry on smartphones.
- Author
-
Ghosh, Surjya, Hiware, Kaustubh, Ganguly, Niloy, Mitra, Bivas, and De, Pradipta
- Subjects
- *
KEYBOARDS (Electronics) , *TOUCH , *DESIGN techniques , *EMOTIONS , *MACHINE learning - Abstract
• We design, develop, deploy an Android application TouchSense, which traces different touch interactions and develop a personalized machine learning model by jointly modeling the different types of touch interaction features. • We trace different touch interactions like typing, swyping in a smartphone and perform a detailed analysis to find the role of individual and combined influence of these interactions on emotion detection. • We carry out a detail user study to find the suitability of this model. We also perform a post-study user analysis to find the usability of the proposed application. There are different modes of interaction with a software keyboard on a smartphone, such as typing and swyping. Patterns of such touch interactions on a keyboard may reflect emotions of a user. Since users may switch between different touch modalities while using a keyboard, therefore, automatic detection of emotion from touch patterns must consider both modalities in combination to detect the pattern. In this paper, we focus on identifying different features of touch interactions with a smartphone keyboard that lead to a personalized model for inferring user emotion. Since distinguishing typing and swyping activity is important to record the correct features, we designed a technique to correctly identify the modality. The ground truth labels for user emotion are collected directly from the user by periodically collecting self-reports. We jointly model typing and swyping features and correlate them with user provided self-reports to build a personalized machine learning model, which detects four emotion states (happy, sad, stressed, relaxed). We combine these design choices into an Android application TouchSense and evaluate the same in a 3-week in-the-wild study involving 22 participants. Our key evaluation results and post-study participant assessment demonstrate that it is possible to predict these emotion states with an average accuracy (AUCROC) of 73% (std dev. 6%, maximum 87%) combining these two touch interactions only. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF