Search

Survey research aims to collect robust and reliable data from respondents. However, despite researchers’ efforts in designing questionnaires, survey instruments may be imperfect, and question structure not as clear as could be, thus creating a burden for respondents. If it were possible to detect such problems, this knowledge could be used to predict problems in a questionnaire during pretesting, inform real-time interventions through responsive questionnaire design, or to indicate and correct measurement error after the fact. Previous research has used paradata, specifically response times, to detect difficulties and help improve user experience and data quality. Today, richer data sources are available, for example, movements respondents make with their mouse, as an additional detailed indicator for the respondent–survey interaction. This article uses machine learning techniques to explore the predictive value of mouse-tracking data regarding a question’s difficulty. We use data from a survey on respondents’ employment history and demographic information, in which we experimentally manipulate the difficulty of several questions. Using measures derived from mouse movements, we predict whether respondents have answered the easy or difficult version of a question, using and comparing several state-of-the-art supervised learning methods. We have also developed a personalization method that adjusts for respondents’ baseline mouse behavior and evaluate its performance. For all three manipulated survey questions, we find that including the full set of mouse movement measures and accounting for individual differences in these measures improve prediction performance over response-time-only models., German Research Foundation (DFG), Peer Reviewed

Survey research aims to collect robust and reliable data from respondents. However, despite researchers’ efforts in designing questionnaires, survey instruments may be imperfect, and question structure not as clear as could be, thus creating a burden for respondents. If it were possible to detect such problems, this knowledge could be used to predict problems in a questionnaire during pretesting, inform real-time interventions through responsive questionnaire design, or to indicate and correct measurement error after the fact. Previous research has used paradata, specifically response times, to detect difficulties and help improve user experience and data quality. Today, richer data sources are available, for example, movements respondents make with their mouse, as an additional detailed indicator for the respondent–survey interaction. This article uses machine learning techniques to explore the predictive value of mouse-tracking data regarding a question’s difficulty. We use data from a survey on respondents’ employment history and demographic information, in which we experimentally manipulate the difficulty of several questions. Using measures derived from mouse movements, we predict whether respondents have answered the easy or difficult version of a question, using and comparing several state-of-the-art supervised learning methods. We have also developed a personalization method that adjusts for respondents’ baseline mouse behavior and evaluate its performance. For all three manipulated survey questions, we find that including the full set of mouse movement measures and accounting for individual differences in these measures improve prediction performance over response-time-only models., German Research Foundation (DFG), Peer Reviewed

In this paper, we present a tourism recommender framework based on the cooperation of ontological knowledge base and supervised learning models. Specifically, a new tourism ontology, which not only captures domain knowledge but also specifies knowledge entities in numerical vector space, is presented. The recommendation making process enables machine learning models to work directly with the ontological knowledge base from training step to deployment step. This knowledge base can work well with classification models (e.g., k-nearest neighbours, support vector machines, or naıve bayes). A prototype of the framework is developed and experimental results confirm the feasibility of the proposed framework. © 2021, Institute of Advanced Engineering and Science. All rights reserved.

In this paper, we present a tourism recommender framework based on the cooperation of ontological knowledge base and supervised learning models. Specifically, a new tourism ontology, which not only captures domain knowledge but also specifies knowledge entities in numerical vector space, is presented. The recommendation making process enables machine learning models to work directly with the ontological knowledge base from training step to deployment step. This knowledge base can work well with classification models (e.g., k-nearest neighbours, support vector machines, or naıve bayes). A prototype of the framework is developed and experimental results confirm the feasibility of the proposed framework. © 2021, Institute of Advanced Engineering and Science. All rights reserved.