Search

A primary objective of panel studies is to analyze change. The same questionnaire is used to compare data recorded at various times. Panel designs assume that the meaning of the questions and the concept of interest are stable over time. Analyses of measurement invariance often show the contrary. A qualitative part supplementing a panel survey can help us understand this phenomenon. In this study, 261 first-year psychology students completed questionnaires about their study motivation on two occasions; we interviewed some students as well. The survey showed that study motivation is not invariant over time. The qualitative data converged with the quantitative outcomes and explained the lack of invariance by the students’ overall transition during the first study year.We conclude that mixing quantitative and qualitative research methods for panel studies helps us understand change in constructs over time. We can study change at the macrolevel and better understand such change at the microlevel. more...

A primary objective of panel studies is to analyze change. The same questionnaire is used to compare data recorded at various times. Panel designs assume that the meaning of the questions and the concept of interest are stable over time. Analyses of measurement invariance often show the contrary. A qualitative part supplementing a panel survey can help us understand this phenomenon. In this study, 261 first-year psychology students completed questionnaires about their study motivation on two occasions; we interviewed some students as well. The survey showed that study motivation is not invariant over time. The qualitative data converged with the quantitative outcomes and explained the lack of invariance by the students’ overall transition during the first study year. We conclude that mixing quantitative and qualitative research methods for panel studies helps us understand change in constructs over time. We can study change at the macrolevel and better understand such change at the microlevel. more...

In mixed-mode surveys, it is difficult to separate sample selection differences from mode-effects that can occur when respondents respond in different interview settings. This paper provides a framework for separating mode effects from selection effects by matching very similar respondents from different survey modes using propensity score matching. The answer patterns of the matched respondents are subsequently compared. We show that matching can explain differences in nonresponse and coverage in two Internet samples. When we repeat this procedure for a telephone and Internet sample however, differences persist between the samples after matching. This indicates the occurrence of mode effects in telephone and Internet surveys. Mode effects can be problematic; hence we conclude with a discussion of designs that can be used to explicitly study mode effects. more...