1. Assessment of Method Effects of Keying and Wording in Instruments: A Mixed-Methods Explanatory Sequential Study
- Author
-
Lin Ma
- Abstract
This dissertation presents an innovative approach to examining the keying method, wording method, and construct validity on psychometric instruments. By employing a mixed methods explanatory sequential design, the effects of keying and wording in two psychometric assessments were examined and validated. Those two self-report psychometric assessments were the Effortful Control assessment (Ellis & Rothbart, 2001) and the Grit assessment (Duckworth & Quinn, 2009). Moreover, the quantitative phase utilized structural equation modeling to analyze 2,104 students' responses and assess the construct of keying and wording. Various hypothetical models were investigated and evaluated. The reliability of each construct in each method was analyzed using several omega coefficients. Simultaneously, construct validity was discussed through model performance and parameter estimates. Following the quantitative phase, six one-on-one interviews were conducted in the qualitative phase. The purpose of the second phase was to gather participants' opinions and gain insights to further interpret and understand the results obtained in the first quantitative phase. By integrating quantitative and qualitative approaches, this study aims to enhance the validity and meaningfulness of its conclusions. The overarching purpose of this study was to assess the keying structure and method effects of keying and wording through a mixed methods explanatory sequential study. To achieve this goal, the study investigated different method factors that impacted the construct validity of psychometric assessments among adolescents. Findings from the first quantitative strand revealed the detection of the keying, although the structure of keying displayed inconsistent probability of variance associated with the reliability across different instruments. Additionally, method effects of keying and wording were identified, and problematic item questions were further examined in the qualitative phase. Results from the qualitative strand led to the conclusion that three factors influenced adolescents' understanding of assessment questions. These were complex item questions, student's characteristics, and school course plan. This study therefore addressed research gaps regarding the effects of keying and wording in psychometric instruments and contributed to understanding their impact on construct validity. The findings draw attention to the keying and wording methods in self-report psychometric instruments. This also impacted the accuracy of assessing the construct of interest and the further parameter estimates. Based on these findings, suggestions for future research were discussed and summarized. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
- Published
- 2024