Search

In this article, we illustrate ways in which generalizability theory (G-theory) can be used with continuous latent response variables (CLRVs) to address problems of scale coarseness resulting from categorization errors caused by representing ranges of continuous variables by discrete data points and transformation errors caused by unequal interval widths between those data points. The mechanism to address these problems is applying structural equation modeling (SEM) as a tool in deriving variance components needed to estimate indices of score consistency and validity. Illustrations include quantification of multiple sources of measurement error, use of non-nested and nested designs, derivation of indices of consistency for norm- and criterion-referenced interpretation of scores, estimation of effects when changing measurement procedures and designs, and disattenuation of correlation coefficients for measurement error. These illustrations underscore the effectiveness of G-theory with continuous latent response variables in providing stable indices of reliability and validity that are reasonably independent of the number of original scale points used, unevenness of scale intervals, and average degree of item skewness. We discuss general distinctions in reliability estimation within G-theory, SEM, and classical test theory; make specific recommendations for using G-theory on raw score and CLRV metrics; and provide computer code in an online supplement for doing all key analyses demonstrated in the article using R and Mplus. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Although widely recognized as a comprehensive framework for representing score reliability, generalizability theory (G-theory), despite its potential benefits, has been used sparingly in reporting of results for measures of individual differences. In this article, we highlight many valuable ways that G-theory can be used to quantify, evaluate, and improve psychometric properties of scores. Our illustrations encompass assessment of overall reliability, percentages of score variation accounted for by individual sources of measurement error, dependability of cut-scores for decision making, estimation of reliability and dependability for changes made to measurement procedures, disattenuation of validity coefficients for measurement error, and linkages of G-theory with classical test theory and structural equation modeling. We also identify computer packages for performing G-theory analyses, most of which can be obtained free of charge, and describe how they compare with regard to data input requirements, ease of use, complexity of designs supported, and output produced. (PsycINFO Database Record