Search

A major deficiency in classical test theory is the reliance on Pearson product-moment (PPM) correlation concepts in the definition of reliability. PPM measures are totally insensitive to first moment differences in tests which leads to the dubious assumption of essential tan-equivalence. Robinson proposed a measure of agreement that is sensitive to different test difficulty and gives a practical statistic to estimate reliability in the presence of known form variation in difficulty. Robinson's measure of agreement appears to be a useful alternative to the generalizability coefficient, as it provides a more conservative estimate of reliability under conditions of parallel form differences in mean. This is likely to be especially useful when examining inter rater reliability when internal consistency of the raters is poor. Robinson's measure does not seem advantageous for highly reliable parallel tests such as are encountered in standardized testing programs. A simulation study is presented to illustrate the degree of the coefficient's sensitivity to form difficulty variance. Robinson's measure of agreement and the intraclass correlation are computed for each simulation and their values are compared. (author/RL)

Generalizability theory is based upon analysis of variance (ANOVA) and requires estimation of variance components for the ANOVA design under consideration in order to compute either G (Generalizability) or D (Decision) coefficients. Estimation of variance components has a number of alternative methods available using SAS, BMDP, and ad hoc procedures. This study examines the effect of these methods on coefficeints for a large sample of subjects under complex designs. The Wechsler Adult Intelligence Scale-Revised national standardization sample provided the data base. Subject (n=1880) variables (age, gender, etc.) provided variety for the construction of complex factoral designs. Three designs were used to investigate variance component estimation and resultant g-coefficeints. The designs included: (1) a two-factor subject x item design; (2) a three-factor subject x item x age design; and (3) a four-factor subject x item x age x gender design. These designs allowed estimation of both within and between subject effects. Suggestions are made concerning equal and unequal situations. (Author/PN)

Samples in research on individual and group differences may be selected based on whole scores which differ from the population mean. Children are diagnosed in clinical practice with a whole score. These procedures produce regression to the population mean which can affect accuracy and adequacy of part score interpretations. (Author/DWH)

Describes methods designed to add new studies to published meta-analysis results, focusing on a procedure using Bayesian analysis and showing how new findings complement previous meta-analysis and extend its conclusions. Also considers how studies are to be weighted, which variables are to be examined, and how often meta-analyses are to be updated. (Author/JN)

A theoretical model is developed for analysis of intervention effects in multiple autocorrelated time-series which may be classified or blocked two ways. The intervention effect is estimated independently for each series. The estimates are then entered as observations in two-factor analysis of variance with one observation per cell, utilizing a pooled variance error estimate for the mean square error. The model was tested empirically through the use of Monte Carlo techniques, and results are presented in tabular form for Type I and Type II error levels. An example is presented with data taken from a Kansas City Police study utilizing the two-factor layout. (Author)