STABILITY OF FOUR ITEM DISCRIMINATION INDICES OVER GROUPS OF DIFFERENT AVERAGE ABILITY.

The purpose of this study was to investigate whether four commonly employed item-discrimination indices are independent of the average achievement level of the examinee group on which they are based. Those indices studied were the Flanagan approximation of the product-moment correlation, the Davis transformation of the product-moment correlation, the Gulliksen item reliability index, and the upper-group vs. lower-group difference in proportion correct. From four subject areas 185 multiple-choice items were constructed, and each item was administered to a random sample of about 350 Iowa high-school students. From these data, it was possible to estimate the item statistics which would have been obtained had the items been used in six different populations: grades 9, 10, 11, 12, grades 9-12 combined, and potential college students, as determined by either ITED or NMSQT. For each reference population, the four item discrimination indices were estimated for each item, using the data from the upper and lower 27 per cent of each population. The over-all results of this study suggest that none of the discrimination indices is independent of the achievement level of the examinee population. Without exception, the mean values of each index differed Significantly among the reference populations. Moreover, the pattern of change in the index, as computed for progressively more able populations, was frequently different for different levels of item difficulty. However, the correlation between item discrimination indices based upon two different reference populations was, on the average, greater than .85. The significant differences among mean Gulliksen and Difference Indices was expected in view of their theoretical bases. The significant differences for the Flanagan and Davis Indices were somewhat more Surprising. It is suggested that, although the theoretical model far the Flanagan Index assumes a linear relationship between item trait and criterion, the relationship may be markedly curvilinear over extended ranges of criterion ability when item data are not corrected for chance success. [ABSTRACT FROM AUTHOR]