Context Bias in the Test of English as a Foreign Language.

This study was undertaken to test the hypothesis that items of the Test of English as a Foreign Language (TOEFL) containing reference to American people, places, customs, etc., tend to favor examinees who have spent some time living in the United States. Two samples of examinees were drawn from the March 1987 TOEFL administration, one tested in the United States consisting of 5,799 persons living in the country more than 1 year, and the other consisting of individuals with less than 1 month residence in the United States tested in their native countries (n=21,652 worldwide). Mantel Haenszel analyses were carried out for each of the 146 operational items of the test. In a separate part of the study, five raters were asked to rate test items to determine explicit references to some aspect of America. Ratings were highly reliable, and all but 16 of the 146 items were unanimously judged to have or not to have some reference to Americana. Of the TOEFL items, however, only one gave a consistent advantage, found in every region studied, to persons with U.S. residence experience. No support was provided for the hypothesis that TOEFL items give an advantage to persons who have lived in the United States for some time. Four appendixes contain the distribution of candidates by region, instructions for a pilot study and the formal study, and a display of raters' assignments of Americana to each test item. (Contains 3 figures, 22 tables, and 11 references.) (Author/SLD)