1. RATER AGREEMENT AND DISAGREEMENT IN THE MEASUREMENT OF ENGLISH ARTICLE ACQUISITION SUPPLIANCE AND ACCURACY
- Author
-
Rose Acen Upor
- Subjects
article acquisition ,inter-rater agreement ,soc ,tlu ,efl ,inter-rater disagreement ,language background effects ,Education (General) ,L7-991 ,Language and Literature ,Philology. Linguistics ,P1-1091 - Abstract
This study combines language assessment processes and interlanguage analysis techniques to determine rater agreement and disagreement in assessing English article acquisition. Employing native English speaking and non-native English speaking raters, picture sequence narratives that were written by English as a Foreign Language (EFL) learners (n=97) were coded and scored for suppliance-in-obligatory context (SOC) and target-like utterance (TLU). Although the kappa statistic revealed a fair agreement between raters (0.17 – 0.33), content analysis methods revealed much higher agreement (88.29% - 94.07%). Furthermore, language background effects between the raters could not be substantiated however the results demonstrated a discernable disagreement pattern between them. Thus, the study recommends the inclusion of a foreign language teaching background as a factor for rater selection to minimize language background effects on rating language assessments.
- Published
- 2021
- Full Text
- View/download PDF