Search

Keystroke logs provide a comprehensive record of observable writing processes. Previous studies examining the keystroke logs of young L1 English writers performing experimental writing tasks have identified writing processes features predictive of the quality of responses. Contrarily, large-scale studies on the dynamic and temporal nature of L2 writing process are scarce, especially in an assessment setting. This study utilized the keystroke logs of adult English as a foreign language (EFL) learners responding to assessment tasks to examine the usefulness of the process features in this new context. We evaluated the features in terms of stability, explored factor structures for their correlations, and constructed models to predict response quality. The results showed that most of the process features were stable and that their correlations could be efficiently represented with a five-factor structure. Moreover, we observed improved response quality prediction over a baseline by up to 48%. These findings have implications for the evaluation and understanding of writing process features and for the substantive understanding of writing processes under assessment conditions.

Scaffolding, an instructional strategy for providing support until learners can perform a task on their own, holds a potential to improve assessments for test takers who need support to demonstrate their abilities, such as English learners (ELs). In this study, we evaluated the benefits of including scaffolded tasks in standardized speaking assessments for young ELs. Our focus was on scaffolded retell tasks that require test takers to retell a given story with procedural scaffolds. We collected responses from 233 third grade ELs to two scaffolded retell tasks, and investigated the relationship between the pre- and post-scaffolding retell performances to learn the impact of the scaffolding on the post-scaffolding performance. We also examined the psychometric added-value of the scaffolding steps and post-scaffolding responses through an application of item response theory (IRT) measurement models. The findings showed that, conditional on the pre-scaffolding performance, the EL test takers' performance on the scaffolding steps were positively associated with their post-scaffolding retell performance. Moreover, the responses to the scaffolding steps and the post-scaffolding retell provided additional information about test takers' oral English proficiency. These findings provide empirical support for the benefits of using scaffolding for EL assessments in a standardized setting.

Pragmatics has been a key component of language competence frameworks. While the majority of second/foreign language (L2) pragmatics tests have targeted productive skills, the assessment of receptive pragmatic skills remains a developing field. This study explores validation evidence for a test of receptive L2 pragmatic ability called the American English Sociopragmatic Comprehension Test (AESCT), which is a Web-based assessment consisting of 54 tasks measuring knowledge of speech acts, routine formulae, and culture-dependent lexical differences. The AESCT is intended to be used as a learning-oriented assessment in university-level applied linguistics classes. This study collected evidence on construct validity supporting the AESCT design as a measure of pragmatic comprehension and for providing feedback in low-stakes learning contexts; 97 university-level English language learners took the AESCT along with the Cambridge Placement Test and a background questionnaire on their exposure to target language input. Descriptive statistics, correlation analyses, and linear regression were used to analyze aspects of construct validity. Results indicate that the AESCT is sufficiently reliable. Overall, learners were found to perform as previous research suggests: sociopragmatic knowledge was related to L2 exposure and L2 proficiency. Alongside future research into L2 pragmatics test validation, implications for instruction-oriented utilization of the AESCT are discussed.

Studies that use structural equation modeling (SEM) techniques are increasingly encountered in the language assessment literature. This popularity has created the need for a set of guidelines that can indicate what should be included in a research report and make it possible for research consumers to judge the appropriateness of the interpretations made from a reported study. This article attempts to fill this void by providing a set of reporting guidelines appropriate for language assessment researchers.