Search

Abstract This study examines how second/foreign language (L2) word difficulty estimates derived from item response theory (IRT) and classical test theory (CTT) frameworks are virtually identical in the context of vocabulary testing. This conclusion is reached via a two-stage process: (a) psychometric assessments of both approaches and (b) L2 word difficulty modelling with lexical sophistication. Using data collected from first/native language (L1) Japanese EFL learners, both approaches led to similar conclusions in terms of the psychometric properties of the construct. Furthermore, CTT- and IRT/Rasch-derived estimates for word difficulty yielded nearly identical results in the predictive models. Although the “CTT-vs-IRT” debates in the past few decades have concluded with a middle ground agreed upon in most educational settings, this study acts as a useful demonstration to L2 vocabulary researchers who appear to rely heavily on Rasch/IRT analysis. The findings have practical applications to the area of L2 word difficulty research. Namely, that Rasch (an IRT procedure) alone might not be sufficient for validation, although it could be preferable when conducting inferential testing because of its potential utility for reducing chances of Type II errors.

In this discussion paper, I examine three exceptional articles presented at the 2023 Symposium of the Japan Association for Language Teaching’s Vocabulary Special Interest Group. These papers are analyzed from the standpoint of providing positive examples for future vocabulary researchers to follow in order to promote rigor in their work. I also discuss how these reports demonstrate a clear self-awareness regarding the conclusions their inquiries can and cannot support. Taken together, these three reports are invaluable for promoting better vocabulary research.

In this brief report, we present a scoping review of 31 second language (L2) vocabulary research syntheses published in the Web of Science from January 1990 to May 2022. The purpose of this undertaking was to understand what L2 vocabulary research synthesizers have aggregated and the investigative foci they intended to address. It was observed that most of the cataloged reports were experimental design meta-analyses, and thus there was room for future research syntheses in the vocabulary space to consider correlational research questions and hypotheses. Because systematic reviews were relatively scarce, there is also a need for these types of syntheses to be conducted in the future, especially those focused upon the observed quality of aggregated L2 vocabulary research.

The substantive component of construct validity requires a confrontation between empirical test results and content relevance. The Vocabulary Size Test (VST) has been extensively validated in terms of empirical results. Less is known, however, about expert judgments of content relevance. The VST was constructed and validated according to the principle that frequency underlies vocabulary acquisition. This does not mean, however, that the two are equivalent. To better understand the construct of word difficulty as it is measured in VSTs, the results of two Rasch analyses, one on the VST, the other a Multi Facet Rasch Model (MFRM) of a questionnaire distributed to education practitioners were correlated (n = 80, r = 0.67, p < 0.001, 95% CI = [0.53, 0.77]). Semi-structured interviews were then conducted to explore how practitioners understand the concept of word difficulty. Findings indicate that word difficulty is understood to encompass more than frequency, validating recent research into the predictive power of lexical sophistication variables.

Self-efficacy has emerged as a popular construct in second language research, especially in the frontline and practitioner-researcher spaces. A troubling trend in the relevant literature is that self-efficacy is often measured in a general or global manner. Such research ignores the fact that self-efficacy is a smaller context-driven construct that should be measured within a specific task or activity where time, place, and purpose domains are considered in the creation of the measurement. Task-based language teaching researchers have also largely neglected the affective factors that may influence task participation, including self-efficacy, despite its potential application to understanding task performance. In this report, we present an instrument specifically developed to measure English as a foreign language students' self-efficacy beliefs when performing a dialogic, synchronous, quasi-formal group discussion task. The instrument's underlying psychometric properties were assessed (N = 130; multisite sample from Japanese universities) and evidence suggested that it could measure a unidimensional construct with high reliability. The aggregate scale constructed from the instrument's items also displayed a central tendency and normal unimodal distribution. This was a positive finding and suggested that the instrument could be useful in producing a self-efficacy measurement for use in the testing designs preferred by second language researchers. The potential applications of this instrument are discussed while highlighting how this report acts as an illustration for investigators to use when researching self-efficacy.

This study investigated how students' self-reported individual differences predicted second language (L2) spoken discussion task output, an objective behavioral outcome, in the Japanese university English as a Foreign Language (EFL) context. Although numerous psychological theories are used as a rationale for task-based language teaching (TBLT), few studies have investigated the impact of individual differences variables on task performance. To address this gap, a cross-validation procedure was used with students (N = 439) from two different universities. They completed questionnaires to measure willingness to communicate (WTC), speaking self-efficacy (SSE), and perceived communicative competence (PCC). They also engaged in a quasiacademic eight-minute group discussion task (TBLT design). This discussion was recorded and transcribed, with the number of words produced used as an objective measure of L2 task production. In the better fitting mediation structural equation model, the influences of SSE and PCC on spoken L2 task production were fully mediated by WTC (R[superscript 2] = 0.21).

At the turn of the new millennium, in an article published in "Language Teaching Research" in 2000, Dörnyei and Kormos proposed that 'active learner engagement is a key concern' for all instructed language learning. Since then, language engagement research has increased exponentially. In this article, we present a systematic review of 20 years of language engagement research. To ensure robust coverage, we searched 21 major journals on second language acquisition (SLA) and applied linguistics and identified 112 reports satisfying our inclusion criteria. The results of our analysis of these reports highlighted the adoption of heterogeneous methods and conceptual frameworks in the language engagement literature, as well as indicating a need to refine the definitions and operationalizations of engagement in both quantitative and qualitative research. Based on these findings, we attempted to clarify some lingering ambiguity around fundamental definitions, and to more clearly delineate the scope and target of language engagement research. We also discuss future avenues to further advance understanding of the nature, mechanisms, and outcomes resulting from engagement in language learning.