Search

The Mantel–Haenszel estimator is one of the most popular techniques for measuring differential item functioning (DIF). A generalization of this estimator is applied to the context of DIF to compare items by taking the covariance of odds ratio estimators between dependent items into account. Unlike the Item Response Theory, the method does not rely on the local item independence assumption which is likely to be violated when one item provides clues about the answer of another item. Furthermore, we use these (co)variance estimators to construct a hypothesis test to assess DIF for multiple items simultaneously. A simulation study is presented to assess the performance of several tests. Finally, the use of these DIF tests is illustrated via application to two real data sets, Peer Reviewed, Postprint (author's final draft)

This research intends to bring insights into learner autonomy in a Malagasy EFL setting. Despite being a topic of research in language education for about four decades, learner autonomy is still almost unknown in countries like Madagascar. Most empirical studies on learner autonomy have taken place in either ESL settings in Western countries or EFL settings in some Asian countries. Very little research has been conducted in African developing countries. In order to foster learner autonomy in a Malagasy setting, the research encompassed two main phases. Phase one focused on exploring the existing conditions for learner autonomy in a Malagasy rural school; while phase two aimed to promote one dimension of learner autonomy with student teachers through a “reflective learning” course. Phase one examined the affordances of learner autonomy in a Malagasy rural school. It investigated three dimensions of learner autonomy, namely self-initiation, self-regulation, and independence, via class observations and interviews with four EFL teachers. The data revealed some elements of autonomy. Self-initiation was fostered through encouragement and opportunities to learn outside class, while independence (from teachers) was mainly promoted through peer collaboration. Though the presence of the elements was not consistent, the fact that they were promoted at all implies possibilities to further exploit them in such a setting. Self-regulation - composed of planning, monitoring, and self-evaluation - was not promoted probably due to the teachers’ unawareness of its importance, and their lack of experience with self-regulation as former learners. The aim of phase two was to promote self-regulation at a Teacher Training College among a group of 22 first-year EFL student teachers as participants. A nine-week “reflective learning” course was designed to achieve three main objectives: (1) to help the student teachers improve their self-regulation skills via reflective journal writing, in or

Learner autonomy, primarily defined as “the ability to take charge of one’s own learning” (Holec, 1981, p. 3), has gained much research interest in the field of language learning and teaching due to its potential contribution to effective language learning. Although the concept of learner autonomy has been extensively discussed in the literature, little research has empirically investigated how this concept is realized in language classrooms. This research explored learner autonomy and its development in the Thai secondary school context. The research was structured in two phases. The first phase of the study was an exploratory phase conducted to identify the Thai teachers’ perspectives towards learner autonomy and how their classroom practices prepared learners for autonomous learning. Data were collected through class observations, teacher interviews and learner group interviews. The findings revealed that although the teachers shared positive views about learner autonomy and regarded it as a useful concept, they did not sufficiently promote autonomous learning in their classroom practices. Learners’ accounts of their learning experiences also indicated that their classrooms did not prepare them methodologically and psychologically to take responsibility for their own learning. Findings from the exploratory phase indicate that the teachers’ use of the teacher-led teaching method as well as the learners’ lack of skills and confidence in their ability to direct their learning process could pose significant challenges to learner autonomy development. Building on findings from the first phase, the second phase of the study featured a strategy-based intervention program designed to promote learner autonomy. This intervention phase involved 30 learners from an intact class in which the strategy-based instruction program was implemented, and 32 learners from a comparison class who received regular English lessons. Data regarding the intervention’s impacts on learners’ de

Idioms are known to cause great difficulty for second language (L2) learners, who may understand the literal meanings of the constituent words of idioms like (be) waiting in the wings, but often fail to interpret the idiomatic, figurative meaning of the expression. Proponents of Cognitive Linguistics (CL) claim that CL provides a pathway to more systematic and insightful learning of figurative expressions like idioms. They advocate that learners should be informed of the literal underpinning of idiomatic expressions and their relationship to the figurative meaning. This is supported by the results of several experimental studies employing ‘etymological elaboration’. However, little is known about how learners actually experience the CL-style explanations, or about how the learning is affected by other factors such as learners’ perceived transparency of the connection between the literal underpinnings and the idiomatic meanings, and their L1. The research reported in this thesis therefore (1) investigates the effectiveness of etymological elaboration in facilitating idiom comprehension and retention; (2) examines the problems that L2 learners, i.e., native-Chinese EFL learners in this study, experience when they encounter English figurative idioms, and identifies the factors influencing success in learning the meanings of idioms. To achieve these objectives, a mixed methods design was employed. Etymological elaboration was implemented in a teaching experiment involving one-on-one interviews, in which 25 Chinese learners of English were presented with idioms whose meaning they were asked to guess first without and then with the aid of information about their literal underpinnings. After the correct figurative meaning was established, participants rated the transparency of the connection between the literal underpinning and the figurative meaning. One week later, the learners were presented with the same idioms and asked to recall their meaning. Follow-up interviews we

More than a decade ago, the Vietnamese Government announced an educational reform to enhance the quality of English language education in the country. An important aspect of this reform is the introduction of the localized test of English proficiency which covers four language skills, namely listening, speaking, reading, and writing. This high-stakes English test is developed and administered by only a limited number of institutions in Vietnam. Although the validity of the test is a considerable concern for test-takers and test score users, it has remained an under-researched area. This study aims to partly address the issue by validating a listening test developed by one of the authorized institutions in Vietnam. In this thesis, the test is referred to as the Locally Created Listening Test or the LCLT. Using the argument-based approach to validation (Kane, 1992, 2013; Chapelle, 2008), this research aims to develop a validity argument for the evaluation, generalization and explanation inferences of the LCLT. Three studies were carried out to elicit evidence to support these inferences. The first study investigated the statistical characteristics of the LCLT test scores, focusing on the evaluation and generalization inference. The second study shed light on the extent to which test items engaged the target construct. The third study examined whether test-takers’ scores on the LCLT correlated well with their scores on an international English test that measured a similar construct. Both the second and third study were carried out to support the explanation inference. These three studies did not provide enough evidence to successfully support the validity argument for the LCLT. The test was found to have major flaws that affected the validity of score interpretations. In light of the research findings, suggestions were given for the betterment of future LCLTs. At the same time, this research helped to uncover the impacts of certain text and task-related factors on the

Deciding on the best statistical method to apply when the response variable is ordinal is essential because the way the categories are ordered in the data is relevant as it could change the results of the analysis. Although the models for continuous variables have similarities to those for ordinal variables, this paper presents the advantages of the use of the ordering information on the outcomes with methods developed for modeling ordinal data such as the ordered stereotype model. The novelty of this article lies in showing the dangers of assigning equally spaced scores to ordered response categories in statistical analysis, which are illustrated with a simulation study and a case study. We propose a new way to use the score parameters, which incorporates the fitted spacing dictated by the data. Additionally, this article uses score parameter estimates in the ordered stereotype model to propose a new measure to calculate continuous medians in the raw data: the adjusted c-median. It benefits the general audience who can easily understand the median as a summary statistic. Supplementary materials for this article are available online., Peer Reviewed, Postprint (published version)

The function of English as an international language (hereafter called EIL) is firmly entrenched in many aspects of modern life, such as academia, aviation, business, education, international relations, science, and tourism. This trend has led high-stakes English language proficiency tests to serve increasingly critical gate-keeping roles for people seeking admission to tertiary institutions, employment overseas, and career advancement. However, while language tests focus on language proficiency, they do not measure the ability over and above core linguistic competence that enables and enhances successful international communication. As more and more English communication occurs between people from different language and cultural backgrounds, research and testing of these skills is warranted. This thesis identifies receptive intelligibility (listening skills), productive intelligibility (speaking skills), intercultural competence, and strategic competence in negotiating meaning as key abilities that enhance the use of EIL. An investigation of these four abilities sets the groundwork for their conceptualization and operationalization into a test battery aimed at government officials from developing Asian and African countries who have gone through English-language training at Victoria University of Wellington. Results from classroom observations and a survey of the participant’s target language use tasks serve to highlight the contexts and purposes for which they use English, as well as providing the contexts for developing test tasks and items and conceptualizing skills used to improve international communication. The four abilities identified above were then operationalized into testable items for an online test battery consisting of four constructs and a total of 76 items. The purpose of this test battery is to serve as a low-stakes assessment tool for making decisions regarding the international communicative competence of government employees who use English for

The University Entrance Exam (UEE) plays a vital role in the Chinese education system. Yet despite it being a high-stakes exam for over nine million senior high school students each year, there is little research demonstrating that the UEE is of an appropriate quality. This is partly due to the very nature of the UEE. Little information about it is publicly available and there has been no significant validation research. This study provides the first comprehensive attempt to validate the University Entrance English Exam. The study is focused on the Shanghai English Language Exam, which is one of the principal tests for university entry in Shanghai. There were two phases in the study. The first was a qualitative content analysis, which compared the past 10 years of UEE papers with English Curriculum Standards, and with Bachman and Palmer’s model of language ability, in both instances with the aim of identifying what the UEE is actually testing. The second phase conducted empirical studies, namely mock exams, classroom observations and interviews, and think-aloud tasks. Six test qualities – reliability, construct validity, authenticity, interactiveness, impact, and practicality – were measured in these studies. The mock exam participants were first-year university students, whereas classroom observations, interviews and think-aloud task participants were senior high school students. In the first phase the study found that by English Curriculum Standards, the UEE adequately tested language skills and language knowledge but failed to test affect and attitude, learning strategies, and cultural understanding by those same standards. Measured against Bachman and Palmer’s model of language ability, the UEE focused on grammatical and textual knowledge and strategic competence but not functional and sociolinguistic knowledge. In the second phase the study found variable responses, with impact registering highest and authenticity registering lowest. The UEE has been the subjec

The importance of teachers’ assessment literacy has been increasingly emphasised in the literature. However, very little research has paid attention to pre-service EFL teachers’ assessment literacy and how they develop this area during teacher training programmes. Moreover, there is a paucity of research on Vietnamese pre-service EFL teachers’ assessment literacy. This study was conducted in three phases to address these gaps: (1) Phase 1 provided a description of current assessment training at four Vietnamese teacher training universities, (2) Phase 2 attempted to map out pre-service EFL teachers’ confidence levels in assessment literacy, and (3) Phase 3 mainly focused on the development of four pre-service EFL teachers’ assessment literacy during their nine-week practicum at Bach Dang University (pseudonym). Phase 1 relied on individual semi-structured interviews with four Vietnamese teacher-trainers to describe the current status of assessment training for pre-service EFL teachers at four key teacher training universities in terms of: (1) teacher-trainers’ background, (2) course content, (3) method of instruction, (4) support for assessment training, and (5) constraints of assessment training. The teacher-trainers noted their lack of professional development in testing and assessment. The method of instruction varied for different teacher training universities. The results showed a greater emphasis on training in summative rather than formative assessment. Also, teacher-trainers identified two main constraints in the current training programmes including: (i) the lack of systematic innovation in language testing and assessment and (ii) the lack of labour, facilities and time for language testing and assessment training. In Phase 2, a questionnaire of pre-service EFL teachers’ confidence levels in assessment literacy was developed and validated. It was then administered to 365 pre-service EFL teachers. The results indicated pre-service EFL teachers’ high confidenc