Search

It is unclear whether children with autism spectrum disorders have atypical semantic fluency and lower memory for the semantics of words. Therefore, we examined semantic typicality, fluency and recall for the categories of fruits and animals in 60 children with autism aged 7-15 years (boys: 48/girls: 12) compared to 60 typically developing controls. Relative to controls, the autism group had reduced animal fluency, fruit typicality and recall for fruits. Notably, these measures were associated with more autistic-like symptoms and/or lower adaptive functioning across the autism and control groups. In conclusion, atypical semantics of fruits in the autism group may reflect development of idiosyncratic semantic networks while their lower semantic fluency and recall suggest impaired executive language functions.

Access to tertiary education is a challenge for many people from refugee-backgrounds. In Aotearoa New Zealand (NZ), resettled refugees are entitled to access education alongside other New Zealanders, and NZ's resettlement policy material recognises education as a key pillar of resettlement. However, refugee-background students are not recognised as 'priority learners' in education policy, so educational institutions are not required to report on refugee-background students' educational access or outcomes. At all levels of the education system, teaching and support practices for refugee-background students vary widely. Many barriers hamper refugee-background students' access to and success in tertiary education. However, refugee-background students are also necessarily-skilful border navigators. In this paper, we share data from an in-progress participatory action research project based in southern NZ that involves collaborating with refugee-background students at the secondary-tertiary education border. After describing the project, its rationale, and our theoretical framework, we draw on insights from seven of the students to illustrate how students represented educational navigation in NZ: as a resistance project, a subterranean project, and a relational project. We conclude by suggesting some implications from our study for research, policy and practice in education.

The design and psychometric methodology of the National Assessment of Educational Progress (NAEP) is constantly evolving to meet the changing interests and demands stemming from a rapidly shifting educational landscape. NAEP has been built on strong research foundations that include conducting extensive evaluations and comparisons before new approaches are adopted. During those evaluations, many lessons are learned and discoveries surface that do not often find their way into widely accessible outlets. This article discusses a number of those insights with the goal to provide an integrated and accessible perspective on the strengths and limitations of NAEP's psychometric methodology and statistical reporting practices. Drawing from a range of technical reports and memoranda, presentations, and published literature, the following topics are covered: calibration, estimation of proficiency, data reduction, standard error estimation, statistical inference, and standard setting.

Several complex sample standard error estimators based on linearization and resampling for the latent regression model of the National Assessment of Educational Progress (NAEP) are studied with respect to design choices such as number of items, number of regressors, and the efficiency of the sample. This paper provides an evaluation of the extent these estimators are appropriate for the models and test lengths often encountered in NAEP and what the effect is on the NAEP imputation model. It is shown that in general the resampling method used in this study provides the most accurate standard errors. However, the differences with the linearization method chosen in this study are relatively small if only small models are used with respect to the independent variables of the latent regression. Illustration is provided through several small simulation studies and NAEP data analysis.

The New Zealand school curriculum was last revised in 2007, at which time a new emphasis was placed on culture in language teaching. The practice of intercultural language teaching is implicit in the curriculum document and explicit in the curriculum guide, which features a set of principles for intercultural communicative language teaching (iCLT). This article presents a study on the extent to which New Zealand language teachers' beliefs and practices are aligned with intercultural language teaching (ILT). A questionnaire administered to New Zealand language teachers included a number of items used in a seminal seven-nation comparative study conducted by Lies Sercu and her colleagues, as well as other items developed from relevant literature. Expanding on previous studies' use of item-by-item analyses, multi-item scales to measure alignment of New Zealand teacher's beliefs and practices with ILT were developed, which yielded good internal reliabilities. The findings revealed an apparent mismatch between beliefs and practices, with teachers being favourably disposed towards ILT but not practising the approach in their classrooms. Interpretation of the data using concepts from teacher cognition research suggested that the differences represented tensions between teachers' abstract, theoretical beliefs and their concrete, practical beliefs. We argue that supporting teachers' applied knowledge of developing intercultural communicative competence (ICC) will allow them to recognize that those beliefs need not be discordant.

This article describes selected results from the Math Online (MOL) study, one of three field investigations sponsored by the National Center for Education Statistics (NCES) to explore the use of new technology in NAEP. Of particular interest in the MOL study was the comparability of scores from paper- and computer-based tests. A nationally representative sample of eighth-grade students was administered a computer-based mathematics test and a test of computer facility, among other measures. In addition, a randomly parallel group of students was administered a paper-based test containing the same math items as the computer-based test. Results showed that the computer-based mathematics test was significantly harder statistically than the paper-based test. In addition, computer facility predicted online mathematics test performance after controlling for performance on a paper-based mathematics test, suggesting that degree of familiarity with computers may matter when taking a computer-based mathematics test in NAEP. (Contains 9 tables, 1 figure and 6 endnotes.)

The present study investigated the factor structure of a field trial sample of the Test of English as a Foreign Language™ Internet-based test (TOEFL® iBT). An item-level confirmatory factor analysis (CFA) was conducted for a polychoric correlation matrix of items on a test form completed by 2,720 participants in the 2003-2004 TOEFL iBT Field Study. CFA-based multitrait-multimethod (MTMM) analyses for the Reading and Listening sections showed that the language abilities assessed in each section were essentially unidimensional, while the factor structure of the entire test was best represented by a higher-order factor model with a general factor (English as a second language/English as a foreign language ability) and four group factors for reading, listening, speaking, and writing. The integrated Speaking and Writing tasks, which require language processing in multiple modalities, well defined the target modalities (speaking and writing). These results broadly support the current reporting of four scores corresponding to the modalities and a total score, as well as the test design where the integrated tasks contribute only to the scores for the target modalities.

The hierarchical latent regression model (HLRM) is a flexible framework for estimating group-level proficiency while taking into account the complex sample designs often found in large-scale educational surveys. A complex assessment design in which information is collected at different levels (such as student, school, and district), the model also provides a mechanism for estimating group differences at various levels and for partitioning variance components among those levels. This study examines parameter recovery in the HLRM and compares it to regular latent regression models (LRMs) through simulation for various levels of cluster variation. Results show that regression effect estimates are similar between the HLRM and the LRM, in particular under small cluster variation. Similarly, student posterior mean estimates and marginal maximum likelihood mean estimates for student groups are comparable across the two model approaches. However, substantial differences are found for the residual variance estimates, the standard errors for regression effect estimates and related standard errors for group estimates, and for students posterior variance estimates. As expected, these differences are larger when the variation across clusters is larger, since a substantial portion of variance remains unexplained in LRM.

In this paper, a data perturbation method for minimizing the possibility of disclosure of participants' identities on a survey is described in the context of the National Assessment of Educational Progress (NAEP). The method distinguishes itself from most approaches because of the presence of cognitive tasks. Hence, a data edit should have minimal impact on both relations among demographic variables and relations between demographic and proficiency variables. Furthermore, since only a few students are at risk to be disclosed in a typical sampling setting common to educational surveys, the proposed data perturbation is governed by a nonuniform probabilistic process. The method is applied to data from NAEP and impact is computed using proficiency averages, demographic proportions, statistical inference results, and loglinear models. Results show that the proposed perturbation method has very little impact on NAEP results, even at relatively large editing rates. Some data coarsening results are reported as well. While the univariate results are relatively unaffected from the coarsening, loglinear models from higher order contingency tables are affected. It is recommended to restrict disclosure limitation techniques to perturbation methods in the case of NAEP.

Two versions of a general method for approximating standard error of regression effect estimates within an IRT-based latent regression model are compared. The general method is based on Binder's (1983) approach, accounting for complex samples and finite populations by Taylor series linearization. In contrast, the current National Assessment of Educational Progress procedure assumes a simple random sample for standard error of regression effects and applies a jackknife estimator to statistics of interest as a way to account for NAEP's complex sample. In this study, the versions of the general method are formally defined and the general method is extended to multiple dimensions. Furthermore, they are applied in an empirical study to the 2004 NAEP long-term trend data comparing both large, nearly saturated, and small models. Subsequently, the results are compared to the operational-based imputation method. Results show no impact on the imputation-based results, limited impact on large models, and reasonable impact on small models. While it is not readily apparent to what this differential impact can be attributed, several explanations are discussed.

Well-known numerical integration methods are applied to item response theory (IRT) with special emphasis on the estimation of the latent regression model of NAEP [National Assessment of Educational Progress]. An argument is made that the Gauss-Hermite rule enhanced with Cholesky decomposition and normal approximation of the response likelihood is a fast, precise, and reliable alternative for the numerical integration in NAEP and in IRT in general.

A hierarchical latent regression model is suggested to estimate nested and nonnested relationships in complex samples such as found in the National Assessment of Educational Progress (NAEP). The proposed model aims at improving both parameters and variance estimates via a two-level hierarchical linear model. This model falls naturally within the set of models used in most large scale surveys, in that all of them are special cases of the hierarchical latent regression model. The model parameter estimates are obtained via the expectation-maximization (EM) algorithm. An example with NAEP data is presented and results of parameter estimation and standard errors are compared with results from operational procedures of NAEP.

Confidence intervals are an important tool to indicate uncertainty of estimates and to give an idea of probable values of an estimate if a different sample from the population was drawn or a different sample of measures was used. Standard symmetric confidence intervals for proportion estimates based on a normal approximation can yield bounds outside the [0,1] scale and poor coverage, because such approximations are generally inappropriate. Many alternative intervals have been proposed to address these issues. This paper discusses a selection of intervals based on scale transformations and continuity corrections and adapts these for use in complex samples. The study expands on the work of Brown, Cai, and DasGupta (2001) and Korn and Graubard (1998) using educational survey designs and complex sample data. Results based on a National Assessment of Educational Progress (NAEP) data resampling study showed that the theoretically appealing Wilson interval yields appropriate coverage with short intervals in most situations.

A multitude of methods has been proposed to estimate the sampling variance of ratio estimates in complex samples (Wolter, 1985). Hansen and Tepping (1985) studied some of those variance estimators and found that a high coefficient of variation (CV) of the denominator of a ratio estimate is indicative of a biased estimate of the standard error of a ratio estimate. Using the same populations, Kovar (1985) and Kovar, Rao, and Wu (1988) repeated the research and showed that the relation between a high CV and bias in standard errors is weak. In light of these conflicting findings, this study uses substantially different populations and design choices taken from the National Assessment of Educational Progress (NAEP) to further investigate the relationship between bias, the CV, and the number of strata, which has also been found to be an indicator of bias (Burke & Rust, 1995). It is found that the CV is a relatively weak indicator of bias, showing poor power properties. Suggestions are made to improve upon statistical suppression rules related to the CV and number of replicate strata.

Intercultural language teaching and learning has increasingly been adopted in state school systems, yet studies have shown that language teachers struggle to include it in their practice. The aim of this study is to use dynamic systems theory to examine how a German as a foreign language teacher in a New Zealand secondary school adopted a project designed to promote intercultural communicative language teaching. It does this with data from a microgenetic analysis of a session in which the teacher was introduced to the project and a qualitatively analyzed interview in which the teacher reported her beliefs and practices for teaching culture. The findings show that the teacher adopted the project by conceptualizing it through the practical challenges involved in using it as a formal assessment. In an interview, she expressed her beliefs about culture in language teaching, and there was some evidence that they reflected the process of adopting the project to be an assessment. The implications focus on the project as a basis for further development of intercultural teaching.

This construct validation study investigated the factor structure of the Test of English as a Foreign Language[TM] Internet-based test (TOEFL[R] iBT). An item-level confirmatory factor analysis was conducted for a test form completed by participants in a field study. A higher-order factor model was identified, with a higher-order general factor (ESL/EFL ability) and four first-order factors for reading, listening, speaking and writing. Integrated Speaking and Writing tasks, which require language processing in multiple modalities, defined the target modalities (speaking and writing). These results broadly supported the current practice of reporting a total score and four scores corresponding to the modalities for the test, as well as the test design that permits the integrated tasks to contribute only to the scores of the target modalities. (Contains 1 footnote, 4 tables, and 5 figures.)

To find population proficiency distributions, a two-level hierarchical linear model may be applied to large-scale survey assessments such as the National Assessment of Educational Progress (NAEP). The model and parameter estimation are developed and a simulation was carried out to evaluate parameter recovery. Subsequently, both a hierarchical and a simple model were applied to NAEP reading data. The impact of using a hierarchical model was found to be relatively modest in this case, mostly due to modest clustering. Several other applications and future studies are discussed. (Contains 10 notes and 16 tables.)