Search

Purpose: Item response theory (IRT) is a modern psychometric framework with several advantageous properties as compared with classical test theory. IRT has been successfully used to model performance on anomia tests in individuals with aphasia; however, all efforts to date have focused on noun production accuracy. The purpose of this study is to evaluate whether the Verb Naming Test (VNT), a prominent test of action naming, can be successfully modeled under IRT and evaluate its reliability. Method: We used responses on the VNT from 107 individuals with chronic aphasia from AphasiaBank. Unidimensionality and local independence, two assumptions prerequisite to IRT modeling, were evaluated using factor analysis and Yen’s Q3 statistic (Yen, 1984), respectively. The assumption of equal discrimination among test items was evaluated statistically via nested model comparisons and practically by using correlations of resulting IRT-derived scores. Finally, internal consistency, marginal and empirical reliability, and conditional reliability were evaluated. Results: The VNT was found to be sufficiently unidimensional with the majority of item pairs demonstrating adequate local independence. An IRT model in which item discriminations are constrained to be equal demonstrated fit equivalent to a model in which unique discrimination parameters were estimated for each item. All forms of reliability were strong across the majority of IRT ability estimates. Conclusions: Modeling the VNT using IRT is feasible, yielding ability estimates that are both informative and reliable. Future efforts are needed to quantify the validity of the VNT under IRT and determine the extent to which it measures the same construct as other anomia tests. [ABSTRACT FROM AUTHOR]

Anomia assessment is a fundamental component of clinical practice and research inquiries involving individuals with aphasia, and confrontation naming tasks are among the most commonly used tools for quantifying anomia severity. While currently available confrontation naming tests possess many ideal properties, they are ultimately limited by the overarching psychometric framework they were developed within. Here, we discuss the challenges inherent to confrontation naming tests and present a modern alternative to test development called item response theory (IRT). Key concepts of IRT approaches are reviewed in relation to their relevance to aphasiology, highlighting the ability of IRT to create flexible and efficient tests that yield precise measurements of anomia severity. Empirical evidence from our research group on the application of IRT methods to a commonly used confrontation naming test is discussed, along with future avenues for test development. [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this study was to verify the equivalence of 2 alternate test forms with nonoverlapping content generated by an item response theory (IRT)--based computer-adaptive test (CAT). The Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996) was utilized as an item bank in a prospective, independent sample of persons with aphasia. Method: Two alternate CAT short forms of the PNT were administered to a sample of 25 persons with aphasia who were at least 6 months postonset and received no treatment for 2 weeks before or during the study. The 1st session included administration of a 30-item PNT-CAT, and the 2nd session, conducted approximately 2 weeks later, included a variable-length PNT-CAT that excluded items administered in the 1st session and terminated when the modeled precision of the ability estimate was equal to or greater than the value obtained in the 1st session. The ability estimates were analyzed in a Bayesian framework. Results: The 2 test versions correlated highly (r = .89) and obtained means and standard deviations that were not credibly different from one another. The correlation and error variance between the 2 test versions were well predicted by the IRT measurement model. Discussion: The results suggest that IRT-based CAT alternate forms may be productively used in the assessment of anomia. IRT methods offer advantages for the efficient and sensitive measurement of change over time. Future work should consider the potential impact of differential item functioning due to person factors and interventionspecific effects, as well as expanding the item bank to maximize the clinical utility of the test. [ABSTRACT FROM AUTHOR]

Background: Item response theory (IRT; Lord & Novick, 1968) is a psychometric framework that can be used to model the likelihood that an individual will respond correctly to an item. Using archival data (Mirman et al., 2010), Fergadiotis, Kellough, and Hula (2015) estimated difficulty parameters for the Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996) using the 1-parameter logistic IRT model. Although the use of IRT in test development is advantageous, its reliance on sample sizes exceeding 200 participants make it difficult to implement in aphasiology. Therefore, alternate means of estimating the item difficulty of confrontation naming test items warrant investigation. In a preliminary study aimed at automatic item calibration, Swiderski, Fergadiotis, and Hula (2016) regressed the difficulty parameters from the PNT on word length, age of acquisition (Kuperman, Stadthagen-Gonzalez, & Brysbaert, 2012), lexical frequency as quantified by the Log10CD index (Brysbaert & New, 2009), and naming latency (Székely et al., 2003). Although the model's predictive utility was high, a substantial proportion (20%) of the response time data were missing. Further, only 39% of the picture stimuli from Székely and colleagues (2003) were identical to those on the PNT. Given that the IRT sample size requirements limit traditional calibration approaches in aphasiology and that the initial attempts in predicting IRT difficulty parameters in our pilot study were based on incomplete response time data this study has two specific aims. Aims: To estimate naming latencies for the 175 items on the PNT, and assess the utility of psycholinguistic variables and naming latencies for predicting item difficulty. Methods and Procedures: Using a speeded picture naming task we estimated mean naming latencies for the 175 items of the PNT in 44 cognitively healthy adults. We then re-estimated the model reported by Swiderski et al (2016) with the new naming latency data. Outcomes and Results: The predictor variables described above accounted for a substantial proportion of the variance in the item difficulty parameters (Adj. R2 = .692). Conclusions: In this study we demonstrated that word length, age of acquisition, lexical frequency, and naming latency from neurotypical young adults usefully predict picture naming item difficulty in people with aphasia. These variables are readily available or easily obtained and the regression model reported may be useful for estimating confrontation naming item difficulty without the need for collection of response data from large samples of people with aphasia. [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this study was to develop a computerized adaptive test (CAT) version of the Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996), to reduce test length while maximizing measurement precision. This article is a direct extension of a companion article (Fergadiotis, Kellough, & Hula, 2015), in which we fitted the PNT to a 1-parameter logistic item-response-theory model and examined the validity and precision of the resulting item parameter and ability score estimates. Method: Using archival data collected from participants with aphasia, we simulated two PNT-CAT versions and two previously published static PNT short forms, and compared the resulting ability score estimates to estimates obtained from the full 175-item PNT. We used a jackknife procedure to maintain independence of the samples used for item estimation and CAT simulation. Results: The PNT-CAT recovered full PNT scores with equal or better accuracy than the static short forms. Measurement precision was also greater for the PNT-CAT than the static short forms, though comparison of adaptive and static nonoverlapping alternate forms showed minimal differences between the two approaches. Conclusion: These results suggest that CAT assessment of naming in aphasia has the potential to reduce test burden while maximizing the accuracy and precision of score estimates. [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this study is to investigate the structure and measurement properties of the Aphasia Communication Outcome Measure (ACOM), a patient-reported outcome measure of communicative functioning for persons with aphasia. Method: Three hundred twenty-nine participants with aphasia responded to 177 items asking about communicative functioning. The data were analyzed using a categorical item factor analysis approach. Validity of ACOM scores on the basis of their convergence with performance-based, clinician-reported, and surrogate-reported assessments of communication was also assessed. Results: Fifty-nine items that obtained adequate fit to a modified bifactor measurement model and functioned similarly across several demographic and clinical subgroupings were identified. The factor model estimates were transformed to item response theory graded response model parameters, and the resulting score estimates showed good precision and moderately strong convergence with other measures of communicative ability and functioning. A free software application for administration and scoring of the ACOM item bank is available from the first author. Conclusions: The ACOM provides reliable measurement of patient-reported communicative functioning in aphasia. The results supported the validity of ACOM scores insofar as (a) factor analyses provided support for a coherent measurement model, (b) items functioned similarly across demographic and clinical subgroups, and (c) scores showed good convergence with measures of related constructs. [ABSTRACT FROM AUTHOR]

Purpose: In this study, we investigated the fit of the Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996) to an item-response-theory measurement model, estimated the precision of the resulting scores and item parameters, and provided a theoretical rationale for the interpretation of PNT overall scores by relating explanatory variables to item difficulty. This article describes the statistical model underlying the computer adaptive PNT presented in a companion article (Hula, Kellough, & Fergadiotis, 2015). Method: Using archival data, we evaluated the fit of the PNT to 1- and 2-parameter logistic models and examined the precision of the resulting parameter estimates. We regressed the item difficulty estimates on three predictor variables: word length, age of acquisition, and contextual diversity. Results: The 2-parameter logistic model demonstrated marginally better fit, but the fit of the 1-parameter logistic model was adequate. Precision was excellent for both person ability and item difficulty estimates. Word length, age of acquisition, and contextual diversity all independently contributed to variance in item difficulty. Conclusions: Item-response-theory methods can be productively used to analyze and quantify anomia severity in aphasia. Regression of item difficulty on lexical variables supported the validity of the PNT and interpretation of anomia severity scores in the context of current word-finding models. [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this study was to identify the most appropriate item response theory (IRT) measurement model for aphasia tests requiring 2-choice responses and to determine whether small samples are adequate for estimating such models. Method: Pyramids and Palm Trees (Howard & Patterson, 1992) test data that had been collected from individuals with aphasia were analyzed, and the resulting item and person estimates were used to develop simulated test data for 3 sample size conditions. The simulated data were analyzed using a standard 1-parameter logistic (1-PL) model and 3 models that accounted for the influence of guessing: augmented 1-PL and 2-PL models and a 3-PL model. The model estimates obtained from the simulated data were compared to their known true values. Results: With small and medium sample sizes, an augmented 1-PL model was the most accurate at recovering the known item and person parameters; however, no model performed well at any sample size. Follow-up simulations confirmed that the large influence of guessing and the extreme easiness of the items contributed substantially to the poor estimation of item difficulty and person ability. Conclusion: Incorporating the assumption of guessing into IRT models improves parameter estimation accuracy, even for small samples. However, caution should be exercised in interpreting scores obtained from easy 2-choice tests, regardless of whether IRT modeling or percentage correct scoring is used. [ABSTRACT FROM AUTHOR]