Search

To evaluate the reliability, construct validity, and responsiveness of the lower extremity-specific Patient-Reported Outcomes Measurement Information System (PROMIS) Mobility (MO) bank in patients who underwent hip arthroscopic surgery for femoroacetabular impingement.Patients who underwent primary hip arthroscopic surgery at a large academic musculoskeletal specialty center between November 2019 and November 2020 completed the following baseline and 6-month measures: PROMIS MO, PROMIS Pain Interference (PI), PROMIS Physical Function (PF), modified Harris Hip Score, International Hip Outcome Tool 33, visual analog scale, and Single Assessment Numeric Evaluation. Construct validity was evaluated using Spearman correlation coefficients. The number of questions until completion was recorded as a marker of test burden. The percentage of patients scoring at the extreme high (ceiling) or low (floor) for each measure was recorded to measure inclusivity. Responsiveness was tested by comparing differences between baseline and 6-month measures, controlling for age and sex, using generalized estimating equations. Magnitudes of responsiveness were assessed through the effect size (Cohen d).In this study, 660 patients (50% female patients) aged 32 ± 14 years were evaluated. PROMIS MO showed a strong correlation with PROMIS PF (r = 0.84, P .001), the International Hip Outcome Tool 33 (r = 0.73, P.001), PROMIS PI (r = -0.76, P.001), and the modified Harris Hip Score (r = 0.73, P.001). Neither PROMIS MO, PROMIS PI, nor PROMIS PF met the conventional criteria for floor or ceiling effects (≥15%). The mean number of questions answered (± standard deviation) was 4.7 ± 2.1 for PROMIS MO, 4.1 ± 0.6 for PROMIS PI, and 4.1 ± 0.6 for PROMIS PF. From baseline to 6 months, the PROMIS and legacy measures exhibited significant responsiveness (P.05), with similar effect sizes between the patient-reported outcome measures.This longitudinal study reveals that in patients undergoing hip arthroscopy, PROMIS MO computerized adaptive testing maintains high correlation with legacy hip-specific instruments, significant responsiveness to change, and low test burden compared with legacy measures, with no ceiling or floor effects at 6-month postoperative follow-up.Level IV, retrospective case series.

Adaptive measurement of change (AMC) is a psychometric method for measuring intra-individual change on one or more latent traits across testing occasions. Three hypothesis tests—a Z test, likelihood ratio test, and score ratio index—have demonstrated desirable statistical properties in this context, including low false positive rates and high true positive rates. However, the extant AMC research has assumed that the item parameter values in the simulated item banks were devoid of estimation error. This assumption is unrealistic for applied testing settings, where item parameters are estimated from a calibration sample before test administration. Using Monte Carlo simulation, this study evaluated the robustness of the common AMC hypothesis tests to the presence of item parameter estimation error when measuring omnibus change across four testing occasions. Results indicated that item parameter estimation error had at most a small effect on false positive rates and latent trait change recovery, and these effects were largely explained by the computerized adaptive testing item bank information functions. Differences in AMC performance as a function of item parameter estimation error and choice of hypothesis test were generally limited to simulees with particularly low or high latent trait values, where the item bank provided relatively lower information. These simulations highlight how AMC can accurately measure intra-individual change in the presence of item parameter estimation error when paired with an informative item bank. Limitations and future directions for AMC research are discussed.

OBJECTIVE Patient-Reported Outcomes Measurement Information System® (PROMIS® ) measures can be administered via computerized adaptive testing (CAT) or fixed short forms (FSF), but the empirical benefits of CAT versus FSF are unknown in juvenile myositis (JM). This study assesses if PROMIS CAT is feasible, precise, correlated with FSF, and less prone to respondent burden and floor/ceiling effects than FSF in JM. METHODS Patients 8-17 yo (self-report and parent proxy) and parents of patients 5-7 yo (only parent proxy) completed PROMIS Fatigue, Pain Interference, Upper Extremity Function, Mobility, Anxiety and Depressive Symptoms measures. Pearson correlations, paired t-tests, and Cohen's D were calculated between PROMIS CAT and FSF. McNemar test assessed floor/ceiling effects between CAT and FSF. Precision and respondent burden were examined across the T-score range. RESULTS Data from 67 patient-parent dyads were analyzed. CAT and FSF mean scores did not significantly differ except in parent proxy Anxiety and Fatigue (effect size: 0.23 and 0.19, respectively). CAT had less pronounced floor/ceiling effects at the less symptomatic extreme in all domains except self-report Anxiety. Increased item burden and higher standard errors were seen in less symptomatic scorers for CAT. Modified stopping rules limiting CAT item administration did not decrease precision. CONCLUSION PROMIS CAT appears to be feasible and correlated with FSF. CAT had less pronounced floor/ceiling effects, allowing detection of individual differences in less symptomatic patients. Modified stopping rules for CAT may decrease respondent burden. CAT can be considered for long-term follow-up of JM patients.

Patient-reported outcome measures are useful tools to quantify patients' pre-treatment and post-treatment symptoms. Historically used "legacy measures", such as the Scoliosis Research Society-22 revised questionnaire (SRS-22r), are often disease-specific and can be time-intensive. Recently developed Patient-Reported Outcomes Measurement Information System (PROMIS) computerized adaptive testing (CAT) measures may reduce administrative burdens and permit more efficient outcome collection within clinic workflows. In an era of medicine where payments are becoming tied to outcomes, we sought to assess the time to completion (TTC) of 8 pediatric PROMIS CAT measures and the SRS-22r in adolescents with idiopathic scoliosis.Patients presenting to a large, urban tertiary referral hospital were prospectively enrolled into the study. Subjects were first-time survey respondents in various phases and types of treatment for adolescent idiopathic scoliosis. In total, 200 patients ranging from 10 to 17 years old completed 8 Pediatric PROMIS CATs and the SRS-22r. PROMIS CATs administered include Physical Activity, Mobility, Anxiety, Depressive symptoms, Peer Relationships, Physical Stress Experiences, Pain Behavior and Pain Interference. TTC was calculated using start and stop timestamps in the REDCap software.The mean (±SD) TTC for each PROMIS CAT was 1.1 (±0.9) minutes with physical activity, mobility, anxiety, depressive symptoms, peer relationships, physical stress experiences, pain behavior, and pain interference taking 1.2, 1.4, 1.0, 0.9, 1.2, 1.0, 1.0, and 1.2 minutes on average to complete, respectively. Mean TTC for the SRS-22r was 5.2 (±3.0) minutes.In this pediatric orthopaedic cohort, completion of 8 PROMIS CATs demonstrated minimal test-taker burden and time required for completion. These findings support rapid and easily integrable PROMIS CATs in clinical practice to aid in increased delivery of efficient, patient-centered care.III, cross-sectional study.

The use of patient-reported outcome measures, especially Patient-Reported Outcomes Measurement Information System (PROMIS) measures, has increased in recent years. Given this growth, it is imperative to ensure that the measures being used are validated for the intended population(s)/disease(s). Our objective was to assess the construct validity of 8 PROMIS computer adaptive testing (CAT) measures among children with adolescent idiopathic scoliosis (AIS).We prospectively enrolled 200 children (aged 10 to 17 y) with AIS, who completed 8 PROMIS CATs (Anxiety, Depressive Symptoms, Mobility, Pain Behavior, Pain Interference, Peer Relationships, Physical Activity, Physical Stress Experiences) and the Scoliosis Research Society-22r questionnaire (SRS-22r) electronically. Treatment categories were observation, bracing, indicated for surgery, or postoperative from posterior spinal fusion. Construct validity was evaluated using known group analysis and convergent and discriminant validity analyses. Analysis of variance was used to identify differences in PROMIS T -scores by treatment category (known groups). The Spearman rank correlation coefficient ( rs ) was calculated between corresponding PROMIS and SRS-22r domains (convergent) and between unrelated PROMIS domains (discriminant). Floor/ceiling effects were calculated.Among treatment categories, significant differences were found in PROMIS Mobility, Pain Behavior, Pain Interference, and Physical Stress Experiences and in all SRS-22r domains ( P0.05) except Mental Health ( P =0.15). SRS-22r Pain was strongly correlated with PROMIS Pain Interference ( rs =-0.72) and Pain Behavior ( rs =-0.71) and moderately correlated with Physical Stress Experiences ( rs =-0.57). SRS-22r Mental Health was strongly correlated with PROMIS Depressive Symptoms ( rs =-0.72) and moderately correlated with Anxiety ( rs =-0.62). SRS-22r Function was moderately correlated with PROMIS Mobility ( rs =0.64) and weakly correlated with Physical Activity ( rs =0.34). SRS-22r Self-Image was weakly correlated with PROMIS Peer Relationships ( rs =0.33). All unrelated PROMIS CATs were weakly correlated (| rs |0.40). PROMIS Anxiety, Mobility, Pain Behavior, and Pain Interference and SRS-22r Function, Pain, and Satisfaction displayed ceiling effects.Evidence supports the construct validity of 6 PROMIS CATs in evaluating AIS patients. Ceiling effects should be considered when using specific PROMIS CATs.Level II, prognostic.

The increasing use of patient-reported outcome (PRO) measures is forcing clinicians and health care systems to decide which to select and how to incorporate them into their records and clinical workflows. This overview addresses 3 topics related to these concerns. First, a literature review summarizes key psychometric and practical factors (such as reliability, responsiveness, computer adaptive testing, and interpretability) in choosing PROs for clinical practice. Second, 3 clinical decision support issues are highlighted: gathering PROs, electronic health record effect on providers, and incorporating PROs into clinical decision support design and implementation. Lastly, the salience of crosscutting domains as well as 9 key pragmatic decisions are reviewed. Crosscutting domains are those that are relevant across most medical and mental health conditions, such as the SPADE symptom pentad (sleep problems, pain, anxiety, depression, low energy/fatigue) and physical functioning. The 9 pragmatic decisions include (1) generic vs disease-specific scales; (2) single- vs multidomain scales; (3) universal scales vs user-choice selection; (4) number of domains to measure; (5) prioritization of domains when multiple domains are assessed; (6) action thresholds; (7) clinical purpose (screening vs monitoring); as well as the (8) frequency and (9) logistical aspects of PRO administration.

Structured Abstract Objective To characterize the ability of the patient-reported Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test (FAMCAT) domains to predict discharge disposition when administered during acute care stays. Design Cohort Study. Logistic regression models were estimated to identify the ability of FAMCAT domains to predict discharge to an institution for post-acute care (PAC). Setting Academic Medical Center Participants Patients admitted to General Medicine Services from June 2016 to June 2019, n = 4,240 Interventions Not applicable. Main Outcome Measure(s) discharge to an institution Results In this sample, 10.5% of patients were discharged to an institution for rehabilitation versus home. FAMCAT domain scores were highly predictive of discharge to institutional PAC. Daily Activity and Basic Mobility domains had excellent discriminative ability for discharge to an institution (c-statistic 0.83 and 0.87, respectively). In best fit models accounting for additional characteristics, discrimination was outstanding for Daily Activity (c-statistic 0.91, 95% CI: 0.89, 0.94) and Basic Mobility (c-statistic 0.92, 95% CI: 0.89, 0.94). Conclusion(s) The FAMCAT Daily Activity and Mobility domains demonstrated excellent discrimination for identifying patients who discharged to an institutional setting for rehabilitation, and outstanding discrimination when adjusted for salient patient factors associated with discharge disposition. Estimates obtained in this investigation are comparable to the best discrimination achieved with clinician-rated measures to identify patients who would require institutional PAC.

Objective To present challenges in assessment of applied cognition and the results of differential item functioning (DIF) analyses used to inform the development of a computerized adaptive test (CAT). Design Measurement evaluation cohort study. DIF analyses of 107 items were conducted across educational, age, and sex groups. DIF hypotheses informed the evaluation of the results. Setting Hospital-based rehabilitation from a single hospital system. Participants A total of 2216 hospitalized patients (N=2216). Interventions Not applicable. Main Outcome Measures Applied cognition item pool from multiple sources. Results Many items were hypothesized to show DIF, particularly for age. Information was moderately high in the lower (cognitive disability) tail of the distribution, but some items were not informative. Reliability estimates were high (>0.89) across all studied groups, regardless of estimation method. There were 35 items with DIF of high magnitude and 19 with accompanying supportive hypotheses. Conclusions A key clinical tool in inpatient rehabilitation medicine is assessment of applied functional cognitive ability to inform patient-centered rehabilitation strategies to improve function. This was the first study to evaluate measurement equivalence of the applied cognition item pool across large samples of hospitalized patients. Although about one-third of the item pool evidenced DIF or low discrimination, results supported placement of most items into the bank and its use across groups differing in education, age, and sex. Six items were classified with salient DIF, defined as consistent DIF of high magnitude and or impact, with confirmatory directional DIF hypotheses, generated by content experts. These were recommended for adjustment or removal from the bank; 4 were deleted from the bank and 2 had lowered CAT exposure (administration frequency) rates. Many items hypothesized to show DIF contained content measuring constructs other than applied cognition such as physical frailty, perceptual difficulties, or skills reflective of greater educational attainment. Challenges in measurement of this construct are discussed.

Objective To develop an item response theory (IRT)–calibrated, patient-reported outcome measure (the PROMIS Cancer Function Brief 3D Profile) of physical function, including associations with fatigue and social participation, in cancer rehabilitation patients. Design Large-scale field testing, graded response model IRT analyses, and multivariate regression analysis. Setting Six cancer rehabilitation clinics associated with cancer centers across the United States. Participants Adults (N=616) treated in outpatient cancer rehabilitation medicine clinics. Intervention Not applicable. Main Outcome Measures: The PROMIS(r) Cancer Function 3D Profile (including existing items from PROMIS(r) item banks). Results A total of 616 patients completed 21 items in the initial item pool. Nine items were removed because of comparatively lower information that they provide according to the IRT item calibrations, low item-total correlations, or bimodal distributions. The remaining items generated a 12-item short form. Regression analyses determined that the items were responsive to and representative of the patient population across trait ranges and multiple domains and subdomains of function. Conclusions This psychometric investigation supports the use of the PROMIS Cancer Function Brief 3D Profile for evaluating function in outpatient cancer rehabilitation patients.

There remains a paucity of information analyzing which factors most influence preoperative expectations for patients undergoing reverse total shoulder arthroplasty (RTSA). The purposes of our study were to characterize preoperative patient expectations for those scheduled to undergo RTSA and to determine the impact of demographic factors, shoulder function, and shoulder pain on these preoperative expectations.Patients were prospectively recruited into the study if they were scheduled to undergo an elective unilateral primary RTSA for a diagnosis of glenohumeral arthritis. Preoperative patient expectations were evaluated using the Hospital for Special Surgery's Shoulder Surgery Expectation Survey. Patients also completed the American Shoulder and Elbow Surgeons shoulder score, the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function-Upper Extremity computer adaptive test (version 2.0), the PROMIS Pain Interference (PI) computer adaptive test (version 1.1), the PROMIS Depression computer adaptive test (version 1.0), visual analog scores, and an itemized satisfaction questionnaire, which paralleled the Hospital for Special Surgery's Shoulder Surgery Expectation Survey. Demographic data and preoperative shoulder range of motion (ROM) were also recorded.A total of 107 patients scheduled to undergo RTSA were included in the study. Relief of daytime pain (n = 91, 85%), improvement in self-care (n = 86, 80%), and improvement in shoulder ROM (n = 85, 79%) were most commonly cited as "very important" expectations. In the item-specific analysis, lower PROMIS Upper Extremity scores were correlated with greater expectations for the ability to reach sideways (P = .015) and the ability to perform daily activities (P = .018). Patients with lower shoulder ROM had greater expectations for improved shoulder ROM (internal rotation with arm at 90°, P = .004) and an improved ability to perform daily activities (forward elevation, P = .038; abduction, P = .009). In the cumulative analysis, a greater number of very important expectations was associated with African American race (P = .013), higher PROMIS PI score (r = 0.351, P = .004), and lower overall preoperative satisfaction (r = 0.334, P.001).Patients scheduled to undergo RTSA have the greatest expectations for relief of daytime pain, improvement in self-care, and improvement in shoulder ROM. Patients with limited preoperative ROM have greater expectations for improvement in self-care and the ability to perform daily activities in addition to expectations for improvement in shoulder ROM. Greater overall expectations for surgery were not associated with preoperative physical function but were instead associated with lower preoperative satisfaction and higher PROMIS PI scores.

In recent years, Service Oriented Architecture (SOA) has been increasingly adopted to develop distributed applications in the context of the Internet. To develop reliable SOA-based applications, an important issue is how to ensure the quality of web services. In this paper, we propose a dynamic random testing (DRT) technique for web services, which is an improvement over the widely-practiced random testing (RT) and partition testing (PT). We examine key issues when adapting DRT to the context of SOA, including a framework, guidelines for parameter settings, and a prototype for such an adaptation. Empirical studies are reported where DRT is used to test three real-life web services, and mutation analysis is employed to measure the effectiveness. Our experimental results show that, compared with the three baseline techniques, RT, Adaptive Testing (AT) and Random Partition Testing (RPT), DRT demonstrates higher fault-detection effectiveness with a lower test case selection overhead. Furthermore, the theoretical guidelines of parameter setting for DRT are confirmed to be effective. The proposed DRT and the prototype provide an effective and efficient approach for testing web services. IEEE

PURPOSE: To assess the psychometric properties of glaucoma-specific health-related quality of life (HRQoL) item banks (IBs), and explore their efficiency using computerized adaptive testing (CAT) simulations. METHODS: In this cross-sectional, clinical study, 300 Asian glaucoma patients answered 221 items within seven IBs: Ocular Comfort Symptoms (OS); Activity Limitation (AL); Lighting (LT); Mobility (MB); Glaucoma Management (GM); Psychosocial (PSY); and Work (WK). Rasch analysis was conducted to assess each IB's psychometric properties (e.g., item "fit" to the construct; unidimensionality) and a set of analytic performance criteria guiding decision making relating to retaining or dropping domains and items was employed. CAT simulations determined the mean number of items for 'high' and 'moderate' measurement precision (stopping rule: SEM 0.3 and 0.387, respectively). RESULTS: Participants' mean age was 67.2 ± 9.2 years (62% male; 87% Chinese). LT, MB, and GM displayed good psychometric properties overall. To optimize AL's psychometric properties, 16 items were deleted due to poor "fit", high missing data, item bias, low discrimination and/or a low clinical/patient importance rating. To resolve multidimensionality in PSY, we rehomed 16 items into a "Concern (CN)" domain. PSY and CN required further amendment, including collapsing of response categories, and removal of poorly functioning items (N = 7). Due to poor measurement precision, low applicability and high ceiling effect, low test information indices, and low item separation index the WK IB was not considered further. In CAT simulations on the final seven IBs (n = 182 items total), an average of 12.1 and 15.7 items per IB were required for moderate and high precision measurement, respectively. CONCLUSIONS: After reengineering our seven IBs, they displayed robust psychometric properties and good efficiency in CAT simulations. Once finalized, GlauCAT™-Asian may enable comprehensive assessment of the HRQoL impact of glaucoma and associated treatments.

Objectives To develop clinically relevant interpretive standards for the Spinal Cord Injury – Functional Index/Capacity (SCI-FI/C) Basic Mobility and Self-Care item bank scores. Design Modified “bookmarking” standard-setting methodology, including 2 stakeholder consensus meetings with individuals with spinal cord injury (SCI) and SCI clinicians, respectively, and a final, combined (consumers and clinicians) “convergence” meeting. Setting Two SCI Model System centers in the United States. Participants Fourteen adults who work with individuals with traumatic SCI and 14 clinicians who work with individuals with SCI. Main Outcome Measures Placement of bookmarks between vignettes based on SCI-FI Basic Mobility and Self-Care T scores. Bookmarks were placed between vignettes representing “No Problems,” “Mild Problems,” “Moderate Problems,” and “Severe Problems” for each item bank. Results Each consensus group resulted in a single set of scoring cut points for the SCI-FI/C Basic Mobility and Self-Care item banks. The cut points were similar but not identical between the consumer and clinician groups, necessitating a final convergence meeting. For SCI-FI/C Basic Mobility, the convergence group agreed on cut scores of 61.25 (no problems/mild problems), 51.25 (mild problems/moderate problems), and 41.25 (moderate problems/severe problems). For SCI-FI/C Self-Care, the convergence group agreed on cut scores of 56.25 (no/mild), 51.25 (mild/moderate), and 38.75 (moderate/severe). Conclusions The results of this study provide straightforward interpretive guidelines for SCI researchers and clinicians using the SCI-FI/C Basic Mobility and Self-Care instruments. These results are appropriate for the full bank, computer adaptive test, and short-form versions of the SCI-FI/C Basic Mobility and Self-Care item banks.

Objective To establish responsiveness of 3 Spinal Cord Injury – Functional Index/Capacity (SCI-FI/C) item banks in the first year following SCI. Design Longitudinal patient-reported outcomes assessment replicated through secondary analysis of an independent data set. Setting 8 SCI Model Systems rehabilitation hospitals in the US. Participants Study 1 participants included 184 adults with recent (≤4 months) traumatic SCI and 221 community-dwelling adults (>1 year post injury). Study 2 participants were 418 individuals with recent SCI (≤4 months). Interventions In Study 1, SCI-FI/C computer adaptive tests were presented in a standardized interview format either in person or by phone call at baseline and 6-month follow-up. Responsiveness was examined by comparing 6-month changes in SCI-FI scores within and across samples (recently-injured vs. community-dwelling), as only the recent injury sample was expected to exhibit change over time. Effect sizes were also computed. In Study 2, the Study 1 results were cross-validated in a second recent SCI sample 1 year after baseline measurement. Study 2 also compared the SCI-FI/C measures’ responsiveness to that of the Self Report Functional Measure (SRFM) and stratified results by injury diagnosis and completeness. Main Outcome Measures The SCI-FI Basic Mobility/C, Self-Care/C and Fine Motor/C item banks (Study 1 and Study 2); Self Report Functional Measure (SRFM; Study 2 only). Results In Study 1, changes in SCI-FI/C scores between baseline and 6-month follow-up were statistically significant (p Conclusion The SCI-FI Basic Mobility/C and Self-Care/C banks demonstrate adequate sensitivity to change at 6 months and 1 year for all individuals with SCI, while the SCI-FI/C Fine Motor item bank is sensitive to change in individuals with tetraplegia or incomplete paraplegia. All SCI-FI/C banks demonstrate stability in a sample not expected to change. Results provide support for the use of these measures for research or clinical use.

With continuously decreasing circuit scale, more and more test patterns (test contents) are added to test set to achieve acceptable defect levels, which seriously affect test time and, consequently, test cost. Theoretically, dropping invalid (pattern that can make the test pass) test patterns can reduce test cost. In fact, When the valid (pattern that can make the test fail) and invalid patterns overlap seriously, identifying effective patterns performs poorly, which will not achieve the expected results. This paper proposes a kind of eliminate redundancy method, so that this LDA-MRMR algorithm which can provide test cost reduction without increasing the defect level obviously. Experimental results demonstrate that the proposed method sacrifices 3% of the predictive accuracy in exchange for 3.7 times time saving compared with traditional methods. In addition, the algorithm is completely software-based and does not require any additional hardware overhead and is directly compatible with traditional integrated circuit testing process.

Purpose: To determine the change in Patient-Reported Outcomes Measurement Information System Computerized Adaptive Testing (PROMIS CAT) scores for physical function, pain interference, and depression that constitute minimum clinically important difference (MCID) using an anchor-based technique and to identify pre-operative clinical thresholds in anchor-based MCID that predict likelihood of achieving MCID following anterior cruciate ligament (ACL) reconstruction. Methods: Adult patients aged 18 years or older undergoing ACL reconstruction that completed both preoperative and postoperative PROMIS CAT assessments and an anchor-based questionnaire were identified over a 23-month period. Anchor-based MCID was determined for PROMIS CAT forms for physical function (PROMIS PF CAT), pain interference (PROMIS PI CAT), and depression (PROMIS D CAT). Results: A total of 137 patients were included for statistical analysis, with pre-operative PROMIS CAT forms completed 27.9 ± 31.2 days before surgery and 492.5 ± 219.9 days postoperatively on average. Statistically significant improvements were observed for all PROMIS CAT domains. PROMIS PF CAT improved from 39.5 ± 8.2 to 55.0 ± 9.7 (P < .0005), PROMIS PI CAT from 59.8 ± 7.2 to 48.2 ± 8.3 (P < .0005), and PROMIS D CAT from 47.9 ± 8.8 to 41.5 ± 8.6 (P < .0005). Anchor-based MCID for each PROMIS CAT form was calculated to be +4.5, –5.4, and –4.1 for PROMIS PF CAT, PROMIS PI CAT, and PROMIS D CAT, respectively. Mean difference between preoperative and postoperative PROMIS CAT scores exceeded MCID for all domains. The percentage of patients achieving MCID for PROMIS PF CAT, PROMIS PI CAT, and PROMIS D CAT was 85%, 72%, and 55%, respectively. After introduction of 95% specificity cutoffs, the percentage of patients achieving MCID for PROMIS PF CAT, PROMIS PI CAT, and PROMIS D CAT increased to 100% (65.7 cutoff score), and 83% (>57.5 cutoff score), respectively. Conclusions: According to anchor-based analysis of PROMIS CAT MCID, ACL reconstruction is effective in improving physical function, pain interference, and depression symptoms. In addition, preoperative PROMIS CAT scores can predict the likelihood of achieving MCID postoperatively. Level of Evidence: Level IV, prognostic case series.

Multiple Administrations Adaptive Testing (MAAT) is an extension of the shadow-test approach to CAT for the assessment framework involving multiple tests administered periodically throughout the year. The maat package utilizes multiple item pools vertically scaled across grades and multiple phases (stages) within each test administration, allowing for transitioning from an item pool to another as deemed necessary to further enhance the quality of assessment.

Computer-based assessment or e-assessment system is an e-learning system where information communication technology is utilized for examination activity, grading, and recording of responses of the examinees. It includes the entire assessment process from the examinees, teachers, institutions, examination agencies, and the public. E-assessment systems have been used extensively in educational and non-educational settings worldwide because of their significant impact on assessment to both the institutions and students. Electronic assessment systems have many significant advantages over conventional paper-pencil-based approaches, like reducing production cost, automatic marking, and fair grading. Other benefits include the ability to conduct adaptive testing, an increase in the frequency of assessment, a large number of people could be examined, low staff requirements during the assessment, and the ability to improve the quality of the evaluation. Text, images, audio, video, and interactive virtual environments in an e-assessment system are all feasible. Nevertheless, given all these advantages, some examinees expressed negative views about electronic assessment in our study due to fear or lack of familiarity with assessment technology and lack of information on e-assessment methods. Despite the increased adoption of e-assessment in higher education and organizations, the examinees' attitudes and opinions need to be studied to implement the system successfully. In an e-assessment environment, several factors account for the evaluation of the system quality. These factors can be categorized into three dimensions: technical, educational, and economical. However, based on literature reviewed, limited studies attempted to map these quality factors to determine user satisfaction with an e-assessment system. This scarcity of e-assessment satisfaction studies is the gap the study intends to fill. Furthermore, this study's findings would help provide practical and theoretical implications for educational institutions and organizations. This study is among the first application of the Delone and McLean information system success model to predict user satisfaction with computer-based assessment in a developing, African country.

Background Social deprivation negatively affects a myriad of physical and behavioral health outcomes. Several measures of social deprivation exist, but it is unclear which measure is best suited to describe patients with orthopaedic conditions. Questions/purposes (1) Which measure of social deprivation, defined as "limited access to society's resources due to poverty, discrimination, or other disadvantage," is most strongly and consistently correlated with patient-reported physical and behavioral health in patients with orthopaedic conditions? (2) Compared with the use of a single measure alone, how much more variability in patient-reported health does the simultaneous use of multiple social deprivation measures capture? Methods Between 2015 and 2017, a total of 79,818 new patient evaluations occurred within the orthopaedic department of a single, large, urban, tertiary-care academic center. Over that period, standardized collection of patient-reported health measures (as described by the Patient-reported Outcomes Measurement Information System [PROMIS]) was implemented in a staged fashion throughout the department. We excluded the 25% (19,926) of patient encounters that did not have associated PROMIS measures reported, which left 75% (59,892) of patient encounters available for analysis in this cross-sectional study of existing medical records. Five markers of social deprivation were collected for each patient: national and state Area Deprivation Index, Medically Underserved Area Status, Rural-Urban Commuting Area code, and insurance classification (private, Medicare, Medicaid, or other). Patient-reported physical and behavioral health was measured via PROMIS computer adaptive test domains, which patients completed as part of standard care before being evaluated by a provider. Adults completed the PROMIS Physical Function version 1.2 or version 2.0, Pain Interference version 1.1, Anxiety version 1.0, and Depression version 1.0. Children ages 5 to 17 years completed the PROMIS Pediatric Mobility version 1.0 or version 2.0, Pain Interference version 1.0 or version 2.0, Upper Extremity version 1.0, and Peer Relationships version 1.0. Age-adjusted partial Pearson correlation coefficients were determined for each social deprivation measure and PROMIS domain. Coefficients of at least 0.1 were considered clinically meaningful for this purpose. Additionally, to determine the percentage of PROMIS score variability that could be attributed to each social deprivation measure, an age-adjusted hierarchical regression analysis was performed for each PROMIS domain, in which social deprivation measures were sequentially added as independent variables. The model coefficients of determination (r2) were compared as social deprivation measures were incrementally added. Improvement of the r2 by at least 10% was considered clinically meaningful. Results Insurance classification was the social deprivation measure with the largest (absolute value) age-adjusted correlation coefficient for all adult and pediatric PROMIS physical and behavioral health domains (adults: correlation coefficient 0.40 to 0.43 [95% CI 0.39 to 0.44]; pediatrics: correlation coefficient 0.10 to 0.19 [95% CI 0.08 to 0.21]), followed by national Area Deprivation Index (adults: correlation coefficient 0.18 to 0.22 [95% CI 0.17 to 0.23]; pediatrics: correlation coefficient 0.08 to 0.15 [95% CI 0.06 to 0.17]), followed closely by state Area Deprivation Index. The Medically Underserved Area Status and Rural-Urban Commuting Area code each had correlation coefficients of 0.1 or larger for some PROMIS domains but neither had consistently stronger correlation coefficients than the other. Except for the PROMIS Pediatric Upper Extremity domain, consideration of insurance classification and the national Area Deprivation Index together explained more of the variation in age-adjusted PROMIS scores than the use of insurance classification alone (adults: r2 improvement 32% to 189% [95% CI 0.02 to 0.04]; pediatrics: r2 improvement 56% to 110% [95% CI 0.01 to 0.02]). The addition of the Medically Underserved Area Status, Rural-Urban Commuting Area code, and/or state Area Deprivation Index did not further improve the r2 for any of the PROMIS domains. Conclusion To capture the most variability due to social deprivation in orthopaedic patients' self-reported physical and behavioral health, insurance classification (categorized as private, Medicare, Medicaid, or other) and national Area Deprivation Index should be included in statistical analyses. If only one measure of social deprivation is preferred, insurance classification or national Area Deprivation Index are reasonable options. Insurance classification may be more readily available, but the national Area Deprivation Index stratifies patients across a wider distribution of values. When conducting clinical outcomes research with social deprivation as a relevant covariate, we encourage researchers to consider accounting for insurance classification and/or national Area Deprivation Index, both of which are freely available and can be obtained from data that are typically collected during routine clinical care. Level of evidence Level III, therapeutic study.

Modern, reliable, and valid outcome measures are essential to understanding the health needs of young children with burn injuries. Burn-specific and age-appropriate legacy assessment tools exist for this population but are hindered by the limitations of existing paper-based instruments. The purpose of this study was to develop item pools comprised of questions appropriate for children aged 1-5 with burn injuries. Item development was based on a framework provided by previous work to develop the Preschool Life Impact Burn Recovery Evaluation (LIBRE) Conceptual Model. The Preschool LIBRE Conceptual Model work established four sub-domains of functioning for children with burns aged 1-5. Item development involved a systematic literature review, a qualitative item review process with clinical experts, and parent cognitive interviews. Four item pools were established: (1) communication and language development; (2) physical functioning; (3) psychological functioning and (4) social functioning for preschool-aged children with burn injuries. We selected and refined candidate items, recall periods, survey instructions, and response option choices through clinical and parental feedback during the qualitative review and cognitive interview processes. Item pools are currently being field-tested as part of the process to calibrate and validate the Preschool1-5 LIBRE Computer Adaptive Test (CAT) Profile.

Background The evaluation of Patient Reported Outcomes Measurement Information System (PROMIS) computerized adaptive test (CAT) in adults with systemic lupus erythematous (SLE) is an emerging field of research. We aimed to examine the test–retest reliability and construct validity of the PROMIS CAT in a Canadian cohort of patients with SLE. Methods Two hundred twenty-seven patients completed 14 domains of PROMIS CAT and seven legacy instruments during their clinical visits. Test–retest reliability of PROMIS was evaluated 7–10 days from baseline using intraclass correlation coefficient (ICC (2; 1)). The construct validity of the PROMIS CAT domains was evaluated against the commonly used legacy instruments, and also in comparison to disease activity and disease damage using Spearman correlations. A multitrait-multimethod matrix (MMM) approach was used to further assess construct validity comparing selected 10 domains of PROMIS and SF-36 domains. Results Moderate to excellent reliability was found for all domains (ICC [2;1] ranging from lowest, 0.66 for Sleep Disturbance and highest, 0.93 for the Mobility domain). Comparing seven legacy instruments with 14 domains of PROMIS CAT, moderate to strong correlations (0.51–0.91) were identified. The average time to complete all PROMIS CAT domains was 11.7 min. The MMM further established construct validity by showing moderate to strong correlations (0.55–0.87) between select PROMIS and SF-36 domains; the average correlations from similar traits (convergent validity) were significantly greater than the average correlations from different traits. Conclusions These results provide evidence on the reliability and validity of PROMIS CAT in SLE in a Canadian cohort.

The first two decades of the twenty-first century witnessed the blossoming of frameworks to conceptualize “21st-century skills” both in school and career. However, there is still a lack of consensus about what we are talking about when using the term “21st-century skills,” as the existing frameworks on “21st-century skills” are different from each other. Drawing upon 471 existing sources published in 2000–2017 regarding “21st-century skills,” identified from six research databases, this study was intended to answer the following research question via a systematic review: What skills had been considered as “21st-century skills” in the existing educational research literature? Seven categories or domains of knowledge and skills emerged when investigating this literature. Implications and future research directions about how adaptive learning and adaptive testing in the new century can empower educational and vocational guidance were briefly discussed at the end of this paper.

BACKGROUND: Measures of unhelpful thoughts and distress correlate with the intensity of pain and the magnitude of incapability among people seeking musculoskeletal specialty care. In this evolving knowledge area, we want to be sure we have not neglected other important mental health factors. This study addressed how measures of confidence in problem solving as well as past and current ability to achieve goals account for variation in symptoms and capability independent of unhelpful thoughts and distress. QUESTIONS/PURPOSES: (1) Are measures of confidence in problem solving ability and past and current ability to achieve goals regarding future outcomes associated with variation in capability, independent of measures of symptoms of depression and anxiety (distress) and measures of unhelpful thoughts (worst-case thinking, negative pain thoughts)? (2) Are these measures independently associated with variation in pain intensity? (3) Are these measures associated with measures of symptoms of depression, symptoms of anxiety, and unhelpful thoughts? METHODS: Over a 7-month period during the pandemic, we enrolled sporadically from the offices of four surgeons treating patients who sought care for various upper and lower extremity conditions. We invited approximately 200 adult new and returning patients to participate (the number of invitations was not formally tracked) and 187 accepted. Thirty-one were excluded due to markedly incomplete entries (related to a problematic attempt to use the patient’s cell phone to complete questionnaires as a pandemic work around), leaving 156 for analysis. Patients completed an 11-point ordinal rating of pain intensity, two measures of unhelpful thoughts (the Pain Catastrophizing Scale and the Negative Pain Thoughts Questionnaire), the Adult Hope Scale to measure past and current ability to achieve goals, the Personal Optimism and Self-Efficacy Optimism Scale to measure confidence in problem solving ability, the Patient-reported Outcomes Measurement Information System (PROMIS) computer adaptive test to measure symptoms of anxiety, the PROMIS computer adaptive test to measure symptoms of depression, and the PROMIS physical function computer adaptive test to assess the magnitude of capability. All questionnaires were validated in previous studies. We used bivariate analyses to identify factors associated with magnitude of capability, pain intensity, confidence in problem solving ability, and past and current ability to achieve goals. All factors with a p value of less than 0.1 were included in multivariable analyses to seek associations between these measures accounting for confounders. We reported partial η(2) as a measure of effect size for all multivariable regression models. The following rules of thumb are used to interpret values for partial η(2): a value of 0.01 = small, 0.06 = medium, and values of 0.14 and higher show large effect size. RESULTS: Greater capability was modestly associated with fewer negative pain thoughts (β = -0.63 [95% CI -1.0 to -0.22]; standard error = 0.20; partial η(2) = 0.06; p = 0.003) and no self-reported comorbidities (β = 2.6 [95% CI 0.02 to 5.3]; standard error = 1.3; partial η(2) = 0.03; p = 0.048) after controlling for education, symptoms of depression and anxiety, worst-case thinking, as well as past and current ability to achieve goals. In a similar multivariable model, greater pain intensity was modestly associated with greater worst-case thinking (β = 0.33 [95% CI 0.20 to 0.45]; standard error = 0.06; partial η(2) = 0.16; p < 0.001) and established patients (β = -1.1 [95% CI -1.8 to -0.31]; standard error = 0.38; partial η(2) = 0.05; p = 0.006). In another similar multivariable model, having more confidence in problem solving ability had a limited association with higher ratings of past and current ability to achieve goals (β = 0.15 [95% CI 0.09 to 0.21]; standard error = 0.03; partial η(2) = 0.13; p < 0.001). In a final multivariable model, lower past and current ability to achieve goals was independently associated with having greater symptoms of depression (β = -0.45 [95% CI -0.67 to -0.23]; standard error = 0.11; partial η(2) = 0.1; p < 0.001) and more negative pain thoughts (β = -0.49 [95% CI -0.89 to -0.09]; standard error = 0.20; partial η2 = 0.04; p = 0.02). CONCLUSION: The observation that unhelpful thoughts about symptoms are more strongly associated with symptom intensity than past and current ability to achieve goals and confidence in problem solving ability add to the evidence that attentiveness to unhelpful thinking is an important aspect of musculoskeletal health. Musculoskeletal specialists can prioritize communication strategies such as relationship building and motivational interviewing that develop trust and facilitate reorientation of common unhelpful thoughts. LEVEL OF EVIDENCE: Level II, prognostic study.

The Washington Assessment of the Risks and Needs of Students (WARNS) is a computer-based assessment created to help courts, schools, and youth service providers determine an adolescent’s risks and needs that may lead to truancy, drop out, or delinquency from school. Users are advised to consider the WARNS total score to work with youth. A total score estimate based on fewer items than the full item set may result in less respondent burden, administration time, and fatigue, while not hindering accurate decisions. This simulation study examined the applicability and efficiency of a computerized adaptive test (CAT) to estimate a WARNS total score under a unidimensional item response theory model. The results demonstrate that the CAT provides an accurate estimate of students’ risks and needs and reduces the number of items administered for each examinee compared with the existing version. Future directions and limitations of CAT development with the WARNS are discussed.

Background Objectives Methods Results and conclusions Major takeaways With the recent pivot to online instruction and/or to mixed online/face‐to‐face (i.e., hybrid) models of teaching necessitated by the novel coronavirus pandemic, the uses of technology to support instruction would seem to have great importance. Whereas effective integration of technology in mathematics classrooms requires recognition of students’ needs and instructional goals that technology could fulfill, very few comprehensive meta‐analytical studies have focused on the roles of instructional technology in support of specific learning goals.In this meta‐analysis, we investigated and reviewed the roles and affordances of technology in mathematics instruction and its effects on students' mathematics achievement.Cross‐searching multiple bibliographic databases generated a total of 3276 potentially relevant full‐text studies. After applying a series of inclusion/exclusion criteria, we kept 77 studies with 151 effect sizes. Two researchers then coded for seven sets of study features and student characteristics: technology roles, publication‐related features (years and resources), research design, sample size, grade levels, duration of interventions, and types of the testing instruments. The coding achieved acceptable levels of inter‐rater reliabilities. Since effect sizes varied across studies and student populations, random‐effects models were used for overall effect estimates and moderator analyses.Overall, this meta‐analysis revealed a small, positive, and statistically significant effect of technology on students' mathematics achievement with an average Hedges' s g (ḡ) of 0.23. The largest significant moderator effects were found when technology was used to design and support collaborative and communicative environments (ḡ = 0.49);the next‐largest moderator effects were found when using technology as supports for problem‐solving (ḡ = 0.39), followed by the roles of as supports for conceptual development (ḡ = 0.31) and as adaptive mathematics practices (ḡ = 0.28). This study also found that the effects of technology were differentiated as a function of publication resources, research design, duration of intervention, and the testing instrument.Focusing on the roles and functions of technology, this study can contribute to guide potential synthetical studies as well as the future design of educational technology. In particular, technology is used effectively when it is used (a) to create and design a collaborative and communicative learning environment where students have chances to collaborate and interact with each other and (b) as supports for mathematics problem‐solving and for mathematics conceptual development. On the other hand, technology is less effective when it is used for monitoring and assessing students' learning processes without follow‐up instructions.The findings suggested that technology intervention lasting for a shorter time is the better choice. When technology is used for longer periods of instruction, maintaining its novelty and students’ curiosity should be considered. This study also suggested that incorporating computer‐adaptive testing in technology‐based intervention programs is more likely to detect technology effects as adaptive testing could continuously adjust to the abilities of individual students. [ABSTRACT FROM AUTHOR] Copyright of Journal of Computer Assisted Learning is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

We are developing an age-related macular degeneration (AMD) health-related quality of life (HRQoL) item bank, applicable to Western and Asian populations. We report primarily on content generation and refinement, but also compare the HRQoL issues reported in our study with Western studies and current AMD-HRQoL questionnaires. In this cross-sectional, qualitative study of AMD patients attending the Singapore National Eye Centre (May–December 2019), items/domains were generated from: (1) AMD-specific questionnaires; (2) published articles; (3) focus groups/semi-structured interviews with AMD patients (n = 27); and (4) written feedback from retinal experts. Following thematic analysis, items were systematically refined to a minimally representative set and pre-tested using cognitive interviews with 16 AMD patients. Of the 27 patients (mean ± standard deviation age 67.9 ± 7.0; 59.2% male), 18 (66.7%), two (7.4%), and seven (25.9%) had no, early-intermediate, and late/advanced AMD (better eye), respectively. Whilst some HRQoL issues, e.g. activity limitation, mobility, lighting, and concerns were similarly reported by Western patients and covered by other questionnaires, others like anxiety about intravitreal injections, work tasks, and financial dependency were novel. Overall, 462 items within seven independent HRQoL domains were identified: Activity limitation, Lighting, Mobility, Emotional, Concerns, AMD management, and Work. Following item refinement, items were reduced to 219, with 31 items undergoing amendment. Our 7-domain, 219-item AMD-specific HRQoL instrument will undergo psychometric testing and calibration for computerized adaptive testing. The future instrument will enable users to precisely, rapidly, and comprehensively quantify the HRQoL impact of AMD and associated treatments, with item coverage relevant across several populations.

Culturally bounded in nature, adaptive behavior is the degree to which a person meets the requirements of personal independence and social responsibilities. This study aimed to develop a computerized adaptive test (CAT) of a culturally appropriate adaptive behavior measure (i.e., the Activities and Participation Rating Scale [APRS]) in the Singapore context for children aged 7 to 12 years, that is, the CAT-APRS-Primary. In Study 1, an item bank consisting of 310 adaptive skills was developed and piloted among 56 caregivers. In Study 2, the refined 310 items were calibrated using Rasch analysis among 352 caregivers. After the removal of five poor items based on the Rasch analysis, the finalized 305 items were used for the CAT development, with these items showing good Rasch person and item fit statistics. Preliminary utility of the CAT-APRS-Primary was established, with participants completing 24 items on average. Limitations and implications for future research were discussed.

Developing an operating systems (OSs) for low-end embedded devices requires continuous adaptation to new hardware architectures and components, while serviceability of features needs to be assured for each individual platform under tight resource constraints. It is challenging to design a versatile and accurate heterogeneous test environment that is agile enough to cover a continuous evolution of the code base and platforms. This mission is even more challenging when organized in an agile open-source community process with many contributors such as for the RIOT OS. Hardware in the Loop (HiL) testing and Continuous Integration (CI) are automatable approaches to verify functionality, prevent regressions, and improve the overall quality at development speed in large community projects. In this paper, we present PHiLIP (Primitive Hardware in the Loop Integration Product), an open-source external reference device together with tools that validate the system software while it controls hardware and interprets physical signals. Instead of focusing on a specific test setting, PHiLIP takes the approach of a tool-assisted agile HiL test process, designed for continuous evolution and deployment cycles. We explain its design, describe how it supports HiL tests, evaluate performance metrics, and report on practical experiences of employing PHiLIP in an automated CI test infrastructure. Our initial deployment comprises 22 unique platforms, each of which executes 98 peripheral tests every night. PHiLIP allows for easy extension of low-cost, adaptive testing infrastructures but serves testing techniques and tools to a much wider range of applications.

Constrained adaptive testing is reviewed as an instance of discrete maximization with the shadow-test approach delivering its solution. The approach may look counterintuitive in that it assumes sequential assembly of full test forms as its basic operation. But it always produces real-time solutions that are optimal and satisfy the set of specifications in effect for the test. Equally importantly, it can be used to run testing programs with different degrees of adaptation for the same set of specifications and/or as a tool to manage programs with simultaneous processes as adaptive item calibration, time management, and/or item-security monitoring.

Patient-reported outcome measures are tools for evaluating symptoms, magnitude of limitations, baseline health status, and outcomes from the patient’s perspective. Healthcare professional organizations and payers increasingly recommend PROMs for clinical care, but there lacks guidance regarding effective communication of PROMs with orthopedic surgery patients. This qualitative study aimed to identify (1) patient attitudes toward the use and communication of PROMs, and (2) what patients feel are the most relevant or important aspects of PROM results to discuss with their physicians. Participants were recruited from a multispeciality orthopedic clinic. Three PROMs: the EuroQol-5 Dimension, the Patient-Specific Functional Scale, and the Patient-Reported Outcome Measurement Information System Physical Function Computer Adaptive Test were shown and a semi-structured interview was conducted to elicit PROMs attitudes and preferences. Interviews were transcribed and inductive-deductively coded. Coded excerpts were aggregated to (1) identify major themes and (2) analyze how themes interacted. Three themes emerged: (1) Beliefs toward the purpose of PROMs, (2) PROMs as a reflection of self, and (3) PROMs to facilitate communication and guide healthcare decisions. These themes informed a framework outlining the patient perspective on communicating PROMs during clinical care. Patient attitudes toward the use and communication of PROMs start with the incorporation of patient beliefs, which can facilitate or act as a barrier to engagement. Patients should ideally believe that PROMs are an accurate reflection of personal health state before incorporation into care. Clinicians should endeavor to communicate the purpose of a chosen PROM in line with a patient’s unique needs and what they feel is most relevant to their own care. Aspects of PROMs results which may be helpful to address include providing context for what scores mean and how they are calculated, and using scores as a way to weigh risks and benefits of treatment and tracking progress over time. Future research can focus on the effect of communication strategies on patient outcomes and engagement in care.

In recent years, fixed forms and computerized adaptive testing (CAT) forms have coexisted in many testing programs and often used interchangeably on the premise that both testing formats meet the same test specifications. In conventional CAT, however, items are selected through computer algorithms to meet primarily statistical criteria, whereas fixed forms are often created focusing heavily on content, non-statistical, and practical requirements. Founded on the optimal test design framework, the shadow-test approach to CAT and its generalization allows for constructing fixed and adaptive test forms to satisfy the same objective and (possibly complex) set of constraints. This approach can render a variety of testing formats with different levels of adaptation and relative efficiency. This paper provides an overview of the optimal test assembly approach and the development of the TestDesign package in R. Several illustrations from real-world testing scenarios are also presented.

Adaptive test assembly entails an added layer of complexity when the test is designed with both discrete and set-based items. Among the challenges is how to interweave discrete and set-based items optimally within each adaptive test. The task is further complicated by the requirements for exposure control of both items and item sets, adjusting adaptivity while administering items within item sets, and maintaining optimality given additional complexity. The present study investigates practical solutions for addressing the challenges and proposes new item selection methods within the optimal test design framework.

Chronic spinal pain poses complex challenges for health care around the world and is in need of effective interventions. Pain neuroscience education (PNE) is a promising intervention hypothesized to improve pain and disability by changing individuals' beliefs, perceptions, and expectations about pain. Pain neuroscience education has shown promise in small, controlled trials when implemented in tightly controlled situations. Exploration of promising interventions through more pragmatic methodologies is a crucial but understudied step towards improving outcomes in routine clinical care. The purpose was to examine the impact of pragmatic PNE training on clinical outcomes in patients with chronic spine pain. The cluster-randomized clinical trial took place in 45 outpatient physical therapist (PT) clinics. Participants included 108 physical therapists (45 clinics and 16 clusters) and 319 patients. Clusters of PT clinics were randomly assigned to either receive training in PNE or no intervention and continue with usual care (UC). We found no significant differences between groups for our primary outcome at 12 weeks, Patient-Reported Outcomes Measurement Information System Physical Function computer adaptive test {mean difference = 1.05 (95% confidence interval [CI]: -0.73 to 2.83), P = 0.25}. The PNE group demonstrated significant greater improvements in pain self-efficacy at 12 and 2 weeks compared with no intervention (mean difference = 3.65 [95% CI: 0.00-7.29], P = 0.049 and = 3.08 [95% CI: 0.07 to -6.09], P = 0.045, respectively). However, a similar percentage of participants in both control (41.1%) and treatment (44.4%) groups reported having received the treatment per fidelity question (yes or no to pain discussed as a perceived threat) at 2 weeks. Pragmatic PT PNE training and delivery failed to produce significant functional changes in patients with chronic spinal pain but did produce significant improvement in pain self-efficacy over UC PT.

Objective To calibrate the Lower Extremity Functional Scale (LEFS) items into a regional lower extremity physical function (LEPF) item bank and assess reliability, validity, and efficiency of computerized adaptive test (CAT) and short form (SF) administration modes. Design Retrospective cohort. Setting Data were collected from patients treated in outpatient rehabilitation clinics for musculoskeletal impairments of the hip, knee, foot, and ankle that responded to all 20 LEFS items at intake. Participants Patients aged 14 years or older who started an episode of care during January 2016-October 2019 and identified the lower extremity region as the source of a primary musculoskeletal complaint. Total cohort included 78,186 patients (mean age, 53±19y, range, 14-89y). Interventions Not applicable. Main Outcome Measures Item response theory (IRT) model assumptions of unidimensionality, local item independence, item fit, and presence of differential item functioning (DIF) were studied. LEPF-CAT– and LEPF-SF–generated scores were evaluated. Results An 18-item solution was supported for its unidimensionality and fit to the IRT model, with reliability estimates >0.9 for all administration modes. No DIF impact on LEPF scores was identified. Scores discriminated between multiple patient groups in clinically logical ways and were highly responsive to change, with negligible floor or ceiling effects. CAT scores were generated using an average of 4.9 items (median, 4). Conclusions The LEPF scores were reliable, valid, and efficient for assessing perceived physical function of patients with musculoskeletal impairments of the hip, knee, foot, and ankle; thus, it was found suitable for research and routine clinical administration. These findings are limited to the type of patients included in this study, with further validation needed to assess their generalizability.

Artificial Intelligence is at the heart of modern society with computers now capable of making process decisions in many spheres of human activity. In education, there has been intensive growth in systems that make formal and informal learning an anytime, anywhere activity for billions of people through online open educational resources and massive online open courses. Moreover, new developments in Artificial Intelligence-related educational assessment are attracting increasing interest as means of improving assessment efficacy and validity, with much attention focusing on the analysis of the large volumes of process data being captured from digital assessment contexts. In evaluating the state of play of Artificial Intelligence in formative and summative educational assessment, this paper offers a critical perspective on the two core applications: automated essay scoring systems and computerized adaptive tests, along with the Big Data analysis approaches to machine learning that underpin them.

Multistage testing (MST), an adaptive test delivery mode that involves algorithmic selection of predefined item modules rather than individual items, offers a practical alternative to linear and fu...

Objective To link scores from 2 condition-specific measures for chronic low back pain (CLBP), the Oswestry Disability Index (ODI) and the Roland-Morris Disability Questionnaire (RMDQ), to Patient Reported Outcomes Measurement Information System (PROMIS) physical function, pain interference, and pain intensity scores. Design Ordinary least squares regression analyses of existing data to link the PROMIS scores with the ODI and RMDQ. Setting Not applicable. Participants Samples of adults with CLBP (N=2279) obtained from the Center for Excellence in Research for Complementary and Integrative Health (CERC) Study (n=1677), the Assessment of Chiropractic Treatment for Low Back Pain and Smoking Cessation in Military Active Duty Personnel (ACT) (n=384), and the pain subsample of the PROMIS 1 Wave 2 Pain and Depression study (PROMIS 1 W2) (n=218). Interventions Not applicable. Main Outcome Measures PROMIS physical function, pain interference, and pain intensity (CERC, ACT, and PROMIS 1 W2), ODI (CERC and PROMIS 1 W2), and RMDQ (ACT and PROMIS 1 W2). Results In predicting PROMIS scores, the ODI model R2 values ranged from 0.26-0.56 and the RMDQ model R2 values ranged from 0.13-0.50. ODI and RMDQ models were the least precise in predicting the PROMIS pain intensity score (R2 value range, 0.13-0.41) relative to the other PROMIS scores. Models with the 3 PROMIS scores as predictors yielded R2 values ranging from 0.64-0.68 and 0.46-0.58 for the ODI and RMDQ, respectively. Models using combined data from 2 studies (ie, PROMIS 1 W2 and ACT, or PROMIS 1 W2 and CERC) tended to be more precise than models using only a single study sample. Conclusions Model results reported here can be used to translate PROMIS physical function, pain interference, and pain intensity scores to and from the ODI and RMDQ. The empirical linkages can facilitate comparisons across CLBP interventions and broaden interpretation of study results.

This study evaluated the psychometric quality of a computerized adaptive testing (CAT) version of the general cognitive ability test (GCAT), using a simulation study protocol put forth by Han, K. T. (2018a). For the needs of the analysis, three different sets of items were generated, providing an item pool of 165 items. Before evaluating the efficiency of the GCAT, all items in the final item pool were linked (equated), following a sequential approach. Data were generated using a standard normal for 10,000 virtual individuals ( M = 0 and SD = 1). Using the measure’s 165-item bank, the ability value (θ) for each participant was estimated. maximum Fisher information (MFI) and maximum likelihood estimation with fences (MLEF) were used as item selection and score estimation methods, respectively. For item exposure control, the fade away method (FAM) was preferred. The termination criterion involved a minimum SE ≤ 0.33. The study revealed that the average number of items administered for 10,000 participants was 15. Moreover, the precision level in estimating the participant’s ability score was very high, as demonstrated by the CBIAS, CMAE, and CRMSE). It is concluded that the CAT version of the test is a promising alternative to administering the corresponding full-length measure since it reduces the number of administered items, prevents high rates of item exposure, and provides accurate scores with minimum measurement error.

The appeal of a shorter testing time makes a computer adaptive testing approach highly desirable for use in multiple assessment and learning contexts. However, for those who have been tasked with designing, configuring, and deploying adaptive tests for operational use at scale, preparing an adaptive test is anything but simple. The process often involves a complex interplay among psychometricians, content experts, and technologists who operate with different vocabularies and subject matter expertise. This paper presents the authors’ experience of developing smart platforms for designing, configuring, and deploying adaptive assessments. We present six design principles for the development of such systems. Through an example of an authoring system now used in production, we discuss smart feedback loops built into the system and how they support efficient iteration. We conclude that it is not possible to overstate the importance of process transition support when launching smart platforms, and that a thoughtful and integrated user experience that allows each of the people in the process to work in a context most accessible to them given their background and experience yields substantial business value.

Computer-adaptive tests use respondents' answers to previous questions to select the subsequent questions. They are gaining popularity for their increased measurement precision and decreased administration time compared to static questionnaires. The purpose of this study was to estimate the test-retest reliability and construct validity of the computer-adaptive test version of a participation measure, the Late Life Disability Instrument (LLDI-CAT) for people with COPD and to compare scores and administration time with those of the static LLDI. Among 76 older adults with COPD, scores on the LLDI-CAT were compared to scores on measures of related constructs, between groups based on symptom severity, prognosis and frailty phenotype, and to scores on the static LLDI. A subsample of 28 people completed the LLDI-CAT a second time within one week of the initial administration for test-retest reliability. The LLDI-CAT had very good test-retest reliability (ICC

Using classical test theory and item response theory, this study applied sequential procedures to a real operational item pool in a variable-length computerized adaptive testing (CAT) to detect items whose security may be compromised. Moreover, this study proposed a hybrid threshold approach to improve the detection power of the sequential procedure while controlling the Type I error rate. The hybrid threshold approach uses a local threshold for each item in an early stage of the CAT administration, and then it uses the global threshold in the decision-making stage. Applying various simulation factors, a series of simulation studies examined which factors contribute significantly to the power rate and lag time of the procedure. In addition to the simulation study, a case study investigated whether the procedures are applicable to the real item pool administered in CAT and can identify potentially compromised items in the pool. This research found that the increment of probability of a correct answer ( p-increment) was the simulation factor most important to the sequential procedures’ ability to detect compromised items. This study also found that the local threshold approach improved power rates and shortened lag times when the p-increment was small. The findings of this study could help practitioners implement the sequential procedures using the hybrid threshold approach in real-time CAT administration.

We present algorithms for testing if a (0,1)-matrix M has Boolean/binary rank at most d, or is 𝜖-far from having Boolean/binary rank at most d (i.e., at least an 𝜖-fraction of the entries in M must be modified so that it has rank at most d). For the Boolean rank we present a non-adaptive testing algorithm whose query complexity is $\tilde {O}\left (d^{4}/ \epsilon ^{6}\right )$ . For the binary rank we present a non-adaptive testing algorithm whose query complexity is O(22d/𝜖2), and an adaptive testing algorithm whose query complexity is O(22d/𝜖). All algorithms are 1-sided error algorithms that always accept M if it has Boolean/binary rank at most d, and reject with probability at least 2/3 if M is 𝜖-far from having Boolean/binary rank at most d.

Design Mixed methods cohort study. Objectives To develop and assess psychometric properties of the pediatric measure of participation (PMoP) short forms (SF) version 2.0. Setting Secondary analyses of data collected from 381 children with spinal cord injury (SCI) of at least 3-month duration living in the community, and 322 parents of children with SCI at three pediatric orthopedic hospitals in the United States. Methods Mixed methods iterative process to customize SF based on, highly relevant items, age and school analysis of item distributions; ceiling and floor effects; internal consistency and group-level reliability; correlation of SF scores with scores derived from the total item bank; and assessment of the degree to which item difficulty matched the abilities of children in the sample. Results PMoP SF V2.0 mean T scores ranged from 47.59 to 51.23. Overall, mean scores were somewhat higher for older children and parent respondents. Group-level reliability values ranged from 0.66 to 0.79; Cronbach's alpha values ranged from 0.79 to 0.90; ICC values ranged from 0.89 to 0.95. Pearson Correlations ranged from 0.80 to 0.95, showing good to strong correlation between scores from the SFs and total item bank for each domain. Test information function demonstrated that score estimates will be less precise at higher ends of the scale. Conclusions PMoP SFs V2.0 contain items relevant to participation among children with SCI, and are tailored for four age groups and school status. They are recommended for use when computer adaptive testing (CAT) is not possible.

Teaching and learning have gone online in response to the pandemic, which reveals the need for accurately tailored educational assessments to ascertain the extent to which learning outcomes or objectives are achieved. Computer Adaptive Testing (CAT) is a technology-driven form of assessment that tailors items to a candidate's ability level with empirically proven benefits over the fixed-form computer based test. A systematic review was employed which shows that item bank is a key requirement for CAT and the items must through a rigorous item development process to ensure and maintain quality  in terms of content, criterion constructs and internal consistency, determining the psychometric validation of behavioural measures while leveraging on variances of Item Response Theory (IRT). Following the item development stage is the need to compile validated items into administrable forms using advanced computer software for automatic test assembly and administration, such as FastTest which allows specifying empirically tried algorithms for CAT from start to termination of the test. This helps to ensure that assessment properly leverages the advantages that CAT holds. Furthermore, the review revealed that CAT has been widely applied with large-scale testing in various fields by educational, health and psychological professionals utilising different IRT models; however only in developed countries. This brings to bear the need for adoption in other parts of the world, for improvements in educational assessments. The interjections of 4IR with AI considering emerging technology aids the CAT algorithm for achieving expert and knowledge-based systems, being a requirement for survival in today’s world. https://doi.org/10.26803/ijlter.20.5.1

Background. Making comprehensive management of myasthenia gravis (MG) is a challenge in clinical practice due to heterogeneity and multiple comorbidities among patients. Aim. To develop an end-to-end instrument for individualized assessment of MG in the perspective of Chinese medicine (TCM) with the application of multidisciplinary quantification approaches. Methods. A self-administrated questionnaire was developed integrating typical symptoms of MG and spleen-kidney deficiency syndrome on basis of the conceptual framework of TCM. With data collected in a multicenter cross-sectional study, confirmatory factor analysis together with multidimensional item response theory (MIRT) was used for evaluating the psychometric property of the questionnaire. A computerized adaptive test was developed based on the MIRT model, and scores of syndrome factors were calculated in simulation. A logistics regression model was also estimated for evaluating the consistency between the quantitative result and the clinical diagnosis of syndrome from clinical practitioners. Result. With 337 patients enrolled and assessed, the 14-item questionnaire was evaluated to be with adequate validity and reliability (Cronbach’s alpha indices = 0.87, AIC = 195.827, BIC = 348.631, CFI = 0.921, RMR = 0.006, GFI = 0.954, RMSEA = 0.048, and χ2/df = 1.782). With adequate factor loadings of symptoms on related syndrome factor, the instrument was evaluated with preliminary interpretation and was suitable for evaluating patients with moderate severity of the spleen and kidney deficiency syndrome. Conclusion. Setting typical symptoms of MG together with systemic discomforts in a computerized adaptive test on the basis of MIRT, this study proposed an innovative research paradigm for quantifying individual condition in the perspective of TCM with application of interdisciplinary approaches.

Large-scale assessments often use a computer adaptive test (CAT) for selection of items and for scoring respondents. Such tests often assume a parametric form for the relationship between item responses and the underlying construct. Although semi- and nonparametric response functions could be used, there is scant research on their performance in a CAT. In this work, we compare parametric response functions versus those estimated using kernel smoothing and a logistic function of a monotonic polynomial. Monotonic polynomial items can be used with traditional CAT item selection algorithms that use analytical derivatives. We compared these approaches in CAT simulations with a variety of item selection algorithms. Our simulations also varied the features of the calibration and item pool: sample size, the presence of missing data, and the percentage of nonstandard items. In general, the results support the use of semi- and nonparametric item response functions in a CAT.