Search

When higher education students are assessed multiple times, teachers need to consider how these assessments can be combined into a single pass or fail decision. A common question that arises is whether students should be allowed to take a resit. Previous research has found little to no clear learning benefits of resits and therefore suggested they might not be advantangeous as they are costly for both students and institutions. However, we conducted a simulation study that shows such a conclusion to be presumptuous. Absence of evidence is not evidence of absence; our results illustrate that if a resit effect were to exist, the analysis used in these studies (i.e. regression discontinuity analysis; RDA) lacked the power to detect such an effect. Power of RDA was only sufficient under extremely implausible conditions (i.e. large sample, large effect size, high correlation between examinations). To adequately compare the effect of assessment policies, researchers are recommended to use other methods than RDA.

Background Self-help therapies are often effective in reducing mental health problems. We developed a new Web-based self-help intervention based on problem-solving therapy, which may be used for people with different types of comorbid problems: depression, anxiety, and work-related stress. Objective The aim was to study whether a Web-based self-help intervention is effective in reducing depression, anxiety, and work-related stress (burnout). Methods A total of 213 participants were recruited through mass media and randomized to the intervention (n = 107) or a waiting list control group (n = 106). The Web-based course took 4 weeks. Every week an automated email was sent to the participants to explain the contents and exercises for the coming week. In addition, participants were supported by trained psychology students who offered feedback by email on the completed exercises. The core element of the intervention is a procedure in which the participants learn to approach solvable problems in a structured way. At pre-test and post-test, we measured the following primary outcomes: depression (CES-D and MDI), anxiety (SCL-A and HADS), and work-related stress (MBI). Quality of life (EQ-5D) was measured as a secondary outcome. Intention-to-treat analyses were performed. Results Of the 213 participants, 177 (83.1%) completed the baseline and follow-up questionnaires; missing data were statistically imputed. Of all 107 participants in the intervention group, 9% (n = 10) dropped out before the course started and 55% (n = 59) completed the whole course. Among all participants, the intervention was effective in reducing symptoms of depression (CES-D: Cohen’s d = 0.50, 95% confidence interval (CI) 0.22-0.79; MDI: d = 0.33, 95% CI 0.03-0.63) and anxiety (SCL-A: d = 0.42, 95% CI 0.14-0.70; HADS: d = 0.33, 95% CI 0.04-0.61) as well as in enhancing quality of life (d = 0.31, 95% CI 0.03-0.60). Moreover, a higher percentage of patients in the intervention group experienced a significant improvement in symptoms (CES-D: odds ratio [OR] = 3.5, 95% CI 1.9-6.7; MDI: OR = 3.7, 95% CI 1.4-10.0; SCL-A: OR = 2.1, 95% CI 1.0-4.6; HADS: OR = 3.1, 95% CI 1.6-6.0). Patients in the intervention group also recovered more often (MDI: OR = 2.2; SCL-A: OR = 2.0; HADS < 8), although these results were not statistically significant. The course was less effective for work-related stress, but participants in the intervention group recovered more often from burnout than those in the control group (OR = 4.0, 95% CI 1.2-13.5). ConclusionsWe demonstrated statistically and clinically significant effects on symptoms of depression and anxiety. These effects were even more pronounced among participants with more severe baseline problems and for participants who fully completed the course. The effects on work-related stress and quality of life were less clear. To our knowledge, this is the first trial of a Web-based, problem-solving intervention for people with different types of (comorbid) emotional problems. The results are promising, especially for symptoms of depression and anxiety. Further research is needed to enhance the effectiveness for work-related stress. Trial RegistrationInternational Standard Randomized Controlled Trial Number (ISRCTN) 14881571

Purpose: We reevaluated the psychometric properties of the Amsterdam Inventory for Auditory Disability and Handicap (AIADH; Kramer, Kapteyn, Festen, & Tobi, 1995) using item response theory. Item response theory describes item functioning along an ability continuum. Method: Cross-sectional data from 2,352 adults with and without hearing impairment, ages 18-70 years, were analyzed. They completed the AIADH in the web-based prospective cohort study "Netherlands Longitudinal Study on Hearing." A graded response model was fitted to the AIADH data. Category response curves, item information curves, and the standard error as a function of self-reported hearing ability were plotted. Results: The graded response model showed a good fit. Item information curves were most reliable for adults who reported having hearing disability and less reliable for adults with normal hearing. The standard error plot showed that self-reported hearing ability is most reliably measured for adults reporting mild up to moderate hearing disability. Conclusions: This is one of the few item response theory studies on audiological self-reports. All AIADH items could be hierarchically placed on the self-reported hearing ability continuum, meaning they measure the same construct. This provides a promising basis for developing a clinically useful computerized adaptive test, where item selection adapts to the hearing ability of individuals, resulting in efficient assessment of hearing disability.

The Center for Epidemiologic Studies-Depression (CES-D) scale is a well-known self-report instrument that is used to measure depressive symptomatology. Respondents who take the full-length version of the CES-D are administered a total of 20 items. This article investigates the use of curtailment and stochastic curtailment (SC), two sequential analysis methods that have recently been proposed for health questionnaires, to reduce the respondent burden associated with taking the CES-D. A post hoc simulation based on 1,392 adolescents' responses to the CES-D was used to compare these methods with a previously proposed computerized adaptive testing (CAT) approach. Curtailment lowered average test lengths by as much as 22% while always matching the classification decision of the full-length CES-D. SC and CAT achieved further reductions in average test length, with SC's classifications exhibiting more concordance with the full-length CES-D than do CAT's. Advantages and disadvantages of each method are discussed. (Contains 5 tables and 2 figures.)

This paper serves as an illustration of the usefulness of structurally incomplete designs as an approach to reduce the length of educational questionnaires. In structurally incomplete test designs, respondents only fill out a subset of the total item set, while all items are still provided to the whole sample. The scores on the unadministered items are subsequently dealt with by using methods for the estimation of missing data. Two structurally incomplete test designs--one recording two thirds, and the other recording a half of the potentially complete data--were applied to the complete item scores on 8 educational psychology scales. The incomplete item scores were estimated with missing data method Data Augmentation. Complete and estimated test data were compared at the estimates of total scores, reliability, and predictive validity of an external criterion. The reconstructed data yielded estimates that were very close to the values in the complete data. As expected the statistical uncertainty was higher in the design that recorded fewer item scores. It was concluded that the procedure of applying incomplete test designs and subsequently dealing with the missing values is very fruitful for reducing questionnaire length.

The role of deliberate practice in predicting performance of adolescent chess players who were in or had dropped out from the national chess training in Netherlands is examined. Findings show that deliberate practice which includes serious chess play and serious chess study alone, has a strong impact on chess performance. The effect of chess players' gender on their performance is also evaluated.

Abstract Background In Routine Outcome Monitoring (ROM) there is a high demand for short assessments. Computerized Adaptive Testing (CAT) is a promising method for efficient assessment. In this article, the efficiency of a CAT version of the Mood and Anxiety Symptom Questionnaire, - Anhedonic Depression scale (MASQ-AD) for use in ROM was scrutinized in a simulation study. Methods The responses of a large sample of patients (N = 3,597) obtained through ROM were used. The psychometric evaluation showed that the items met the requirements for CAT. In the simulations, CATs with several measurement precision requirements were run on the item responses as if they had been collected adaptively. Results CATs employing only a small number of items gave results which, both in terms of depression measurement and criterion validity, were only marginally different from the results of a full MASQ-AD assessment. Conclusions It was concluded that CAT improved the efficiency of the MASQ-AD questionnaire very much. The strengths and limitations of the application of CAT in ROM are discussed.

Abstract Background Although previous studies using non- elderly groups have assessed the factorial invariance of the Center for Epidemiological Studies Depression Scale (CES-D) across different groups with the same social-cultural backgrounds, few studies have tested the factorial invariance of the CES-D across two elderly groups from countries with different social cultures. The purposes of this study were to examine the factorial structure of the CES-D, and test its measurement invariance across two different national elderly populations. Methods A total of 6806 elderly adults from China (n = 4903) and the Netherlands (n = 1903) were included in the final sample. The CES-D was assessed in both samples. Three strategies were used in the data analysis procedure. First, a confirmatory factor analysis (CFA) was carried out to determine the factor structures of the CES-D that best fitted the two samples. Second, the best fitting model was incorporated into a multi-group CFA model to test measurement invariance of the CES-D across the two population groups. Third, latent mean differences between the two groups were tested. Results The results of confirmatory factor analysis (CFA) showed: 1) in both samples, Radloff's four-factor model resulted in a significantly better fit and the four dimensions (somatic complaints, depressed affect, positive affect, and interpersonal problems) of the CES-D seem to be the most informative in assessing depressive symptoms compared to the single-, three-, and the second-order factor models; and 2) the factorial structure was invariant across the populations under study. However, only partial scalar and uniqueness invariance of the CES-D items was supported. Latent means in the partial invariant model were lower for the Dutch sample, compared to the Chinese sample. Conclusions Our findings provide evidence of a valid factorial structure of the CES-D that could be applied to elderly populations from both China and the Netherlands, producing a meaningful comparison of total scores between the two elderly groups. However, for some specific factors and items, caution is required when comparing the depressive symptoms between Chinese and Dutch elderly groups.

Abstract Background The Youden index, the sum of sensitivity and specificity minus one, is an index used for setting optimal thresholds on medical tests. Discussion When using this index, one implicitly uses decision theory with a ratio of misclassification costs which is equal to one minus the prevalence proportion of the disease. It is doubtful whether this cost ratio truly represents the decision maker's preferences. Moreover, in populations with a different prevalence, a selected threshold is optimal with reference to a different cost ratio. Summary The Youden index is not a truly optimal decision rule for setting thresholds because its cost ratio varies with prevalence. Researchers should look into their cost ratio and employ it in a decision theoretic framework to obtain genuinely optimal thresholds.

Abstract Background Panic disorder (PD) is a serious DSM-IV axis I disorder affecting up to 3% of the adult population each year. It is associated with a large burden of disease and extensive economic costs. This study aims to examine the effects and feasibility of the 'Don't Panic' course, a preventive cognitive behavioural intervention in sub-threshold and mild PD. It also compares the effectiveness of two modifications of the course (8 vs. 12 sessions). Methods The method used was a quasi-experimental two-group pre-post design with a baseline measurement (T0) and two follow-up measurements. Follow-ups were at the end of the intervention (T1) and six months later (T2). Primary outcome measure was the Panic Disorder Severity Scale-Self Report. A total of 114 participants suffering from panic attacks (mean age 42 years; 78% female) entered the study. Results The course participants showed a significant effect on the outcome measures at follow-up. Large effect sizes were found on panic symptoms, on symptoms of agoraphobia and on mental health-related quality of life at T1 and T2. Overall, the course leaders and the participants evaluated the course positively. There were no significant differences in outcome measures between the short and the long version of the course. Conclusion The study suggests that people with sub-threshold PD and mild PD could benefit from this preventive intervention and that the intervention might be feasible. Furthermore, the short version could be as effective as the long version.

Abstract Background The Major Depression Inventory (MDI) is a new, brief, self-report measure for depression based on the DSM-system, which allows clinicians to assess the presence of a depressive disorder according to the DSM-IV, but also to assess the severity of the depressive symptoms. Methods We examined the sensitivity, specificity, and psychometric qualities of the MDI in a consecutive sample of 258 psychiatric outpatients. Of these patients, 120 had a mood disorder (70 major depression, 49 dysthymia). A total of 139 subjects had a comorbid axis-I diagnosis, and 91 subjects had a comorbid personality disorder. Results Crohnbach's alpha of the MDI was a satisfactory 0.89, and the correlation between the MDI and the depression subscale of the SCL-90 was 0.79 (p < .001). Subjects with major depressive disorder (MDD) had a significantly higher MDI score than subjects with anxiety disorders (but no MDD), dysthymias, bipolar, psychotic, other neurotic disorders, and subjects with relational problems. In ROC analysis we found that the area under the curve was 0.68 for the MDI. A good cut-off point for the MDI seems to be 26, with a sensitivity of 0.66, and a specificity of 0.63. The indication of the presence of MDD based on the MDI had a moderate agreement with the diagnosis made by a psychiatrist (kappa: 0.26). Conclusion The MDI is an attractive, brief depression inventory, which seems to be a reliable tool for assessing depression in psychiatric outpatients.

We investigated longitudinal measurement invariance in the Dutch–Flemish PROMIS adult v1.0 item banks for Depression and Anxiety using two clinical samples with mood and anxiety disorders (n = 640 and n = 528, respectively). Factor analysis was used to evaluate whether the item banks were sufficiently unidimensional at two test-occasions and whether the measured constructs remained the same over time. The results indicated that the item banks were sufficiently unidimensional, but the thresholds and residual variances of the constructs changed over time. However, using tentative rules of thumb, these invariance violations did not substantially affect the endorsement of a specific response category of a specific item at a specific test-occasion. Furthermore, the impact on the mean latent change scores of the item banks remained below the proposed cutoff value for substantial bias. These findings suggest that the invariance violations lacked practical significance for test-users, meaning that the item banks provide sufficiently invariant latent factor scores for use in clinical practice. [ABSTRACT FROM AUTHOR]

In psychological assessment of children, it is pivotal to establish from what age on self-reports can complement or replace informant reports. We introduce a psychometric approach to estimate the minimum age for a child to produce self-report data that is of similar quality as informant data. The approach makes use of statistical validity indicators such as person-fit and long-string indices, and can be readily applied to data commonly collected in psychometric studies of child measures. We evaluate and illustrate the approach, using self-report and informant-report data of the PedsQL, a pediatric health-related quality of life measure, from 651 child–mother pairs. To evaluate the approach, we tested various hypotheses about the validity of the self-report data, using the G n p person-fit index as the validity indicator and the mother informant-data as a benchmark for validity. Results showed that G n p discriminated between self-reports of younger and older children, between self-reports of children that completed the PedsQL alone or with a parent, and between self-reports and informant reports. We conclude that the validity-index approach has good potential for future applications. Future research should further evaluate the approach for different types of questionnaires (e.g., personality inventories) and using different validity indices (e.g., response-bias indices). [ABSTRACT FROM AUTHOR]