Search

Multiple measures administered in repeated waves within a nonequivalent dependent variables quasi-experimental design were used to test the effects of a reform-oriented instructional method called Enhanced Anchored Instruction (EAI) on the math achievement of 128 middle school students, including students with learning disabilities (LD). EAI problems are presented in multimedia and hands-on formats, a potential benefit for students with low skills in both reading and math. Overall, students of all ability levels benefited from EAI with effect sizes (eta[squared]) ranging from 0.53 to 0.59. Results revealed that although students with LD scored lower on pretests, their learning trajectories matched those of students without LD. A maintenance test administered several weeks after instruction showed that students with LD retained what they had learned. Implications for instruction and suggestions for future research are provided.

The sense that replicability is an important aspect of empirical science led Killeen (2005a) to define "p[subscript rep]," the probability that a replication will result in an outcome in the same direction as that found in a current experiment. Since then, several authors have praised and criticized 'p[subscript rep]," culminating in the 3 articles in the current issue of "Psychological Methods." In this article, Killeen's "p[subscript rep]" is reviewed, and the contributions of the current articles are summarized and discussed. An examination of the role of a measure of theoretical support such as "p[subscript rep]" in the acquisition of knowledge leads me to concur with Senn (2002) that "p[subscript rep]" is of little epistemological value.

This study estimates pretreatment-posttreatment effect size benchmarks for the treatment of major depression in adults that may be useful in evaluating psychotherapy effectiveness in clinical practice. Treatment efficacy benchmarks for major depression were derived for 3 different types of outcome measures: the Hamilton Rating Scale for Depression (M. A. Hamilton, 1960, 1967), the Beck Depression Inventory (A. T. Beck, 1978; A. T. Beck & R. A. Steer, 1987), and an aggregation of low reactivity-low specificity measures. These benchmarks were further refined for 3 conditions: treatment completers, intent-to-treat samples, and natural history (wait-list) conditions. The study confirmed significant effects of outcome measure reactivity and specificity on the pretreatment-posttreatment effect sizes. The authors provide practical guidance in using these benchmarks to assess treatment effectiveness in clinical settings.

Quantitative and qualitative inquiry are sometimes portrayed as distinct and incompatible paradigms for research in education. Approaches to combining qualitative and quantitative research typically "integrate" the two methods by letting them co-exist independently within a single research study. Here we describe intra-sample statistical analysis (ISSA) as a general technique for using quantitative tools to support qualitative inquiry so as to simultaneously provide warrants from qualitative and quantitative traditions. In certain circumstances ISSA makes it possible to relax the requirement that individual participants be treated as the unit of analysis in statistical models, and thus provides justification for coding qualitative observations and drawing statistically based conclusions about observations in a qualitative context. We developed ISSA and describe it here both because it can be used as a tool for qualitative research, and because it illuminates the relationship between method and interpretation in the research traditions that it bridges. In this article, we (a) summarize key features of qualitative and quantitative research relevant to ISSA; (b) describe ISSA as an analytical technique; (c) discuss the quantitative and qualitative justification for ISSA and the nature of the conclusions that can be drawn based on it; and (d) explore the more general implications of ISSA for qualitative and quantitative inquiry.

It is well-known that for normally distributed errors parametric tests are optimal statistically, but perhaps less well-known is that when normality does not hold, nonparametric tests frequently possess greater statistical power than parametric tests, while controlling Type I error rate. However, the use of nonparametric procedures has been limited by the absence of easily performed tests for complex experimental designs and analyses and by limited information about their statistical behavior for realistic conditions. A Monte Carlo study of tests of predictor subsets in multiple regression analysis indicates that various nonparametric tests show greater power than the F test for skewed and heavy-tailed data. These nonparametric tests can be computed with available software.

Investigated the effect of item parameter drift on ability estimates under item response theory, simulating item response data for two testing occasions for the two-parameter logistic model under several crossed conditions. Results show that item parameter drift under the simulated conditions had a small effect on ability estimates. (SLD)

Developed a family wise approach for evaluating the significance of copying indices designed to hold the Type I error rate constant for each examinee. Examined the Type I error rate and power of two indices under a variety of copying situations. Results indicate the superiority of a family wise definition of Type I error rate over a pair-wise definition. Discusses conditions under which Type I error control is lost. (SLD)

Revises an earlier experiment that revealed age-related increases in sensitivity to rhythmic information when judging the difference between a theme and selected pitch and rhythm variations. Tested the possibility that the developmental differences found earlier were due to characteristics of the test melody. Results supported the earlier study. (MJP)

Treatment data from a university counseling center (UCC) that utilized the Outcome Questionnaire--45.2 (OQ-45; M. J. Lambert et al., 2004), a self-report general clinical symptom measure, was compared against treatment efficacy benchmarks from clinical trials of adult major depression that utilized similar measures. Statistical analyses suggested that the treatment effect size estimate obtained at this counseling center with clients whose level of psychological distress was above the OQ-45 clinical cutoff score was similar to treatment efficacy observed in clinical trials. Analyses on OQ-45 items suggested that clients elevated on 3 items indicating problematic substance use resulted in poorer treatment outcomes. In addition, clients who reported their relational status as separated or divorced had poorer outcomes than did those who reported being partnered or married, and clients reporting intimacy issues resulted in greater numbers of sessions. Although differential treatment effect due to training level was found where interns and other trainees had better pre-post outcome than did staff, interpretation of this result requires great caution because clients perceived to have complicated issues are actively reassigned to staff. More effectiveness investigations at UCCs are warranted. Keywords: university counseling centers, benchmarking, treatment effectiveness, therapist effect, clinical assessment

This preliminary study evaluated the effectiveness of psychotherapy treatment for adult clinical depression provided in a natural setting by benchmarking the clinical outcomes in a managed care environment against effect size estimates observed in published clinical trials. Overall results suggest that effect size estimates of effectiveness in a managed care context were comparable to effect size estimates of efficacy observed in clinical trials. Relative to the 1-tailed 95th-percentile critical effect size estimates, effectiveness of treatment provided in this setting was observed to be between 80% (patients with comorbidity and without antidepressants) and 112% (patients without comorbidity concurrently on antidepressants) as compared to the benchmarks. Because the nature of the treatments delivered in the managed care environment were unknown, it was not possible to make conclusions about treatments. However, while replications are warranted, concerns that psychotherapy delivered in a naturalistic setting is inferior to treatments delivered in clinical trials appear unjustified. Keywords: benchmarking, effectiveness, managed care, clinical trials, depression

This study estimates pretreatment--posttreatment effect size benchmarks for the treatment of major depression in adults that may be useful in evaluating psychotherapy effectiveness in clinical practice. Treatment efficacy benchmarks for major depression were derived for 3 different types of outcome measures: the Hamilton Rating Scale for Depression (M. A. Hamilton, 1960, 1967), the Beck Depression Inventory (A. T. Beck, 1978; A. T. Beck & R. A. Steer, 1987), and an aggregation of low reactivity--low specificity measures. These benchmarks were further refined for 3 conditions: treatment completers, intent-to-treat samples, and natural history (wait-list) conditions. The study confirmed significant effects of outcome measure reactivity and specificity on the pretreatment--posttreatment effect sizes. The authors provide practical guidance in using these benchmarks to assess treatment effectiveness in clinical settings. Keywords: effectiveness, psychotherapy, outcome, benchmarking, depression

Individual looks at television vary enormously in length, and this has previously indicated differences in ongoing cognitive processes. Furthermore, the relative frequency of looks of different lengths may indicate styles of attention to television. This article compares visual attention of 152 subjects across a variety of genres and examines differences located by situation, person, and media perception variables. Attention style was not consistent for individuals but varied for different types of programming and between-program breaks. Situational, person, and media belief variables did not predict the proportions of four types of looks. They did, however, predict differences in the proportion of moderately short (orienting) looks across types of content, but not the previously more important very short (monitoring) or moderately long (engaged) looks. The overall importance of type of content in these results suggests that further research should examine within-program differences in message construction and assess attentional style as patterns or sequences of looks.

Looks at television that cross content boundaries (both between and within programming) provide an opportunity to examine the causes of attentional inertia--that looks at television become very much more stable after the first few seconds. Previous research left unresolved whether this inertia is due to expectations or biologic processes (strategic vs. nonstrategic processes), and this study allows direct comparisons. The strength of the inertial relationship varied considerably for different kinds of program boundaries, and also for within-program boundaries, with the latter varying as well by the genre in which they were contained. Taken together, the results provided no evidence for nonstrategic, biological processes causing attentional inertia. Instead, several genre-specific explanations based on expectations and cognitive demands are proposed.

Administering a diagnostic test for detecting mental disorders using interactive voice response (IVR) technology appears to be effective. IVR technology involves a patient dialing an 800 telephone number and using a touch-tone telephone to answer questions. The Primary Care Evaluation of Mental Disorders (PRIME-MD) was administered over the telephone by a physician and by IVR to 200 patients. A subset of patients were also interviewed in person. About 60% of the patients were diagnosed with a psychiatric disorder by both techniques. However, the prevalence of alcohol abuse was twice as high when using IVR., Context. -- Common mental disorders are often not identified in primary care settings. Objective. -- To evaluate the validity and clinical utility of a telephone-assisted computer-administered version of Primary Care Evaluation of Mental Disorders (PRIME-MD), a brief questionnaire and interview instrument designed to identify psychiatric disorders in primary care patients. Design. -- Comparison of diagnoses obtained by computer over the telephone using interactive voice response (IVR) technology vs those obtained by a trained clinician over the telephone using the Structured Clinical Interview for DSM-IV [Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition] Diagnosis (SCID). A subsample also received the clinician-administered version of PRIME-MD. Patients. -- Outpatients (N=200) from 4 primary care clinics, an eating disorders clinic, an alcohol treatment facility, psychiatric outpatients, and community controls. Setting. -- Interviews conducted by telephone, except for face-to-face administration of PRIME-MD, which was conducted at either the primary care clinic or a research clinic. Measurements and Main Results. -- Prevalence rates for any psychiatric disorder were similar between diagnoses made by the computer and those made by a mental health professional using the SCID (60.0% vs 58.5%). Prevalence rates for individual diagnoses were generally similar across versions. However, primary care patients reported twice as much alcohol abuse on the computer (15.0%) as on either the SCID (7.5%) or the clinician-administered PRIME-MD (7.5%). Using the SCID as the criterion, both the computer- and clinician-administered versions of PRIME-MD demonstrated high and roughly equivalent levels of sensitivity and specificity. Overall agreement ([Kappa]) for any diagnosis was 0.67 for the computer-administered PRIME-MD and 0.70 for the clinician-administered PRIME-MD. Conclusions. -- the computer-administered PRIME-MD appears to be a valid instrument for assessing psychopathology in primary care patients. Interactive voice response technology allows for increased availability, and provides primary care physicians with information that will increase the quality of patient care without additional physician time and at minimal expense.