Search

Author(s): Aczel, Balazs; Szaszi, Barnabas; Sarafoglou, Alexandra; Kekecs, Zoltan; Kucharský, Simon; Benjamin, Daniel; Chambers, Christopher D; Fisher, Agneta; Gelman, Andrew; Gernsbacher, Morton A; Ioannidis, John P; Johnson, Eric; Jonas, Kai; Kousta, Stavroula; Lilienfeld, Scott O; Lindsay, D Stephen; Morey, Candice C; Munafo, Marcus; Newell, Benjamin R; Pashler, Harold; Shanks, David R; Simons, Daniel J; Wicherts, Jelte M; Albarracin, Dolores; Anderson, Nicole D; Antonakis, John; Arkes, Hal R; Back, Mitja D; Banks, George C; Beevers, Christopher; Bennett, Andrew A; Bleidorn, Wiebke; Boyer, Ty W; Cacciari, Cristina; Carter, Alice S; Cesario, Joseph; Clifton, Charles; Conroy, Ronan M; Cortese, Mike; Cosci, Fiammetta; Cowan, Nelson; Crawford, Jarret; Crone, Eveline A; Curtin, John; Engle, Randall; Farrell, Simon; Fearon, Pasco; Fichman, Mark; Frankenhuis, Willem; Freund, Alexandra M; Gaskell, M Gareth; Giner-Sorolla, Roger; Green, Don P; Greene, Robert L; Harlow, Lisa L; de la Guardia, Fernando Hoces; Isaacowitz, Derek; Kolodner, Janet; Lieberman, Debra; Logan, Gordon D; Mendes, Wendy B; Moersdorf, Lea; Nyhan, Brendan; Pollack, Jeffrey; Sullivan, Christopher; Vazire, Simine; Wagenmakers, Eric-Jan | Abstract: An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Report cards classifying physicians into performance tiers are central to health care quality improvement initiatives. Misclassification is a concern since physicians often have small patient panels for standard performance measures. Given that report cards are used for different purposes by different stakeholders, we specify loss functions and evaluate the potential cost of misclassification for physician report card designs. Monte Carlo simulation to explore misclassification risk and cost for four illustrative physician report card designs and three loss functions representing overall misclassification, patient, and pay-for-performance program perspectives. True physician performance is simulated under a beta-binomial model with parameters yielding simulated true scores resembling previously reported estimates. Misclassification risk varied across report card designs. Overall misclassification risk increased with the number of performance tiers for our simulated scenarios. However, the relationship between misclassification cost and number of tiers was inconsistent across the loss functions. The report card with the lowest misclassification cost varied across stakeholders. Within stakeholder, the costs of a two-tier report card with a high or low hurdle (25th and 75th percentile, respectively) varied. Loss functions and report card designs are illustrative and not intended to exhaustively catalog all possibilities. Little guidance exists on misclassification costs from the patient perspective. Misclassification cost depends on how performance information will be used and by whom. Selecting the lowest-cost design for a given stakeholder could maximize the usefulness of physician performance data. Misclassification cost could guide report card design, improving the usefulness of a report card for one stakeholder without disadvantaging others .

This paper presents the results of a three-year evaluation performed on the first cohorts of Chile Solidario, the most important anti-poverty program in Chile. The paper presents a description of the program, emphasizing the mechanism by which people were admitted into the program. We then propose evaluation strategies and discuss their validity. The final evaluation is conducted using a Matching estimator, and we discuss the principles surrounding the potential for this to be a valid evaluation method. The initial results using the Chile Solidario Panel suggest that the program had positive effects on psychosocial welfare and on take-up of subsidies and social programs. However, it is not possible to obtain reliable results due to data shortcomings, particularly the lack of baseline data. In order to solve the problem, we generated a database using six years of administrative data, including around 1,000,000 family records per year. A method for overcoming the treatment substitution problem is discussed and implemented. Results are much more robust than those of the Chile Solidario Panel and show small, but clearly positive effects for several variables, especially the number of workers in the family, the percentage of workers in the family and the employment of the head of the family