Search

This paper studies the determinants of college major choice using a unique "information" experiment embedded in a survey. We first ask respondents their "self" beliefs--beliefs about their own expected earnings and other major-specific outcomes conditional on various majors, their "population" beliefs--beliefs about the population distribution of these characteristics, as well as their subjective beliefs that they will graduate with each major. After eliciting these baseline beliefs, we provide students with information on the true population distribution of these characteristics, and observe how this new information causes respondents to update their beliefs. Our experimental design creates unique panel data. We first show that respondents make substantial errors in population beliefs, and logically revise their self beliefs in response to the information. Subjective beliefs about future major choice are positively and strongly associated with beliefs about self earnings, ability, and spouse's earnings. However, cross-sectional estimates are severely biased upwards because of the positive correlation of tastes with earnings and ability. The experimental variation in beliefs allows us to identify a rich model of college major choice, with which we estimate the relative importance of earnings and earnings uncertainty on the choice of college major versus other factors such as ability to complete coursework, spouse's characteristics, and tastes for majors. While earnings are a significant determinant of major choice, tastes are the dominant factor in the choice of field of study. We also investigate why males and females choose different college majors. The following are appended: (1) Figures A1 and A2; and Tables A1, A2, and A3; (2) Information on Survey Design and Information Treatments; (3) and Estimation Details.

This chapter examines the core analytic elements of randomized experiments for social research. Its goal is to provide a compact discussion for faculty members, graduate students, and applied researchers of the design and analysis of randomized experiments for measuring the impacts of social or educational interventions. Design issues considered include choosing the size of a study sample and its allocation to experimental groups, using covariates or blocking to improve the precision of impact estimates, and randomizing intact groups instead of individuals. Analysis issues considered include estimating impacts when not all sample members comply with their assigned treatment and estimating impacts when groups are randomized. (Contains 2 tables, 2 figures, and 32 footnotes.) [This working paper will be published as a chapter in the forthcoming "Handbook of Social Research" by Sage Publications, Inc. It was supported by the Judith Gueron Fund for Methodological Innovation in Social Policy Research at MDRC, which was created through gifts from the Annie E. Casey, Rockefeller, Jerry Lee, Spencer, William T. Grant, and Grable Foundations.]

Confidence intervals are beginning to play an increasing role in the reporting of research findings within the social and behavioral sciences and, consequently, are becoming more prevalent in beginning classes in statistics and research methods. Confidence intervals are an attractive means of conveying experimental results, as they contain a considerable amount of information in a concise format. The objectives of this study are to (1) show that the theory of conceptual metaphor as delineated in contemporary embodied cognition is a useful framework for describing statistics students' conceptions of confidence intervals; and (2) provide empirical evidence from discourse and gesture that graduate students in social science use at least two competing conceptual metaphors for confidence limits that have important implications for the understanding and application of statistics and for the reform of statistics education. In the first metaphor--Confidence Intervals Are Changing Rings Around a Fixed Point ("Changing Ring metaphor")--confidence intervals are moving disks of various diameters covering a fixed but unknown point, like horseshoes of varying widths pitched at a fixed stake. Key to this correct conceptual metaphor is that the interval is a property of a sample but not of the population. Here, the diameter of the disk (i.e., the length of the confidence interval) changes from sample to sample, whereas the location of the stake (i.e., the population parameter or population mean) is fixed across samples but generally unknown. In contrast, the second metaphor--Confidence Intervals Are Changing Points on a Fixed Disk ("Fixed Disk metaphor")--conceptualizes confidence intervals as fixed-diameter disks onto which successive points are placed. In this incorrect metaphor, the population parameter can change from sample to sample. The interval is of a fixed length, and each experiment results in placing a new parameter somewhere onto the fixed-diameter disk. One possible source of this second metaphor is a suspected confusion between acceptance regions in hypothesis testing and confidence intervals, concepts that tend to be taught in close proximity to one another in statistics textbooks. The Fixed Disk metaphor will generally lead to a misinterpretation of the confidence interval that results in inaccurate problem solving. By better understanding students' mental representations of confidence intervals and appealing to the metaphors they convey, the authors hope to improve both statistics instruction and education researchers' uses of statistical tests. Interview items are appended. (Contains 5 tables and 4 figures.)

This paper explores the impact that the evaluation industry has had on the development and implementation of social policy and programs, primarily as carried out by the U.S. Departments of Labor and Health and Human Services. In addition, major tools evaluators have developed and used, and the institutional arrangements through which they have worked are reviewed. Three principal approaches that have emerged in the evaluation industry funded by executive and congressional agencies and by the grant and contract establishment are microsimulation, experimental and quasi-experimentation, and qualitative studies. With regard to experimentation and quasi-experimentation, a case for randomization, selection modeling, the New Jersey income support experiment, selecting a probabilistic sample, the elusiveness of the target population, attrition, the need to generalize to untested treatments, biases arising from the limited duration of experiments, feedback effects, Hawthorne effects, and the elusiveness of consensus are considered. Advocates of each method may claim that their method of estimating program impact offers policymakers better insights into the effects of policy decisions than those of alternative methods. Analysts have been unable to agree about the impact of social policy evaluation. Current practices, including those used by the General Accounting Office, are generally not adequate as guides to making choices among social policy alternatives. Improved results will be obtained only by making more effective use of various methodologies and by achieving a better balance in funding them. There is cause for optimism that program evaluation will justify its continued funding. (SLD)

This paper posits a taxonomy for categorizing issues that arise in the evaluation of Intelligent Tutoring Systems (ITSs). The taxonomy has three dimensions: Life Cycle of Evaluation, Research Issues, and Methodological Issues. The Life Cycle dimension has four levels: pre-experimental, laboratory study, field study, and initial operational test and evaluation. The three levels of the Research Issues dimension--functionality, effectiveness, and cost--are subsequently further divided into several sublevels. The Methodological Issues dimension is discussed in the context of each of the Research Issues levels. A recommendation from this work is that ITS evaluation studies should adopt multi-dimensional, multi-method designs. (Author)

This paper reports an experiment in over 3,000 Indonesian villages designed to test the role of performance incentives in improving the efficacy of aid programs. Villages in a randomly-chosen one-third of subdistricts received a block grant to improve 12 maternal and child health and education indicators, with the size of the subsequent year's block grant depending on performance relative to other villages in the subdistrict. Villages in remaining subdistricts were randomly assigned to either an otherwise identical block grant program with no financial link to performance, or to a pure control group. We find that the incentivized villages performed better on health than the non-incentivized villages, particularly in less developed areas, but found no impact of incentives on education. We find no evidence of negative spillovers from the incentives to untargeted outcomes, and no evidence that villagers manipulated scores. The relative performance design was crucial in ensuring that incentives did not result in a net transfer of funds toward richer areas. Incentives led to what appear to be more efficient spending of block grants, and led to an increase in labor from health providers, who are partially paid fee-for-service, but not teachers. On net, between 50-75% of the total impact of the block grant program on health indicators can be attributed to the performance incentives.

College students and college recruiters rated bogus resumes of twelve senior industrial management students who were allegedly seeking employment. In addition to subject population, three variables were systematically varied in each resume: applicant's sex, physical attractiveness, and scholastic standing. The dependent variable was subjects' perceptions of the applicant's suitability for the position of head of a furniture department in a large department store. Significant effects showed that applicants with high scholastic standing were preferred to applicants with low scholastic standing, male applicants were preferred to female applicants, attractive applicants were preferred to unattractive applicants, and applicants were rated more favorably by college students than by recruiters. It appears that the training and experience of college recruiters did not reduce the tendency to discriminate among job applicants on the basis of sex or physical attractiveness. (Author)

Educational research and development (educational R&D), more recently called educational knowledge production and utilization, has become a prominent feature of the educational scene with many of its salient characteristics having taken shape during the last ten years. The federal role in educational R&D began in 1867 with the formation of the National Department of Education. The passage of the Cooperative Research Act of 1954, the National Defense Education Act of 1958, and the creation of the National Institute of Education in 1972 expressed federal concern for stepping up efforts to improve the educational system. Although funds have decreased during the last decade, educational R&D laboratories and centers have prospered. One benefit gained recently from educational R&D is that the monolithic structure of the school system is giving way to a greater recognintion of differences among individuals. In general, the effects of educational R&D upon educational practice have been disappointing because the effort has been too small, the trained researchers too few, and the resources too limited. An 11-point agenda for educational R&D during the next decade includes: (1) building a constituency; (2) broadening the collegial base; (3) strengthening all parts of the educational R&D enterprise; (4) recognizing that education is a total system; (5) shifting emphasis from correlational studies and single-variable experimental studies to more complex experimental studies, interventions, and clinical analyses; (6) making more modest claims; (7) building a better national educational R&D agenda; (8) effectively advancing the interdisciplinary claims of educational R&D; (9) strengthening university participation in educational R&D; (10) increasing attention to cost/benefit consideration; (11) reestablishing the upward trend of expenditures for educational R&D. (SK)

Investigates the effect of delay of reinforcement upon human discrimination learning with particular emphasis on the form of the gradient within the first few seconds of delay. In previous studies subjects are usually required to make an instrumental response to a stimulus, this is followed by the delay interval, and finally, the reinforcement occurs. The present procedure did not require any instrumental response during the experimental treatment. Instead, a pair of stimuli was presented for a brief period, a delay interval occured, and then a light appeared over the stimulus that the subject was instructed to remember. In the present situation the subject is required to associate a stimulus with the reinforcement light. Classical conditioning literature indicates that an interstimulus interval of about .50 sec. produces optimal conditioning. Even small changes in either direction from this interval produce performance decrements. Thus, evidence from both animal discrimination learning and classical conditioning support the importance of the first second or two of the interval between the occurrence of two stimuli that are to become associated. In this series of experiments, particular attention was given to delay intervals between 0 and 2.0 sec. with other delays ranging up to 5.0 sec. (Author/RC)

Econometric cost functions have begun to appear in education adequacy cases with greater frequency. Cost functions are superficially attractive because they give the impression of objectivity, holding out the promise of scientifically estimating the cost of achieving specified levels of performance from actual data on spending. By contrast, the opinions of education stakeholders form the basis of the most common approach to estimating the cost of adequacy, the professional-judgment method. The problem is that education cost functions do not in fact tell us the cost of achieving any specified level of performance. Instead, they provide estimates of average spending for districts of given characteristics and current performance. It is a huge and unwarranted stretch to go from this interpretation of regression results to the claim that they provide estimates of the minimum cost of achieving current performance levels, and it is even more problematic to extrapolate the cost of achieving at higher levels. In this paper we review the cost function technique and provide evidence that draws into question the usefulness of the cost function approach for estimating the cost of an adequate education. (Contains 28 footnotes and 7 figures.) [Funding for this paper was provided by the Missouri Show-Me Institute.]

This paper presents results from a randomized control trial in which approximately 1,000 OLPC XO laptops were provided for home use to children attending primary schools in Lima, Peru. The intervention increased access and use of home computers, with some substitution away from computer use outside the home. Beneficiaries were more likely to complete domestic chores but less likely to read books. Treatment children scored almost one standard deviation higher in a test of XO proficiency, though there were no effects on objective and self-reported skills for using a Windows-based PC and Internet. There were positive impacts on the Raven's Progressive Matrices test among children who did not have a home computer before the intervention, but no significant effects for the sample as a whole. Finally, there was little evidence for spillovers within schools, although close friends and classmates of laptop recipients did exhibit higher proficiency with the XO computer.

The purpose of this paper is to study identification and estimation of causal effects in experiments with multiple sources of noncompliance. This research design arises in many applications in education when access to oversubscribed programs is partially determined by randomization. Eligible households decide whether or not to comply with the intended treatment. The paper treats program participation as the outcome of a decision process with five latent household types. We show that the parameters of the underlying model of program participation are identified. Our proofs of identification are constructive and can be used to design a GMM estimator for all parameters of interest. We apply our new methods to study the effectiveness of magnet programs in a large urban school district. Our findings show that magnet programs help the district to attract and retain students from households that are at risk of leaving the district. These households have higher incomes, are more educated, and have children that score higher on standardized tests than households that stay in district regardless of the outcome of the lottery.

This paper: i) estimates the effect that going to a better school has on students' academic achievement, and ii) explores whether this intervention induces behavioral responses on the part of children, their parents, and the school system. For the first task, we exploit almost 2,000 regression discontinuity quasi-experiments observed in the context of Romania's high school educational system. For the second, we use data from a specialized survey of children, parents, teachers and principals that we implemented in 59 Romanian towns. The first finding is that students do benefit from access to higher achieving schools and tracks within schools. A second set of results suggests that the stratification of schools by quality in general, and the opportunity to attend a better school in particular, result in significant behavioral responses on the part of teachers, parents, and students. Although we do not expect the magnitude or even the direction of these responses to hold everywhere, their existence has a number of implications for evaluation, particularly since some of them change over time, and some would seem to be relevant only once interventions reach a certain scale.

An important element in considering school finance policies is that households are not passive but instead respond to policies. Household behavior is especially important in considering how households affect the spatial structure of metropolitan areas where different jurisdictions incorporate bundles of advantages and disadvantages. This paper adds richness to existing urban models by incorporating multiple workplace locations, alternative public services by jurisdiction (school qualities), and voter- determined school expenditure. In our general equilibrium model of residential location and community choice, households base optimizing decisions on commuting costs, school quality, and land rents. The resulting equilibrium has heterogeneous communities in terms of income and tastes for schools. This basic model is used to analyze a series of conventional policy experiments, including school district consolidation and district power utilization. The important conclusion within our range of simulations is that welfare falls for all families with the restrictions on choice that are implied by these approaches.

This essay reviews progress in empirical economics since Leamer'rs (1983) critique. Leamer highlighted the benefits of sensitivity analysis, a procedure in which researchers show how their results change with changes in specification or functional form. Sensitivity analysis has had a salutary but not a revolutionary effect on econometric practice. As we see it, the credibility revolution in empirical work can be traced to the rise of a design-based approach that emphasizes the identification of causal effects. Design-based studies typically feature either real or natural experiments and are distinguished by their prima facie credibility and by the attention investigators devote to making the case for a causal interpretation of the findings their designs generate. Design-based studies are most often found in the microeconomic fields of Development, Education, Environment, Labor, Health, and Public Finance, but are still rare in Industrial Organization and Macroeconomics. We explain why IO and Macro would do well to embrace a design-based approach. Finally, we respond to the charge that the design-based revolution has overreached.

A long line of research on behavioral economics has established the importance of factors that are typically absent from the standard economic framework: reference dependent preferences, hyperbolic preferences, and the value placed on non-financial rewards. To date, these insights have had little impact on the way the educational system operates. Through a series of field experiments involving thousands of primary and secondary school students, we demonstrate the power of behavioral economics to influence educational performance. Several insights emerge. First, we find that incentives framed as losses have more robust effects than comparable incentives framed as gains. Second, we find that non-financial incentives are considerably more cost-effective than financial incentives for younger students, but were not effective with older students. Finally, and perhaps most importantly, consistent with hyperbolic discounting, all motivating power of the incentives vanishes when rewards are handed out with a delay. Since the rewards to educational investment virtually always come with a delay, our results suggest that the current set of incentives may lead to underinvestment. For policymakers, our findings imply that in the absence of immediate incentives, many students put forth low effort on standardized tests, which may create biases in measures of student ability, teacher value added, school quality, and achievement gaps.

We investigate short and long-term effects of early childhood education using variation created by a unique policy experiment in British Columbia, Canada. Our findings imply starting Kindergarten one year late substantially reduces the probability of repeating the third grade, and meaningfully increases in tenth grade math and reading scores. Effects are highest for low income students and males. Estimates suggest that entering kindergarten early may have a detrimental effect on future outcomes.