Search

We review 100 years of research on performance appraisal and performance management, highlighting the articles published in JAP, but including significant work from other journals as well. We discuss trends in eight substantive areas: (1) scale formats, (2) criteria for evaluating ratings, (3) training, (4) reactions to appraisal, (5) purpose of rating, (6) rating sources, (7) demographic differences in ratings, and (8) cognitive processes, and discuss what we have learned from research in each area. We also focus on trends during the heyday of performance appraisal research in JAP (1970-2000), noting which were more productive and which potentially hampered progress. Our overall conclusion is that JAP's role in this literature has not been to propose models and new ideas, but has been primarily to test ideas and models proposed elsewhere. Nonetheless, we conclude that the papers published in JAP made important contribution to the filed by addressing many of the critical questions raised by others. We also suggest several areas for future research, especially research focusing on performance management. (PsycINFO Database Record, ((c) 2017 APA, all rights reserved).)

Context is a central construct in the study of performance appraisal (PA) systems, but to date studies linking the distal context within which organizations operate to PA have been limited in two ways. First, these studies have focused predominantly on national culture, and have paid less attention to the legal/political and economic environment within which organizations. Second, few studies reflect the growing international nature of business organizations, which create multiple layers of context (e.g., home and host country context). Drawing on both situational strength and institutional theories, we hypothesize the links between the "big three" facets of distal context (i.e., culture, legal/political, and economic) and the processes and purposes of PA systems. Using data from 472 multinational enterprises (MNEs) in 22 countries, we show that all three facets of the distal context of the host country influence decisions about the conduct of PA systems. In addition, we show that the culture and legal/political systems of the home country from which the MNE originated are also linked to the PA processes and purposes. Finally, we find evidence that MNEs are more likely to develop hybridized PA systems that depart from host country norms. [ABSTRACT FROM AUTHOR]

A wide range of systems for evaluating performance have been used in organisations, ranging from traditional annual performance appraisals to performance management systems built around informal, real‐time evaluations, and these systems almost always fail. Rather than continuing to make cosmetic adjustments to this system, organisations should consider dropping the practice of regularly evaluating the performance of each of their employees, focusing rather on the small subset of situations in which evaluations of performance and performance feedback are actually useful. Four barriers to successful performance evaluation are reviewed: (a) the distribution of performance, (b) the continuing failure to devise reliable and valid methods for obtaining judgments about performance, (c) the limited utility of performance feedback to employees, and (d) the limited utility of performance evaluations to organisations. In this paper, I propose ways of managing performance without relying on regular performance evaluation, refocusing managers' activities from performance management to performance leadership. [ABSTRACT FROM AUTHOR]

Content-oriented validation strategies establish the validity of selection tests as predictors of performance by comparing the content of the tests with the content of the job. These comparisons turn out to have little if any bearing on the predictive validity of selection tests. There is little empirical support for the hypothesis that the match between job content and test content influences validity, and there are often structural factors in selection (e.g., positive correlations among selection tests) that strongly limit the possible influence of test content on validity. Comparisons between test content and job content have important implications for the acceptability of testing, the defensibility of tests in legal proceedings, and the transparency of test development and validation, but these comparisons have little if any bearing on validity. [ABSTRACT FROM AUTHOR]

The 12 papers commenting on K. R. Murphy (2009a) raise a number of important issues, most of which can be subsumed in one of four themes. First, papers examining content-oriented validation strategies are still necessary and useful, in part because of the frequent use of these strategies in the practice of industrial–organizational (I–O) psychology. Second, the term “content validity” means many different things both within and beyond the field of I–O psychology, and it is useful to understand what sorts of inferences examinations of test content do and do not support. Third, these 12 papers present very little evidence that content validation, as typically carried out by I–O psychologists, actually provides information about the likelihood that people who do well on the test will do well on the job. Finally, I believe that the best use of content-related evidence in validating selection tests is in developing hypotheses about relationships between test scores and criteria rather than in testing these hypotheses. [ABSTRACT FROM AUTHOR]

Ratings of job performance are widely viewed as poor measures of job performance. Three models of the performance–performance rating relationship offer very different explanations and solutions for this seemingly weak relationship. One-factor models suggest that measurement error is the main difference between performance and performance ratings and they offer a simple solution—that is, the correction for attenuation. Multifactor models suggest that the effects of job performance on performance ratings are often masked by a range of systematic nonperformance factors that also influence these ratings. These models suggest isolating and dampening the effects of these nonperformance factors. Mediated models suggest that intentional distortions are a key reason that ratings often fail to reflect ratee performance. These models suggest that raters must be given both the tools and the incentive to perform well as measurement instruments and that systematic efforts to remove the negative consequences of giving honest performance ratings are needed if we hope to use performance ratings as serious measures of job performance. [ABSTRACT FROM AUTHOR]

The comments and suggestions prompted by K. R. Murphy’s (2008) description of alternate models of the relationship between job performance and ratings of job performance reflect 3 broad themes: (a) the relationship between performance appraisal and performance measurement, (b) the best psychometric models for understanding performance ratings, and (c) the appropriateness of static measures for dynamic phenomena. This paper comments on these 3 themes and suggests directions for future research and practice in performance appraisal that focuses on rater goals, organizational interventions to improve the accuracy and value of ratings, and assessments of the value of performance ratings as criteria. [ABSTRACT FROM AUTHOR]

Despite recent enthusiasm for the use of personality inventories in personnel selection, many of the problems cited in Guion and Gottier's (1965) review have yet to be resolved. The validity of measures of broad personality traits is still low, personality tests used in organizations are still poorly chosen, and links between personality and jobs are poorly understood. Personality measures are unlikely to achieve the degree of acceptance given to cognitive tests because of differences in the domains, differences in the tests, and differences in the environments in which cognitive tests versus personality inventories are developed. [ABSTRACT FROM AUTHOR]

Recent performance appraisal research has highlighted the important role played by contextual and individual factors in shaping rating behavior. This article reviews cumulated empirical data supporting the proposition that in factors and constraints present in the organization, contexts in which appraisal systems reside and rater attributes, such as personality factors or beliefs, systematically affect rating behavior. The effects of these context and rater factors are reflected in ratings accuracy, ratings discrimination among raters/dimensions, and rating elevation. [ABSTRACT FROM AUTHOR]

A simulation was used to explore the effects of variations in the rate at which applicants drop out of selection processes on racial differences in selection outcomes. Archival data was used to simulate a realistic range of selection scenarios in which test score differences between groups and selection ratios varied. The basis for dropping out was manipulated in two separate studies. Study 1 simulated dropout decisions that occurred at random within racial subgroups; in this study, dropout rates of minority versus White candidates were varied. Study 2 examined dropout decisions that occurred as a function of test scores. Results from both studies showed that mean test score differences between White and minority applicants have the largest influence on adverse impact. Interventions designed to reduce the tendency of minority applicants to withdraw from selection are likely to have, at best, small effects on the adverse impact of selection tests. [ABSTRACT FROM AUTHOR]

A particular form of test score banding, in which bands are based on the reliability of the test and in which selection within bands takes into account criteria that are likely to enhance workforce diversity, has been proposed as an alternative to the traditional top-down (rank-order) hiring systems, but it has been hotly debated among both scientists and practitioners. In a question-and-answer format, this article presents three different viewpoints (proponents, critics, and neutral observers) on the scientific, legal, and practical issues. The article also attempts to seek some consensus among experts on this controversial procedure. [ABSTRACT FROM AUTHOR]

Interrater correlations are widely interpreted as estimates of the reliability of supervisory performance ratings, and are frequently used to correct the correlations between ratings and other measures (e.g., test scores) for attenuation. These interrater correlations do provide some useful information, but they are not reliability coefficients. There is clear evidence of systematic rater effects in performance appraisal, and variance associated with raters is not a source of random measurement error. We use generalizability theory to show why rater variance is not properly interpreted as measurement error, and show how such systematic rater effects can influence both reliability estimates and validity coefficients. We show conditions under which interrater correlations can either overestimate or underestimate reliability coefficients, and discuss reasons other than random measurement error for low interrater correlations. [ABSTRACT FROM AUTHOR]

Murphy and DeShon (2000) show that interrater correlations do not provide reasonable estimates of the reliability of job performance ratings, and suggest that better estimates can be obtained by applying the methods of generalizability theory, Schmidt, Viswesvaran, and Ones (2000) criticize our suggestions as radical, and argue that: (a) the reliability of ratings should be evaluated using the parallel test model rather than the more general and more realistic generalizability model, (b) reliability and validity are distinct concepts that should not be confused, and (c) measurement models have little to do with substantive models of the processes that generate scores on a test or measure. All three of these ideas were once part of the psychometric mainstream, but progress in psychometrics over the last 3 decades has moved the field well beyond these assumptions and approaches. Modern psychometric theory calls for close linkages between measurement models and substantive models of the phenomena being measured. [ABSTRACT FROM AUTHOR]

The effects of rating scale formats on several indices of the usefulness of performance appraisal for employee development were examined, The job performance of 96 police officers was rated using simple graphic scales or one of two behaviorally oriented rating formats: behaviorally anchored rating scales (BARS) and behavior observation scales (BOS). As predicted, ratees' satisfaction with performance appraisal was highest and their perceptions of performance goals most favorable when using SOS. In addition, performance improvement goals for officers rated using SOS were judged by experts to be most observable and specific. Contrary to the authors' predictions, graphic rating scales were generally as good as BOS and as good as or better than BARS when evaluated in terms of ratee attitudes and goal characteristics. The results suggest that different behaviorally-oriented rating formats can enhance or inhibit the developmental applications of performance appraisal. [ABSTRACT FROM AUTHOR]

The attitudes of twenty-nine managers toward performance appraisal and toward their organization were used to predict differences in mean ratings and measures of discrimination among ratees and performance dimensions. Substantial multiple correlations were obtained for all three dependent variables (between 32 and 46% of the variance in each measure was explained), and several individual correlations were statistically significant. Raters who showed low levels of confidence in the appraisal system, high levels of discomfort, or high levels of instrumental commitment were more likely to provide ratings that were unusually high or that did not discriminate well among ratees and/or dimensions. Raters who showed higher levels of attitudinal commitment or who perceived more risks associated with distorting ratings tended to provide lower ratings and to discriminate among ratees and/or dimensions. [ABSTRACT FROM AUTHOR]

The relationship between attitudes and beliefs regarding performance appraisal and its organizational context and three measures of rating quality (i.e., rating level, discrimination among ratees, discrimination among rating dimensions) were examined. Eighteen of twenty-four directional predictions about the correlations between attitudinal measures and rating behavior measures were supported. However, because of the expected weak to moderate relationships between predictors and criteria and the use of small samples, statistical power was generally low, and many of the individual observed correlations failed to reach significance. Nevertheless, the consistent pattern of support for the directional predictions suggests that the attitudes and beliefs studied here provide a useful basis for understanding performance appraisal behaviors. [ABSTRACT FROM AUTHOR]

Cleveland and Shore (1992) suggested that four perceptual age measures could be grouped into person-oriented and context-oriented factors. This study examined longitudinal data from their same sample, and tested three propositions related to the distinctiveness and usefulness of the age measures. Confirmatory factor analyses showed that the factor structure proposed by Cleveland and Shore was invariant over time, and that new multi-item scales measuring two types of age loaded on appropriate factors. As hypothesized, context-oriented measures showed less temporal stability than person- oriented measures, and the temporal relationships among person-oriented measures were more easily explained in terms of a strict simplex structure than was the case for context-oriented measures. Perceptual age measures accounted for variance in self-ratings and managers' ratings of employee health, self-ratings of retirement intentions, and managers' ratings of promotability not accounted for by chronological age. [ABSTRACT FROM AUTHOR]

This article describes the application of a number of test score banding procedures in personnel selection. Equations are developed illustrating the relationship between the width of test score bands and test reliability. When reliability is moderate to low, bands are likely to be larger than the standard deviation of the test, and are likely to include a large proportion of the applicant pool. The relationships between band widths and the differences between higher scoring and lower scoring groups are also examined. When the band is smaller than the differences between groups, banding may not by itself prove effective as a means of reducing the adverse impact of tests, even when banding systems that maximize opportunities for members of the lower scoring group are used.

The goal of focal articles in Industrial and Organizational Psychology: Perspectives on Science and Practice is to present new ideas or different takes on existing ideas and stimulate a conversation in the form of comment articles that extend the arguments in the focal article or that present new ideas stimulated by those articles. The two focal articles in this issue stimulated a wide range of reactions and a good deal of constructive input. [ABSTRACT FROM PUBLISHER]