Search

A recent attempt to generate an updated ranking for the operational validity of 25 selection procedures, using a process labeled “conservative estimation” (Sackett et al., 2022), is flawed and misleading. When conservative estimation's treatment of range restriction (RR) is used, it is unclear if reported validity differences among predictors reflect (i) true differences, (ii) differential degrees of RR (different u values), (iii) differential correction for RR (no RR correction vs. RR correction), or (iv) some combination of these factors. We demonstrate that this creates bias and introduces confounds when ranking (or comparing) selection procedures. Second, the list of selection procedures being directly compared includes both predictor methods and predictor constructs, in spite of the substantial effect construct saturation has on validity estimates (e.g., Arthur & Villado, 2008). This causes additional confounds that cloud comparative interpretations. Based on these, and other, concerns we outline an alternative, “considered estimation” strategy when comparing predictors of job performance. Basic tenets include using RR corrections in the same manner for all predictors, parsing validities of selection methods by constructs, applying the logic beyond validities (e.g., ds), thoughtful reconsideration of prior meta‐analyses, considering sensitivity analyses, and accounting for nonindependence across studies. [ABSTRACT FROM AUTHOR]

A significant percentage of veterans suffer from post‐traumatic stress disorder (PTSD). Veterans are often directed to social media platforms to seek support during their transition to civilian life. However, social media platforms are increasingly used to aid in hiring decisions, and these platforms may make veterans' PTSD more discoverable during the hiring process. Based on social identity theory and identity management theory, the integrated suspicion model, and the stigma literature, we conducted four studies that examine veterans' PTSD disclosures on social media and the consequences in the hiring process. Study 1 suggests that 16%–34% of veterans included cues related to PTSD status on social media. Study 2, based on 290 upper‐level business students, shows that veterans with PTSD were more stigmatized than veterans without PTSD, and stigmatization is associated with more suspicion and lower hiring‐related ratings (of expected task performance, expected organizational citizenship behaviors (OCB), expected counterproductive work behaviors (CWB), and intention to interview). Study 3, based on 431 working professionals with hiring experience, further supports relationships from Study 2. Study 4, based on 298 working professionals, identifies peril (i.e., perceptions regarding danger) as an additional mediator for the effects of PTSD on hiring‐related ratings. In sum, we identify and explore the identity management conundrum that social media disclosure poses for veterans with PTSD in the hiring process and discuss potential remedies and avenues for future research. [ABSTRACT FROM AUTHOR]

To promote diversity in organizations it is important to have accurate knowledge about subgroup differences associated with selection procedures. However, current estimates of subgroup differences in situational judgment tests (SJTs) are overwhelmingly based on range‐restricted incumbent samples that are downwardly biased. This study provides much‐needed applicant level estimates of SJT subgroup differences (N = 37,530). As a key finding, Black‐White differences (d = 0.66) were higher than in incumbent samples (d = 0.38). Overall, sex differences were small. Females scored higher for management jobs (d = −0.13) and males scored higher for administrative jobs (d = 0.15). By analyzing applicant samples that do not suffer from range restriction, this study adds knowledge about subgroup differences in SJTs. [ABSTRACT FROM AUTHOR]

Organizations frequently screen or select job applicants based on their work experience. Despite this, surprisingly little is known about the criterion‐related validity of prehire experience, which reflects the amount, duration, or type of experience workers have acquired before they enter a new organization. To address this critical gap in the literature, we used meta‐analysis to synthesize data from 81 independent samples that reported relations between prehire experience and performance or turnover. Results revealed overall corrected correlations of.06 for job performance (k = 44, n = 11,785),.11 for training performance (k = 21, n = 8,176), and.00 for turnover (k = 32, n = 11,676). Measures that capture prehire experience with tasks, jobs, or occupations relevant to workers' current position also are only weakly related to the outcomes (e.g., ρ¯̂ = .07 for job performance). Two exceptions to our main findings are that (a) prehire experience is somewhat more predictive of job performance when workers first start a new job and (b) measures of task‐level experience predict training performance, although these results are based on small subsets of primary studies. Overall, the present findings suggest that the types of prehire experience measures organizations currently use to screen job applicants generally are poor predictors of future performance and turnover. We therefore caution organizations from selecting employees based on such measures unless more positive evidence emerges. [ABSTRACT FROM AUTHOR]

Situational judgment tests ( SJTs) have become popular in recent years. Yet, there is little research on how the constructs targeted within SJTs influence standardized ethnic group differences. Further, most research in this area is subject to differential range restriction concerns that hinder understanding of the role of constructs. We report on scale-level data from four jobs in which SJTs were part of the first major hurdle of selection, thus providing an analysis of how constructs might relate to standardized group differences when range restriction concerns are minimized. Results indicate that cognitively saturated scales were associated with d values of 0.56 and 0.76, while interpersonal items were associated with d values of 0.07, 0.20, and 0.50. Based on the obtained ds, we simulated hiring and obtained adverse impact ratios to help interpret these values of d for decision-makers. Overall, we demonstrate the importance of examining constructs and gathering data at the applicant level of analyses when attempting to understand SJT d values (and any associated adverse impact). Practitioner points The constructs targeted by SJTs may influence ethnic group differences, such that cognitively-related items/scales may lead to larger group differences than scales targeting interpersonal skills., It can be useful to compare SJT ds based on both job applicant samples and targeted constructs during test development and during consideration of alternative predictors. [ABSTRACT FROM AUTHOR]

In both theoretical and applied literatures, there is confusion regarding accurate values for expected Black-White subgroup differences in personnel selection test scores. Much confusion arises because empirical estimates of standardized subgroup differences ( d) are subject to many of the same biasing factors associated with validity coefficients (i.e., d is functionally related to a point-biserial r). To address such issues, we review/cumulate, categorize, and analyze a systematic set of many predictor-specific meta-analyses in the literature. We focus on confounds due to general use of concurrent, versus applicant, samples in the literature on Black-White d. We also focus on potential confusion due to different constructs being assessed within the same selection test method, as well as the influence of those constructs on d. It is shown that many types of predictors (such as biodata inventories or assessment centers) can have magnitudes of d that are much larger than previously thought. Indeed, some predictors (such as work samples) can have ds similar to that associated with paper-and-pencil tests of cognitive ability. We present more realistic values of d for both researcher and practitioner use. Implications for practice and future research are noted. [ABSTRACT FROM AUTHOR]

Split-hand/split-foot malformation (SHFM1) has been reported to be caused by deletions, duplications or rearrangements involving the 7q21.3 region harboring DSS1, DLX5, and DLX6. We report on a female patient with unilateral syndactyly of the third and fourth fingers of the right hand and overgrowth and lateral deviation of the right great toe. There was a split foot malformation on the right, with absent fifth toe. The left hand was apparently normal and left foot was intact. The patient has no hearing loss. We performed conventional G-banding karyotype analysis, array comparative genomic hybridization (aCGH) and fluorescence in situ hybridization (FISH). G-banding karyotype result was normal 46,XX. However, a duplication of 719 kb (96,303,736-97,022,335; NCBI build36/hg18, March 2006) was identified at the 7q21.3 region by aCGH. The array result was also confirmed by FISH analysis. The duplicated region harbors only DLX5 and DLX6, which are known for their role in SHFM1. Additionally, FISH analysis of parental samples showed de novo origin of this abnormality in the patient. This is the first report that highlights the duplication of 719 kb at 7q21.3, harboring only DLX5 and DLX6 associated with the SHFM1 phenotype. © 2012 Wiley Periodicals, Inc. [ABSTRACT FROM AUTHOR]

Simulations and analyses based on meta-analytic matrices are fairly common in human resource management and organizational behavior research, particularly in staffing research. Unfortunately, the meta-analytic values estimates for validity and group differences (i.e., ρ and δ, respectively) used in such matrices often vary in the extent to which they are affected by artifacts and how accurately the values capture the underlying constructs and the appropriate population. We investigate how such concerns might influence conclusions concerning key issues such as prediction of job performance and adverse impact of selection procedures, as well as noting wider applications of these issues. We also start the process of building a better matrix upon which to base many such simulations and analyses in staffing research. Finally, we offer guidelines to help researchers/practitioners better model human resources processes, and we suggest ways that researchers in a variety of areas can better assemble meta-analytic matrices. [ABSTRACT FROM AUTHOR]

We clarify the existing literature on gender differences in work sample exams by disentangling such differences on work sample exams from other predictors (e.g., situational judgment tests). In fact, we note that there are only two articles specifically related to gender differences in work sample exams. Based on theory and literature from social psychology, neuropsychology, and applied psychology, we propose that the three constructs of self-concept, social skills, and writing skills are likely to influence work sample gender differences. We tested our hypotheses on two samples of managers. We found in one instance that males scored higher on a technical exercise, but there was stronger support for females, on average, scoring higher on exercises that involved social skills and on exercises that involved writing skills. Work samples that targeted a broad array of knowledge, skills, and abilities were associated with higher overall scores for females ( ds of −.37 and −.34) and, thus, were unlikely to be associated with gender-based adverse impact against females. [ABSTRACT FROM AUTHOR]

Work sample exams are generally thought to have either low or comparatively low levels of ethnic group differences when used for personnel selection. Such exams are sometimes called “simulation exercises” and involve having applicants perform a set of tasks that are similar to those performed on the job. The nearly ubiquitous meta-analytic value of Black–White subgroup differences in the literature is d = .38. Unfortunately, this estimate is plagued by a variety of problems (e.g., range restriction, inclusion of nonwork sample tests). Further, there are virtually no analyses that examine how the saturation of different constructs influence work sample tests. We gathered available data for Black–White ethnic group differences and found that overall work sample differences were markedly larger for samples of job applicants (d = .73) than previously thought. We also examined how different exercises and saturation of different constructs influenced work sample ds. For example, work sample test ratings of cognitive and job knowledge skills were associated with a mean observed d = .80, whereas ratings of various social skills were associated with mean observed ds that varied from .21 to .27. We urge scientists and practitioners to consider both the method and the constructs that are targeted when forecasting predictor ds. [ABSTRACT FROM AUTHOR]

De Corte, Lievens, and Sackett add to the literature on selection test validity and adverse impact (AI). Their Pareto-based weighting scheme essentially asks organizations if they are willing to give up some validity to hopefully achieve some reduction in AI. We considered their approach and conclusions in relation to the regression weighting method we used, and we offer five points that reflect our observations as well as our shared goals. We hope our comments, like their work in this field, will invigorate the pursuit of new ways of examining, and one day resolving, the persistent concern regarding the AI associated with valid selection tests. [ABSTRACT FROM AUTHOR]

Videotaped interviews are used for both research and for making selection decisions in organizations. However, little research has examined the extent to which the psychometric characteristics of ratings of videotaped interviews are comparable with those of ratings made on the basis of face-to-face (FTF) interviews. Within a simulated selection setting, we compared ratings of interviewers who conducted FTF structured interviews to ratings of interviewers who viewed videotapes of those interviews. Results revealed that FTF ratings were significantly higher than video ratings of the same interviewees. We also found that the two sets of interviewers rated the relative performance of interviewees differently. For example, the correlation between FTF and video ratings ( r=.31) was significantly smaller than the correlation between ratings of interviewers who conducted FTF panel interviews with the same interviewees ( r=.73). Overall results suggest that researchers and practitioners should be cautious about generalizing research findings and selection decisions made on the basis of videotaped interviews to FTF interviews. [ABSTRACT FROM AUTHOR]

Applied psychologists have long been interested in the relationship between applicant personality and employment interview ratings. Analysis of data from two studies, one using a situational interview and one using a behavioral interview, suggests that the correlations of structured interview ratings with self-report measures of personality factors are generally rather low. Further, a small meta-analysis integrates these two studies and the limited previous literature to arrive at a similar conclusion – there is relatively little relationship between structured interviews and self-reported personality factors. [ABSTRACT FROM AUTHOR]

Work sample tests have been used in applied psychology for decades as important predictors of job performance, and they have been suggested to be among the most valid predictors of job performance. As we examined classic work sample literature, we found the narrative review by Asher and Sciarrino (1974) to be plagued by many methodological problems. Further, it is possible that data used in this study may have influenced the results (e.g., r= .54) reported by Hunter and Hunter in their seminal work in 1984. After integrating all of the relevant data, we found an observed mean correlation between work sample tests and measures of job performance of .26. This value increased to .33 when measures of job performance (e.g., supervisory ratings) were corrected for attenuation. Our results suggest that the level of the validity for work sample tests may not be as large as previously thought (i.e., approximately one third less than previously thought). Further, our work also summarizes the relationship of work sample exams to measures of general cognitive ability. We found that work sample tests were associated with an observed correlation of .32 with tests of general cognitive ability. [ABSTRACT FROM AUTHOR]

Although there has been empirical attention paid to the criterion-related validity of predictor composites, there has been much less attention paid to the standardized ethnic group differences associated with these composites. One important area of inquiry in predictor composite research is the influence of adding predictors to a test of general mental ability. The limited empirical literature on this practice is mixed, but the prevailing expectation is that there is likely to be higher validity and less adverse impact. Unfortunately, much of the previous work is limited by the presence of inaccurate validity and standardized ethnic group difference values. In this analysis we formed meta-analytic matrices to more accurately estimate the validity and standardized ethnic group differences of several composites that combine a measure of cognitive ability with measures of conscientiousness, a structured interview, or biodata. While results were somewhat complex, we found that adding alternative predictors does not result in a situation in which validity automatically goes up and adverse impact potential automatically goes down. In fact, the reductions in adverse impact (if any) from adding “non-cognitive” predictors were more modest than much of the literature suggests. [ABSTRACT FROM AUTHOR]

The use of education and experience minimum qualifications (MQs) is nearly ubiquitous in employment settings, yet it appears to be rare that such MQs are validated by the end user (either via content validity or criterion-related validation approaches). In this article, we present a method of content-validating MQs that is related to a procedure noted by , and we demonstrate the method's application for an upper-level management position in a state agency. Our procedure is based on adherence to the Uniform Guidelines and sound professional practice. In addition, the procedure was heard in a federal court proceeding (deposition and expert testimony in relation to 3 jobs), was approved by that court, and we discuss the court's findings. In particular, the court found, “The MQ development process used by the State Personnel Department (SPD) is consistent with the requirements of the Guidelines and leads to content-valid MQs.” Given data available to us for another job, we also show that the obtained MQs may result in less adverse impact than a previously determined, task-based set of MQs. [ABSTRACT FROM AUTHOR]

Almost all book chapters, review articles, and textbooks in the field of personnel selection suggest that work sample tests are associated with lower levels of ethnic group adverse impact than paper-and-pencil tests of cognitive ability. However, the empirical literature is heavily dependent on adverse impact estimates obtained from incumbent samples rather than applicant samples. As such, parameter estimates are subject to range restriction from prior selection and on-the-job experiences. Further, an emerging consensus in the selection literature indicates that any method of assessment can be associated with highorlow levels of adverse impact– depending on the nature of the construct(s) being measured. To begin to examine these issues, we present two recent sets of applicant data from public sector jobs (for a management and entry-level job, each with technical and interpersonal skill requirements) and show that adverse impact of work sample exams might be more extensive than realized. We discuss the mismatch between what the field of employee selection“knows” and what is said in articles/summaries about work samples. Employers and other practitioners who depend on advice in academic overview articles may be overly optimistic and eventually disappointed by minimal reduction in adverse impact. Implications for workforce diversity and future research needs are also discussed. [ABSTRACT FROM AUTHOR]

The method of selecting among job applicants using statistically based banding has been proposed over the last 10 years as a way to increase workforce diversity. The method continues to be reviewed by academics and considered by practitioners. Although the goal of increasing workforce diversity is important, statistical banding of scores remains controversial. We present a set of unique, statistically and theoretically based criticisms of a form of banding (top-score-referenced banding) that is widely used in hundreds of jobs in the public sector throughout the United States. We suggest that even within the premises of such banding, the wrong formula is used to estimate the standard error of measurement and standard error of the difference. One consequence is that too many individuals are labeled as essentially equal with respect to test scores. A related consequence is that test scores within a single band are statistically different and should therefore be treated as such for selection purposes. A more logically and statistically defensible procedure for responding to diversity concerns is to continue to attend to adverse impact issues at each step of the recruiting and test development process. [ABSTRACT FROM AUTHOR]

The primary purpose of this investigation was to test two key characteristics hypothesized to influence the validity of situational (SI) and behavior description (BDI) structured interviews. A meta-analysis of 54 studies with a total sample size of 5536 suggested that job complexity influences the validity of SIs, with decreased validity for high-complexity jobs, but does not influence the validity of BDIs. And, results indicated a main effect for study design across both SIs and BDIs, with predictive studies having 0.10 lower validity on average than concurrent studies. Directions for future research are discussed. [ABSTRACT FROM AUTHOR]

Assessment of standardized ethnic group differences (d) on predictors of job performance has become an important issue for applied psychologists. A number of studies have used an experimental design in which the predictor of interest was administered after an initial screening predictor. We examined the influence of prior selection on a first predictor on observed ds for second predictors in multiple-hurdle selection systems. Results of a Monte Carlo simulation indicate observed ds on the second predictor are underestimated in the presence of prior selection on another predictor. More important, "downward bias" in observed standardized ethnic group difference is substantial (30-70%) when selection ratios are low, standardized ethnic group differences on the screening predictor are high, and when the first and second predictors correlate above .30. Researchers should consider the influence of range restriction in designing studies of ethnic group differences and comparing ds across predictors, particularly when data are collected under a multiple-hurdle design. [ABSTRACT FROM AUTHOR]

The cognitive ability levels of different ethnic groups have interested psychologists for over a century. Many narrative reviews of the empirical literature in the area focus on the Black--White differences, and the reviews conclude that the mean difference in cognitive ability (g) is approximately 1 standard deviation; that is, the generally accepted effect size is about 1.0. We conduct a meta-analytic review that suggests that the one standard deviation effect size accurately summarizes Black--White differences for college application tests (e.g., SAT) and overall analyses of tests of g for job applicants in corporate settings. However, the 1 standard deviation summary of group differences fails to capture many of the complexities in estimating ethnic group differences in employment settings. For example, our results indicate that job complexity, the use of within job versus across job study design, focus on applicant versus incumbent samples, and the exact construct of interest are important moderators of standardized group differences. In many instances, standardized group differences are less than 1 standard deviation. We conduct similar analyses for Hispanics, when possible, and note that Hispanic--White differences are somewhat less than Black--White differences. [ABSTRACT FROM AUTHOR]

A variety of recent articles in the personnel selection literature have used analyses of meta-analytically derived matrices to draw general conclusions for the field. The purpose of this article is to construct a matrix that incorporates as complete information as possible on the relationships among cognitive ability measures, three sets of alternative predictors, and job performance. We build upon a starting matrix used by Schmitt, Rodgers, Chan, Sheppard, and Jennings (1997). Mean differences, by race, for each of the measures and the potential for adverse impact of predictor composites are also considered. We demonstrate that the use of alternative predictors alone to predict job performance (in the absence of cognitive ability) lowers the potential for adverse impact. However, in contrast to recent claims, adverse impact continues to occur at many commonly used selection ratios. Future researchers are encouraged to use our matrix and to expand upon it as new primary research becomes available. We also report and reaffirm many methodological lessons along the way, including the many judgment calls that appear in an effort of this magnitude and a reminder that the field could benefit from even greater conceptual care regarding what is labeled an "alternative predictor." Directions for future meta-analyses and for future primary research activities are also derived. [ABSTRACT FROM AUTHOR]

Examines the predictive power of panel interview and pre-employment tests. Prediction of job performance and promotions; Validity of selection interviews; Description of predictors and criteria.

This article reviews the use of missing data techniques (MDTs) for problems in applied psychology and provides several recommendations for missing data researchers (MDRs). Researchers need to develop taxonomies of missing data. They need to determine where the losses occur and why. Research is needed to investigate the impact of MDR on the decisions or interpretations that follow the use of MDTs. Relevant professional organizations might examine the research in this area and draft guidelines for researchers and practitioners to help them make choices among the various MDTs in their own research.

This research describes a new approach to the measurement and enhancement of organizational productivity entitled the Productivity Measurement and Enhancement System (ProMES). Various conceptual issues regarding productivity are briefly reviewed and the system is described. Research is reported in which productivity measures using this approach were developed and evaluated in five organizational units. The measures from the system were then used to give group-level feedback, followed by group goal setting and group incentives. Results indicated that unit personnel were cooperative in developing and using the system and that the system had good psychometric characteristics. Feedback increased productivity 50%. and feedback with the addition of goal setting and incentives increased productivity 15% and 76%, respectively. The approach to measuring productivity has several unique features. For example, it combines all the functions of a unit into a single index of productivity that reflects differential importance, allows for direct comparison of different units, identifies priorities for increasing productivity, and provides for aggregating measures to higher levels of the organization. Suggestions for future research and additional applications of this approach are offered. [ABSTRACT FROM AUTHOR]