Search

A potential voter must incur a number of costs in order to successfully cast an in-person ballot, including the costs associated with identifying and traveling to a polling place. In order to investigate how these costs affect voter turnout, we introduce two quasi-experimental designs that can be used to study how the political participation of registered voters is affected by differences in the relative distance that registrants must travel to their assigned Election Day polling place and whether their polling place remains at the same location as in a previous election. Our designs make comparisons of registrants who live on the same residential block, but are assigned to vote at different polling places. We find that living farther from a polling place and being assigned to a new polling place reduce in-person Election Day voting, but that registrants largely offset for this by casting more early in-person and mail ballots.

As the Web has become integrated into daily life, understanding how individuals spend their time online impacts domains ranging from public policy to marketing. It is difficult, however, to measure even simple aspects of browsing behavior via conventional methods---including surveys and site-level analytics---due to limitations of scale and scope. In part addressing these limitations, large-scale Web panel data are a relatively novel means for investigating patterns of Internet usage. In one of the largest studies of browsing behavior to date, we pair Web histories for 250,000 anonymized individuals with user-level demographics---including age, sex, race, education, and income---to investigate three topics. First, we examine how behavior changes as individuals spend more time online, showing that the heaviest users devote nearly twice as much of their time to social media relative to typical individuals. Second, we revisit the digital divide, finding that the frequency with which individuals turn to the Web for research, news, and healthcare is strongly related to educational background, but not as closely tied to gender and ethnicity. Finally, we demonstrate that browsing histories are a strong signal for inferring user attributes, including ethnicity and household income, a result that may be leveraged to improve ad targeting.

Past studies have found that racial and ethnic minorities are more likely than White drivers to be pulled over by the police for alleged traffic infractions, including a combination of speeding and equipment violations. It has been difficult, though, to measure the extent to which these disparities stem from discriminatory enforcement rather than from differences in offense rates. Here, in the context of speeding enforcement, we address this challenge by leveraging a novel source of telematics data, which include second-by-second driving speed for hundreds of thousands of individuals in 10 major cities across the United States. We find that time spent speeding is approximately uncorrelated with neighborhood demographics, yet, in several cities, officers focused speeding enforcement in small, demographically nonrepresentative areas. In some cities, speeding enforcement was concentrated in predominantly non-White neighborhoods, while, in others, enforcement was concentrated in predominately White neighborhoods. Averaging across the 10 cities we examined, and adjusting for observed speeding behavior, we find that speeding enforcement was moderately more concentrated in non-White neighborhoods. Our results show that current enforcement practices can lead to inequities across race and ethnicity.

Summary Judges, doctors and managers are among those decision makers who must often choose a course of action under limited time, with limited knowledge and without the aid of a computer. Because data-driven methods typically outperform unaided judgements, resource-constrained practitioners can benefit from simple, statistically derived rules that can be applied mentally. In this work, we formalize long-standing observations about the efficacy of improper linear models to construct accurate yet easily applied rules. To test the performance of this approach, we conduct a large-scale evaluation in 22 domains and focus in detail on one: judicial decisions to release or detain defendants while they await trial. In these domains, we find that simple rules rival the accuracy of complex prediction models that base decisions on considerably more information. Further, comparing with unaided judicial decisions, we find that simple rules substantially outperform the human experts. To conclude, we present an analytical framework that sheds light on why simple rules perform as well as they do.

We assessed racial disparities in policing in the United States by compiling and analysing a dataset detailing nearly 100 million traffic stops conducted across the country. We found that black drivers were less likely to be stopped after sunset, when a ‘veil of darkness’ masks one’s race, suggesting bias in stop decisions. Furthermore, by examining the rate at which stopped drivers were searched and the likelihood that searches turned up contraband, we found evidence that the bar for searching black and Hispanic drivers was lower than that for searching white drivers. Finally, we found that legalization of recreational marijuana reduced the number of searches of white, black and Hispanic drivers—but the bar for searching black and Hispanic drivers was still lower than that for white drivers post-legalization. Our results indicate that police stops and search decisions suffer from persistent racial bias and point to the value of policy interventions to mitigate these disparities. Pierson et al. compiled and analysed a dataset detailing nearly 100 million traffic stops conducted across the United States, finding evidence of persistent racial bias against black and Hispanic drivers in police stops and search decisions.

Significance Automated speech recognition (ASR) systems are now used in a variety of applications to convert spoken language to text, from virtual assistants, to closed captioning, to hands-free computing. By analyzing a large corpus of sociolinguistic interviews with white and African American speakers, we demonstrate large racial disparities in the performance of five popular commercial ASR systems. Our results point to hurdles faced by African Americans in using increasingly widespread tools driven by speech recognition technology. More generally, our work illustrates the need to audit emerging machine-learning systems to ensure they are broadly inclusive., Automated speech recognition (ASR) systems, which use sophisticated machine-learning algorithms to convert spoken language to text, have become increasingly widespread, powering popular virtual assistants, facilitating automated closed captioning, and enabling digital dictation platforms for health care. Over the last several years, the quality of these systems has dramatically improved, due both to advances in deep learning and to the collection of large-scale datasets used to train the systems. There is concern, however, that these tools do not work equally well for all subgroups of the population. Here, we examine the ability of five state-of-the-art ASR systems—developed by Amazon, Apple, Google, IBM, and Microsoft—to transcribe structured interviews conducted with 42 white speakers and 73 black speakers. In total, this corpus spans five US cities and consists of 19.8 h of audio matched on the age and gender of the speaker. We found that all five ASR systems exhibited substantial racial disparities, with an average word error rate (WER) of 0.35 for black speakers compared with 0.19 for white speakers. We trace these disparities to the underlying acoustic models used by the ASR systems as the race gap was equally large on a subset of identical phrases spoken by black and white individuals in our corpus. We conclude by proposing strategies—such as using more diverse training datasets that include African American Vernacular English—to reduce these performance differences and ensure speech recognition technology is inclusive.

In domains ranging from computer vision to natural language processing, machine learning models have been shown to exhibit stark disparities, often performing worse for members of traditionally underserved groups. One factor contributing to these performance gaps is a lack of representation in the data the models are trained on. It is often unclear, however, how to operationalize representativeness in specific applications. Here we formalize the problem of creating equitable training datasets, and propose a statistical framework for addressing this problem. We consider a setting where a model builder must decide how to allocate a fixed data collection budget to gather training data from different subgroups. We then frame dataset creation as a constrained optimization problem, in which one maximizes a function of group-specific performance metrics based on (estimated) group-specific learning rates and costs per sample. This flexible approach incorporates preferences of model-builders and other stakeholders, as well as the statistical properties of the learning task. When data collection decisions are made sequentially, we show that under certain conditions this optimization problem can be efficiently solved even without prior knowledge of the learning rates. To illustrate our approach, we conduct a simulation study of polygenic risk scores on synthetic genomic data -- an application domain that often suffers from non-representative data collection. We find that our adaptive sampling strategy outperforms several common data collection heuristics, including equal and proportional sampling, demonstrating the value of strategic dataset design for building equitable models., 15 pages, 2 figures

Background: The transversus abdominis plane block (TAP block) in the petit triangle is being used since 2001 for analgesia by blocking the T6 to T12 nerves, which is devoid of the sympathetic blockade and has opioid-sparing effect during and after abdominal operations. The appropriateness and efficacy of using double pop blind transversus abdominis plane block were studied in abdominal surgeries using bupivacaine with butorphanol as additive. Subjects and Methods: A total of 78 adult patients of ASA I and II were included, who were to undergo laparoscopic cholecystectomy under general anesthesia. The control group patients received tramadol, diclofenac, and paracetamol in the perioperative period. The study group patients received TAP block by double pop blind technique after induction of anesthesia but before surgical incision as preemptive analgesia. The p-value, the mean and the confidence interval were calculated by using Student t-test with the use of online software by graphpad.com. Results: Each of the two groups had 39 patients, and none met the exclusion criteria. Patients of the study (TAP) group remained pain-free for a longer time by 439 (416 – 463) minutes more than the control (IV, Intravenous) group. Rescue analgesia in the study group was required 640 minutes after the end of the surgery, but in the control group, rescue analgesia was required earlier at 200 minutes only after the surgery. Conclusion: Double pop blind technique for TAP block is appropriate and without complications if done with carefulness as to avoid penetration of the blunted green needle beyond the fascia between the internal oblique and the transversus abdominis muscle.

Jurisdictions across the country, including the federal government through its recently enacted First Step Act, have begun using statistical algorithms (also called “instruments”) to help determine an arrestee’s or an offender’s risk of reoffending. These risk assessment instruments (RAIs) might be used at a number of points in the criminal process, including at the front-end by judges to impose a sentence after conviction, at the back-end by parole boards to make decisions about prison release, or in between these two points by correctional authorities determining the optimal security and service arrangements for an offender. At the pretrial stage, RAIs might come into play at the time of the bail or pretrial detention determination by a judge, which usually takes place shortly after arrest. The increased use of RAIs in the criminal justice system has given rise to several criticisms. RAIs are said to be no more accurate than clinical assessments, racially biased, lacking in transparency and, because of their quantitative nature, dehumanizing. This chapter critically examines a number of these concerns. It also highlights how the law has, and should, respond to these issues.

A new virus (SARS-CoV-2) or COVID-19, was identified in China in late 2019 and since then it has spread across the Globe. On March 11, 2020, the World Health Organization (WHO) declared it as a pandemic. On 24th March, India imposed a three-week long nationwide lockdown for its 1.3 billion people to curb the spread of the coronavirus pandemic [1]. This was quoted as the biggest isolation effort taken by any government at that point of time. With a lot of extensions for the lockdown phase in between, the nationwide lockdown was gradually and finally lifted by 1st of June with multiple government guidelines for living with the new normal. This study aimed at measuring the efficacy of the nationwide lockdown by observing the growth rate of daily confirmed and active cases in pre-Iockdown, during lockdown and post lockdown stages. The study also statistically reviewed the progress of the disease at a state level, especially for the top 5 affected states of India, to comment over the lockdown's effectiveness in those states. Statistical analysis of confirmed COVID-19 cases reported between January 30, 2020 and July 24, 2020. ‘Lockdown’ to impose strict social distancing measures was effective in reducing incidence of spread in India. Country level lockdown has restricted the growth rate of spread of virus and provided initial benefits to the healthcare infrastructure of states. However, augmentation of healthcare infrastructure to an appropriate level is key to continue gain from lockdown measures.

Statistical algorithms can outperform human predictions of recidivism., Dressel and Farid recently found that laypeople were as accurate as statistical algorithms in predicting whether a defendant would reoffend, casting doubt on the value of risk assessment tools in the criminal justice system. We report the results of a replication and extension of Dressel and Farid’s experiment. Under conditions similar to the original study, we found nearly identical results, with humans and algorithms performing comparably. However, algorithms beat humans in the three other datasets we examined. The performance gap between humans and algorithms was particularly pronounced when, in a departure from the original study, participants were not provided with immediate feedback on the accuracy of their responses. Algorithms also outperformed humans when the information provided for predictions included an enriched (versus restricted) set of risk factors. These results suggest that algorithms can outperform human predictions of recidivism in ecologically valid settings.

It is well known among researchers and practitioners that election polls suffer from a variety of sampling and nonsampling errors, often collectively referred to as total survey error. Reported mar...

In an effort to bring greater efficiency, equity, and transparency to the criminal justice system, statistical risk assessment tools are increasingly used to inform bail, sentencing, and parole decisions. We examine New York City's stop-and-frisk program, and propose two new use cases for personalized risk assessments. First, we show that risk assessment tools can help police officers make considerably better real-time stop decisions. Second, we show that such tools can help audit past actions; in particular, we argue that a sizable fraction of police stops were conducted on the basis of little evidence, in possible violation of constitutional protections.

It is widely thought that news organizations exhibit ideological bias, but rigorously quantifying such slant has proven methodologically challenging. Through a combination of machine learning and crowdsourcing techniques, we investigate the selection and framing of political issues in 15 major U.S. news outlets. Starting with 803,146 news stories published over 12 months, we first used supervised learning algorithms to identify the 14% of articles pertaining to political events. We then recruited 749 online human judges to classify a random subset of 10,950 of these political articles according to topic and ideological position. Our analysis yields an ideological ordering of outlets consistent with prior work. We find, however, that news outlets are considerably more similar than generally believed. Specifically, with the exception of political scandals, we find that major news organizations present topics in a largely non-partisan manner, casting neither Democrats nor Republicans in a particularly favorable or unfavorable light. Moreover, again with the exception of political scandals, there is little evidence of systematic differences in story selection, with all major news outlets covering a wide variety of topics with frequency largely unrelated to the outlet's ideological position. Finally, we find that news organizations express their ideological bias not by directly advocating for a preferred political party, but rather by disproportionately criticizing one side, a convention that further moderates overall differences.

Viral products and ideas are intuitively understood to grow through a person-to-person diffusion process analogous to the spread of an infectious disease; however, until recently it has been prohibitively difficult to directly observe purportedly viral events, and thus to rigorously quantify or characterize their structural properties. Here we propose a formal measure of what we label “structural virality” that interpolates between two conceptual extremes: content that gains its popularity through a single, large broadcast and that which grows through multiple generations with any one individual directly responsible for only a fraction of the total adoption. We use this notion of structural virality to analyze a unique data set of a billion diffusion events on Twitter, including the propagation of news stories, videos, images, and petitions. We find that across all domains and all sizes of events, online diffusion is characterized by surprising structural diversity; that is, popular events regularly grow via both broadcast and viral mechanisms, as well as essentially all conceivable combinations of the two. Nevertheless, we find that structural virality is typically low, and remains so independent of size, suggesting that popularity is largely driven by the size of the largest broadcast. Finally, we attempt to replicate these findings with a model of contagion characterized by a low infection rate spreading on a scale-free network. We find that although several of our empirical findings are consistent with such a model, it fails to replicate the observed diversity of structural virality, thereby suggesting new directions for future modeling efforts. This paper was accepted by Lorin Hitt, information systems.

There has been a marked shift towards learning and consuming information through video. Most academic research, however, is still distributed only in text form, as researchers often have limited time, resources, and incentives to create video versions of their work. To address this gap, we propose, deploy, and evaluate a scalable, end-to-end system for crowdsourcing the creation of short, 5-minute research videos based on academic papers. Doing so requires solving complex coordination and collaborative video production problems. To assist coordination, we designed a structured workflow that enables efficient delegation of tasks, while also motivating the crowd through a collaborative learning environment. To facilitate video production, we developed an online tool with which groups can make micro-audio recordings that are automatically stitched together to create a complete talk. We tested this approach with a group of volunteers recruited from 52 countries through an open call. This distributed crowd produced over 100 video talks in 12 languages based on papers from top-tier computer science conferences. The produced talks consistently received high ratings from a diverse group of non-experts and experts, including the authors of the original papers. These results indicate that our crowdsourcing approach is a promising method for producing high-quality research talks at scale, increasing the distribution and accessibility of scientific knowledge.

Election forecasts have traditionally been based on representative polls, in which randomly sampled individuals are asked for whom they intend to vote. While representative polling has historically proven to be quite eective, it comes at considerable nancial and time costs. Moreover, as response rates have declined over the past several decades, the statistical benets of representative sampling have diminished. In this paper, we show that with proper statistical adjustment, non-representative polls can be used to generate accurate election forecasts, and often faster and at less expense than traditional survey methods. We demonstrate this approach by creating forecasts from a novel and highly non-representative survey dataset: a series of daily voter intention polls for the 2012 presidential election conducted on the Xbox gaming platform. After adjusting the Xbox responses via multilevel regression and poststratication , we obtain estimates in line with forecasts from leading poll analysts, which were based on aggregating hundreds of traditional polls conducted during the election cycle. We conclude by arguing that non-representative polling shows promise not only for election forecasting, but also for measuring public opinion on a broad range of social, economic and cultural issues.

In a response to our article “Political Ideology and Racial Preferences in Online Dating,” Kevin Lewis (2015) has raised concerns about various aspects of our analysis of user behavior on a national online dating site. The core of Lewis’ critique is that we do not report specifics of the sample, the website, and the analysis. Lewis also details numerous smaller points with respect to details of the analysis. The goal of this response is not to reply to every point that Lewis makes. As with any research project, our design decisions require tradeoffs, and readers should evaluate the evidence on their own. Rather, we use this space to make broader points about the general value for sociology of research using online data and how such research should be practiced.

Outcome tests are a popular method for detecting bias in lending, hiring, and policing decisions. These tests operate by comparing the success rate of decisions across groups. For example, if loans made to minority applicants are observed to be repaid more often than loans made to whites, it suggests that only exceptionally qualified minorities are granted loans, indicating discrimination. Outcome tests, however, are known to suffer from the problem of infra-marginality: even absent discrimination, the repayment rates for minority and white loan recipients might differ if the two groups have different risk distributions. Thus, at least in theory, outcome tests can fail to accurately detect discrimination. We develop a new statistical test of discrimination---the threshold test---that mitigates the problem of infra-marginality by jointly estimating decision thresholds and risk distributions via a hierarchical Bayesian latent variable model. Applying our test to a dataset of 4.5 million police stops in North Carolina, we find that the problem of infra-marginality is more than a theoretical possibility, and can cause the outcome test to yield misleading results in practice., To appear in The Annals of Applied Statistics, 2017

From doctors diagnosing patients to judges setting bail, experts often base their decisions on experience and intuition rather than on statistical models. While understandable, relying on intuition over models has often been found to result in inferior outcomes. Here we present a new method-select-regress-and-round-for constructing simple rules that perform well for complex decisions. These rules take the form of a weighted checklist, can be applied mentally, and nonetheless rival the performance of modern machine learning algorithms. Our method for creating these rules is itself simple, and can be carried out by practitioners with basic statistics knowledge. We demonstrate this technique with a detailed case study of judicial decisions to release or detain defendants while they await trial. In this application, as in many policy settings, the effects of proposed decision rules cannot be directly observed from historical data: if a rule recommends releasing a defendant that the judge in reality detained, we do not observe what would have happened under the proposed action. We address this key counterfactual estimation problem by drawing on tools from causal inference. We find that simple rules significantly outperform judges and are on par with decisions derived from random forests trained on all available features. Generalizing to 22 varied decision-making domains, we find this basic result replicates. We conclude with an analytical framework that helps explain why these simple decision rules perform as well as they do.

Algorithms are now regularly used to decide whether defendants awaiting trial are too dangerous to be released back into the community. In some cases, black defendants are substantially more likely than white defendants to be incorrectly classified as high risk. To mitigate such disparities, several techniques recently have been proposed to achieve algorithmic fairness. Here we reformulate algorithmic fairness as constrained optimization: the objective is to maximize public safety while satisfying formal fairness constraints designed to reduce racial disparities. We show that for several past definitions of fairness, the optimal algorithms that result require detaining defendants above race-specific risk thresholds. We further show that the optimal unconstrained algorithm requires applying a single, uniform threshold to all defendants. The unconstrained algorithm thus maximizes public safety while also satisfying one important understanding of equality: that all individuals are held to the same standard, irrespective of race. Because the optimal constrained and unconstrained algorithms generally differ, there is tension between improving public safety and satisfying prevailing notions of algorithmic fairness. By examining data from Broward County, Florida, we show that this trade-off can be large in practice. We focus on algorithms for pretrial release decisions, but the principles we discuss apply to other domains, and also to human decision makers carrying out structured decision rules., Comment: To appear in Proceedings of KDD'17

What explains the relative persistence of same-race romantic relationships? One possible explanation is structural–this phenomenon could reflect the fact that social interactions are already stratified along racial lines–while another attributes these patterns to individual-level preferences. We present novel evidence from an online dating community involving more than 250,000 people in the United States about the frequency with which individuals both express a preference for same-race romantic partners and act to choose same-race partners. Prior work suggests that political ideology is an important correlate of conservative attitudes about race in the United States, and we find that conservatives, including both men and women and blacks and whites, are much more likely than liberals to state a preference for same-race partners. Further, conservatives are not simply more selective in general; they are specifically selective with regard to race. Do these stated preferences predict real behaviors? In general, we find that stated preferences are a strong predictor of a behavioral preference for same-race partners, and that this pattern persists across ideological groups. At the same time, both men and women of all political persuasions act as if they prefer same-race relationships even when they claim not to. As a result, the gap between conservatives and liberals in revealed same-race preferences, while still substantial, is not as pronounced as their stated attitudes would suggest. We conclude by discussing some implications of our findings for the broader issues of racial homogamy and segregation.

With the availability of social network data, it has become possible to relate the behavior of individuals to that of their acquaintances on a large scale. Although the similarity of connected individuals is well established, it is unclear whether behavioral predictions based on social data are more accurate than those arising from current marketing practices. We employ a communications network of over 100 million people to forecast highly diverse behaviors, from patronizing an off-line department store to responding to advertising to joining a recreational league. Across all domains, we find that social data are informative in identifying individuals who are most likely to undertake various actions, and moreover, such data improve on both demographic and behavioral models. There are, however, limits to the utility of social data.In particular, when rich transactional data were available, social data did little to improve prediction.

Recent studies have examined racial disparities in stop-and-frisk, a widely employed but controversial policing tactic. The statistical evidence, however, has been limited and contradictory. We investigate by analyzing three million stops in New York City over five years, focusing on cases where officers suspected the stopped individual of criminal possession of a weapon (CPW). For each CPW stop, we estimate the ex ante probability that the detained suspect has a weapon. We find that in more than 40% of cases, the likelihood of finding a weapon (typically a knife) was less than 1%, raising concerns that the legal requirement of “reasonable suspicion” was often not met. We further find that blacks and Hispanics were disproportionately stopped in these low hit rate contexts, a phenomenon that we trace to two factors: (1) lower thresholds for stopping individuals—regardless of race—in high-crime, predominately minority areas, particularly public housing; and (2) lower thresholds for stopping minorities relative to similarly situated whites. Finally, we demonstrate that by conducting only the 6% of stops that are statistically most likely to result in weapons seizure, one can both recover the majority of weapons and mitigate racial disparities in who is stopped. We show that this statistically informed stopping strategy can be approximated by simple, easily implemented heuristics with little loss in efficiency.

Machine learning is used increasingly in clinical care to improve diagnosis, treatment selection, and health system efficiency. Because machine-learning models learn from historically collected data, populations that have experienced human and structural biases in the past—called protected groups—are vulnerable to harm by incorrect predictions or withholding of resources. This article describes how model design, biases in data, and the interactions of model predictions with clinicians and patients may exacerbate health care disparities. Rather than simply guarding against these harms passively, machine-learning systems should be used proactively to advance health equity. For that goal to be achieved, principles of distributive justice must be incorporated into model design, deployment, and evaluation. The article describes several technical implementations of distributive justice—specifically those that ensure equality in patient outcomes, performance, and resource allocation—and guides clinicians as to when they should prioritize each principle. Machine learning is providing increasingly sophisticated decision support and population-level monitoring, and it should encode principles of justice to ensure that models benefit all patients.

It is often asserted that friends and acquaintances have more similar beliefs and attitudes than do strangers; yet empirical studies disagree over exactly how much diversity of opinion exists within local social networks and, relatedly, how much awareness individuals have of their neighbors' views. This article reports results from a network survey, conducted on the Facebook social networking platform, in which participants were asked about their own political attitudes, as well as their beliefs about their friends' attitudes. Although considerable attitude similarity exists among friends, the results show that friends disagree more than they think they do. In particular, friends are typically unaware of their disagreements, even when they say they discuss the topic, suggesting that discussion is not the primary means by which friends infer each other's views on particular issues. Rather, it appears that respondents infer opinions in part by relying on stereotypes of their friends and in part by projecting their own views. The resulting gap between real and perceived agreement may have implications for the dynamics of political polarization and theories of social influence in general.

Respondent-driven sampling (RDS) is a recently introduced, and now widely used, technique for estimating disease prevalence in hidden populations. RDS data are collected through a snowball mechanism, in which current sample members recruit future sample members. In this paper we present RDS as Markov chain Monte Carlo importance sampling, and we examine the effects of community structure and the recruitment procedure on the variance of RDS estimates. Past work has assumed that the variance of RDS estimates is primarily affected by segregation between healthy and infected individuals. We examine an illustrative model to show that this is not necessarily the case, and that bottlenecks anywhere in the networks can substantially affect estimates. We also show that variance is inflated by a common design feature in which the sample members are encouraged to recruit multiple future sample members. The paper concludes with suggestions for implementing and evaluating RDS studies.

Recent studies have examined racial disparities in stop-and-frisk, a widely employed but controversial policing tactic. The statistical evidence, however, has been limited and contradictory. We investigate by analyzing three million stops in New York City over five years, focusing on cases where officers suspected the stopped individual of criminal possession of a weapon (CPW). For each CPW stop, we estimate the ex ante probability that the detained suspect has a weapon. We find that in more than 40% of cases, the likelihood of finding a weapon (typically a knife) was less than 1%, raising concerns that the legal requirement of “reasonable suspicion” was often not met. We further find that blacks and Hispanics were disproportionately stopped in these low hit rate contexts, a phenomenon that we trace to two factors: (1) lower thresholds for stopping individuals — regardless of race — in high-crime, predominately minority areas, particularly public housing; and (2) lower thresholds for stopping minorities relative to similarly situated whites. Finally, we demonstrate that by conducting only the 6% of stops that are statistically most likely to result in weapons seizure, one can both recover the majority of weapons and mitigate racial disparities in who is stopped. We show that this statistically informed stopping strategy can be approximated by simple, easily implemented heuristics with little loss in efficiency.

Summary Steroids and calcineurin inhibitors (CNI) have been mainstays of immunosuppression but both have numerous side effects that are associated with substantial morbidity and mortality. This study was carried out to determine if steroids can be eliminated with early discontinuation of cyclosporine A (CsA) and later discontinuation of mycophenolate mofetil (MMF). Ninety-six patients with kidney transplants were entered into four subgroups of two pilot studies. All patients received Thymoglobulin® induction, rapamycin (RAPA), and the immunonutrients arginine and an oil containing ω-3 fatty acids. Mycophenolate mofetil was started in standard doses and discontinued by 2 years. CsA was given in reduced doses for either 4, 6, or 12 months. Follow-up was 12–36 months. Thirteen first rejection episodes occurred during the first year (14%). Combining all patients, 86% were rejection-free at 1 year, 80% at 2 years and 79% at 3 years. No kidney has been lost to acute rejection. Ninety percent of the 84 patients at risk at the end of the study were steroid-free and 87% were off CNI. Fifty-seven percent of 54 patients with a functioning kidney at 3 years were receiving monotherapy with RAPA. We conclude that this therapeutic strategy is worthy of a prospective multi-center clinical trial.

Logarithmic Sobolev inequalities are a well-studied technique for estimating rates of convergence of Markov chains to their stationary distributions. In contrast to continuous state spaces, discrete settings admit several distinct log Sobolev inequalities, one of which is the subject of this paper. Here we derive modified log Sobolev inequalities for some models of random walk, including the random transposition shuffle and the top-random transposition shuffle on S n , and the walk generated by 3-cycles on A n . As an application, we derive concentration inequalities for these models.

Respondent Driven Sampling (RDS) is a novel variant of link tracing sampling that has primarily been used to estimate the characteristics of hard-to-reach groups, such as the HIV prevalence of drug users.1 ‘Seeds’ are selected by convenience from a population of interest (target population) and given coupons. Seeds then use these coupons to recruit other people, who themselves become recruiters. Recruits are given compensation, usually money, for taking part in the survey and also an incentive for recruiting others. This process continues in recruitment ‘waves’ until the survey is stopped. Estimation methods are then applied to account for the biased recruitment, for example, the presumed over-recruitment of people with more acquaintances, in an attempt to generate estimates for the underlying population. RDS has quickly become popular and relied on by major public health organisations, including the US Centers for Disease Control and Prevention and Family Health International, chiefly because it is often found to be an efficient method of recruitment in hard-to-reach groups, but also because of the availability of custom written software incorporating inference methods that are designed to generate estimates that are representative of the wider population of interest, despite the biased sampling. As demonstrated by RDS's popularity,1 there was a clear need for new methods of data collection on hard-to-reach groups. However, RDS has not been without its critics. Its reliance on the target population for recruitment introduced ethicalw1 and sampling concerns.w2 If RDS estimates are overly biased or the variance is unacceptably high, then RDS will be little more than another method of convenience sampling. If these errors can be minimised however, then RDS has the potential to become a very useful survey methodology. In this editorial we highlight that ‘RDS’ includes both data collection and statistical inference methods, discuss the limitations …

Online social networking platforms regularly support hundreds of millions of users, who in aggregate generate substantially more data than can be stored on any single physical server. As such, user data are distributed, or sharded, across many machines. A key requirement in this setting is rapid retrieval not only of a given user's information, but also of all data associated with his or her social contacts, suggesting that one should consider the topology of the social network in selecting a sharding policy. In this paper we formalize the problem of efficiently sharding large social network databases, and evaluate several sharding strategies, both analytically and empirically. We find that random sharding---the de facto standard---results in provably poor performance even when frequently accessed nodes are replicated to many shards. By contrast, we demonstrate that one can substantially reduce querying costs by identifying and assigning tightly knit communities to shards. In particular, our theoretical analysis motivates a novel, scalable sharding algorithm that outperforms both random and location-based sharding schemes.

Carbamylated hemoglobin: A potential marker for the adequacy of hemodialysis therapy in end-stage renal failure. Urea can dissociate in vivo to form isocyanic acid which can react with hemoglobin to form carbamylated hemoglobin. Previous work has shown that formation of carbamylated hemoglobin depends upon both the severity and the duration of renal failure. To determine whether carbamylated hemoglobin can be used as an assessment of the adequacy of hemodialysis treatment, we prospectively studied 55 stable patients who regularly attended our hospital dialysis program. Carbamylated hemoglobin was greater in those patients with a Kt/V of ≤ 1.1 compared to those with a Kt/V of > 1.1 (120 ± 8 µgVH/gHb versus 99 ± 7, P < 0.01), and there was a negative correlation with Kt/V (r = -0.37, P = 0.007). There were positive correlations between carbamylated hemoglobin and the time-averaged urea concentration (r = 0.4, P = 0.004), and a negative correlation with the urea reduction ratio (r = -0.37, P = 0.01). Carbamylated hemoglobin may therefore be a useful marker of the degree of uremia, just as glycosylated hemoglobin is used in the assessment of patients with diabetes mellitus.

Models of networked diffusion that are motivated by analogy with the spread of infectious disease have been applied to a wide range of social and economic adoption processes, including those related to new products, ideas, norms and behaviors. However, it is unknown how accurately these models account for the empirical structure of diffusion over networks. Here we describe the diffusion patterns arising from seven online domains, ranging from communications platforms to networked games to microblogging services, each involving distinct types of content and modes of sharing. We find strikingly similar patterns across all domains.In particular, the vast majority of cascades are small, and are described by a handful of simple tree structures that terminate within one degree of an initial adopting "seed." In addition we find that structures other than these account for only a tiny fraction of total adoptions; that is, adoptions resulting from chains of referrals are extremely rare. Finally, even for the largest cascades that we observe, we find that the bulk of adoptions often takes place within one degree of a few dominant individuals. Together, these observations suggest new directions for modeling of online adoption processes.

Many factors can affect interpretation of glycohemoglobin (GHB/HbA1c) measurements in patients with chronic renal failure (CRF). Several reports have suggested that erythrocyte survival is substantially lowered in most patients with CRF; this would be expected to lower GHB results (1)(2)(3)(4)(5)(6). Several reports have also suggested that GHB methods, especially those based on charge separation (e.g., ion-exchange HPLC), may have interference by carbamylated hemoglobin that would be expected to falsely increase GHB results (7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17). Many of these reports evaluated older assay methods; newer ion-exchange methods may show improved separation of the HbA1c fraction from other hemoglobin adducts (15)(17). Because renal failure is common in patients with diabetes and GHB is widely used as an index of mean blood glucose in these patients, we examined GHB results in patients with CRF by several different GHB assay methods. We also investigated the impact of shortened erythrocyte lifespan by comparing the GHB results obtained for nondiabetic patients with and without CRF. Fifty-five patients with CRF (blood urea nitrogen >400 mg/L) were recruited for this study. Twenty-nine were not receiving dialysis and …

Recent work has demonstrated that Web search volume can “predict the present,” meaning that it can be used to accurately track outcomes such as unemployment levels, auto and home sales, and disease prevalence in near real time. Here we show that what consumers are searching for online can also predict their collective future behavior days or even weeks in advance. Specifically we use search query volume to forecast the opening weekend box-office revenue for feature films, first-month sales of video games, and the rank of songs on the Billboard Hot 100 chart, finding in all cases that search counts are highly predictive of future outcomes. We also find that search counts generally boost the performance of baseline models fit on other publicly available data, where the boost varies from modest to dramatic, depending on the application in question. Finally, we reexamine previous work on tracking flu trends and show that, perhaps surprisingly, the utility of search data relative to a simple autoregressive model is modest. We conclude that in the absence of other data sources, or where small improvements in predictive performance are material, search queries provide a useful guide to the near future.

Respondent-driven sampling (RDS) is a network-based technique for estimating traits in hard-to-reach populations, for example, the prevalence of HIV among drug injectors. In recent years RDS has been used in more than 120 studies in more than 20 countries and by leading public health organizations, including the Centers for Disease Control and Prevention in the United States. Despite the widespread use and growing popularity of RDS, there has been little empirical validation of the methodology. Here we investigate the performance of RDS by simulating sampling from 85 known, network populations. Across a variety of traits we find that RDS is substantially less accurate than generally acknowledged and that reported RDS confidence intervals are misleadingly narrow. Moreover, because we model a best-case scenario in which the theoretical RDS sampling assumptions hold exactly, it is unlikely that RDS performs any better in practice than in our simulations. Notably, the poor performance of RDS is driven not by the bias but by the high variance of estimates, a possibility that had been largely overlooked in the RDS literature. Given the consistency of our results across networks and our generous sampling conditions, we conclude that RDS as currently practiced may not be suitable for key aspects of public health surveillance where it is now extensively applied.

In sponsored search auctions advertisers typically pay a fixed amount per click that their advertisements receive. In particular, the advertiser and the publisher enter into a contract (e.g., the publisher displays the ad; the advertiser pays the publisher 10 cents per click), and each party's subjective value for such a contract depends on their estimated click-through rates (CTR) for the ad. Starting from this motivating example, we define and analyze a class of contract auctions that generalize the classical second price auction. As an application, we introduce impression-plus-click pricing for sponsored search, in which advertisers pay a fixed amount per impression plus an additional amount if their ad is clicked. Of note, when the advertiser's estimated CTR is higher than the publisher's estimated CTR, both parties find negative click payments advantageous, where the advertiser pays the publisher a premium for the impression but the publisher then pays the advertiser per click.

In a prediction market, agents trade assets whose value is tied to a future event, for example the outcome of the next presidential election. Asset prices determine a probability distribution over the set of possible outcomes. Typically, the outcome space is small, allowing agents to directly trade in each outcome, and allowing a market maker to explicitly update asset prices. Combinatorial markets, in contrast, work to estimate a full joint distribution of dependent observations, in which case the outcome space grows exponentially. In this paper, we consider the problem of pricing combinatorial markets for single-elimination tournaments. With $n$ competing teams, the outcome space is of size 2n-1. We show that the general pricing problem for tournaments is P-hard. We derive a polynomial-time algorithm for a restricted betting language based on a Bayesian network representation of the probability distribution. The language is fairly natural in the context of tournaments, allowing for example bets of the form "team i wins game k". We believe that our betting language is the first for combinatorial market makers that is both useful and tractable. We briefly discuss a heuristic approximation technique for the general case.

A deck of $n$ cards is shuffled by repeatedly moving the top card to one of the bottom $k_n$ positions uniformly at random. We give upper and lower bounds on the total variation mixing time for this shuffle as $k_n$ ranges from a constant to $n$. We also consider a symmetric variant of this shuffle in which at each step either the top card is randomly inserted into the bottom $k_n$ positions or a random card from the bottom $k_n$ positions is moved to the top. For this reversible shuffle we derive bounds on the $L^2$ mixing time. Finally, we transfer mixing time estimates for the above shuffles to the lazy top to bottom-$k$ walks that move with probability 1/2 at each step., Published at http://dx.doi.org/10.1214/10505160500000062 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

On complete, non-compact manifolds and infinite graphs, Faber-Krahn inequalities have been used to estimate the rate of decay of the heat kernel. We develop this technique in the setting of finite Markov chains, proving upper and lower mixing time bounds via the spectral profile. This approach lets us recover and refine previous conductance-based bounds of mixing time (including the Morris-Peres result), and in general leads to sharper estimates of convergence rates. We apply this method to several models including groups with moderate growth, the fractal-like Viscek graphs, and the torus, to obtain tight bounds on the corresponding mixing times., 26 pages

Respondent-driven sampling is a network-based technique for estimating traits in hard-to-reach populations, for example, the prevalence of HIV among drug injectors. In recent years RDS has been used in more than 120 studies in more than 20 countries, and by leading public health organisations, including the Centers for Disease Control and Prevention in the USA. Despite the widespread use and growing popularity of RDS, there has been little empirical validation of the methodology. In this talk, I investigate the performance of RDS by simulating sampling from 85 known, network populations. Across a variety of traits we find that RDS is substantially less accurate than generally acknowledged, and that reported RDS CI are misleadingly narrow. Moreover, it is unlikely RDS performs any better in practice than in our simulations as we model a best-case scenario in which the theoretical RDS sampling assumptions hold exactly. Notably, the poor performance of RDS is driven not by the bias, but by the high variance of estimates, a possibility that had been largely overlooked in the RDS literature. Given the consistency of our results across networks and our generous sampling conditions, we conclude that RDS as currently practiced may not be suitable for key aspects of public health surveillance where it is now extensively applied. This work is joint with Matthew Salganik.