Search

Your search keyword '"Ioannidis, John P A"' showing total 4,129 results
Start Over Author "Ioannidis, John P A" Search Limiters Academic (Peer-Reviewed) Journals
4,129 results on '"Ioannidis, John P A"'

1. Metrics to Detect Small-Scale and Large-Scale Citation Orchestration

Author

Evdaimon, Iakovos, Ioannidis, John P. A., Nikolentzos, Giannis, Chatzianastasis, Michail, Panagopoulos, George, and Vazirgiannis, Michalis

Subjects
Computer Science - Digital Libraries
Abstract

Citation counts and related metrics have pervasive uses and misuses in academia and research appraisal, serving as scholarly influence and recognition measures. Hence, comprehending the citation patterns exhibited by authors is essential for assessing their research impact and contributions within their respective fields. Although the h-index, introduced by Hirsch in 2005, has emerged as a popular bibliometric indicator, it fails to account for the intricate relationships between authors and their citation patterns. This limitation becomes particularly relevant in cases where citations are strategically employed to boost the perceived influence of certain individuals or groups, a phenomenon that we term "orchestration". Orchestrated citations can introduce biases in citation rankings and therefore necessitate the identification of such patterns. Here, we use Scopus data to investigate orchestration of citations across all scientific disciplines. Orchestration could be small-scale, when the author him/herself and/or a small number of other authors use citations strategically to boost citation metrics like h-index; or large-scale, where extensive collaborations among many co-authors lead to high h-index for many/all of them. We propose three orchestration indicators: extremely low values in the ratio of citations over the square of the h-index (indicative of small-scale orchestration); extremely small number of authors who can explain at least 50% of an author's total citations (indicative of either small-scale or large-scale orchestration); and extremely large number of co-authors with more than 50 co-authored papers (indicative of large-scale orchestration). The distributions, potential thresholds based on 1% (and 5%) percentiles, and insights from these indicators are explored and put into perspective across science.

Published
2024

2. The Costs of Competition in Distributing Scarce Research Funds

Author

Schweiger, Gerald, Barnett, Adrian, Besselaar, Peter van den, Bornmann, Lutz, De Block, Andreas, Ioannidis, John P. A., Sandström, Ulf, and Conix, Stijn

Subjects
Economics - General Economics
Abstract

Research funding systems are not isolated systems - they are embedded in a larger scientific system with an enormous influence on the system. This paper aims to analyze the allocation of competitive research funding from different perspectives: How reliable are decision processes for funding? What are the economic costs of competitive funding? How does competition for funds affect doing risky research? How do competitive funding environments affect scientists themselves, and which ethical issues must be considered? We attempt to identify gaps in our knowledge of research funding systems; we propose recommendations for policymakers and funding agencies, including empirical experiments of decision processes and the collection of data on these processes. With our recommendations we hope to contribute to developing improved ways of organizing research funding., Comment: 29 pages

Published
2024

4. Recommendations on data sharing in HIV drug resistance research.

Author

Avila-Rios, Santiago, Ayitewala, Alisen, Bosch, Ronald, Calvez, Vincent, Ceccherini-Silberstein, Francesca, Charpentier, Charlotte, Descamps, Diane, Eshleman, Susan, Fokam, Joseph, Frenkel, Lisa, Gupta, Ravindra, Ioannidis, John, Kaleebu, Pontiano, Kantor, Rami, Kassaye, Seble, Kosakovsky Pond, Sergei, Kouamou, Vinie, Kouyos, Roger, Kuritzkes, Daniel, Lessells, Richard, Marcelin, Anne-Genevieve, Mbuagbaw, Lawrence, Inzaule, Seth, Siedner, Mark, Minalga, Brian, Ndembi, Nicaise, Neher, Richard, Paredes, Roger, Pillay, Deenan, Raizes, Elliot, Rhee, Soo-Yon, Ruxrungtham, Kiat, Sabeti, Pardis, Schapiro, Jonathan, Sirivichayakul, Sunee, Steegen, Kim, Sugiura, Wataru, van Zyl, Gert, Vandamme, Anne-Mieke, Wensing, Annemarie, Wertheim, Joel, Gunthard, Huldrych, Jordan, Michael, Shafer, Robert, Little, Susan, and Richman, Douglas

Abstract

• Human immunodeficiency virus (HIV) drug resistance has implications for antiretroviral treatment strategies and for containing the HIV pandemic because the development of HIV drug resistance leads to the requirement for antiretroviral drugs that may be less effective, less well-tolerated, and more expensive than those used in first-line regimens.  • HIV drug resistance studies are designed to determine which HIV mutations are selected by antiretroviral drugs and, in turn, how these mutations affect antiretroviral drug susceptibility and response to future antiretroviral treatment regimens.  • Such studies collectively form a vital knowledge base essential for monitoring global HIV drug resistance trends, interpreting HIV genotypic tests, and updating HIV treatment guidelines.  • Although HIV drug resistance data are collected in many studies, such data are often not publicly shared, prompting the need to recommend best practices to encourage and standardize HIV drug resistance data sharing.  • In contrast to other viruses, sharing HIV sequences from phylogenetic studies of transmission dynamics requires additional precautions as HIV transmission is criminalized in many countries and regions.  • Our recommendations are designed to ensure that the data that contribute to HIV drug resistance knowledge will be available without undue hardship to those publishing HIV drug resistance studies and without risk to people living with HIV.

Published
2023

7. Author Correction: Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19 from an international collaborative meta-analysis of randomized trials

Author

Axfors, Cathrine, Schmitt, Andreas M., Janiaud, Perrine, van’t Hooft, Janneke, Abd-Elsalam, Sherief, Abdo, Ehab F., Abella, Benjamin S., Akram, Javed, Amaravadi, Ravi K., Angus, Derek C., Arabi, Yaseen M., Azhar, Shehnoor, Baden, Lindsey R., Baker, Arthur W., Belkhir, Leila, Benfield, Thomas, Berrevoets, Marvin A. H., Chen, Cheng-Pin, Chen, Tsung-Chia, Cheng, Shu-Hsing, Cheng, Chien-Yu, Chung, Wei-Sheng, Cohen, Yehuda Z., Cowan, Lisa N., Dalgard, Olav, de Almeida e Val, Fernando F., de Lacerda, Marcus V. G., de Melo, Gisely C., Derde, Lennie, Dubee, Vincent, Elfakir, Anissa, Gordon, Anthony C., Hernandez-Cardenas, Carmen M., Hills, Thomas, Hoepelman, Andy I. M., Huang, Yi-Wen, Igau, Bruno, Jin, Ronghua, Jurado-Camacho, Felipe, Khan, Khalid S., Kremsner, Peter G., Kreuels, Benno, Kuo, Cheng-Yu, Le, Thuy, Lin, Yi-Chun, Lin, Wu-Pu, Lin, Tse-Hung, Lyngbakken, Magnus Nakrem, McArthur, Colin, McVerry, Bryan J., Meza-Meneses, Patricia, Monteiro, Wuelton M., Morpeth, Susan C., Mourad, Ahmad, Mulligan, Mark J., Murthy, Srinivas, Naggie, Susanna, Narayanasamy, Shanti, Nichol, Alistair, Novack, Lewis A., O’Brien, Sean M., Okeke, Nwora Lance, Perez, Léna, Perez-Padilla, Rogelio, Perrin, Laurent, Remigio-Luna, Arantxa, Rivera-Martinez, Norma E., Rockhold, Frank W., Rodriguez-Llamazares, Sebastian, Rolfe, Robert, Rosa, Rossana, Røsjø, Helge, Sampaio, Vanderson S., Seto, Todd B., Shahzad, Muhammad, Soliman, Shaimaa, Stout, Jason E., Thirion-Romero, Ireri, Troxel, Andrea B., Tseng, Ting-Yu, Turner, Nicholas A., Ulrich, Robert J., Walsh, Stephen R., Webb, Steve A., Weehuizen, Jesper M., Velinova, Maria, Wong, Hon-Lai, Wrenn, Rebekah, Zampieri, Fernando G., Zhong, Wu, Moher, David, Goodman, Steven N., Ioannidis, John P. A., and Hemkens, Lars G.

Published
2024
Full Text
View/download PDF

9. Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics

Author

Bartoš, František, Maier, Maximilian, Wagenmakers, Eric-Jan, Nippold, Franziska, Doucouliagos, Hristos, Ioannidis, John P. A., Otte, Willem M., Sladekova, Martina, Deresssa, Teshome K., Bruns, Stephan B., Fanelli, Daniele, and Stanley, T. D.

Subjects
Statistics - Applications
Abstract

Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

Published
2022
Full Text
View/download PDF

13. Constructive and obsessive criticism in science

Author

Prasad, Vinay and Ioannidis, John PA

Subjects
Biomedical and Clinical Sciences, Cardiovascular Medicine and Haematology, Clinical Sciences, Guilt, Humans, Social Media, cancel culture, criticism, peer-review, social media, Twitter, General Clinical Medicine, Cardiovascular medicine and haematology, Clinical sciences
Abstract

Social media and new tools for engagement offer democratic platforms for enhancing constructive scientific criticism which had previously been limited. Constructive criticism can now be massive, timely and open. However, new options have also enhanced obsessive criticism. Obsessive criticism tends to focus on one or a handful of individuals and their work, often includes ad hominem aspects, and the critics often lack field-specific skills and technical expertise. Typical behaviours include: repetitive and persistent comments (including sealioning), lengthy commentaries/tweetorials/responses often longer than the original work, strong degree of moralizing, distortion of the underlying work, argumentum ad populum, calls to suspend/censor/retract the work or the author, guilt-by-association, reputational tarnishing, large gains in followers specifically through attacks, finding and positing sensitive personal information, anonymity or pseudonymity, social media campaigning, and unusual ratio of criticism to pursuit of one's research agenda. These behaviours may last months or years. Prevention and treatment options may include awareness, identifying and working around aggravating factors, placing limits on the volume by editors, constructive pairing of commissioned editorials, incorporation of some hot debates from unregulated locations such as social media or PubPeer to the pages of scientific journals, preserving decency and focusing on evidence and arguments and avoiding personal statements, or (in some cases) ignoring. We need more research on the role of social media and obsessive criticism on an evolving cancel culture, the social media credibility, the use/misuse of anonymity and pseudonymity, and whether potential interventions from universities may improve or further weaponize scientific criticism.

Published
2022

21. Gender-related variables for health research

Author

Nielsen, Mathias W, Stefanick, Marcia L, Peragine, Diana, Neilands, Torsten B, Ioannidis, John PA, Pilote, Louise, Prochaska, Judith J, Cullen, Mark R, Einstein, Gillian, Klinge, Ineke, LeBlanc, Hannah, Paik, Hee Young, and Schiebinger, Londa

Subjects
Behavioral and Social Science, Clinical Research, Basic Behavioral and Social Science, Generic health relevance, Good Health and Well Being, COVID-19, Culture, Female, Gender Identity, Humans, Male, SARS-CoV-2, Sex Factors, Social Factors, Surveys and Questionnaires, Gender measures, Biomedical outcomes, Sex differences
Abstract

BackgroundIn this paper, we argue for Gender as a Sociocultural Variable (GASV) as a complement to Sex as a Biological Variable (SABV). Sex (biology) and gender (sociocultural behaviors and attitudes) interact to influence health and disease processes across the lifespan-which is currently playing out in the COVID-19 pandemic. This study develops a gender assessment tool-the Stanford Gender-Related Variables for Health Research-for use in clinical and population research, including large-scale health surveys involving diverse Western populations. While analyzing sex as a biological variable is widely mandated, gender as a sociocultural variable is not, largely because the field lacks quantitative tools for analyzing the influence of gender on health outcomes.MethodsWe conducted a comprehensive review of English-language measures of gender from 1975 to 2015 to identify variables across three domains: gender norms, gender-related traits, and gender relations. This yielded 11 variables tested with 44 items in three US cross-sectional survey populations: two internet-based (N = 2051; N = 2135) and a patient-research registry (N = 489), conducted between May 2017 and January 2018.ResultsExploratory and confirmatory factor analyses reduced 11 constructs to 7 gender-related variables: caregiver strain, work strain, independence, risk-taking, emotional intelligence, social support, and discrimination. Regression analyses, adjusted for age, ethnicity, income, education, sex assigned at birth, and self-reported gender identity, identified associations between these gender-related variables and self-rated general health, physical and mental health, and health-risk behaviors.ConclusionOur new instrument represents an important step toward developing more comprehensive and precise survey-based measures of gender in relation to health. Our questionnaire is designed to shed light on how specific gender-related behaviors and attitudes contribute to health and disease processes, irrespective of-or in addition to-biological sex and self-reported gender identity. Use of these gender-related variables in experimental studies, such as clinical trials, may also help us understand if gender factors play an important role as treatment-effect modifiers and would thus need to be further considered in treatment decision-making.

Published
2021

22. Exact Inference for Disease Prevalence Based on a Test with Unknown Specificity and Sensitivity

Author

Cai, Bryan, Ioannidis, John P. A., Bendavid, Eran, and Tian, Lu

Subjects
Statistics - Methodology, Statistics - Applications
Abstract

To make informative public policy decisions in battling the ongoing COVID-19 pandemic, it is important to know the disease prevalence in a population. There are two intertwined difficulties in estimating this prevalence based on testing results from a group of subjects. First, the test is prone to measurement error with unknown sensitivity and specificity. Second, the prevalence tends to be low at the initial stage of the pandemic and we may not be able to determine if a positive test result is a false positive due to the imperfect specificity of the test. The statistical inference based on large sample approximation or conventional bootstrap may not be sufficiently reliable and yield confidence intervals that do not cover the true prevalence at the nominal level. In this paper, we have proposed a set of 95% confidence intervals, whose validity is guaranteed and doesn't depend on the sample size in the unweighted setting. For the weighted setting, the proposed inference is equivalent to a class of hybrid bootstrap methods, whose performance is also more robust to the sample size than those based on asymptotic approximations. The methods are used to reanalyze data from a study investigating the antibody prevalence in Santa Clara county, California, which was the motivating example of this research, in addition to several other seroprevalence studies where authors had tried to correct their estimates for test performance. Extensive simulation studies have been conducted to examine the finite-sample performance of the proposed confidence intervals.

Published
2020

24. A Case Study in Model Failure? COVID-19 Daily Deaths and ICU Bed Utilisation Predictions in New York State

Author

Chin, Vincent, Samia, Noelle I., Marchant, Roman, Rosen, Ori, Ioannidis, John P. A., Tanner, Martin A., and Cripps, Sally

Subjects
Quantitative Biology - Populations and Evolution, Physics - Physics and Society
Abstract

Forecasting models have been influential in shaping decision-making in the COVID-19 pandemic. However, there is concern that their predictions may have been misleading. Here, we dissect the predictions made by four models for the daily COVID-19 death counts between March 25 and June 5 in New York state, as well as the predictions of ICU bed utilisation made by the influential IHME model. We evaluated the accuracy of the point estimates and the accuracy of the uncertainty estimates of the model predictions. First, we compared the "ground truth" data sources on daily deaths against which these models were trained. Three different data sources were used by these models, and these had substantial differences in recorded daily death counts. Two additional data sources that we examined also provided different death counts per day. For accuracy of prediction, all models fared very poorly. Only 10.2% of the predictions fell within 10% of their training ground truth, irrespective of distance into the future. For accurate assessment of uncertainty, only one model matched relatively well the nominal 95% coverage, but that model did not start predictions until April 16, thus had no impact on early, major decisions. For ICU bed utilisation, the IHME model was highly inaccurate; the point estimates only started to match ground truth after the pandemic wave had started to wane. We conclude that trustworthy models require trustworthy input data to be trained upon. Moreover, models need to be subjected to prespecified real time performance tests, before their results are provided to policy makers and public health officials.

Published
2020
Full Text
View/download PDF

25. The importance of transparency and reproducibility in artificial intelligence research

Author

Haibe-Kains, Benjamin, Adam, George Alexandru, Hosny, Ahmed, Khodakarami, Farnoosh, Board, MAQC Society, Waldron, Levi, Wang, Bo, McIntosh, Chris, Kundaje, Anshul, Greene, Casey S., Hoffman, Michael M., Leek, Jeffrey T., Huber, Wolfgang, Brazma, Alvis, Pineau, Joelle, Tibshirani, Robert, Hastie, Trevor, Ioannidis, John P. A., Quackenbush, John, and Aerts, Hugo J. W. L.

Subjects
Statistics - Applications
Abstract

In their study, McKinney et al. showed the high potential of artificial intelligence for breast cancer screening. However, the lack of detailed methods and computer code undermines its scientific value. We identify obstacles hindering transparent and reproducible AI research as faced by McKinney et al and provide solutions with implications for the broader field.

Published
2020
Full Text
View/download PDF

27. Retrospective Median Power, False Positive Meta-Analysis and Large-Scale Replication

Author

Stanley, T. D., Doucouliagos, Hristos, and Ioannidis, John P. A.

Abstract

Recent, high-profile, large-scale, preregistered failures to replicate uncover that many highly-regarded experiments are "false positives"; that is, statistically significant results of underlying null effects. Large surveys of research reveal that statistical power is often low and inadequate. When the research record includes selective reporting, publication bias and/or questionable research practices, conventional meta-analyses are also likely to be falsely positive. At the core of research credibility lies the relation of statistical power to the rate of false positives. This study finds that high (>50%-60%) median retrospective power (MRP) is associated with credible meta-analysis and large-scale, preregistered, multi-lab "successful" replications; that is, with replications that corroborate the effect in question. When median retrospective power is low (<50%), positive meta-analysis findings should be interpreted with great caution or discounted altogether.

Published
2022
Full Text
View/download PDF

28. Detecting Publication Selection Bias through Excess Statistical Significance

Author

Stanley, T. D., Doucouliagos, Hristos, Ioannidis, John P. A., and Carter, Evan C.

Abstract

We introduce and evaluate three tests for publication selection bias based on excess statistical significance (ESS). The proposed tests incorporate heterogeneity explicitly in the formulas for expected and ESS. We calculate the expected proportion of statistically significant findings in the absence of selective reporting or publication bias based on each study's SE and meta-analysis estimates of the mean and variance of the true-effect distribution. A simple proportion of statistical significance test (PSST) compares the expected to the observed proportion of statistically significant findings. Alternatively, we propose a direct test of excess statistical significance (TESS). We also combine these two tests of excess statistical significance (TESSPSST). Simulations show that these ESS tests often outperform the conventional Egger test for publication selection bias and the three-parameter selection model (3PSM).

Published
2021
Full Text
View/download PDF

29. Exploration, inference and prediction in neuroscience and biomedicine

Author

Bzdok, Danilo and Ioannidis, John

Subjects
Quantitative Biology - Neurons and Cognition, Statistics - Applications
Abstract

The last decades saw dramatic progress in brain research. These advances were often buttressed by probing single variables to make circumscribed discoveries, typically through null hypothesis significance testing. New ways for generating massive data fueled tension between the traditional methodology, used to infer statistically relevant effects in carefully-chosen variables, and pattern-learning algorithms, used to identify predictive signatures by searching through abundant information. In this article, we detail the antagonistic philosophies behind two quantitative approaches: certifying robust effects in understandable variables, and evaluating how accurately a built model can forecast future outcomes. We discourage choosing analysis tools via categories like 'statistics' or 'machine learning'. Rather, to establish reproducible knowledge about the brain, we advocate prioritizing tools in view of the core motivation of each quantitative analysis: aiming towards mechanistic insight, or optimizing predictive accuracy.

Published
2019

30. Author Correction: A consensus-based transparency checklist.

Author

Aczel, Balazs, Szaszi, Barnabas, Sarafoglou, Alexandra, Kekecs, Zoltan, Kucharský, Šimon, Benjamin, Daniel, Chambers, Christopher D, Fisher, Agneta, Gelman, Andrew, Gernsbacher, Morton A, Ioannidis, John P, Johnson, Eric, Jonas, Kai, Kousta, Stavroula, Lilienfeld, Scott O, Lindsay, D Stephen, Morey, Candice C, Munafò, Marcus, Newell, Benjamin R, Pashler, Harold, Shanks, David R, Simons, Daniel J, Wicherts, Jelte M, Albarracin, Dolores, Anderson, Nicole D, Antonakis, John, Arkes, Hal R, Back, Mitja D, Banks, George C, Beevers, Christopher, Bennett, Andrew A, Bleidorn, Wiebke, Boyer, Ty W, Cacciari, Cristina, Carter, Alice S, Cesario, Joseph, Clifton, Charles, Conroy, Ronán M, Cortese, Mike, Cosci, Fiammetta, Cowan, Nelson, Crawford, Jarret, Crone, Eveline A, Curtin, John, Engle, Randall, Farrell, Simon, Fearon, Pasco, Fichman, Mark, Frankenhuis, Willem, Freund, Alexandra M, Gaskell, M Gareth, Giner-Sorolla, Roger, Green, Don P, Greene, Robert L, Harlow, Lisa L, de la Guardia, Fernando Hoces, Isaacowitz, Derek, Kolodner, Janet, Lieberman, Debra, Logan, Gordon D, Mendes, Wendy B, Moersdorf, Lea, Nyhan, Brendan, Pollack, Jeffrey, Sullivan, Christopher, Vazire, Simine, and Wagenmakers, Eric-Jan

Abstract

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Published
2020

31. A consensus-based transparency checklist

Author

Aczel, Balazs, Szaszi, Barnabas, Sarafoglou, Alexandra, Kekecs, Zoltan, Kucharský, Šimon, Benjamin, Daniel, Chambers, Christopher D, Fisher, Agneta, Gelman, Andrew, Gernsbacher, Morton A, Ioannidis, John P, Johnson, Eric, Jonas, Kai, Kousta, Stavroula, Lilienfeld, Scott O, Lindsay, D Stephen, Morey, Candice C, Munafò, Marcus, Newell, Benjamin R, Pashler, Harold, Shanks, David R, Simons, Daniel J, Wicherts, Jelte M, Albarracin, Dolores, Anderson, Nicole D, Antonakis, John, Arkes, Hal R, Back, Mitja D, Banks, George C, Beevers, Christopher, Bennett, Andrew A, Bleidorn, Wiebke, Boyer, Ty W, Cacciari, Cristina, Carter, Alice S, Cesario, Joseph, Clifton, Charles, Conroy, Ronán M, Cortese, Mike, Cosci, Fiammetta, Cowan, Nelson, Crawford, Jarret, Crone, Eveline A, Curtin, John, Engle, Randall, Farrell, Simon, Fearon, Pasco, Fichman, Mark, Frankenhuis, Willem, Freund, Alexandra M, Gaskell, M Gareth, Giner-Sorolla, Roger, Green, Don P, Greene, Robert L, Harlow, Lisa L, de la Guardia, Fernando Hoces, Isaacowitz, Derek, Kolodner, Janet, Lieberman, Debra, Logan, Gordon D, Mendes, Wendy B, Moersdorf, Lea, Nyhan, Brendan, Pollack, Jeffrey, Sullivan, Christopher, Vazire, Simine, and Wagenmakers, Eric-Jan

Subjects
Behavioral Research, Checklist, Consensus, Delphi Technique, Guidelines as Topic, Humans, Information Dissemination, Periodicals as Topic, Social Sciences
Abstract

We present a consensus-based checklist to improve and document the transparency of research reports in social and behavioural research. An accompanying online application allows users to complete the form and generate a report that they can submit with their manuscript or post to a public repository.

Published
2020

32. Machine learning and AI research for Patient Benefit: 20 Critical Questions on Transparency, Replicability, Ethics and Effectiveness

Author

Vollmer, Sebastian, Mateen, Bilal A., Bohner, Gergo, Király, Franz J, Ghani, Rayid, Jonsson, Pall, Cumbers, Sarah, Jonas, Adrian, McAllister, Katherine S. L., Myles, Puja, Granger, David, Birse, Mark, Branson, Richard, Moons, Karel GM, Collins, Gary S, Ioannidis, John P. A., Holmes, Chris, and Hemingway, Harry

Subjects
Computer Science - Computers and Society, Computer Science - Machine Learning, Statistics - Applications, Statistics - Machine Learning, 68T01
Abstract

Machine learning (ML), artificial intelligence (AI) and other modern statistical methods are providing new opportunities to operationalize previously untapped and rapidly growing sources of data for patient benefit. Whilst there is a lot of promising research currently being undertaken, the literature as a whole lacks: transparency; clear reporting to facilitate replicability; exploration for potential ethical concerns; and, clear demonstrations of effectiveness. There are many reasons for why these issues exist, but one of the most important that we provide a preliminary solution for here is the current lack of ML/AI- specific best practice guidance. Although there is no consensus on what best practice looks in this field, we believe that interdisciplinary groups pursuing research and impact projects in the ML/AI for health domain would benefit from answering a series of questions based on the important issues that exist when undertaking work of this nature. Here we present 20 questions that span the entire project life cycle, from inception, data analysis, and model evaluation, to implementation, as a means to facilitate project planning and post-hoc (structured) independent evaluation. By beginning to answer these questions in different settings, we can start to understand what constitutes a good answer, and we expect that the resulting discussion will be central to developing an international consensus framework for transparent, replicable, ethical and effective research in artificial intelligence (AI-TREE) for health., Comment: 25 pages, 2 boxes, 1 figure

Published
2018

35. Toward unrestricted use of public genomic data

Author

Amann, Rudolf I, Baichoo, Shakuntala, Blencowe, Benjamin J, Bork, Peer, Borodovsky, Mark, Brooksbank, Cath, Chain, Patrick SG, Colwell, Rita R, Daffonchio, Daniele G, Danchin, Antoine, de Lorenzo, Victor, Dorrestein, Pieter C, Finn, Robert D, Fraser, Claire M, Gilbert, Jack A, Hallam, Steven J, Hugenholtz, Philip, Ioannidis, John PA, Jansson, Janet K, Kim, Jihyun F, Klenk, Hans-Peter, Klotz, Martin G, Knight, Rob, Konstantinidis, Konstantinos T, Kyrpides, Nikos C, Mason, Christopher E, McHardy, Alice C, Meyer, Folker, Ouzounis, Christos A, Patrinos, Aristides AN, Podar, Mircea, Pollard, Katherine S, Ravel, Jacques, Muñoz, Alejandro Reyes, Roberts, Richard J, Rosselló-Móra, Ramon, Sansone, Susanna-Assunta, Schloss, Patrick D, Schriml, Lynn M, Setubal, João C, Sorek, Rotem, Stevens, Rick L, Tiedje, James M, Turjanski, Adrian, Tyson, Gene W, Ussery, David W, Weinstock, George M, White, Owen, Whitman, William B, and Xenarios, Ioannis

Subjects
Information and Computing Sciences, Law and Legal Studies, Library and Information Studies, Human Genome, Genetics, Access to Information, Databases, Genetic, Genome, Human, Genomics, Humans, Information Dissemination, General Science & Technology
Abstract

Publication interests should not limit access to public data

Published
2019

36. An Empirical Comparison of Three Methods for Multiple Cutoff Diagnostic Test Meta-Analysis of the Patient Health Questionnaire-9 (PHQ-9) Depression Screening Tool Using Published Data vs Individual Level Data

Author

Benedetti, Andrea, Levis, Brooke, Rücker, Gerta, Jones, Hayley E., Schumacher, Martin, Ioannidis, John P. A., and Thombs, Brett

Abstract

Selective cutoff reporting in primary diagnostic accuracy studies with continuous or ordinal data may result in biased estimates when meta-analyzing studies. Collecting individual participant data (IPD) and estimating accuracy across all relevant cutoffs for all studies can overcome such bias but is labour intensive. We meta-analyzed the diagnostic accuracy of the Patient Health Questionnaire-9 (PHQ-9) depression screening tool. We compared results for two statistical methods proposed by Steinhauser and by Jones to account for missing cutoffs, with results from a series of bivariate random effects models (BRM) estimated separately at each cutoff. We applied the methods to a dataset that contained information only on cutoffs that were reported in the primary publications and to the full IPD dataset that contained information for all cutoffs for every study. For each method, we estimated pooled sensitivity and specificity and associated 95% confidence intervals for each cutoff and area under the curve (AUC). The full IPD dataset comprised data from 45 studies, 15 020 subjects, and 1972 cases of major depression and included information on every possible cutoff. When using data available in publications, using statistical approaches outperformed the BRM applied to the same data. AUC was similar for all approaches when using the full IPD dataset, though pooled estimates were slightly different. Overall, using statistical methods to fill in missing cutoff data recovered the receiver operating characteristic (ROC) curve from the "full IPD dataset" well when using only the "published" subset. All methods performed similarly when applied to the full IPD dataset. [This article was written with the DEPRESsion Screening Data (DEPRESSD) Collaboration.]

Published
2020
Full Text
View/download PDF

38. Redefine statistical significance.

Author

Benjamin, Daniel J, Berger, James O, Johannesson, Magnus, Nosek, Brian A, Wagenmakers, E-J, Berk, Richard, Bollen, Kenneth A, Brembs, Björn, Brown, Lawrence, Camerer, Colin, Cesarini, David, Chambers, Christopher D, Clyde, Merlise, Cook, Thomas D, De Boeck, Paul, Dienes, Zoltan, Dreber, Anna, Easwaran, Kenny, Efferson, Charles, Fehr, Ernst, Fidler, Fiona, Field, Andy P, Forster, Malcolm, George, Edward I, Gonzalez, Richard, Goodman, Steven, Green, Edwin, Green, Donald P, Greenwald, Anthony G, Hadfield, Jarrod D, Hedges, Larry V, Held, Leonhard, Hua Ho, Teck, Hoijtink, Herbert, Hruschka, Daniel J, Imai, Kosuke, Imbens, Guido, Ioannidis, John PA, Jeon, Minjeong, Jones, James Holland, Kirchler, Michael, Laibson, David, List, John, Little, Roderick, Lupia, Arthur, Machery, Edouard, Maxwell, Scott E, McCarthy, Michael, Moore, Don A, Morgan, Stephen L, Munafó, Marcus, Nakagawa, Shinichi, Nyhan, Brendan, Parker, Timothy H, Pericchi, Luis, Perugini, Marco, Rouder, Jeff, Rousseau, Judith, Savalei, Victoria, Schönbrodt, Felix D, Sellke, Thomas, Sinclair, Betsy, Tingley, Dustin, Van Zandt, Trisha, Vazire, Simine, Watts, Duncan J, Winship, Christopher, Wolpert, Robert L, Xie, Yu, Young, Cristobal, Zinman, Jonathan, and Johnson, Valen E

Subjects
Data Interpretation, Statistical, Models, Statistical, Reproducibility of Results, Scholarly Communication
Abstract

We propose to change the default P-value threshold for statistical significance for claims of new discoveries from 0.05 to 0.005.

Published
2018

47. When to replicate systematic reviews of interventions : consensus checklist

Author

Tugwell, Peter, Welch, Vivian Andrea, Karunananthan, Sathya, Maxwell, Lara J, Akl, Elie A, Avey, Marc T, Bhutta, Zulfiqar A, Brouwers, Melissa C, Clark, Jocalyn P, Cook, Sophie, Cuervo, Luis Gabriel, Curran, Janet Agnes, Ghogomu, Elizabeth Tanjong, Graham, Ian G, Grimshaw, Jeremy M, Hutton, Brian, Ioannidis, John P A, Jordan, Zoe, Jull, Janet Elizabeth, Kristjansson, Elizabeth, Langlois, Etienne V, Little, Julian, Lyddiatt, Anne, Martin, Janet E, Marušić, Ana, Mbuagbaw, Lawrence, Moher, David, Morton, Rachael L, Nasser, Mona, Page, Matthew J, Pardo, Jordi Pardo, Petkovic, Jennifer, Petticrew, Mark, Pigott, Terri, Pottie, Kevin, Rada, Gabriel, Rader, Tamara, Riddle, Alison Y, Rothstein, Hannah, Schüneman, Holger J, Shamseer, Larissa, Shea, Beverley J, Simeon, Rosiane, Siontis, Konstantinos C, Smith, Maureen, Soares-Weiser, Karla, Thavorn, Kednapa, Thavorn, David Tovey, Vachon, Brigitte, Valentine, Jeffery, Villemaire, Rebecca, Walker, Peter, Weeks, Laura, Wells, George, Wilson, David B, and White, Howard

Published
2020

Catalog

Books, media, physical & digital resources
See catalog results