Search

BP eer review is faith-based (not evidence-based), slow, wasteful, ineffective, largely a lottery, easily abused, prone to bias, doesn’t detect fraud and irrelevant.^ —Richard Smith, former BMJ editor, remarks to the Royal Society, 20 April 2015 Peer review in academic publishing has come under increasing scrutiny. Richard Smith’s accusations of haplessness and irrelevancy are not universally embraced; however, a growing number of scholars and thought leaders have increasingly questioned accepted practices and supported alternatives, some of which call into question the basic assumptions of the current peer review process. For example, ArXiv now has over 1 million research papers in Mathematics, Physics, Computer Science and similar subjects stored in an electronic archive. There is no peer review process; it is up to the reader to assess the integrity of the methods and conclusions. Other projects have emphasized so-called Bobjective^ peer review focusing on technical soundness rather than importance or originality; community peer review conducted via a preprint depository; an open collaborative process in which a group of peer reviewers share critiques and sign the final review jointly; and granting authors Bownership^ of peer reviews, which can then be shared with subsequent journals to which the paper is submitted. At JGIM, our take on peer review mirrors Churchill’s view of democracy: it is the worst form of scientific quality assurance except for all the others that have been tried. We are fully aware that our review process is not flawless (and in fact we have shown using our own data that it is not), nor are we averse to experimentation (our cooperative venture with the Annals of Internal Medicine is one example: articles reviewed and rejected by Annals can, at the authors’ discretion, be sent directly to JGIM along with the Annals peer reviews). However, we continue to believe that our outstanding group of JGIM reviewers and deputy editors add value to the process of selecting and preparing manuscripts for publication. In 2014–2015, JGIM reviewers volunteered their time and expertise to review about one-third of the approximately 1,500 manuscripts JGIM receives each year. During this period, 845 reviewers provided a total of 1061 reviews with a mean quality score of 4.35 on a scale of 1–6 (as judged by JGIM Deputy Editors). Of these, 191 (25 more than last year) provided at least two reviews and 24 provided three or more. We are indebted to them for their service. Among this group of dedicated peer reviewers, there is a group of top performers who stand out. Reviewers included in this group performed at least two reviews between July 2014 and June 2015, returned all reviews within 30 days, and received a quality score of four or greater on all reviews. An asterisk identifies the 101 reviewers who met these criteria. We congratulate them on their service to the academic community and thank them for their efforts on behalf of the journal. In our first BFrom the Editor’s Desk^ in January 2010, we wrote: BIn steering the journal forward...we will be guided by the principle that JGIM is a journal for generalists committed to improving the world in which they practice and teach. Thus, we will seek to publish data derived from settings where real patients live and real doctors practice, as well as reviews and tools that clinicians and educators can use to do their jobs more effectively, efficiently, and humanely.^ We remain committed to this vision and hope that you, our readers, will join us by continuing to submit to, review for, read, and respond to JGIM. Whether in print or on-line, JGIM is your journal and we need to hear from you. Send us an e-mail and share your thoughts on the future of peer review, medical journalism, and general internal medicine. 30(12):17 – 17 21

Peer review is fundamental to the editorial process, but the vital work of our reviewers is only one component of many in that process. As we begin the new year, this seems a suitable time to give both our readers and our authors a look behind the curtain: What do we know about peer review’s efficacy, can it be improved, and what happens at Clinical Orthopaedics and Related Research® after the reviews have been completed? We can trace the origins of peer review as far back as the 18th century [6], but consistent, contemporary-looking peer review of scientific and medical manuscripts became common only in the late-20th century. Perhaps the most influential general interest science journal of our time, Nature, only instituted peer review in 1967 [9]. Today, peer review is ubiquitous among the better journals of our specialty, and at CORR®, we work hard to get it right. Thoughtful observers have raised questions about its efficacy [4, 11], but good evidence substantiates that peer review improves both article quality and clarity [3, 10]. Insofar as peer review is a decentralized system that depends in large measure on the work of volunteers, no doubt it can be improved, and this remains an area of active research. Simple reviewer training does not seem to make an impact [5], while certain labor-intensive add-ons might [1]. At CORR®, we are developing a systematic refinement to guide reviewers through the editorial process. The idea is to create a tool that can walk a reviewer through the key elements of each article in such a way that the reviewer does not need to remember lessons learned in a lecture or training session. We hope that this tool, which will be used during the “routine” review process, may result in improved quality and consistency of reviews compared to simple mentoring or the implementation of still-more-rigid guidelines. Since we depend on peer review to make good decisions, and because the process requires a considerable investment of time and effort, it is important to ask whether peer review can be improved upon. Alternatives to traditional review include open peer review (allowing authors to know the identities of reviewers), postpublication peer review, and portable peer review, among many others. We need to remain mindful of the shortcomings of traditional peer review, open minded about other options, and willing to invest in thoughtful experiments on the topic when appropriate. CORR® is currently involved with a large-scale effort to investigate modifiable nonscientific elements of manuscript presentation that can influence reviewers’ decisions, and we hope to have results to share on the topic in the next year or so. Although high-quality peer review is essential to a robust editorial process, it is neither a direct democracy nor an end in itself. On the former, thoughtful editors must do more than count votes, if for no other reason than reviewers often disagree about the applicability or validity of a paper’s methods, research questions, and conclusions. Rather than tallying opinions, editors need to read the reviews in detail, and use their content to determine whether articles are interesting, important, and methodologically sound. Quality reviews provide essential subject matter expertise to guide good editorial decisions. But reviewers’ manuscript-disposition recommendations are advisory to the editors; they do not constitute a binding vote of any kind. One substantive review identifying an article’s irremediable methodological flaws can and should sway a decision, even if two other more-cursory reviews missed those flaws and favored publication. While we commonly request additional reviews even after several have come in, we generally do so not to “break ties,” but to ensure we have a balanced perspective from the best possible experts on whether a paper is interesting enough to deserve the attention of our readers, or to verify the validity of subspecialized methodological elements. An effective editorial process begins with peer review, but it does not end there. Reviewers with special subject-matter knowledge help us decide when papers on unusual topics are important or interesting enough to warrant attention from our readers. They also offer invaluable technical guidance on complex or uncommon methodological elements. But most papers study familiar topics using straightforward study designs. Specifically, the large majority of orthopaedic papers evaluate treatment approaches, and the large majority of those are case series [2], the “bones” of which are relatively similar [8] regardless of whether the topic is hip replacement or carpal tunnel release. Selection bias and study inclusion criteria, loss to followup, and outcomes assessment are essential to the validity of studies about surgical treatments, and good editors must ask about these elements whether or not the reviewers have. Editors should ensure that studies do not overstate their conclusions, or fail to disclose and discuss key limitations. Finally, editors work with authors to ensure that the presentation is as clear as it can be. Corporate standards for methodological rigor, completeness, and clarity are central to the integrity of a journal’s brand, and so are squarely within the purview of its editors. Editors who ask questions beyond those posed by reviewers are not violating the spirit of peer review, they are completing an important process. Scientists, reviewers, and editors, will inevitably err. The last step in the process – no less important than any other – is the evaluation of new work by a thoughtful community of readers. Ideas (including those in papers that have passed through peer review), in time, get replaced – either incrementally or in revolutionary fashion – through further experiments that may fill in the blanks, correct the errors, or even shift the paradigm [7]. Unless something changes substantially about the way scientists evaluate new ideas, these new ideas themselves probably will pass through peer review. We think that is a good thing. Often, the first inkling we get that an idea in an article is not quite right comes in the form of a letter to the editor. Send yours to eic@clinorthop.org.

Today, information and communication technology has spread into our workplaces and living surroundings. In other words, the development of pervasive and ubiquitous Internet enables people to connect to services in places it was not possible a few years ago. This paper analyses and discusses social media as a means of peer support in www-based discussion forums that enable newspaper readers express their opinions and arguments in the media. At the time of swine flu approaching the country, we examined how the readers discussed the intriguing topic. During the study period, altogether 1,361 comments were given. The results reveal that people want to be heard or read and that they also seem to be eager to comfort each other.