How Many Reviewer Comments in a Journal Peer Review

• Loading metrics

Open Access

Peer-reviewed

Research Article

Transparency in peer review: Exploring the content and tone of reviewers' confidential comments to editors

• Bridget C. O'Brien,
• Anthony R. Artino Jr,
• Joseph A. Costello,
• Erik Driessen,
• Lauren A. Maggio

x

• Published: November 29, 2021
• https://doi.org/10.1371/journal.pone.0260558

Figures

Abstract

Purpose

Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer-review process. Peer-review systems often let confidential comments to editors that could reduce transparency to authors, nonetheless this pick has escaped scrutiny. Our study explores one) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone.

Methods

Our dataset included 358 reviews of 168 manuscripts submitted between Jan 1, 2022 and Baronial 24, 2022 to a health professions education journal with a single blind review process. We get-go identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consequent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended "reject," nosotros coded for alignment between reviewers' comments to the editor and to authors. We report descriptive statistics.

Results

49% of reviews contained comments to the editor (north = 176). Most of these comments summarized the reviewers' impression of the article (85%), which included explicit reference to their recommended conclusion (44%) and suitability for the journal (ten%). The majority of comments addressed argument quality (56%) or enquiry design/methods/data (51%). The tone of comments tended to exist critical (40%) or constructive (34%). For the 86 reviews recommending "pass up," the majority of comments to the editor contained content that besides appeared in comments to the authors (80%); boosted content tended to exist irrelevant to the manuscript. Tone frequently aligned (91%).

Determination

Findings point variability in how reviewers use the confidential comments to editor section in online peer-review systems, though generally the manner they utilise them suggests integrity and transparency to authors.

Citation: O'Brien BC, Artino AR Jr, Costello JA, Driessen E, Maggio LA (2021) Transparency in peer review: Exploring the content and tone of reviewers' confidential comments to editors. PLoS ONE xvi(xi): e0260558. https://doi.org/x.1371/journal.pone.0260558

Editor: Alberto Baccini, Academy of Siena, Italy, Italian republic

Received: July 29, 2021; Accepted: October 26, 2021; Published: November 29, 2021

This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The piece of work is fabricated available under the Creative Commons CC0 public domain dedication.

Data Availability: Data available at: https://doi.org/x.5281/zenodo.5128723.

Funding: The author(due south) received no specific funding for this work.

Competing interests: Three authors accept editorial roles in Perspectives on Medical Education: Erik Driessen (Editor-in-Chief), Lauren Maggio (Deputy Editor-in-Primary), and Anthony Artino (Associate Editor). Driessen and Maggio receive an honorarium for their editorial roles. This does not alter our adherence to PLOS One policies on sharing information and materials.

Introduction

The peer-review process serves the dual purposes of assisting editors with publication decisions and providing authors with constructive feedback [ane–3]. Journals vary in their approach to peer review, particularly in decisions well-nigh revealing the identity of reviewers to authors and authors to reviewers [ane, 4, 5]. For many journals, masking reviewer identities became the norm as a style to encourage honest critique without fear of retaliation from authors. However, the soundness of this approach has come into question as such masking may inadvertently encourage reviewers to be less careful virtually providing constructive and respectful feedback to authors [6]. To mitigate this business concern, many journals provide guidelines for reviewers that set expectations for quality feedback [7, 8]. In addition, to enhance transparency and accountability, some journals have an open peer-review process that either requires or gives reviewers the choice to share their identity and/or brand their review publicly bachelor [iv, nine]. While such efforts to increase transparency are thought to improve review quality past increasing reviewers' sense of accountability, a serial of studies conducted by the BMJ found no difference in the quality of masked versus unmasked peer reviews [10–12]. The only statistically significant finding was that reviewers were more probable to decline the invitation to review if they knew their identity would exist revealed to authors [11, 12]. An of import caveat in the transparency hypothesis is that about journals provide space for reviewers to provide confidential comments to the editor, which effectively allows authors to express concerns they feel uncomfortable sharing with authors. Reviewers' decision recommendations are also ofttimes not shared with authors. Concerns can ascend among authors when confidential comments and recommendations to the editor event in an editor'south decision to reject a manuscript and the rationale for the decision is not shared with the authors.

Reviewer guidelines provide variable guidance most how to use the confidential comments to the editor section. The Committee On Publication Ethics (COPE) upstanding guidelines for peer reviewers [13] advises reviewers to "ensure your confidential comments and recommendations for the editor are consequent with your report for the authors; nearly feedback should be put in the study that the authors will see." The guidelines further remind reviewers that these comments "should non be a identify for denigration or imitation accusation, washed in the knowledge that the authors will not run across your comments" [thirteen p.iv]. The journal Academic Medicine provides a comprehensive guide for reviewers of research manuscripts that describes how reviewers can use that department [xiv]. The guide suggests using the confidential comments to the editor to:

"recommend additional review by someone with specific expertise, make specific comments on the quality of the manuscript, provide opinions nigh the relevance or significance of the work, or raise potential ethical concerns. The reviewer can likewise use this space to give the editor more nuanced and detailed data and to explain the severity of any problems detected in the manuscript, along with the likelihood that the authors can address the problems through a revision. The reviewer should ensure that his or her confidential comments to the editor do non contradict the comments directed toward the authors and that they provide information that is relevant just to the editor." [14 p.10]

Past dissimilarity, PLOS One explicitly prohibits confidential comments to the editor and instructs authors to only use this section to declare competing interests [fifteen]. "Confidential concerns relating to publication or research ethics" are handled outside the peer review system, through electronic mail to the journal [15]. Despite these guidelines, we know lilliputian nigh how reviewers use this section, which leaves a crucial gap in understanding transparency and accountability in peer review.

From authors' perspectives, submitting a manuscript for publication is, at minimum, an opportunity to receive feedback to better their work. However, for reasons described in a higher place, this feedback can be confusing or unhelpful when the feedback received does non align with an editor's determination to refuse the manuscript or offers little insight into their conclusion. This lack of transparency and alignment warrants farther consideration for ii reasons. Offset, honest and effective feedback from reviewers plays a key role in comeback of scholarship. 2nd, non-transparency (whether warranted or non) raises doubts well-nigh the legitimacy and fairness of the peer-review procedure and suggests an surface area for process improvement.

To better sympathize issues related to transparency and quality feedback in peer review, nosotros examined how reviewers use the confidential comments section of an online peer-review system. Our specific research questions are:

1. How exercise reviewers utilise the confidential comments to the editor section during peer review?
2. To what extent does the content and tone of reviewers' comments to the editor marshal with their comments to the authors when reviewers make a "reject" recommendation?

Materials and methods

Design

Nosotros conducted a descriptive study of reviewers' comments on manuscripts submitted to the health professions education periodical Perspective on Medical Education (PME). We selected to study PME, which is an established medical education journal (founded in 1988), in light of its commitment to the study of its ain processes and because our research team had permission to utilize the data necessary to answer the research questions. Additionally, our writer squad, AA (Associate Editor), ED (Editor-in-Chief), LM (Deputy Editor-in-Chief), all hold editorial roles at PME. Together we accept over xx years of experience running the periodical, which affords our writer team with knowledge near the journal'due south processes and history. Our study was approved by the Ethical Review Lath of the Netherlands Society for Medical Education, and the need for informed consent of reviewers and authors was waived. File number: NERN 2020.1.5. Date of decision 18-07-2020.

Data source

PME is a peer-reviewed journal published by Bohn Stafleu Loghum, a subsidiary of Springer Publishing. The journal aims to support and enrich collaborative scholarship between educators and clinicians, and to advance knowledge regarding clinical education practices. PME believes in taking an evidence-based arroyo to publishing and thus notifies authors and reviewers on its website and in its author instructions that the periodical reserves the right to acquit enquiry on all materials submitted to the journal to amend its processes.

PME publishes on boilerplate 65 articles per year, which are a combination of publication types, including original research. In 2019, the journal received 402 submissions. The periodical'due south two deputy editors screen submitted manuscripts and reject a substantial number of the publications prior to peer review. An acquaintance editor handles the manuscripts selected to continue with peer review and requests at least ii external reviews. Some manuscripts receive more than than 2 reviews when an editor feels they need more information about the quality and/or importance of the review. The journal uses unmarried-blind peer review such that the external reviewers are enlightened of the writer's identity, just the authors exercise not know the reviewers' identity. Reviewers must submit a decision of take, minor revisions, revisions, major revisions, or reject and provide a narrative argument. They as well have the option to submit confidential comments to the editors. The reviewers' comments to the editor are visible to the associate editor, deputy editor-in-chief, and editor-in-chief.

Inclusion criteria

We included reviews of three peer-reviewed article types: Original Research Articles, Reviews, or Prove and Tell (descriptions of educational innovations). All included reviews were of manuscripts submitted between Jan 2022 and August 2020. We but included reviews of new submissions; we did not include whatsoever manuscripts that were revised and reviewed again.

Data collection

The information prepare was downloaded from PME's Editorial Manager system on August 24, 2022 every bit an Excel spreadsheet. Information technology included: manuscript ID, submission appointment, reviewer ID, reviewer country of residence, editor concluding determination (if available), reviewer comments to the editor, and reviewer comments to the authors. We did not download the names or contact data of reviewers or authors, manuscript title, or manuscript abstruse to minimize the likelihood that members of the research team would be able to identify authors or reviewers. The de-identified comments are available on Zenodo (https://doi.org/ten.5281/zenodo.5128723).

Information analysis

Nosotros conducted a combination of conventional and directed content assay on reviewer comments [16]. To reflect the purpose and content covered in confidential comments to the editor, we developed a coding scheme based on studies of peer review [six, 17, 18] and guidelines for peer review (e.g., COPE Guidelines [xiii], Academic Medicine review guide[8]). Our chief codes covered content, section of the manuscript, tone, and alignment betwixt the content and tone of comments to the authors and editors when the reviewer recommended "reject." Nosotros added sub-categories as needed via conventional content analysis (deriving categories from the information). Content codes included overall verdict, quality of statement, structure and linguistic communication, blueprint and methods, reference to comments to authors, summary of manuscript, and other. Nosotros also coded the section of the manuscript referenced in the comments. We coded the tone of comments as disquisitional, constructive, supportive, or neutral and noted comments in which the reviewer addressed the editor by name or in a mode that conveyed a familiar tone.

To evaluate the alignment between reviewers' comments to the editor and comments to authors, we restricted our assay to reviews in which the reviewer recommended "refuse," as we predictable that these reviews had the greatest potential for discrepancies between what the reviewers wrote to the authors and to the editors. We coded these reviews for alignment of content and tone. Content was considered not to align if the reviewers mentioned concerns near the manuscript to the editor that they did not mention to the authors. We expected that comments to authors would embrace more content than comments to editors and did non consider this a lack of alignment. Tone was based on overall tone. We coded all reviews that did not contain comments to the editor as aligned with comments to the writer. This approach allowed us to calculate the overall incidence of alignment in reviews where reviewers recommended rejecting the manuscript.

To facilitate coding, BCO created an initial code book as informed by the above. BCO, LAM, ARA, ED and then each independently coded 15 reviews and then met to discuss their coding, reconcile differences and adapt the code book. Using the revised code volume, BCO and LAM independently each coded an boosted 35 reviews then met to talk over and revise the code volume. The remaining reviews were then assigned to BCO, LAM, ARA and JAC such that each review was coded by two authors. BCO and LAM met with all authors to reconcile codes past discussion.

Nosotros calculated descriptive statistics for a) key features of the articles in our review (Tabular array 1) and b) cardinal features of reviews with and without comments to the editor (Table two), and c) characteristics of reviews with comments to the editor, including: content (Table 3), department of the manuscript, and tone (Tabular array 4). Our study is descriptive, then nosotros did not acquit statistical tests to compare reviews with and without comments to the editor. Rather, we provide this information to situate our findings in context.

Table 2. Descriptive statistics for 358 reviews of 168 manufactures submitted to Perspectives on Medical Education in 2022 or 2020, past reviews with comments to editor and reviews with no comments to editor.

https://doi.org/10.1371/journal.pone.0260558.t002

Results

Our final dataset included 358 reviews of 168 manufactures submitted and peer reviewed from January i, 2022 to Baronial 24, 2020. Each article had 1 to 5 reviews. Manufactures with only i review reverberate situations in which the article was all the same under review and pending an additional reviewer when we extracted the data or rare occasions when an acquaintance editor provided a second review (not captured in the database) due to difficulty recruiting reviewers. The bulk of articles were original research articles (65%) and most manufactures had two or more reviews (2.1 reviews, on average). Amid the 144 articles with a final conclusion from the editor, more than half were rejected (88/144, 61%). Table 1 displays descriptive information on included manufactures.

Characteristics of reviews with and without confidential comments to the editor

Our analysis focuses on reviewer comments, so for the remainder of the results the unit of analysis is the number of reviews (north = 358). Slightly less than half of the reviews included confidential comments to the editor (49%). Table 2 displays descriptive data on reviews with and without comments. Show and Tell manuscripts had a higher percent of comments to the editor than Original Research and Review manuscripts. Among reviews that had comments to the editor, a smaller percentage recommended minor revisions (fourteen%) and a larger percentage recommended decline (27%) compared to reviews with no comments to the editor (25% minor revisions, 21% turn down). The proportion of reviews with and without comments to the editor that recommended accept, revisions, and major revisions were similar. The proportion of reviews in which the editor ultimately rejected the manuscript was similar for reviews with and without comments (56% with comments, 52% without).

Reviews included in our report came from 233 unique reviewers from 20 countries. Reviewers completed 1.v reviews on average within the timeframe of our report (range ane–9 reviews) and 75 reviewers completed 2 or more reviews. 136 unique reviewers provided comments to the editor, though 29 of these reviewers as well provided reviews that did not include comments to the editor. A higher proportion of reviews with comments to the editor came from Canada and the USA compared to reviews without comments to the editor.

RQ1. How reviewers utilize the "comments to the editor" section of online peer-review platforms

To reply our first enquiry question, nosotros analyzed the content of the 176 reviews containing comments to the editor (49% of all reviews). Reviewers' comments to the editor were much briefer, on boilerplate, than their comments to authors (Grand = 77 words, SD = 87 and M = 433 words, SD = 199, respectively).

We organized our analysis around iii dimensions of the comments: Content, Department of the Manuscript, and Tone. Tables 3 and 4 display the frequency of categories under Content and Tone and provide instance comments that reflect common ideas in reviewers' comments.

Content (Table three).

Less than one-half the comments to the editor explicitly stated the reviewers' recommendation (44%), though most provided overall evaluative statements that provided a rationale for the recommendation. More than one-half the comments remarked on the quality of the authors' statement (56%), which included reference to relevant literature, application of a conceptual framework, or lack thereof. Roughly half the comments addressed inquiry design, methods, and information (51%) which included comments on the need for more data or description of methods. A small number of comments (x%) raised concerns about suitability for the periodical based on quality standards and readership or concerns nigh fit with the type of commodity. Few comments mentioned ethical concerns related to the study or raised potential conflicts of involvement for the reviewer or authors (3%). Many comments (forty%) included content not covered in our initial coding framework. Nosotros organized this content into three categories: comments nigh a) the reviewer (business organisation nearly own limitations as a reviewer, being too harsh, being new to reviewing, and performing a group review or review with a mentee), b) the periodical's editorial process (feedback, suggestions, and questions related to the journal and the peer review procedure), c) engagement with the journal (gratitude for the opportunity to review, willingness to review a revision, or offer to write an accompanying commentary).

Section of the manuscript.

Most half the comments (49%) were likewise full general to tell what part of the manuscript was mentioned. When the section could exist determined, Methods (37%) and Results (25%) were virtually mutual, followed by Discussion (16%) and Introduction (12%). Comments rarely mentioned championship (1%), abstract (ii%), and manuscript exhibits (i.e., tables, figures, supplemental materials) (5%).

Tone (Table iv).

The most mutual tone of comments to the editor was critical (40%). However, tone varied by reviewer recommendation—amidst comments with a critical tone, about half coincided with a recommendation to refuse (48%) and nearly one-third with major revisions (32%). Comments with a constructive tone were likewise frequent (34%), peculiarly when reviewers recommended revisions (33%), major revisions (30%), or pocket-size revisions (xviii%). The remaining comments had a supportive (fifteen%) or neutral (11%) tone. A few comments (6%) suggested the reviewer knew the editor (due east.g. "Hi [associate editor name], Always welcome feedback on how to make my reviews better…Happy to discuss further as needed. [reviewer proper name]").

RQ2. Alignment betwixt the content and tone of reviewers' comments to the authors and comments to the editor when reviewers recommend reject

Amidst the reviews in which the reviewer recommended "reject" (n = 86), 48 had comments to the editor (56%). Nosotros considered reviews with no comments to the editor every bit aligned in both content and tone. For content, we constitute that more than comments aligned with the comments to the authors than did not (80% and twenty%, respectively). Amid the 17 reviews with content discrepancies, we institute comments to the editor that could reduce transparency to the authors. These comments included overall verdict statements such as "We find this manuscript not suitable for publication" (54-R2) which contrasted comments to the authors that suggested potential changes with statements like "We suggest to…" and "This tin exist overcome by…" as well every bit specific concerns about lack of value (due east.g. "I don't find the findings meaningful"(13-R1) and "I fear that this study and these outcomes are of petty involvement to the readers of this journal."(23-R1)) and inadequacy of data (e.g. "I recollect that the idea of this paper could be salvaged with boosted data gathering and reporting as reflected above, which might suggest a "major revisions" recommendation, just the current information don't seem adequate for publication"(41-R1)). Nosotros as well found comments to the editor that provided context or commentary on the review (east.g. "I hope this review is not as well harsh and if and so please let me know and nosotros can accommodate." (24-R1) and "I've given my views here as a programme managing director and I hope they're fair." (33-R2), as well as comments that did not pertain to the manuscript (e.grand. questions about the manuscript rating system or whether institution proper name should exist blinded).

The tone of reviewers' comments to the editor and the author generally aligned (91%). We constitute that the tone in the7 reviews with discrepant tones were more critical in comments to the editor than to the author.

Discussion

Our study sought to ameliorate understand issues related to transparency in peer review by examining how reviewers use the confidential comments to editors. Our analysis shows that roughly half of reviews independent no additional comments to the editor. Amidst those that did, most were brief and summarized the reviewers' overall evaluation of the manuscript, often referencing their comments to the authors. Comments nearly often addressed the quality of the arguments and aspects of the research blueprint, methods, or data. The vast majority of comments to the editor had a critical or effective tone, which roughly aligns with the decision terms recommended by reviewers. In reviews that recommended rejecting the manuscript, the content aligned more oftentimes than not and, in many cases, the additional content was non related to the quality of the manuscript. Tone too aligned in most of these comments.

Overall our findings suggest that virtually reviewers are transparent in their reviews, sharing their feedback with authors fifty-fifty when they estimate the manuscript unsuitable for publication and rarely making additional comments to the editor that heighten concerns not shared with the authors. These findings add an important piece of information to the current literature on innovations in peer review. A number of journals have shifted to processes that unmask reviewers' identity equally either an optional or required part of peer review [19]. Studies examining reviewers' preferences for sharing their identity with authors [20–22] and the result of such signed reviews on the quality, tone, and time spent on reviews have shown mixed effects [10, 11, 22–24], perhaps reflecting unlike norms across different fields and reviewer pools. None of these studies indicate whether these unmasked reviewers can write confidential comments to editors. This option provides a mechanism for reviewers to raise concerns regarding ethical issues that warrant further investigation by editors (due east.g., plagiarism, duplicate publication) via a confidential channel that likely feels more comfortable to reviewers. Yet this pick means that a piece of the peer-review process remains invisible to authors, which could fuel suspicion near bias in the review process. Findings from our exploratory report propose such suspicions may exist unwarranted. Some journals aim to alleviate such suspicions by instructing reviewers to use the comments to editors for a limited set up of purposes such as concerns about conflict of interest or unethical breeches [fifteen]. Nosotros institute very few instances of comments for these purposes, suggesting that such guidance would likely result in infrequent use of confidential comments to editors.

The periodical nosotros selected provided minimal guidance to reviewers for how to use the confidential comments to editors, which gives united states insight into reviewers' interpretations and potential norms in the field nigh how one might use this section. That said, the variety of comments included in this section, and the fact that 51% of reviewers did not use the confidential comments surface area, suggests that journals might do good from giving clearer guidance or training to their reviewer pool. Many journals provide guidance and checklists to reviewers, the contents of which were catalogued in a contempo scoping review of biomedical periodical peer review guides [7]. Unfortunately, this review did non report the number of guides that included content about how to use the confidential comments section, but may be worth exploring to empathise the norms around use of this section and to help standardize language beyond peer review guides.

Our findings revealed some potential benefits of confidential comments to editors that warrant consideration. Lee and colleagues narrate peer review equally an inherently social and partial process—one that cannot and arguably should not endeavor to exist objective [25]. Our findings align with this characterization and reveal means that reviewers use these comments to signal engagement in the scholarly community and reflect on their positionality. We found reviewers using this department to express gratitude for the opportunity to review the manuscript and willingness to review a revision. Editors may find value in these expressions of goodwill and commitment to the community. Reviewers besides showed signs of cocky-awareness and want for comeback past noting concern virtually the tone of their review and indicating a desire for feedback. Such comments reveal reviewers' thoughtfulness in crafting reviews and may advise a manner to reward and sustain their efforts past giving them ways to hone their skills and acquire from editors. Reviewers also occasionally used the confidential comments department to alert editors to their perspective on a topic or limitations regarding evaluation of the content or methodology. Such information may provide helpful context to editors, particularly when needing to brand a decision based on disparate reviews. Eliminating or restricting use of this department without providing alternative means and guidance about how best to communicate such data to editors might reduce opportunities for reviewers to communicate with editors in means that enrich relationships in what can otherwise feel like an impersonal process.

Several articles have described diverse types of bias and partiality that may employ to peer review, including bias for or against the author based on personal characteristics (race, ethnicity, gender, status) or emotions toward the author (e.k. sympathy for their state of affairs), the condition or prestige of the writer's establishment, and the ideology, philosophical orientation, or methodology put forth by the author [25–27]. We did not evaluate comments specifically for such biases and partiality, though our coding of "other" content could have captured some of this content. Since authors and institutions were deidentified in our dataset, our power to evaluate this content was limited. That said, this blazon of analysis could be a fruitful direction for future research, particularly to understand when reviewer partiality is problematic and how comments to the editor may help or hinder awareness of such partiality.

Our findings should be considered in low-cal of several study limitations. Commencement, we analyzed comments from a unmarried periodical inside the field of medical education. Information technology is possible that in another medical pedagogy periodical, or ane outside of the field, comments to the editor differ from those analyzed here, which limits the generalizability of our findings. Second, nosotros focused on determining alignment between comments to the author and editor for those manuscripts that reviewers recommended rejecting. Hereafter inquiry should consider exploring alignment for those manuscripts recommended to take and revise. Despite the fact that the majority of comments were written by authors located in English language-speaking countries, this study also included comments from reviewers for which English was probable non their showtime language. This point could have introduced subtle differences in the comments that our predominantly North American enquiry team did not notice. Therefore, we encourage investigators to consider broadening their research team in future work, or engaging with stakeholders from the reviewers' countries to obtain feedback on the findings.

Determination

Findings from our exploratory study indicate that nearly one-half of reviews comprise no confidential comments to the editor and those that do use them in a variety of means. Virtually comments included a summary or rationale for the reviewer's overall evaluation of the manuscript and carried a critical or constructive tone. Reviewers' commitment to transparency, based on lack of employ of confidential comments and alignment betwixt the content and tone of reviewers' comments to editors and authors, appears to be strong and in many cases lack of alignment in content reflected concerns unrelated to the manuscript or circumstances the reviewer felt important call to the editors' attention. These findings suggest opportunities for further investigation and word about the pros and cons of retaining or clarifying the purpose of confidential comments to the editors every bit part of peer review.

Acknowledgments

Disclaimer: The views expressed in this commodity are those of the authors and do non necessarily reflect the official policy or position of the Uniformed Services University of the Wellness Sciences, the Department of Defense, or the U.S. Government.

References

1. 1. Tennant JP, Dugan JM, Graziotin D, Jacques DC, Waldner F, Mietchen D, et al. A multi-disciplinary perspective on emergent and future innovations in peer review. F1000Research. 2017;6: 64. pmid:29188015
   • View Commodity
   • PubMed/NCBI
   • Google Scholar
2. 2. Glonti K, Cauchi D, Cobo Eastward, Boutron I, Moher D, Hren D. A scoping review on the roles and tasks of peer reviewers in the manuscript review procedure in biomedical journals. BMC Med. 2019;17: 118. pmid:31217033
   • View Commodity
   • PubMed/NCBI
   • Google Scholar
3. 3. Azer SA, Ramani South, Peterson R. Becoming a peer reviewer to medical didactics journals. Medical Teacher. 2012;34: 698–704. pmid:22643022
   • View Article
   • PubMed/NCBI
   • Google Scholar
4. 4. Open up Peer Review. In: PLOS [Internet]. [cited 23 Jul 2021]. Available: https://plos.org/resource/open-peer-review/
5. 5. Tvina A, Spellecy R, Palatnik A. Bias in the Peer Review Process: Can We Do Better? Obstetrics & Gynecology. 2019;133: 1081–1083. pmid:31135720
   • View Article
   • PubMed/NCBI
   • Google Scholar
6. half dozen. Le Sueur H, Dagliati A, Buchan I, Whetton Advertisement, Martin GP, Dornan T, et al. Pride and prejudice–What tin can we larn from peer review? Medical Teacher. 2020;42: 1012–1018. pmid:32631121
   • View Article
   • PubMed/NCBI
   • Google Scholar
7. 7. Vocal E, Ang L, Park J-Y, Jun Eastward-Y, Kim KH, Jun J, et al. A scoping review on biomedical journal peer review guides for reviewers. PLOS 1. 2021;xvi: e0251440. pmid:34014958
   • View Article
   • PubMed/NCBI
   • Google Scholar
8. 8. Durning SJ, Carline JD, editors. Review criteria for research manuscripts. 2nd ed. Washington, D.C.: Clan of American Medical Colleges; 2015. Available: https://store.aamc.org/review-criteria-for-research-manuscripts.html
9. ix. Groves T, Loder E. Prepublication histories and open peer review at The BMJ. BMJ. 2014;349: g5394–g5394. pmid:25186622
   • View Commodity
   • PubMed/NCBI
   • Google Scholar
10. ten. van Rooyen S, Godlee F, Evans Southward, Smith R, Blackness Due north. Result of Blinding and Unmasking on the Quality of Peer Review: A Randomized Trial. JAMA. 1998;280: 4. pmid:9676666
    • View Article
    • PubMed/NCBI
    • Google Scholar
11. 11. van Rooyen S, Godlee F, Evans S, Black N, Smith R. Event of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial. BMJ. 1999;318: 23–27. pmid:9872878
    • View Article
    • PubMed/NCBI
    • Google Scholar
12. 12. van Rooyen Southward, Delamothe T, Evans SJW. Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. BMJ. 2010;341: c5729–c5729. pmid:21081600
    • View Article
    • PubMed/NCBI
    • Google Scholar
13. thirteen. COPE Council. COPE Ethical guidelines for peer reviewers—English. Version 2: September 2017; https://doi.org/10.24318/cope.2019.ane.9
14. 14. Dine CJ, Caelleigh Every bit, Shea JA. Chapter 3: Review forms and reviewer comments. In Durning SJ, Carline JD, editors. Review criteria for enquiry manuscripts, second ed. Washington, DC: Association of American Medical Colleges; 2015: pp 9–11.
15. 15. PLOS ONE: Guidelines for reviewers. [cited 23 Jul 2021]. Available: https://journals.plos.org/plosone/due south/reviewer-guidelines
16. 16. Hsieh H-F, Shannon SE. Three Approaches to Qualitative Content Assay. Qual Health Res. 2005;15: 1277–1288. pmid:16204405
    • View Article
    • PubMed/NCBI
    • Google Scholar
17. 17. Hyland K, Jiang F (Kevin). "This work is antonymous to the spirit of research": An anatomy of harsh peer reviews. Journal of English for Bookish Purposes. 2020;46: 100867.
    • View Article
    • Google Scholar
18. 18. Yakhontova T. "The authors accept wasted their time…": Genre features and linguistic communication of anonymous peer reviews. Topics in Linguistics. 2019;twenty: 67–89.
    • View Article
    • Google Scholar
19. 19. Ross-Hellauer T. What is open up peer review? A systematic review. F1000Res. 2017;6: 588. pmid:28580134
    • View Commodity
    • PubMed/NCBI
    • Google Scholar
20. 20. Baggs JG, Broome ME, Dougherty MC, Freda MC, Kearney MH. Blinding in peer review: the preferences of reviewers for nursing journals. Journal of Advanced Nursing. 2008;64: 131–138. pmid:18764847
    • View Article
    • PubMed/NCBI
    • Google Scholar
21. 21. Snell L, Spencer J. Reviewers' perceptions of the peer review process for a medical education journal. Med Educ. 2005;39: ninety–97. pmid:15612905
    • View Article
    • PubMed/NCBI
    • Google Scholar
22. 22. Walsh E, Rooney M, Appleby 50, Wilkinson G. Open peer review: A randomised controlled trial. The British Periodical of Psychiatry. 2000;176: 47–51. pmid:10789326
    • View Article
    • PubMed/NCBI
    • Google Scholar
23. 23. Bruce R, Chauvin A, Trinquart L, Ravaud P, Boutron I. Touch on of interventions to amend the quality of peer review of biomedical journals: a systematic review and meta-analysis. BMC Med. 2016;fourteen: 85. pmid:27287500
    • View Article
    • PubMed/NCBI
    • Google Scholar
24. 24. Nobarany S, Booth KS. Use of politeness strategies in signed open peer review. Journal of the Association for Data Science and Applied science. 2015;66: 1048–1064.
    • View Article
    • Google Scholar
25. 25. Lee CJ, Sugimoto CR, Zhang Grand, Cronin B. Bias in peer review. Journal of the American Society for Informatics and Engineering science. 2013;64: 2–17.
    • View Article
    • Google Scholar
26. 26. D'Andrea R, O'Dwyer JP. Can editors relieve peer review from peer reviewers? PLOS I. 2017;12: e0186111. pmid:29016678
    • View Commodity
    • PubMed/NCBI
    • Google Scholar
27. 27. Tvina A, Spellecy R, Palatnik A. Bias in the Peer Review Procedure: Can We Do Better? Obstetrics & Gynecology. 2019;133: 1081–1083. pmid:31135720
    • View Article
    • PubMed/NCBI
    • Google Scholar

mcmilloncamle2002.blogspot.com

Source: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0260558

Share this post