Being a Great Peer Reviewer

Great peer reviewers combine academic knowledge, strong communication, and a desire to do good work.

They see being a reviewer as a way to help authors publish quality research and to advance their field of study. They want to make a difference and feel they need recognition for their work.

Great peer reviewers must also be good time managers, not taking on review requests when they won’t have the time and focus to make useful contributions.

As a researcher, you’ve probably already been at the end of good and not-so-good reviews. Maybe you’re already a reviewer, but feel something is missing in your reviews.

This post covers the traits of great peer reviewers, who journal editors and authors love and respect. If you have these traits, you’ll be one of those peer review legends. You’ll build a great relationship with a journal editor and set yourself up for success.

Contents (click to expand)

Have subject-matter expertise

Identify gaps in the study’s methods, observations, and analyses. Ensure the results are interpreted and discussed well and make logical sense. Peer review problems often center on methods sections, so these warrant extra attention and insight.

A time series plot with three time points, for instance, is not acceptable. All it takes to change the conclusions of the analysis is for the middle data point to shift up or down and transform a linear trend to a unimodal one. This should be something you recognize as a qualified reviewer.

You also need to know the current literature. Your own body of knowledge ensures that your peer review aligns with existing knowledge and currently accepted research design, focus, and technologies.

In STEM fields, technologies developed and applied in research studies evolve rapidly. Standards accepted just a few years ago may no longer suit today’s papers. For example, many journals from higher impact journals may only accept microbial community profiling studies that have more than 10,000 DNA reads per sample, However, 3,000 reads per sample was enough to publish in PNAS 10 years ago, and a couple of hundred reads got your paper published in the early 2000s.

Be unbiased and objective

Your peer review must have no biases related to your or the authors’ cultural and educational background and personal opinions.

For example, poor grammar is a common problem in manuscripts because scientific research brings experts from many countries together and English is the commonly used language of communication. Manuscripts with typos or unnatural word choices can be frustrating to read, and some reviewers put in less effort or even vent their frustrations on the authors. However, your evaluation must be based on the content and its quality.

Language does need to be suitable for publication, but it can often be corrected with a good-quality edit and shouldn’t be a source of bias. Many authors are trying their best with their limited resources. Help them; don’t punish them.

Here are some questions you could ask yourself during the review process to help objectively assess the written work.

- - Does the literature presented in the introduction align with the authors’ research questions?
  - Are the methods sound?
  - Do the data support the conclusions?
  - Are the authors’ claims objective and credible?
  - Are complex concepts explained clearly?

Provide constructive criticism

Your feedback shouldn’t be a rant about the manuscript’s shortcomings, even if it truly is lacking. Your peer review should include a balance of all of the following:

- - Listing and praising the manuscript’s strengths.
  - Listing aspects that are faulty and explaining why they need improving.
  - Providing ample constructive feedback that will help the authors improve their submission.

Some reviewers only write a short paragraph or two stating the general problematic areas. Such evaluations assess whether the manuscript is suited for publication, but truly helpful and constructive feedback should point out specific details about the manuscript and thoroughly explain how to improve each part.

Use attention to detail

Reviewers often fail to capture all the details of a manuscript’s study. They may point out concepts as missing even though they had been mentioned elsewhere in the manuscript. This is often because they hadn’t paid enough attention while reading the paper.

Sometimes, authors also receive useless requests for changes because the reviewers misunderstood the manuscript. Maybe it was the authors’ fault for writing poorly, but as a reviewer and expert, it’s better to give them the benefit of the doubt and write helpful feedback. Alternatively, make it clear that the writing was confusing. Then, suggest changes given your understanding of what they wrote.

The Methods section must be detailed enough so others can replicate the study and draw credible conclusions. Reviewers should catch and report inconsistencies and methodological oversights so the authors can fix them. Authors often forget to include key steps in a workflow or details about an analysis added at the last minute. Copy-pasted sentences also need to be modified according to each experiment’s context. The data and analysis must make sense, and the results must be interpreted correctly.

At times, misusing a single word or phrase could also change the meaning of a statement entirely or reduce the clarity of the authors’ report. For example: “the results indicated that the enzyme can now bind substrate C” can also make logical sense if written as “the results indicated that the enzyme cannot bind substrate C.” It’s a simple, careless error that even spellcheck might not catch.

Practice clear communication

Use straightforward language and structure your comments logically to guide the authors through your reasoning. Many journals adopt peer review systems that don’t allow authors to ask for clarification from reviewers, so this is important.

You must communicate your concerns about the manuscript clearly. Otherwise, authors and editors will dismiss any unclear feedback you provide, and your efforts will go to waste.

For instance, you may review a manuscript that repeated work published in another uncited study. During revision, the authors will make the changes you expect if you tell them exactly which figure, table, or analysis is repeated from the past paper and how you want this problem resolved. Simply stating “concept already well documented in past literature” could be taken in multiple ways. That’s unhelpful.

Be on time

Delaying your peer review hinders the authors, the journal and its editor, and the scientific community. Missing deadlines affects journal scheduling, especially if the journal publishes articles as issues. You may also get negative ratings as an unreliable reviewer at that journal. Especially here in Reviewer Credits, your rating speaks to your skills and reputation.

Holding up the review process means the study also risks getting “scooped” by other authors if they publish first. This was a particular problem with COVID-19, as it was such a “hot” and dynamic topic. Even though we like to think the collective scientific community was searching for vaccines, therapeutics, and greater understanding, it was also a race for researchers to boost their reputations.

And one more… delayed release to the research community can impact the authors’ work if another study, someone’s doctoral defense, or a grant relies on the publication of the work you’re reviewing.

Maintain ethical integrity

Outside medical fields, the security of manuscripts under review tends to be largely overlooked. Research staff aren’t always well-informed about how to safeguard confidential data. A few institutes have mandatory security and DGPR training for all staff and students, but most don’t. You must diligently follow strict security protocols on the system hosting the manuscript. Speak to nobody about the manuscript’s content without permission from the journal.

Uploading the manuscript to an external server and allowing generative AI tools to review the manuscript for you is also frowned upon. This practice breaches confidentiality, security protocols, and ethics.

You must also declare conflicts of interest to ensure your review is fair, trustworthy, and transparent.

Feel a sense of obligation

We typically feel a strong sense of obligation toward our own projects. However, as peer reviewers, we also have a broader obligation as responsible researchers within our fields of expertise. Reviewers who only prize their own work tend to give less merit to the work of others. This arrogance becomes evident in some peer reviewers’ low-quality evaluations.

Make it your priority to contribute to your field and disseminate high-quality research work. Your diligence will support rigorous scientific advancement that will benefit society and the environment.

Commit yourself to quality

As a peer reviewer, you’re helping authors improve the quality of their manuscript through objective and detailed peer review. In doing so, you enhance the credibility and impact of that study on your field of expertise.

To provide impactful feedback, you must put aside enough hours to do the reviewing task. Journals recommend about 6 hours, according to one study, but some online forum posters recommend as much as 10 hours … or more!

Take A Peer Reviewer Power-Up Course

Also seriously consider declining a peer review invitation if you know you won’t have the time to finish the job. There are many viable reasons to decline a review. Editors won’t hate you for it (better than doing it late or poorly), and authors will appreciate quality feedback from someone with the time to do a thorough review.

Mentor the authors

Peer review allows you to share your knowledge and guide less-experienced authors through the publication process.

You may encounter many different flavors of reviews and writing if you write and peer review papers. The least insightful reviews include brief sentence fragments with no guidance on how to revise the manuscript. Authors may sense that the reviewer is unhappy with parts of the manuscript, but they also expect to know what will satisfy the reviewers. Even worse, the editor may reject the paper based on what they perceive as a negative review.

Rude and condescending reviews are also not uncommon. Younger reviewers may wrongly view such evaluations as evidence that it’s OK to be uninhibited and overly honest, especially in anonymous peer review.

At the other extreme are evaluations written by caring and thoughtful peer reviewers. They spend time pointing out the commendable and problematic issues in the study and give suggestions for improvement. They may even provide a list of additional references to include in the revised manuscript to help the authors deepen their understanding of certain concepts.

Professionalism, always

Have you already been at the receiving end of rude, inconsiderate, insubstantial, and narcissistic peer reviews? These reviewers clearly imply that their work is better than that of the authors.

Reviewers may even write that they had stopped reading the manuscript because they didn’t like the study’s quality. One reviewer wrote that the “author’s last name sounds Spanish” and that they “didn’t read the manuscript because [they were] sure it’s full of bad English.” It hurts to be treated like that. It’s bullying or flat-out discrimination.

A survey study found that 58% of the respondents indicated receiving what they considered as unprofessional reviews. This study defines unprofessional peer review as “any statement that is unethical or irrelevant to the nature of the work,” including:

- - Lack of constructive criticism
  - Comments directed at the author(s) instead of their work
  - Opinion-based comments not based on objective evidence
  - Deliberately hurtful language

Communication with editors and authors should always be done respectfully. Avoid biases, socio-political attitudes, and dismissive tones. Always maintain a friendly and business-formal tone. Even less-well-written manuscripts deserve to be treated with respect. If you strongly sense that authors are taking advantage of your peer review efforts as a free manuscript-editing tool, state your observation kindly, but give them the benefit of the doubt. Ultimately, these authors are your colleagues and potential future collaborators.

Read the guidelines and scope

Journals distinguish themselves by focusing on different aspects of research, publication procedure, and delivery of studies. For example, Nature and Science reject studies that aren’t novel and state-of-the-art. Meanwhile, PLOS ONE and Scientific Reports accept studies showing methodological rigor in their papers despite the lack of novelty.

Misguided feedback could make manuscripts less well-aligned with the journal’s scope after revision. It can even delay their publication. Some types of peer review also have a higher risk of misaligning a manuscript. These include third-party peer review, where third-party reviewers may not have access to the journal’s scope and manuscript’s sensitive data.

Learn more about being a great reviewer and getting recognized for it

Reviewer Credits aims to make outstanding peer reviewers available to journals and for the reviewers to be recognized and rewarded for their work.

Sign up with our community of great reviewers today and access free resources to build the quality of your reviews, raise your reputation, and make sure you get recognition for your hard work.