How to be an effective peer reviewer and help your career

When authors are appointed by academic journals to become peer reviewers, the benefits to them are multi-fold, in terms of career development and networks, participants at a recent webinar titled Excellence in Peer Review: How to be an effective peer reviewer were told.

The event held online by the publisher Francis and Taylor on 24 September was attended by academics from all over the world and was addressed by Dr Dafne Solera, desk review manager, and Dr Diana Marshall, head of reviewer programmes at the Taylor and Francis Group, both based in London, United Kingdom.

Solera referred to Mike J Smith, editor-in-chief of Journal of Maps who said that “after authors, reviewers are the lifeblood of any journal”.

She said they provide input on errors, inconsistencies in arguments and gaps. They assist in making the article clear and credible to readers. Overall, they provide an “essential service” as each article published has to be trustworthy and of sufficient quality.

But while peer reviewers contribute to the quality of articles, the experience also helps support their own career development, she argued. This is because they remain up to date with what is happening in their fields, build networks and connections, enhance their writing skills and become recognised as experts.

Furthermore, the peer review process could assist authors in how they prepare their own papers, thereby strengthening their standing in their field, Solera said.

Marshall added that being an author who has become a peer reviewer entails “improving your own work and you tend think about what a reviewer will be looking at in your work”.

Recognition takes varying forms

Solera said that recognition takes various forms such as discounts on publication charges or free access to information. Journals also have annual acknowledgements of their reviewers and could link their contributions to their ORCID profiles.

In order to track down potential peer reviewers, editors search within their own networks, as well as journal databases or they use artificial intelligence. “It is a good idea to keep your online profile current and to include key words or descriptions of your expertise and current research interests.”

Journals have different ways of assessing whether a submitted article is suitable for publication. The editor makes the initial assessment and decides whether it is suitable for peer review or if it should be rejected.

For Solera, if an article makes the cut, at least two independent reviewers who are experts will be approached for comments. The editor will make a decision based on the comments. “Very few articles will be ready for publication immediately.” Reviewers make comments and this informs the author where it should be revised or rejected.

Solera points out that there are three models of review:

• The single blind review is when the reviewer knows who the author is. Knowing the identity of the author could prevent the reviewer from being objective.

• The double-blind review is when the author and reviewer do not know each other. This might not always work to get rid of bias as reviewers might be familiar with the writing style of some academics and could guess who they are. If the field of research is small or niched, they could figure out who the author is.

• Open peer review is completely transparent. The reviewers know who the authors are, and once peer review is complete, the authors know who the reviewers are. There is evidence that reviewers provide more detailed comments, which are beneficial to the authors.

In some cases, journals have editors-in-chief, and others may have section or associate editors who oversee peer review of articles within own specialist area. They could also appoint guest editors. Editors expect reviewers to provide a report on whether the article is suitable for publication in the journal.

“Readers will expect the reviewers to have assessed the paper thoroughly so that they can trust the paper to be reliable and described clearly,” Solera said.

According to Solera, once an author has been invited to review an article, the email will include the name and title, type of review, deadline and abstract. “Consider whether you have the right expertise.” If you cannot assess the whole article, indicate so.

It is important to consider whether there is a conflict of interest on the part of the reviewer – this could result in a biased, unobjective peer review. If you decline the offer, a quick decline is preferable. The editor could extend the deadline or come back with another offer later.

Avoiding bias

Peer review is confidential, and reviewers should try to avoid “conscious and unconscious bias”.

Solera points out four key areas to consider when reviewing an article:

• The validity of the study – make sure the research described is scientific and sound, that the methods are appropriate and answer the research questions that the findings support the conclusion.

• The quality of language and presentation. While the article needs to be understandable and clearly presented, it is not the job of the peer reviewer to copy edit it.

• Is it adding to body of knowledge or is it advancing on what has already been published?

• Is it a unique contribution or novel idea? “Research that confirms previous findings can be very significant even if it is not very novel, depending on the journal,” she explained.

In most cases the request can be to review empirical research, which means looking at results, methods, data collection and analysis and how well the methods are suited to the research questions.

In terms of reviewing theoretical research, it is important to look at the conceptual framework and how the arguments are presented and discussed in the context of existing literature. In all cases a key question to consider is whether the conclusions drawn by the authors are appropriate from their results or arguments and not over-stated or exaggerated.

According to Solera, “it is a good idea to get an overview of the paper, before going into details of your review.”

She recommends that one works in layers, starting from the surface and then going deeper, starting with skim-reading.

“Focus on the abstract, method, figures and conclusion. This will give you a general idea of what the paper is about, what the main findings are and what the author’s conclusions are.”

Then read it from the beginning to the end without taking detailed notes – what has been done, why and how the findings can be interpreted. Then go through it section by section and write detailed notes for your report.

Marshall provides these tips for reviewing empirical research:

• There are four main sections – the introduction, methods, results and discussion – be clear about why the study was relevant, what the authors did, what they found and then a balanced discussion based on the results obtained.

• Assess if the methods were suitable for the research question that was posed and that the study design did not introduce any bias.

• Comments should be in the form of questions.

• Correcting grammar should not be your main priority as the editor needs your expertise to focus on the details of that research. Copy editing can be done later by the journal.

Fraud and misconduct issues

She stressed that “you should not feel under pressure to identify all issues related to fraud or misconduct. Assess the quality and validity of the research presented and do not be expected to identify the different ethical issues which include the manipulation of data and overlapping of previous work.”

Marshall points out that sometimes authors dabble in publishing misconduct whereby the same article is sent to several journals at the same time or there is an attempt to conceal a conflict of interest.

Reviewers should alert the editor and use neutral language in raising issues, so you are not using accusatory language about the author. Let the journal investigate.

“Be neutral and consider the research done, not what you wish should have been done instead. Comment on good things and what needs to improve,” said Marshall. Keep in touch with the journal if you have any concerns.

She points out the reviewers need a clear flow to their report and that they should start with a brief summary of the research. One approach is to have major essential points that the author needs to address, like problems with study design. Minor comments, which are not essential for validation or reliability of the study, include for example incorrect referencing.

Numbering comments and outlining page numbers and lines are examples of a clearly structured report.

“It is good to review as you would want to be reviewed. The review process can feel challenging from the author’s perspective, and you will also be receiving many comments on your own work.”

She referred to a comment made by Dr Tony Bertram at a training event: “You don’t have to stand on someone else’s work to make yourself feel taller,” meaning ‘you don’t have to put someone else down to make yourself feel better’.

All review reports are sent to the editor who will make the final decision. He or she will consider the views of all editors and their own assessment. A reviewer might be asked to re-review. “Focus on assessing the author’s response to concerns. Consider it as part of your original commitment to review.”