If you plan to review scientific articles, research reports, or even colleagues’ papers in-progress, I hope that you will find some value in this post…
First and foremost, make sure that you have enough time and expertise to give due diligence to reviewing the paper. It should take you no less than a few hours of dedicated work to review a paper for a scientific journal. It will take even more time if you don’t have a lot of experience reviewing or if you are unfamiliar with the topic/methods described in the paper (you should allot time for some extra background reading in this case). If you would have to take a course on statistics or complete an extensive literature review to be able to understand the basics of the paper, it might not be in your best interest to review it. If you accept the invitation to review the paper, you should plan to express your limitations to the editor (e.g., not familiar with the content area, not familiar with these statistical methods), rather than giving the impression that you are an authority on all aspects of the paper.
No matter what you are reviewing, authors should be able to expect some constructive feedback. Constructive is the operative word there. Even when you discover that a paper is a disorganized mess and has obvious methodological errors (this will sometimes be the case), you should assume that the paper was submitted in good faith and that the authors need your advice in order to improve it. It is never acceptable to admonish authors’ abilities or intentions in your review. Your goal is to evaluate and improve the work, not criticize the authors. Even if you don’t believe that this paper should be accepted to the journal, you do have a responsibility to provide constructive feedback so that the authors can revise and resubmit to another journal. They will appreciate your diligence!
So as not to make the process too impersonal, always remember that your audience includes individuals with very different experiences from your own. Focus on the good parts of the paper, first and foremost, because you would hope that your reviewers would give you the same courtesy. There are people with advanced degrees who don’t have great technical writing skills or learned English as a second- or third-language. A reviewer should help authors to bring out the best qualities in their work, but not directly instruct them on the stylistics of good writing. Remember that there are differences between correctness and style. Most of us prefer our own writing styles, so it is important that reviewers can sometimes take a “good enough” stance when considering what feedback is important, rather than nit-picking grammar (that is the copy editor’s job anyway).
If the paper has major shortcomings that need addressed, don’t spend your time rewriting it for the authors . Rather, think of ways to improve the overall work and address major issues. Major issues include:
- precedents and methods are not clear or are clearly inappropriate (i.e., not scientific)
- the work does not not propose or verify unique insights (i.e., not novel)
- information is redundant or unnecessary (i.e., not concise)
- information is scattered or disorganized (i.e., not clear)
In the case of a scientific article, some criteria are more important than others. Failure to meet the first two criteria might warrant a recommendation to reject the paper outright (provide constructive feedback to the authors and make this recommendation to the editor only). If the paper can be better framed or steps might be taken to remedy methodological problems, do provide clear feedback on what improvements can be made. The last two points (#3 and 4) are important, but are more easily forgivable, and clear suggestions should be made to reorganize and/or reduce extraneous information. These observations should be brought forth constructively, framed as “potential improvements” rather than “repairs”, and noting specific examples of what could be done to improve the work.
If there are no major issues, then it is appropriate to be more detailed in the review. Mention that the paper is good overall (scientific, novel, concise, and clear), and that you have a few “minor suggestions” might improve readers’ understanding of the topic and methods at hand. Focus on specific parts of the paper.
There are no right or wrong ways to approach a review. This is what works for me:
- Print out and skim the paper to get an understanding of what it is all about, how the project was approached, and what the overall goal/conclusions were.
- Type up an abstract-length summary of how you understand of the paper. This summary will help authors to get an outside-looking-in perspective.
- Put the paper away and give yourself about a day to gain some distance from it. It will help to have a clear head and be able to take a fresh look at it for the next step. While you’re waiting, you might want to re-read the instructions from the editor as well as some of the “helpful resources” below.
- Read the paper a second time (paying attention to the details) and note specific issues. Revise your summary as necessary.
- Provide recommendations to improve upon any glaring issues, noting examples and page numbers where appropriate. If you have serious concerns, especially about methodology, it is appropriate to recommend to the editor that the paper be rejected. If there are less grievous issues that can be remedied, you might recommend that the editor ask the authors to revise and resubmit. If there are only minor issues, make specific suggestions as appropriate, and recommend that the editor accept with revisions or accept outright. In any event, it is typically not appropriate to share your judgments about acceptance/rejection with the authors directly, only with the editor. The editor will make the final determination of how to handle this paper.
- Take a break (give it a day, or at least a few hours while you make dinner, go for a walk, whatever).
- Review your recommendations and revise to make sure that they are well-organized, actionable, and inoffensive. This is an important step!
- Submit your review and don’t forget to update your CV if this is the first time that you have reviewed for this journal. Formal reviewing experience is important in academic advancement and a variety of career fields.
Benos, D. J., Kirk, K. L., & Hall, J. E. (2003). How to review a paper. Advances in Psychology Education, 27(2), 47-52.
Lee, A. S. (1995). Reviewing a manuscript for publication. Journal of Operations Management, 13(1), 87-92.
Roberts, L. W., Coverdale, J., Edenharder, K., & Louie, A. (2004). How to review a manuscript: A “Down-to-Earth” approach. Academic Psychiatry, 28(2), 81-87.
Spigt, M., & Arts, I. C. W. (2010). How to review a manuscript. Journal of Clinical Epidemiology, 63, 1385-1390.