Quick tips on how to be a successful reviewer
By Ashwin N. Ananthakrishnan, MD, MPH from Massachusetts General Hospital and Harvard Medical School
Being a peer reviewer has several important benefits. It exposes one to the editorial process and encourages critical appraisal of work. It exposes you to the cutting edge of scientific advances in any particular field. In addition, I always feel like I learn something from every manuscript I review. It may also be a useful resource for networking with senior faculty in the field who are the editors for various journals.
There are several important steps in the peer review process that I’ve found helpful.
- Perform a quick scan of the literature to get a sense of what already exists in the field. This is important to both inform you of the current state of knowledge in the field and how this manuscript could potentially advance it, but also the strengths and weaknesses of the studies that exist including size of the cohort, study design (RCT vs observation; prospective vs retrospective). This ensures that in addition to evaluating the manuscript in itself, you are also evaluating it against the field of existing knowledge and how it may offer an added contribution.
- A couple of quick read-throughs of the manuscript will give you a broad sense of the research questions, study methods, and key results and message.
- Highlight both the strengths and weaknesses you identify. The purpose of the review is not necessarily just to find flaws in the research, so it is important to also note any particular strengths.
- List the major comments that you would like addressed and which concern you about the validity of the study. These may include some of the following, but in no particular order. Assess the appropriateness of selection of the study population – is this representative of those with the particular disease? Would patients in your practice be eligible as “candidates” for this study? While all studies including clinical trials may not be applicable to the entire spectrum of patients with the disease, having a realistic inclusion criteria is important. Next, see if the intervention (if an interventional study) is appropriate. Similarly also assess the outcomes – whether this is clinically and biologically meaningful, and if it is appropriate for the disease. For example, for a chronic disease like Crohn’s disease or ulcerative colitis, it is important for an intervention to provide both short and long-term benefits. In this case, endpoints that extend 1 year or longer are important, whereas for an acute illness like ulcer bleed, in-hospital outcomes may be appropriate. Again, one study may not address all relevant outcomes, but what you would consider important for that time frame should be included. For observational studies, assess if the study design is appropriate to answer the question – for example case control vs. cohort? If the former, have the controls been adequately selected? For both studies, are possible confounders inquired about and adjusted for in the analysis? Is the statistical analysis appropriate (for example, univariate and adjusted multivariate models or time to event analysis)? Are effect sizes (odds ratio, hazard ratio, relative risks, confidence intervals) also provided in addition to just p-values (which do not convey magnitude of association)?
- Identify any minor typos, spelling errors. Separating out your major from minor concerns ensures that your most important questions get the emphasis they deserve and allows the author to address them appropriately.
- Write a quick note to the Associate Editor regarding its suitability for publication. This should not merely repeat the detailed points already listed in your review, but convey a global impression.