j o u r n a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 7 ) 1 e5
Available online at www.sciencedirect.com
ScienceDirect
journal homepage: www.JournalofSurgicalResearch.com
Editorial
How to review a paper: suggestions from the editors of Surgery
and the Journal of Surgical Research
Introduction
The following are the proceedings of a breakfast session
sponsored by Elsevier, held on February 4, 2016, during the
11th Annual Academic Surgical Congress in Jacksonville,
Florida. The four invited speakers were the current editors of
Surgery and the Journal of Surgical Research (JSR). A selected
reference list follows these proceedings.1e14
How to review a manuscript
The first speaker was Michael Sarr, MD, former and Emeritus
Professor of Surgery at the Mayo Clinic and the current Co-
Editor-in-Chief of Surgery. The topic of his talk was “How to
Review a Manuscript.” Dr. Sarr began with several questions
to the audience related to the relevance of the overall session:
1. When asked how many had done a review of a scientific
article, an overwhelming majority raised their hands.
2. When asked how many had done more than five reviews,
again, most audience members raised their hands.
3. When asked whether they had been coached or taught
how to do a review, <20% of the audience voted
affirmatively.
4. When asked whether they had ever been “offended” by
the callousness of reviewers’ comments, more than half
of the audience agreed.
5. And when asked whether they had ever felt that a
reviewer had done a bad job, had not read their article, or
had not understood their article, again, more than half of
the audience agreed.
Having set the stage for the session, Dr Sarr first focused on
what an editor wants from reviewers. Although the editor does
want the reviewer’s opinion on whether the article should be
published, what is most important is a critical, unbiased,
comprehensive review of both the good and bad points of the
article. The review should include constructive, positive sug-
gestions to the authors. In addition, in a section usually titled
“confidential comments to the editor,” the editor wants a very
clear opinion on the article’s importance and whether it
falls into the following categories: (1) warrants publication,
(2) requires a minor revision, (3) requires a major revision, or (4)
requires a major revision and rereview. Notably, the timely
submission of the review (<2 wk) is always encouraged!
Dr Sarr continued his talk by describing the sections of the
review expected by the editor. These include the major sec-
tions usually titled “confidential comments to the editor” and
“comments to the author.” The latter section usually begins
with a short summary of the article (three to five sentences).
Then, when indicated, the reviewer should list by numbering
the major points (these represent important points that the
author must address or change) and the minor points (these
are often correctable by the author or editor with simple
editing). Numbering these points facilitates the authors’ re-
sponses to each point (see suggested outline for a scientific
review in Table). When the review is given as a full paragraph
with multiple suggestions imbedded within the paragraph, it
is very difficult for the author to address all the criticisms in a
logical, easily understood manner; likewise, it is very difficult
for the editor to review the authors’ responses.
In the “confidential comments to the editor” section, the
reviewer should begin with a brief description of the study (two
to three sentences). Next, the reviewer should provide com-
ments about the importance of the topic for that journal, fol-
lowed by a brief description of the strengths and weaknesses of
the article, as well as any limitations or problems that would
preclude its acceptance; these comments do not need to be as
detailed as the comments provided to the authors, but they
may be used to help determine the study’s importance and
potential for publication. Finally, the reviewer can add his or
her very candid opinion on whether the article should or should
not be published, remembering that the authors will not see
this or be able to identify the reviewer! Optionally, if a reviewer
enjoys doing reviews, this is a good place to thank the editor
and encourage the editor to continue sending articles for re-
view. The reviewer can also sign or type their name in this part
of the review; this is good public relations for the reviewer and
shows the valuing of what we call “academic citizenship.”
In the “comments to the authors” section, the reviewer
provides evidence to the authors (and the editor) that he or
she critically read the article and got the “big picture.” In a
short paragraph (two to three sentences), the reviewer
conveys his or her understanding of the study’s goals or
aims and its methods and results. The reviewer’s goals are
2 j o u r n a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 7 ) 1 e5

Table e Suggested outline for a scientific review.

 Summary: The initial paragraph (two to three sentences) should mention the name and location of the author(s) with a
short summary of the aims and goals of the article.
 Major points: These are critical points that the authors must address; they should be numbered separately to help the
author, as well as the editor to address a revision in an orderly manner.
1.
2.
3.
etc.
 Minor points: These are points that should be easy to address by the authors; again the individual numbering helps
authors and editors.
1.
2.
etc.
 Finally, the reviewer may elect to sign the review with his or her name; if so, then his or her name will be communicated
to the author with the review. However, note that some journals keep the review process completely anonymous.

to review the underlying science even if the English is poor,
identify things that must be changed before publication, and
make constructive suggestions for important improvements.
Also, the reviewer should try to be the authors’ advocate
(i.e., certainly what we would all want when someone re-
views our own submissions) and start out by being compli-
mentary to the authors, such as with comments like “the
authors identified a potentially important topic” or “the au-
thors did a nice study investigating.” The goal of a reviewer
is not to refute and criticize everything in the article. A
reviewer should always be courteous, nonoffensive, and
constructive and remember that, with constructive criti-
cism, there is always a way to say that the study is not good
or appropriate in a nice way, such as “the authors attempted
to determine the genomic signature, but unfortunately.,”
followed by a description of limitations and suggestions for
improvement. Overall, the reviewers should convey to the
authors that they understood the work and are providing a
fair assessment.
In the subsequent detailed scientific review, the reviewer
discusses the following matters. Was the methodology un-
derstandable? Were the approach and design correct? Was the
statistical analysis understandable, and was there adequate
statistical power? Was institutional review board (IRB)
approval needed and confirmed? Was the results section well
organized? Did the discussion section discuss all pertinent
prior literature and the authors’ data in the proper context? Do
the data support the conclusions? Are the figures and tables
appropriate? Are there too many figures and tables or too few,
and do they merely repeat information from the text? Should
these sections be reconfigured?
Dr Sarr next discussed reviews of clinical articles. Whereas
basic science articles should be hypothesis driven (always
remembering the null hypothesis!), clinical articles and re-
views are often goal or aim driven. A reviewer should check
for the national or international registration of randomized
controlled trials that allows other investigators to access
the study and should check for mention of adherence to
the guidelines of the Consolidated Standards of Reporting
Trials Statement. Reports of randomized controlled trials
must include a power calculation, describe an appropriate
randomization process, and ask an important question.
The following special circumstances were then discussed.
1. In database studies, the database needs to be described by
the authors. Is the database administrative or clinical? Is
the database risk adjusted? Can the database really
answer the question? If so, how can the findings be used
to improve outcomes?
2. Systematic reviews and meta-analyses should ideally
conform to Preferred Reporting Items for Systematic
Reviews and Meta-Analyses Statement guidelines. The
reviewer should ask the following questions:
(a) Is the topic important enough?
(b) Does the article include primary reviews of more than
just titles?
(c) Does the review feature only articles written in
English?
(d) Is there bias or heterogeneity? (This must be addressed
in the results section.)
(e) Is there a true evidence-based conclusion?
(f) Are the conclusions useful?
(g) Does the review end with the conclusion that there are
not enough data to make any evidence-based conclu-
sions? If so, is the study really worth publishing?
Unfortunately, systematic reviews and meta-
analyses are only as good as the literature, and the
authors cannot fully assess this point until after the
study is completed. Because the authors put a great
deal of work into doing the study, they all too often
submit the article even if it is not really helpful. Alas, if
the literature is not robust enough to answer the
question, then the study is not helpful and probably
does not warrant publication.
(h) When evaluating articles from nonenative English-
speaking authors, reviewers should start by looking
at the science, not the grammar. If the science is good,
does the article need to be rewritten by an English
editing service, or can the editor or publisher edit it
satisfactorily?
Dr Sarr summarized his presentation with the following:
what an editor wants is for the reviewer to be fair and
comprehensive, with the ultimate goal of publishing solid,
 j o u r n a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 7 ) 1 e5
potentially important work. Sometimes the editor and/or the
authors want to publish only “discoveries,” but more
commonly in surgery journals, the goal is usually to publish
true translational work.
Finally, the editor (and the author) wants a timely sub-
mission of all reviews.
What an author wants from reviewers
The next discussant was Kevin Behrns, MD. At the time,
Dr Behrns was the Edward R. Woodward Professor and
Chairman of the Department of Surgery at the University of
Florida, Gainesville. Since January 1, 2017, he has become the
Vice President for Medical Affairs and Dean of the Saint Louis
University School of Medicine. The topic of Dr Behrns’ talk was
“What an author wants.” His objectives were as follows:
1. To review the reasons an author chooses to submit to a
particular journal,
2. To discuss the expectations an author has of a reviewer
and an editor,
3. To provide examples of reviews that meet an author’s
expectations, and
4. To highlight additional ways to improve surgical articles
through increased communication and collaboration.
The first objective for authors is to select the right journal for
their article. Factors that should be weighed include whether
the journal has a national or international reputation and
whether the journal has the appropriate readership and con-
tent areas for the science. Authors also prefer an easy sub-
mission process, timely reviews, and a seamless publication
process if the article is accepted. Guidance from the managing
editor and an understanding publisher are also plusses.
From the authors’ perspective, the review should make it
clear that the article has been read and understood and that
the work has been evaluated in context with the literature. An
organized review will help authors understand which specific
issues must be addressed and whether the deficiencies pre-
clude publication. Dr Behrns’ discussion reiterated Dr Sarr’s
suggestions in that the review should provide a brief summary
of the work, comment about the novelty versus addition of
incremental knowledge, cite major and minor issues with the
work, provide examples of how the work can be improved,
indicate whether the study design is substantially flawed, and
discuss whether the methodology and/or conclusions can be
changed to potentially allow publication.
Using an example of an article review from Surgery, Dr
Behrns provided examples of a brief summary, major com-
ments, and minor comments. In the brief summary (see in the
following), the reviewer identified the first author, reiterated the
hypothesis, and briefly highlighted the methods and whether
they were standard or novel. The reviewer went on to summa-
rize the important results and interpret the authors’ conclusion.
Unknown author et al. conducted a meta-analysis of
randomized clinical trials that compared pancreaticogas-
trostomy (PG) versus pancreaticojejunostomy (PJ) recon-
struction during a pancreatoduodenectomy. Seven
3
randomized trials including more than 1100 patients were
included. Standardized criteria for meta-analyses were
applied. The studies exhibited moderately strong hetero-
geneity. The results demonstrated that PG was associated
with a lower pancreatic fistula rate than PJ. Abdominal fluid
collections were more common after PJ, but postoperative
bleeding was more frequent after PG. The authors conclude
that PG decreases postoperative pancreatic fistulae but does
not influence mortality, morbidity, or reoperation rate.
In the major comments section, the authors want the re-
viewers to state why any issue is problematic, as well as to
suggest potential alternatives to improve the work. In this sec-
tion (and in the minor comments section), authors prefer
comments that denote the location of the issue in the article (i.e.,
in the results, methods, and so on; page and line numbers also
really help the authors to focus on the specific issues raised). In
the minor comments section, the reviewer should highlight
even small errors to demonstrate the depth of the review.
Authors tacitly implore that reviewers consider the native
language of the authors and hope that they do not reject good
science because of poor English. Dr Behrns’ examples included
some context of “how annoying” these minor issues are to the
reviewer. He made the following suggestions for reviewers:
1. If there are only a few issues, reviewers can just point
them out.
2. If there are many issues, reviewers can indicate that the
issues detract from what would otherwise be good work
and require the help of an English-speaking scientific
editor to make the work readable and interpretable.
3. If the work is sloppy overall and it is evident that the
article was not reviewed carefully before final submis-
sion, a good way to inform the authors is to say that the
carelessness in the review of the work submitted is a
potential reflection of the carefulness in the in-
vestigators’ experimentsdthis usually gets the attention
of the author. A comment about whether the senior
author has reviewed the work also is in order. After all,
the authors attest that they all have reviewed and
approved the final submission.
In his conclusion, Dr Behrns opined that communication,
collaboration, and continual improvement of our scientific
work are the goals of the academic surgery community. The
reviewers can help achieve these goals by signing their re-
views (we understand that there are pros and cons to signing a
review), providing helpful examples from experts or senior
investigators, and even offering to discuss the article with a
junior author who has done good work but may need scientific
guidance. This approach can establish a novel type of men-
toring relationship. The authors’ motivation should be to
advance the body of knowledge, report results, show progress,
perhaps aid with their promotion and tenure, and market
their program and themselves.
In essence, the authors expect and deserve the reviewers to
deliver a clear, concise, organized, and timely review that pla-
ces their work in the context of the literature. The review should
offer constructive criticism and options for improvements.
Overall, it must satisfy the authors’ sense of a fair review.
4 j o u r n a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 7 ) 1 e5
Top 10 list of reviewer “not to do’s”
The next discussant was David McFadden, MD, Chairman of
Surgery at the University of Connecticut and Co-Editor-in-
Chief of the JSR (now Editor Emeritus). The topic of Dr
McFadden’s talk was “Dave’s top 10 list of reviewer ‘not to
do’s.’” Dr McFadden started out by exhorting the group to
remember that although reviewing articles is a voluntary,
unpaid service, it is part of what we think of as a “culture of
service” to our profession. Peer review is an age-old process
established to ensure that a published article contributes to
scientific truth and that the work reflects a substantial
contribution. It was first used in the modern sense in the
Philosophical Transactions of The Royal Society in 1665 and was
then used routinely beginning in 1731 in Medical Essays and
Observations published by the Royal Society of Edinburgh.
Since the inception of peer review, numerous practical and
theoretic arguments opposing it have been suggested; each
argument has counterarguments, and none has prevented its
use in scientific disciplines.
Dr McFadden then stated that reviewers should not forget
the golden rule, that is, avoid the temptation to dismember
the authors with their insightful analysis. Remembering Lord
Acton’s comments about power tending to corrupt, reviewers
must always refrain from taking out their unrelated irritations
on the well-meaning, hapless authors and should bear in
mind that the writing of even a dreadful article is still hard
work. The authors unquestionably view their “baby” with love
and hopefully believe that they did the best they could in
raising it to adulthood.
It is uniformly better for the reviewer to direct all state-
ments to be about the article (e.g., “This article suffers from a
lack of attention to detail”) rather than about the authors
(e.g., “The authors should have paid more attention to
detail”).
The next caveat discussed by Dr McFadden was conflict of
interest. Reviewers must acknowledge any personal conflict of
interest and should not review work with which they have
such a conflict. Such personal conflicts of interest can include
competing research interests, a competing author, an article
from the reviewer’s institution, or an article written by a
colleague or friend (or enemy, for that matter). The review
must be unbiased, and if a reviewer thinks there may any
semblance of a conflict of interest, the reviewer should
politely refuse the review and inform the editor as to the
reason; this protects both the reviewer and the journal.
The next recommendation mentioned by Dr McFadden
was that reviewers should not simply respond with “accept”
or “reject,” or provide a noninformative review. An example of
this is, “I read the manuscript on hepatocyte ischemia that
you sent me. I found no problems with it. I think it is the first
time that this work has been done. This manuscript should be
published. Recommendation: accept.” Another example is, “I
read the manuscript on hepatocyte ischemia that you sent
me. I found major problems with it. This is not the first time
that this work has been done. This manuscript should be
rejected. Recommendation: reject.” Such reviews are of no
help to the authors or the editors.
Dr McFadden next made a plea for reviewers to not accept
an invitation to review an article unless they intend to do the
review in a timely manner. Also, reviewers should respond
“yes” or “no” to the invitation quickly to allow the online
article management service to solicit the next reviewer in
queue, if necessary. Most journals strive for rapid turnover, so
2 wk is generally an accepted standard. If for some reason a
reviewer cannot meet the deadline in a timely manner, the
editor should be notified. The editor will understand, as
everyone is busy.
Also, reviewers should try not to magnify or mention
their own work in the field. Reviewers are presumably asked
to review articles because of their track record and expertise
and should leave their egos out of the review. Stanley S.
Siegelman, an editor for Radiology, once suggested that re-
viewers could be broadly categorized as “Zealots” or “As-
sassins.” Zealots were those who support their field and
believe that any article in that area deserves publication.
Assassins are those who tend to believe only their work is
important and that the work of others in their field is infe-
rior. Both do a disservice to the time-honored process of peer
review: the former because they are inadequately critical,
and the latter because they fail to acknowledge the contri-
butions of others.
It is important that reviewers do not send mixed messages
to authors and reviewers; in other words, reviewers should
avoid being inconsistent with the comments they make to
either party. Reviews should not laud an article in the com-
ments to the authors while disparaging it in the confidential
comments to the editor. The comments to the authors should
match the recommendation in the confidential comments to
the editor; it is particularly challenging for an editor if a
reviewer lauds an article in the written review to the authors
but then chooses “major revisions” or “reject” as the recom-
mendation regarding publication. Also, reviewers should
specifically avoid commenting to the authors that the article is
or should be accepted or rejected; that is the editor’s decision,
and any decision communicated by the reviewer that may
differ from the ultimate decision of the editor makes it diffi-
cult for everyone.
Finally, the ability to “cut and paste” is a blessing and a
curse. Reviewers can use the antiplagiarism software of the
editorial platform to assess overlap with the literature. Many
journals currently use such a software program to check for
plagiarism. If there is reason for concern, the reviewer should
notify the editor immediately.
Why is reviewing articles important for your
career?
The final panelist was Scott A. LeMaire, MD. Dr LeMaire is a
Professor and Vice-Chair of Research in the Michael E. DeBa-
key Department of Surgery at Baylor College of Medicine in
Houston, Texas. He also serves as Professor in the Department
of Molecular Physiology and Biophysics, as well as the Director
of Research in the Division of Cardiothoracic Surgery at Baylor
College of Medicine, and is a member of the Professional Staff
of the Texas Heart Institute. He is currently the Editor-in-Chief
 j o u r n a l o f s u r g i c a l r e s e a r c h x x x ( 2 0 1 7 ) 1 e5
of the JSR. The topic of Dr LeMaire’s talk was “Why is
reviewing articles important for your career?”
Dr LeMaire first noted that reviewing articles enables re-
viewers to develop their own science. Reviewing articles will
help them learn about relevant cutting-edge science. They will
gain knowledge and insights into their field, related fields,
others’ work, and others’ perspectives on the field.
Second, reviewers learn how to write from a reviewer’s
perspective. Reading and reviewing others’ work, including
their articles and their responses to reviews, will improve
their own writing. Dr LeMaire likened this opportunity to
serving on a study section, which helps grant reviewers
improve their own grant writing.
Third, being a reviewer facilitates one’s academic career. In
many institutions, national and international recognition are
crucial criteria for promotion. Reviewing articles can provide
excellent visibility toward documenting one’s reputation as
an expert in the field, especially if the review is comprehen-
sive and insightful. After developing a track record with an
editor, a reviewer may consider asking the editor to write a
letter of support for an appointment or promotion. Reviewing
may also be a path toward becoming an editorial board
member and beyond. Reviewers who write prompt, thorough,
and insightful reviews are often invited to join editorial
boards, which can be the first step to greater leadership roles
with a journal and academic organizations. Dr LeMaire pre-
sented evidence that in 1998, the JSR had four associate editors
and 50 editorial board members. Of these, 10 went on to
become President of the Association for Academic Surgery or
Society of University Surgeons!
Serving as a journal reviewer also expands one’s network.
Editors and associate editors do notice people who provide
good reviews. Other opportunities may ensue. Serving as a
reviewer helps with career development by expanding one’s
CV and increasing national and international visibility.
In conclusion, being a reviewer contributes to a funda-
mental academic process and a key element of academic
citizenship. Reviewing articles helps academic surgeons
develop their science, helps them develop their careers, and
engages them in a fundamental academic endeavor. When
any of us submits an article for potential publication, someone
must review it, so we all need to participate in the process.
Acknowledgment
Author contributions: D.W.M., S.A.L., M.G.S., and K.E.B
contributed to conception and design. D.W.M., S.A.L., M.G.S.,
and K.E.B. drafted the article. D.W.M., S.A.L., M.G.S., and K.E.B.
made the critical revision of the article. The authors thank
Nicole Stancel, PhD, ELS, from the Texas Heart Institute, for
assistance with editing.
Disclosure
The authors reported no proprietary or commercial interest in
any product mentioned or concept discussed in the article.
5
references
1. Schulz KF, Altman DG, Moher D. for the CONSORT Group.
CONSORT 2010 statement: updated guidelines for reporting
parallel group randomised trials. Ann Int Med 2010;152:726e32.
2. Moher D, Liberati A, Tetzlaff J, Altman DG. The PRISMA Group.
Preferred reporting Items for systematic reviews and meta-
analyses: the PRISMA statement. Int J Surg 2010;8:336e41.
3. Ali PA, Watson R. Peer review and the publication process.
Nurs Open 2016;3:193e202.
4. Baig S, Ahmed S, Attique H. Reviewing a manuscript: Disparity
amongst peer reviewers’ priorities from basic health sciences
and clinicians. J Coll Physicians Surg Pak 2016;26:677e80.
5. Ezeala C, Nweke I, Ezeala M. Common errors in manuscripts
submitted to medical science journals. Ann Med Health Sci Res
2013;3:376e9.
6. Bordage G. Reasons reviewers reject and accept manuscripts:
the strengths and weaknesses in medical education reports.
Acad Med 2001;76:889e96.
7. Stahel PF, Moore EE. How to review a surgical paper: a guide for
junior referees. BMC Med 2016;14:29.
8. Kurdi MS. ’Scholarly peer reviewing’: the art, its joys and woes.
Indian J Anaesth 2015;59:465e70.
9. Alam M. How to review a manuscript. Dermatol Surg 2015;41:883e8.
10. Kapoor S, Sikka P, Saxena K. Publication of research article: an
art or science? Ann Med Health Sci Res 2013;3:96e8.
11. Cornelius JL. Reviewing the review process: identifying sources
of delay. Australas Med J 2012;5:26e9.
12. Govender P, Buckley O, McAuley G, et al. Does online submission of
manuscripts improve efficiency? JBR-BTR 2008;91:231e4.
13. Lovejoy TI, Revenson TA, France CR. Reviewing manuscripts
for peer-review journals: a primer for novice and seasoned
reviewers. Ann Behav Med 2011;42:1e13.
14. Alexandrov AV, Hennerici MG, Norrving B. Suggestions for
reviewing manuscripts. Cerebrovasc Dis 2009;28:243e6.
David W. McFadden, MD
Department of Surgery
University of Connecticut School of Medicine
Farmington, Connecticut
Scott A. LeMaire, MD
Michael E. DeBakey Department of Surgery
Baylor College of Medicine, and Department of Cardiovascular
Surgery, Texas Heart Institute
Houston, Texas
E-mail address: slemaire@bcm.edu
Michael G. Sarr, MD
Department of Surgery
Mayo Clinic, Rochester, Minnesota
Kevin E. Behrns, MD
Department of Surgery
Saint Louis University School of Medicine
St. Louis, Missouri
0022-4804/$ e see front matter
ª 2017 Published by Elsevier Inc..
http://dx.doi.org/10.1016/j.jss.2017.05.007