Original ResearchGeneral Otolaryngology

Developing a Registry for Complications

in Otorhinolaryngologic Surgery: Tonsil
Surgery as a Pilot Cohort

Otolaryngology
Head and Neck Surgery
2015, Vol. 153(1) 3440
American Academy of
OtolaryngologyHead and Neck
Surgery Foundation 2015
Reprints and permission:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/0194599815582156
http://otojournal.org

Johanna Ruohoalho, MD1, Antti A. Makitie, MD, PhD1,2,

Timo Atula, MD, PhD1, Annika Takala, MD, PhD3,
Harri Keski-Santti, MD, PhD1, Katri Aro, MD, PhD1,
Aaro Haapaniemi, MD1, Mari Markkanen-Leppanen, MD, PhD1, and
Leif J. Back, MD, PhD1

Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.

Keywords
complication registration, quality assurance, postoperative
complications, tonsillectomy, tonsillotomy

Abstract
Objective. To find a suitable method to prospectively register
all tonsil surgeryrelated complications.

Received September 1, 2014; revised February 4, 2015; accepted

March 26, 2015.

Study Design. Prospective cohort study.

Setting. Tertiary care center.
Subjects and Methods. From September 2011 to February
2012, patients undergoing tonsillectomy or tonsillotomy
were enrolled. A wide range of demographic and clinical
data including incidents of postoperative complications was
recorded prospectively, and patient records were reviewed
9 months after the end of study period. We evaluated the
coverage of prospective data recording, analyzed the complication rates, and assessed the process of registration.
Results. A total of 573 patients were recruited. The study
registry including 57 variables required the completion of
missing data before analysis. Of all 79 patients with a complication, 69.6% were captured prospectively at the emergency department, and the rest were found when reviewing
the patient records. The proportion of prospectively captured complications was highest for the most common complications (eg, 81.1% for secondary hemorrhage). The
overall complication rate was 13.8%. Secondary hemorrhage
was the most common complication, with the incidence of
9.6%.
Conclusion. We have demonstrated the initial feasibility of a
prospective complication registry for otorhinolaryngology
procedures, and the results can be applied accordingly. We
also present 5 practical recommendations when initiating a
functional registry. Particular attention should be paid to
recognition and registration of both rare and serious events.
Regular analysis of the results is required in order to
respond to possible changes in the incidence or nature of
complications.

ostoperative complications may cause significant harm

to patients and considerable costs to society. Relevant
information on the amount and type of surgical complications and the possible predictive factors is important when
assessing the quality of surgery. Systematic registration of surgical complications provides a tool to evaluate and improve
clinical outcome and to compare health care units, techniques,
materials, and instruments used. Prospective registering allows
monitoring of the quality of treatment in real time and prompt
reactions to any detected changes. Qualitative aspects of care
are also attaining increasing attention in research. In recent
years, several reports from various specialties focusing on registration of complications have been published.1-4
In otorhinolaryngologyhead and neck surgery (ORLHNS), publications addressing surgical complication registration systems are sparse. Some reported national projects
of registering ORL-HNS complications exist,5-7 as well as
some quality improvement programs,8 but continuous

Department of OtorhinolaryngologyHead and Neck Surgery, University

of Helsinki and Helsinki University Hospital, Helsinki, Finland
2
Division of Ear, Nose and Throat Diseases, Department of Clinical
Sciences, Intervention and Technology, Karolinska Institutet and Karolinska
University Hospital, Stockholm, Sweden
3
Department of Anesthesiology, Intensive Care, Emergency Medicine and
Pain Medicine, University of Helsinki and Helsinki University Hospital,
Helsinki, Finland
Corresponding Author:
Johanna Ruohoalho, MD, Department of OtorhinolaryngologyHead and
Neck Surgery, Helsinki University Hospital, PO Box 220, FI00029 HUH,
Helsinki, Finland.
Email: johanna.ruohoalho@hus.fi

Downloaded from oto.sagepub.com by guest on October 15, 2016

Ruohoalho et al

35

systematic registration databases reported in literature are

scarce. The spectrum of complications in ORL-HNS is
rather different from those arising from other fields of surgery, and general complication registries do not always take
site-specific factors into consideration. Therefore, development of registration systems for otorhinolaryngological
needs is important.
Tonsillectomy is one of the most commonly performed surgical operations.9 In view of being an elective procedure, tonsillectomy is associated with a relatively high rate of postoperative
complications, most commonly hemorrhage. The reported occurrence of primary hemorrhage is between 0.1% and 1.5%5,10-12
and that of secondary hemorrhage between 0.8% and 40%,12-16
depending on the age of the target population, the definition of
bleeding used, sample size, and methodological factors. Studies
based on analysis of national registries seem to underestimate the
risk of haemorrhage,11,17,18 indicating the need for a comprehensive registration system.
Our aim was to establish a feasible method to prospectively register all complications related to otorhinolaryngologic surgery with tonsil surgery as a pilot cohort. Our longterm objective is to create a surgical complication registration database, which would provide valuable information for
quality assurance and give us a benchmarking tool to
improve the results of surgery in our department.

Materials and Methods

The study protocol was approved by the Research Ethics
Committee of Helsinki University Hospital (HUH; DNRO
89/13/03/02/2011). From September 2011 to February 2012,
all patients undergoing tonsillectomy or tonsillotomy with
or without adenoidectomy at the Department of ORL-HNS,
HUH, were assessed for enrollment. Procedures carried out
due to malignancy, as a part of multilevel surgery for
obstructive sleep apnea, retonsillectomies, and acute operations were excluded. All operations were performed under
general anesthesia. Demographic and clinical data were
recorded prospectively in a database, which included 57
variables, such as age, sex, comorbidities (Charlson
Comorbidity Index),19,20 body mass index (BMI), American
Society of Anesthesiologists Physical Status Classification,
indication for surgery, educational level of surgeon (consultant vs resident), surgical technique, duration of operation,
length of hospital stay, and information regarding medication and laboratory results.
All patients received both written and oral aftercare education with instruction to contact our emergency department
(ED) in the event of any adverse event. These instructions
are part of our clinical routine, and they include ED contact
information and the description of symptoms and signs of
common tonsil surgery complications. Only complications
warranting a visit to the outpatient department or hospital
admission were registered, including the complications listed
and defined in Table 1. ED personnel with 1 to 2 doctors
and 2 to 4 nurses on duty at a time recorded the name and ID
of the patient, date, and suspected complication in a hard-copy
form. Only authors had access to transfer detailed information

on patients characteristics and complications from the medical

records into a complication database. Adenoidectomy-related
hemorrhage was not recorded. All the complications were
documented in terms of timing in relation to surgery, and
adverse postoperative events occurring within 9 months of the
operation were recorded.
To evaluate the accuracy of the data in our registry, we
reviewed all clinical charts retrospectively 9 months after
the end of the study period and completed missing information into the database. Discrepancies in the clinical findings
assessed from the medical records and the database were
resolved by consensus of the first and last author.
The comprehensiveness of prospective data recording
was examined, and postoperative complication rates were
evaluated.
The normality of data distribution was evaluated using
the Kolmogorov-Smirnov test. As all the continuous variables were nonparametric, we report values with median
and range for descriptive purposes. Statistical analyses were
performed using SPSS software (version 19.0, Armonk,
New York).

Results
During the 6-month study period, 794 patients underwent
tonsillectomy or tonsillotomy at the Department of ORLHNS, HUH. Of those excluded, 8 had tonsillectomy performed as part of the workup or treatment of head and
neck malignancy, 4 received simultaneous palatal surgery, 2 received retonsillectomy, and 64 underwent an
acute tonsillectomy. The remaining 143 did not give their
consent to participate. The final study group consisted of
573 patients (Figure 1), of whom 315 (55.0%) were
women. The median age was 19 years (range, 2-70). Four
hundred fifty-four (79.2%) patients underwent tonsillectomy, and 119 (20.8%) had tonsillotomy performed.
Patient characteristic and perioperative factors are presented in Table 2.
A total of 85 complications were registered in 79
patients. In the prospective data recording, we captured
64.7% of complications (69.6% of the patients with complications), and the rest were found in the revision of patient
charts. The most common complications were recorded
most comprehensively in prospective data collection; for
example, the prospectively registered proportion of minor
postoperative hemorrhages was 85.4% (Table 3).
The overall complication rate was 13.8%. Seventeen
patients (3.0%) had a major complication that necessitated
hospital admission. There was no mortality in the present
patient series. No medical complications were registered in
the database or noted when reviewing the patient records.
The total number of patients with postoperative hemorrhage was 55 (9.6%); 13 patients had 2 separate bleeding
episodes. All of the bleedings were secondary: 48 (87.2%)
were minor and 7 (12.7%) were major. However, none of
the patients needed blood transfusion. The median time
interval between the operation and the first bleeding episode
was 7 days (range, 2-13).

Downloaded from oto.sagepub.com by guest on October 15, 2016

36

OtolaryngologyHead and Neck Surgery 153(1)

Table 1. Definitions of Recorded Complications.

Complication
Hemorrhage
Primary
Secondary
Minor
Major
Infection
Minor
Major
Dehydration
Pain, major
Taste disturbance
Other otorhinolaryngologyhead and neck
surgery complications
Medical complications
Cardiovascular event
Cerebrovascular event
Thromboembolic complication
Delirium
Organ failure
Sepsis

Definition

Bleeding 24 h postoperatively

Bleeding .24 h postoperatively
Hemorrhage treated in outpatient clinic
Hemorrhage treated in operating theater
Surgical site infection treated in outpatient clinic
Surgical site infection needing hospital readmission
Dehydration needing hospital readmission
Pain needing hospital readmission
Impaired or altered sense of taste leading to an outpatient contact within 9 mo of
the surgery
Any surgical site problem manifested within 9 mo of the surgery leading to an
outpatient contact or hospital admission, for example, functional problems of the
palate or pharynx, tension sensations, or soft palate perforations
Within
Within
Within
Within
Within
Within

30
30
30
30
30
30

d of the
d of the
d of the
d of the
d of the
d of the

Table 3 presents the incidence of complications and proportions of prospectively registered events. Other complications not specified in Table 3 included 2 soft palate
perforations, 2 globus sensations, 1 functional disorder of
the palate, 1 worsening of tinnitus, 1 sensation of tongue
stiffness, and 1 case of prolonged pharyngeal pain, which
was relieved after 7 months. One 5-year-old girl was admitted to the hospital due to urticaria and fever 2 days after
tonsillectomy and one 6-year-old boy due to unclear muscular fatigue on the first postoperative day.

Discussion
The objective of registering surgical complications is to
improve the quality and safety of surgery. Our intention was
to find a feasible method to prospectively register surgical
complications with a view to specific features of ORL surgery. In this pilot project, we took tonsillectomies and tonsillotomies under scrutiny, since they are among the most
commonly performed surgical procedures at our institution.
This was a prospective cohort study of 573 patients undergoing tonsil surgery at our department during a 6-month
study period. We evaluated the comprehensiveness of data
collection and occurrence of postoperative complications in
terms of finding appropriate registration mode for continuous clinical use. We recorded 64.7% of complications prospectively, and the remaining 35.3% were observed when

operation
operation
operation
operation
operation
operation

reviewing the patient records 9 months after the end of the

study period.
Some publications addressing the accuracy of prospective
complication registration in other fields of surgery exist, but
we found no reports on the comprehensiveness of ORLHNS complication registries. Veen et al21 evaluated the
accuracy of their prospective registration system at the
department of surgery and found the rate of missed complications to be 27%. Another study assessing the completeness of 3 different complication databases in colorectal
cancer surgery found the proportion of missed events to
vary between 7% and 69%.22 In our pilot study, one-third of
complications were not registered prospectively. This rate is
disappointing, but it seems to be comparable with the
above-mentioned studies.
Morbidity and mortality conferences have been part of
quality assurance and education in surgery departments
since the early 1900s23 and are now a requirement for
departments with surgical education programs, including
ORL-HNS, in many countries.24 However, the systematic
registration of surgical complications in departments of
ORL-HNS is scarce. The Swedish Association for
Otorhinolaryngology and Head and Neck Surgery has
launched several national registries that broadly reflect ear,
nose, and throat care. One of those is National Tonsil
Surgery Register started in 1997.5 The Association of
Surgeons of the Netherlands has developed a nationwide

Downloaded from oto.sagepub.com by guest on October 15, 2016

Ruohoalho et al

37

Figure 1. Flow chart of study participants and complication registering.

reporting system for adverse surgical events,4,7 also covering head and neck procedures. In the United States, the
National Surgical Quality Improvement Program (NSQIP)
collects data in a standardized manner on 30-day postoperative outcomes for a randomized sample of patients
undergoing surgical procedures.25 In 2008, 47 ORL-HNS
programs submitted data.8 In a recent publication analyzing NSQIP data, the most common postoperative complications after tonsillectomy were pneumonia, urinary tract
infection, and superficial site infections,26 reflecting the
inability of general surgical complication registries to
catch ORL site-specific complications.
Our aim was to create a registry covering special features
of ORL surgery as well as possible. We included a large
number of variables to be collected into our pilot registry,
but many of them appeared to be rather useless in this relatively young and healthy patient population. Factors
reported to have an impact on complication occurrence in
tonsil surgery seem to be conventional (eg, age, sex,
BMI),11-13 and relevant background information is directly
available in the patient information system used in our operating rooms. We did exploit existing patient data, but they

were transferred manually to the complication database. In

an optimal situation, the registry would retrieve all relevant
background information automatically from the other hospital databases, and we are developing our system toward
that. In addition, a structured Internet-based questionnaire
would enable patient feedback to be incorporated into a
registry.
As we noticed, there will be incomplete information if
the responsibility of data collection is not precisely assigned
to a specific staff member. Although the effort needed from
our ED personnel was minimal, almost one-third of the
patients suffering from a complication were not registered
in the system. As the reliability and quality of the registry
are strongly dependent on the consistency of definitions and
accuracy of data,27 we tried to minimize reporting errors
and subjective interpretation by keeping the responsibility
for complication data input with the researchers at this pilot
stage. Since the vast majority of tonsil operations is carried
out as day-case surgery and patients are rarely admitted to
ward care, this aforementioned strategy turned out to be
unsuccessful. To achieve a comprehensive registration
system, all physicians involved in the treatment of surgical

Downloaded from oto.sagepub.com by guest on October 15, 2016

38

OtolaryngologyHead and Neck Surgery 153(1)

Table 2. Patient Characteristics and Surgery-Related Factors.

Total number of patients
Age, y (median, range)
\16, n (%)
.16, n (%)
Sex, n (%)
Male
Female
BMI (median, range)
CCI, n (%)
Unadjusted
0
1
2
3
Age adjusted
0
1
2
3
4
ASA, n (%)
1
2
3
Performed surgery, n (%)
Tonsillectomy
Tonsillotomy
Indication for surgery, n (%)
Chronic tonsillitis
Recurrent acute tonsillitis
History of peritonsillar abscess
Upper airway obstruction
Periodic fever
Other
Operation technique, n (%)
Bipolar dissection
Cold steel dissection
RFA
Use of local anesthetics, n (%)
Surgeon, n (%)
Consultant
Resident
Duration of operation (median, range)
Postoperative follow-up, n (%)
Day case surgery
Short stay surgery

573
19 (2-70)
250 (43.6)
323 (56.4)
258 (45.0)
315 (55.0)
21.0 (12.1-41.6)

527
42
3
1

(92.0)
(7.3)
(0.5)
(0.2)

527
38
4
3
1

(92.0)
(6.6)
(0.7)
(0.5)
(0.2)

457 (79.8)
107 (18.7)
9 (1.6)
454 (79.2)
119 (20.8)
216
79
48
216
5
9

(37.7)
(13.8)
(6.4)
(37.7)
(0.9)
(1.6)

323
147
103
157

(56.4)
(25.7)
(18.0)
(27.4)

397 (69.3)
176 (30.7)
0:20 (0:06-1:03)
558 (97.4)
15 (2.6)

Abbreviations: BMI, body mass index; CCI, Charlson Comorbidity Index;

ASA, American Society of Anesthesiologists Physical Status Classification;
RFA, radiofrequency ablation.

patients should be engaged in the registration process, and

registration methods and definitions used should be clear
and uniform. In NSQIP, surgical risk assessment nurses are
responsible for data collection. We agree with Veen et al7

Table 3. Complication Rates and Proportion of Prospectively

Registered Events.
N = 573

Secondary hemorrhage
Minor
Major
Infection minor
Infection major
Dehydration
Pain major
Taste disturbance
Other complication
Any complication

n
55
48
7
6
1
4
4
5
10
79

%
9.6
8.4
1.2
1.0
0.2
0.7
0.7
0.9
1.7
13.8

Prospectively
Registered Events
n
45
41
4
4
1
0
1
2
2
55

%
81.8
85.4
57.1
33.3
100.0
0
25.0
40.0
20.0
69.6

that the best person to report complications is the surgeon,

and the role of trained nurses in this aspect should be
supportive.
A significant limitation of systems managed by hospital
personnel other than the surgeons is their retrospective
nature. We strongly believe that to attain the best benefit,
efficacy, and accuracy, the registration of data should be
made prospectively. We found that rare complications were
not recorded prospectively as comprehensively as the most
common and obvious postsurgical adverse events. In prospective registration process, particular attention should be
paid to the early recognition of both serious and unexpected
events, as their detection is valuable in identifying the
whole spectrum of complications. Therefore, it is essential
to educate personnel involved in the registration process to
recognize also the rare complications. To detect and respond
to trends in the incidence of complications, prospectively
collected data should be analyzed in a real-time manner.
Because a part of our goal in registration of complications is measuring and improving the quality of care, allocating the resources, and achieving financial savings in
terms of decreasing unnecessary treatment costs, the complications that require hospital resources are the most relevant
ones in this respect. To use a registry for those purposes, it
is important to ensure the reliability of registered data and
implement a quality monitoring system. As our stated scope
was to register only the complications needing outpatient
ED or hospital contact, we concluded the retrospective chart
review to be an appropriate means of data quality control
and did not contact the patients directly. In the future, the
audit system of our continuous complication registry will be
based on a randomized chart review.
Our study has some limitations. Unfortunately, a significant proportion of patients had to be excluded in the
absence of a written informed consent, which the Research
Ethics Committee required because of the prospective study
design. This high dropout rate is a major limitation for the

Downloaded from oto.sagepub.com by guest on October 15, 2016

Ruohoalho et al

39

study. Second, the intention was that some of authors would

collect data from the ED records on a daily basis and enter
them into the database, but that aim was not fully achieved.
We screened all medical records afterward for quality assurance purposes and noticed that the coverage of prospective
complication recording was not optimal. As our department
is the only ORL-HNS center in the area and patients were
advised to contact our ED in the case of any adverse event,
we are confident of the accuracy of the operation-related
complications registered. In a possible event of a medical
complication (cardiovascular events, cerebrovascular events,
thromboembolic complications, delirium, organ failure,
sepsis), patients might end up in various health care units in
the area. As all units patient records were not available for
revision through our current electronic patient record
system, these medical complications might be susceptible to
underreporting. On the other hand, in this specific registry
with a relatively young and healthy population, we believe
that these events are uncommon, and their relation to the
performed surgery remains uncertain.
In summary, our practical suggestions for initiating clinically functioning complication registry would include 5
steps. First, the responsibility of data recording should be
strictly assigned to the physician initiating the treatment of
complication. Second, the registry database should be easily
accessible on every workstation and integrated into other
patient information systems. Third, only essential information should be included in the registry, so that additional
work would be minimal and the threshold for registration
would remain low. Fourth, as the complications are somewhat procedure specific, necessary procedure-specific data
should be incorporated into the registered variables. Finally,
a regular audit process should be designed to ensure the
comprehensiveness and quality of collected data.

Conclusion
We have demonstrated the initial feasibility of a prospective
complication registry for otolaryngology procedures, and
the results can be applied accordingly. We also present 5
practical recommendations when initiating a functional
complication registry. Particular attention should be paid to
registration of both serious and rare events. Regular analysis
of the results is required in order to respond to possible
changes in the incidence or nature of complications.
Acknowledgment
Matti Westman, MD, took part in designing the work and acquisition of data.

Authors Note
This trial was registered at clinicaltrials.gov. NCT 02049372.

Author Contributions
Johanna Ruohoalho, acquisition, analysis and interpretation of
data, main responsibility for writing the manuscript; Antti A.
Makitie, designing the work, analysis and interpretation of data,

took part drafting of the work and revised it critically; Timo

Atula, acquisition of data, critical revision of manuscript; Annika
Takala, acquisition and analysis of data, critical revision of manuscript; Harri Keski-Santti, acquisition of data, critical revision of
manuscript; Katri Aro, acquisition of data, critical revision of
manuscript; Aaro Haapaniemi, acquisition of data, critical revision
of manuscript; Mari Markkanen-Leppanen, acquisition of data,
critical revision of manuscript; Leif J. Back, acquisition, analysis,
and interpretation of data, main responsibility for designing the
work, took part in drafting the work and revised it critically.

Disclosures
Competing interests: None.
Sponsorships: None.
Funding source: The Helsinki University Hospital Research Fund
(no role in study).

References
1. Adler A, Lieberman D, Aminalai A, et al. Data quality of the
German screening colonoscopy registry. Endoscopy. 2013;45:
813-818.
2. Lawson EH, Louie R, Zingmond DS, et al. A comparison of
clinical registry versus administrative claims data for reporting
of 30-day surgical complications. Ann Surg. 2012;256:973-981.
3. Rebasa P, Mora L, Luna A, Montmany S, Vallverdu H,
Navarro S. Continuous monitoring of adverse events: influence
on the quality of care and the incidence of errors in general
surgery. World J Surg. 2009;33:191-198.
4. Marang-van de Mheen PJ, Stadlander MC, Kievit J. Adverse
outcomes in surgical patients: implementation of a nationwide
reporting system. Qual Saf Health Care. 2006;15:320-324.
5. Hessen Soderman AC, Ericsson E, Hemlin C, et al. Reduced
risk of primary postoperative hemorrhage after tonsil surgery
in Sweden: results from the national tonsil surgery register in
sweden covering more than 10 years and 54,696 operations.
Laryngoscope. 2011;121:2322-2326.
6. Lowe D, van der Meulen J, Cromwell D, et al. Key messages
from the national prospective tonsillectomy audit. Laryngoscope.
2007;117:717-724.
7. Veen EJ, Janssen-Heijnen ML, Leenen LP, Roukema JA. The
registration of complications in surgery: a learning curve.
World J Surg. 2005;29:402-409.
8. Stachler RJ, Yaremchuk K, Ritz J. Preliminary NSQIP results:
a tool for quality improvement. Otolaryngol Head Neck Surg.
2010;143:26-30, 30.e1-3.
9. Rautiainen H, Rasilainen J. Episodes of care: procedures and interventions 2010. National Institute for Health and Welfare. http://
www.thl.fi/tilastoliite/tilastoraportit/2011/Tr41_11.pdf.
Accessed
February 4, 2015.
10. Liu JH, Anderson KE, Willging JP, et al. Posttonsillectomy
hemorrhage: what is it and what should be recorded? Arch
Otolaryngol Head Neck Surg. 2001;127:1271-1275.
11. Tomkinson A, Harrison W, Owens D, Harris S, McClure V,
Temple M. Risk factors for postoperative hemorrhage following tonsillectomy. Laryngoscope. 2011;121:279-288.
12. Tolska HK, Takala A, Pitkaniemi J, Jero J. Post-tonsillectomy
haemorrhage more common than previously describedan
institutional chart review. Acta Otolaryngol. 2013;133:181-186.

Downloaded from oto.sagepub.com by guest on October 15, 2016

40

OtolaryngologyHead and Neck Surgery 153(1)

13. Sarny S, Ossimitz G, Habermann W, Stammberger H.

Hemorrhage following tonsil surgery: a multicenter prospective study. Laryngoscope. 2011;121:2553-2560.
14. Sarny S, Habermann W, Ossimitz G, Schmid C, Stammberger
H. Tonsilar haemorrhage and re-admission: a questionnaire
based study. Eur Arch Otorhinolaryngol. 2011;268:1803-1807.
15. Alexander RJ, Kukreja R, Ford GR. Secondary posttonsillectomy haemorrhage and informed consent. J Laryngol
Otol. 2004;118:937-940.
16. Evans AS, Khan AM, Young D, Adamson R. Assessment of
secondary haemorrhage rates following adult tonsillectomya
telephone survey and literature review. Clin Otolaryngol
Allied Sci. 2003;28:489-491.
17. Kvaerner KJ. Benchmarking surgery: secondary post-tonsillectomy
hemorrhage 1999-2005. Acta Otolaryngol. 2009;129:195-198.
18. McKenna M. Postoperative tonsillectomy/adenoidectomy
hemorrhage: a retrospective chart review. ORL Head Neck
Nurs. 1999;17:18-21.
19. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new
method of classifying prognostic comorbidity in longitudinal
studies: development and validation. J Chronic Dis. 1987;40:
373-383.
20. Hall WH, Ramachandran R, Narayan S, Jani AB, Vijayakumar
S. An electronic application for rapidly calculating Charlson
comorbidity score. BMC Cancer. 2004;4:94.

21. Veen EJ, Janssen-Heijnen ML, Bosma E, de Jongh MA,

Roukema JA. The accuracy of complications documented in a
prospective complication registry. J Surg Res. 2012;173:54-59.
22. Gunnarsson U, Seligsohn E, Jestin P, Pahlman L. Registration
and validity of surgical complications in colorectal cancer surgery. Br J Surg. 2003;90:454-459.
23. Gordon LA. Gordons Guide to the Surgical Morbidity and
Mortality Conference. Philadelphia, PA: Hanley & Belfus;
1994.
24. Accreditation Council of Graduate Medical Education. Program
requirements for graduate medical education in otolaryngology. IV.
A.3.b. http://www.acgme.org/acgmeweb/Portals/0/PFAssets/Program
Requirements/280_otolaryngology_07012014.pdf. Updated July 1,
2013. Accessed February 4, 2015.
25. Khuri SF, Daley J, Henderson W, et al. The Department of
Veterans Affairs NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the
measurement and enhancement of the quality of surgical care.
national VA surgical quality improvement program. Ann Surg.
1998;228:491-507.
26. Chen MM, Roman SA, Sosa JA, Judson BL. Safety of adult
tonsillectomy: a population-level analysis of 5968 patients.
JAMA Otolaryngol Head Neck Surg. 2014;140:197-202.
27. Levine MN, Julian JA. Registries that show efficacy: good,
but not good enough. J Clin Oncol. 2008;26:5316-5319.

Downloaded from oto.sagepub.com by guest on October 15, 2016