H e a l t h C a r e Po l i c y a n d Q u a l i t y O r i g i n a l R e s e a r c h

Patterson et al.
Brain CT for Atraumatic Headache and 30-Day ED Revisits

Downloaded from www.ajronline.org by 114.121.237.109 on 01/04/17 from IP address 114.121.237.109. Copyright ARRS. For personal use only; all rights reserved

Health Care Policy and Quality

Original Research

The Association Between Use

of Brain CT for Atraumatic
Headache and 30-Day Emergency
Department Revisitation
Brian W. Patterson1
Peter S. Pang2
Lora AlKhawam 3
Azita G. Hamedani1
Eneida A. Mendonca4
Ying-Qi Zhao5
Arjun K. Venkatesh6
Patterson BW, Pang PS, AlKhawam L, et al.

Keywords: appropriate use, CT brain, emergency,

headache

OBJECTIVE. The purpose of this article is to describe the association between initial CT
for atraumatic headache and repeat emergency department (ED) visitation within 30 days of
ED discharge.
MATERIALS AND METHODS. A retrospective observational study was performed
at an academic urban ED with more than 85,000 annual visits. All adult patients with a chief
complaint of headache from January through December 2010 who were discharged after ED
evaluation were included in the analysis. Patients were excluded if they were transferred, died
in the ED, or had a diagnosis indicating a traumatic mechanism. A propensity scorematched
logistic regression model was used to determine whether the use of brain CT was associated
with the primary outcome of ED revisitation within 30 days, controlling for potential confounding variables.
RESULTS. Of 80,619 total patient visits to the ED during the study period, 922 ED discharges with a chief complaint of headache were included. A total of 139 (15.1%) patients revisited within 30 days. The return rate was 11.2% among patients who underwent CT at their
initial visit and 21.1% among those who did not. In the adjusted analysis, controlling for age,
race, sex, insurance status, triage vital signs, laboratory values, and triage pain level, the odds
ratio for revisitation given CT performance was 0.49 (95% CI, 0.270.86).
CONCLUSION. After adjustment for clinical factors, we found that patients who underwent a brain CT examination for atraumatic headache at an initial ED visit were less likely to
return to the ED within 30 days. Future appropriate use quality metrics regarding ED imaging use may need to incorporate downstream health care use.

DOI:10.2214/AJR.16.16330
Received February 16, 2016; accepted after revision
April 27, 2016.
P. S. Pang is a consultant for Janssen, Medtronic, Novartis,
Trevena, scPharmaceuticals, Cardioxyl, Roche
Diagnostics, and Relypsa and has received honoraria from
Palatin Technologies and research support from Roche and
Novartis. A. K. Venkatesh works under contract with the
Centers for Medicare & Medicaid Services to develop
hospital outcome and efficiency measures related to the
use of medical imaging and also works under contract with
the Centers for Medicare & Medicaid Innovation to lead a
national quality collaborative regarding imaging use in the
emergency department.
Based on a presentation at the Society for Academic
Emergency Medicine 2013 annual meeting, Dallas, TX.
WEB
This is a web exclusive article.
AJR 2016; 207:W117W124
0361803X/16/2076W117
American Roentgen Ray Society

mericans increasingly receive

unscheduled acute care in the
emergency department (ED) [1].
During these visits, there has
been a concurrent rapid increase in the use of
CT imaging [2]. Many nonemergency physicians and policy makers have criticized this
trend in both the lay press [3] and medical literature [2]. A variety of efforts have aimed to
reduce CT use within the ED, given the costs
and risks associated with ionizing radiation.
Specifically, the use of CT of the brain for
patients with atraumatic headache has been

identified as potentially inefficient, as evidenced by wide variation in ordering rates

between ED providers and low diagnostic
yields [4, 5]. In response, the Centers for
Medicare & Medicaid Services proposed a
quality measure, known as OP-15, to reduce
unnecessary CT for atraumatic headache [6],
using a definition based on administrative
claims data. The ability to evaluate the appropriateness of imaging by use of administrative claims data has been called into question [7], and the implementation of OP-15
has since been delayed.

1BerbeeWalsh Department of Emergency Medicine, University of WisconsinMadison School of Medicine and Public

Health, 800 University Bay Dr, Ste 310, Mail Code 9123, Madison, WI 53705. Address correspondence to B. Patterson
(bpatter@medicine.wisc.edu).
2 Department of Emergency Medicine, Indiana University School of Medicine, Indianapolis, IN.
3 Department of Emergency Medicine, Northwestern University, Chicago, IL.
4 Department of Biostatistics and Medical Informatics, Department of Pediatrics, University of Wisconsin School of
Medicine and Public Health, Madison, WI.
5 Public Health Sciences Division, Fred Hutchinson Cancer Research Center, Seattle, WA.
6 Department of Emergency Medicine and Center for Outcomes Research and Evaluation, Yale University School of
Medicine, New Haven, CT.

AJR:207, December 2016 W117

Patterson et al.

Downloaded from www.ajronline.org by 114.121.237.109 on 01/04/17 from IP address 114.121.237.109. Copyright ARRS. For personal use only; all rights reserved

All Initial Visits With Chief

Complaint of Headache
n = 1268
Admitted
289 (22.8%)
Discharged
n = 979
Transferred
Left Against Medical Advice
Left Hospital Without Being Seen
57 (5.8%)

Died Within
30 Days
0 (0.0%)
Analyzed Discharges
922

No CT Performed
360

30-Day Revisit
76 (21.1%)

No Revisit
284 (78.9%)

CT Performed
562

30-Day Revisit
63 (11.2%)

Had CT, Matched to No CT Group

According to Propensity Score: 360

No Revisit
499 (88.8%)

Unmatched
Comparison

30-Day Revisit
37 (10.3%)

No Revisit
323 (89.7%)

Matched
Comparison

Fig. 1Patient flow diagram.

Beyond the limitations of available data

sources, there is a lack of consensus regarding
an approach for evaluating the appropriateness of ED interventions. Although overuse is
a major quality concern, a clear definition of
appropriate use does not exist for most emergency care services [8]. Furthermore, prior
efforts to study and reduce imaging overuse
have focused on a single ED visit without consideration of downstream consequences, such
as delayed diagnoses, ED revisits, or rehospitalization. With up to 25% of all ED visits
generating a repeat ED visit within 30 days
[9], a better understanding of the relationship
between imaging use and these downstream
outcomes is necessary to evaluate the effect
of emergency care diagnostics on future or

W118

downstream health care use by patients. Specifically, we hypothesized that initial use of
CT for atraumatic headache would be associated with a lower likelihood of ED revisitation
and potentially reveal the inherent tradeoffs
between distinct measures of resource use.
Prior studies investigating ED returns
have been limited to administrative claims
data and therefore largely sought to identify demographic factors associated with revisits, including advanced age [10] and lack
of access to primary care [11]. Although
these analyses reveal important disparities
in access to care, they do not identify easily modifiable factors or describe the specific
relationship between emergency care interventions and ED revisitation.

Examining the relationship between index CT (i.e., CT at the first ED visit) and
subsequent health care use, defined as ED
revisits, will better inform providers seeking to better manage acute care resources
for a population of patients and policy makers developing quality measures that seek
to reduce exposure to ionizing radiation
and reduce health care costs. A better understanding of this relationship would help
identify outcomes of importance as well as
potential targets for quality improvement efforts. Although administrative claims databases are limited in their ability to evaluate
this relationship, data created by an electronic health record (EHR) can provide additional information on clinical variables

AJR:207, December 2016

Brain CT for Atraumatic Headache and 30-Day ED Revisits

Age

Downloaded from www.ajronline.org by 114.121.237.109 on 01/04/17 from IP address 114.121.237.109. Copyright ARRS. For personal use only; all rights reserved

Male Sex
Black Race
Other Race
Private Insurance
Medicaid Insurance
Medicare Insurance
Sodium Low (< 134 mEq/L)
Sodium High (> 142 mEq/L)
Sodium Not Evaluated
Low Glucose
WBC Low
WBC High
WBC Not Evaluated
Low Hemoglobin
High Platelets
Triage Pulse Oximetry Not Recorded
Heart Rate > 100 beats/min
Heart Rate Not Recorded
Temperature > 100.4F
Temperature Not Recorded
Hypertensive
Blood Pressure Not Recorded
Triage Pain Score
(Each additional 1/10)
CT Brain Performed

Unmatched OR
1

Matched OR

Fig. 2Odds ratios (ORs) for 30-day revisit.

relevant to an initial ED visit: chief complaints, vital signs, and use of specific laboratory and imaging tests [12, 13]. Showing
the feasibility and benefit of using clinically enriched data to evaluate ED imaging use
will also inform future efforts seeking to
use health care information technology for
the measurement of health system efficiency. We have performed a prior retrospective
study that identified an association between
the use of abdominal CT and lower 30-day
revisit rates [14]. This study uses a propensity score analysis to evaluate a similar association among a different population within the same dataset and builds on the prior
work examining the effects of imaging use
in the ED on revisit rates.

Our primary objective is to describe the

relationship between CT of the brain at first
ED visit and ED revisitation within 30 days
for patients who present to the ED with atraumatic headache.
Materials and Methods
Study Design and Setting

We conducted a retrospective observational

study using EHR data from consecutive patients
from a single academic urban ED with more than
85,000 annual visits.

Selection of Participants

All patients 18 years old or older with a chief

complaint of headache seen between January and
December 2010 who were treated and discharged

from the ED were included in the analysis (data

were collected from December 1, 2009, through
January 31, 2011, for a 30-day lead-in and 30-day
follow-up period). Patients were excluded if at the
initial visit they were admitted to the hospital or an
observation unit, left against medical advice, or if an
ED discharge diagnosis was not entered (assumed
to represent patients who left without being seen).
Because the EHR uses a free-text field for chief
complaint (as documented by the triage nurse),
headache chief complaints were defined as those
containing any of the words headache, head
pain, or HA. Patients were excluded from analysis if their final ED discharge diagnosis indicated traumatic presentation (International Classification of Diseases9 codes containing the terms
concussion, head injury, fracture, or fall).

AJR:207, December 2016 W119

Downloaded from www.ajronline.org by 114.121.237.109 on 01/04/17 from IP address 114.121.237.109. Copyright ARRS. For personal use only; all rights reserved

Patterson et al.
No patients meeting these inclusion and exclusion criteria died in the ED, precluding enrollment. The chief complaint was chosen to capture
patients defined by their presentation, evaluation,
and management, as opposed to their final diagnosis. Discharge diagnoses were not included in the
dataset because these have poor concordance with
ED patients reason for visit or their need for admission or further advanced care [15].

Methods and Measurements

Analysis was conducted on data abstracted

from the Enterprise Data Warehouse [16, 17], a
comprehensive repository of all data contained
within all EHRs within a single health care system. The Enterprise Data Warehouse includes all
clinical components of the EHR with linkage to
other institutional databases that contain claims
and use data [16, 17]. Variables were abstracted by
Enterprise Data Warehouse programming staff;
individual charts were not reviewed. The following variables were analyzed for each patient encounter: age, race, sex, insurance status, triage
vital signs, laboratory values, triage pain level,
performance of brain CT, disposition, and ED revisit, as well as patient death identified within the
Social Security Death Index. These variables were
chosen a priori by author consensus, consistency
with cited previous literature, and availability
for analysis. This investigation received approval
from the institutional review board of Northwestern University.

Outcomes

The primary measured outcome was return visit to the ED for any reason within 30 days of initial
ED visit for atraumatic headache.

Analysis

A multiple logistic regression model was used

to examine the association between performance
of CT and the primary outcome of return visits
to the ED within 30 days of initial visit. This was
performed on both the overall dataset and a subset
of patients matched using a propensity score. Figure 1 shows the patient allocation into treatment
groups. The propensity score analysis was performed to control for the fact that many of the covariates collected for analysis possibly influenced
both the decision to perform CT of the brain and
the likelihood of return to the ED. We performed
propensity score matching using MatchIt R software [16]. Specifically, a logistic regression model
was used to estimate the propensity score, defined
as the probability of performing CT of the brain,
conditional on age, sex, insurance, triage pain value, sodium level, WBC count, alcohol level, glucose level, platelet count, triage temperature, tri-

W120

age blood pressure, triage heart rate, and triage

pulse oximetry. These variables were selected a
priori by the authors on the basis of their potential
to influence both the choice to perform CT and
the likelihood of returning to the ED. Our overall
goal was to compare two groups, whose who underwent CT and those who did not, with regard to
the outcome of return visits. In the matched analysis, we selected a subset of patients from the larger
group (those who underwent CT) for comparison
with the group that did not undergo CT matched
on this propensity score. The matching procedure
can reduce the dependence of causal inference on
statistical modeling assumptions [1719]. With the
preprocessed matched data, the subsequent analysis is more accurate and less sensitive to model
specification [20, 21].
Where appropriate, continuous variables were
categorized on the basis of more clinically relevant cutoffs (i.e., continuous sodium values were
replaced by low, high, normal, and not evaluated). Missing values were given their own category
where they existed; because the database was abstracted from the EHR, missing laboratory values
were thought to be much more likely to reflect tests
or procedures not performed rather than information available to clinicians that was not entered into
the database. Because it is not necessarily appropriate to order laboratory tests for all patients presenting with headache, the choice to not perform them
was thought to be most accurately reflected by creating a not evaluated value and, thus, avoiding biasing the study away from sick patients by censoring those without complete information [22].
Given the structure of our model, multicollinearity between predictor variables was an expected finding (e.g., the decision not to obtain a sodium level is likely collinear with the decision not
to obtain a WBC count). This would be expected
to broaden the 95% CIs of these predictors. However, because this would bias the model conservatively and the primary purpose of the study was to
evaluate the effect of decision to perform CT as
opposed to examine the other variables as independent predictors, we did not further complicate
the model by assessing or controlling for interaction or collinearity.

Results
Characteristics of Study Subjects
Figure 1 graphically depicts patient flow
within the study. Table 1 describes the collected demographic variables for the study
group. Overall, compared with the patients
who did not undergo CT, those who did undergo imaging at the initial ED visit were
more likely to be older, have Medicare or private insurance, and have laboratory tests per-

formed but had similar vital signs, laboratory

test results (among those evaluated), and triage pain scores.
Main Results
Among the 922 discharged patients meeting the inclusion criteria, the 30-day revisit
rate was 15.1% (n = 139). The return rate was
11.2% among patients who underwent CT at
their initial visit and 21.1% among those who
did not, an absolute difference of 9.9% (95%
CI, 4.914.9%). On revisits, patients had similar rates of undergoing CT (17.7% among
those who already underwent CT at the initial visit vs 16.1% among those who did not).
Figure 2 depicts odds ratios (ORs) and
95% CIs for 30-day revisitation in both the
matched and unmatched regression models.
The unmatched logistic regression shows an
OR of 0.52 (95% CI, 0.340.77) for ED revisitation if imaging was performed. When
we included only cases matched according
to propensity score (360 sets of matched patients) for undergoing CT, the OR was 0.49
(95% CI, 0.270.86). ROC curves were created to evaluate the fit of both regressions;
the areas under the unmatched and matched
curves were 0.71 and 0.72, respectively.
Hosmer-Lemeshow statistics were calculated for both regressions and were greater
than 0.05, with values of 0.25 for the unadjusted model and 0.11 for the adjusted model. In the matched analysis, only head CT
and private insurance were significantly
associated with lower return rates, whereas increasing triage pain value was associated with higher rates of return. In the unmatched analysis, male sex and increasing
age were also significantly associated with
increased rates of 30-day revisit.
Discussion
In this sample, more than one in seven patients discharged after a chief complaint of
headache revisited the ED within 30 days.
Patients undergoing CT were approximately half as likely to return as those who did
not, both before and after using a propensity
score to adjust for the likelihood of undergoing CT. A possible explanation for this finding is that CT provided reassurance to patients, thereby preventing return ED visits.
Another possible explanation is that outpatient providers, particularly primary care clinicians, may be able to better focus on and
manage the symptoms of acute headache
given the knowledge of negative CT findings. Ideally, future studies would be able to

AJR:207, December 2016

Brain CT for Atraumatic Headache and 30-Day ED Revisits

TABLE 1: Baseline Population Characteristics
Variable
Downloaded from www.ajronline.org by 114.121.237.109 on 01/04/17 from IP address 114.121.237.109. Copyright ARRS. For personal use only; all rights reserved

Age (y)

CT Imaging Performed

CT Imaging Not
Performed (n = 360)

Unmatched (n = 562)

Matched (n = 360)

39.0 (37.440.5)

42.5 (41.243.7)

45.9 (44.347.5)

Sex
Female

70.8 (65.975.3)

71.0 (67.174.6)

71.7 (66.876.0)

Male

29.2 (24.734.1)

29.0 (25.432.9)

28.3 (23.933.2)

White

42.5 (37.547.8)

48.4 (44.252.5)

49.3 (44.154.5)

Black

45.9 (40.851.1)

35.1 (31.339.2)

35.0 (30.240.1)

Other

11.5 (8.615.3)

16.5 (13.619.8)

15.7 (12.319.9)

18.1 (14.422.4)

14.9 (12.218.1)

14.2 (10.918.2)

8.3 (2.911.7)

11.7 (9.314.7)

14.4 (11.218.5)

Race

Insurance status
Self-pay or uninsured
Medicare
Medicaid

19.4 (15.723.9)

12.1 (9.615.1)

8.9 (6.312.3)

Private or other

54.2 (49.059.3)

61.2 (57.165.2)

62.5 (57.467.4)

Normal (50100 beats/min)

83.6 (79.487.1)

88.1 (85.190.5)

91.7 (88.394.1)

Low (< 50 beats/min)

No observations

0.05 (0.011.6)

0.8 (0.32.6)

High (> 100 beats/min)

15.0 (11.719.1)

10.9 (8.513.7)

7.2 (5.010.4)

1.4 (0.63.3)

0.5 (0.21.6)

0.3 (0.02.0)

93.6 (90.695.7)

89.9 (87.192.1)

87.8 (84.090.8)

Triage heart rate

Not recorded
Triage blood pressure
Normotensive
Hypotensive (systolic blood pressure < 80 mm Hg)

Hypertensive (systolic blood pressure > 180 or diastolic blood pressure > 100 mm Hg)

4.7 (3.07.5)

9.3 (7.111.9)

11.4 (8.515.1)

Not recorded

1.7 (0.73.7)

0.9 (0.42.1)

0.8 (0.32.6)

Triage temperature
Normal (< 100.4F [38C])

93.0 (89.995.3)

96.6 (94.897.8)

93.1 (89.995.3)

High ( 100.4F [38C])

3.6 (2.16.1)

1.8 (1.03.3)

No observations

Not recorded

3.3 (1.95.8)

1.6 (0.83.0)

0.8 (0.32.6)

95.3 (92.597.0)

95.4 (93.396.8)

94.4 (91.596.4)

Triage pulse oximetry

Normal ( 90%)
Low (< 90%)

0.3 (0.02.0)

0.2 (0.01.3)

0.3 (0.01.9)

Not recorded

4.4 (2.77.1)

4.4 (3.06.5)

5.3 (3.48.1)

Triage pain score, numeric value between 110

6.6 (6.36.9)

6.5 (6.26.7)

6.3 (6.06.6)

36.1 (31.341.2)

58.0 (53.962.0)

86.1 (82.189.3)

3.1 (1.75.4)

4.8 (3.36.9)

6.4 (4.39.4)

Sodium level
Normal (134142 mEq/L)
Low (< 134 mEq/L)
High (> 142 mEq/L)
Not evaluated

0.3 (0.041.9)

1.1 (0.52.4)

1.7 (0.73.7)

60.6 (55.465.5)

36.1 (32.240.2)

5.8 (3.88.8)

Glucose level
Normal (65100 mg/dL)
Low (< 65 mg/dL)
Not evaluated

38.9 (34.044.0)

63.9 (59.867.8)

94.2 (91.296.2)

0.6 (0.12.2)

No observations

No observations

60.6 (55.465.5)

36.1 (32.240.2)

5.8 (3.88.8)

(Table 1 continues on next page)

AJR:207, December 2016 W121

Patterson et al.
TABLE 1: Baseline Population Characteristics (continued)

Downloaded from www.ajronline.org by 114.121.237.109 on 01/04/17 from IP address 114.121.237.109. Copyright ARRS. For personal use only; all rights reserved

Variable

CT Imaging Performed

CT Imaging Not
Performed (n = 360)

Unmatched (n = 562)

Matched (n = 360)

0.3 (0.02.0)

0.2 (0.01.3)

No observations

Blood alcohol level

Normal ( 80 mg/dL)
High (> 80 mg/dL)

No observations

0.4 (0.11.4)

0.6 (0.012.2)

Not evaluated

99.7 (98.0100.0)

99.5 (98.499.8)

99.4 (97.899.9)

31.7 (27.136.7)

54.1 (49.958.2)

80.3 (75.884.1)

WBC count
Normal (3.510.5 K/L)
Low (< 3.5 K/L)

0.8 (0.32.6)

1.6 (0.83.1)

2.2 (1.14.4)

High (> 10.5 K/L)

7.2 (5.010.4)

7.5 (5.610.0)

9.2 (6.612.6)

60.3 (55.165.2)

36.8 (32.940.9)

8.3 (5.911.7)

31.3 (26.836.4)

55.3 (51.259.4)

80.8 (76.484.6)

Not evaluated
Hemoglobin level
Normal ( 11.6 g/dL)
Low (< 11.6 g/dL)
Not evaluated

8.3 (5.911.7)

7.8 (5.910.4)

10.8 (8.014.5)

60.3 (55.165.2)

36.8 (32.940.9)

8.3 (5.911.7)

38.3 (33.443.5)

60.3 (56.264.3)

87.5 (83.790.5)

Platelet count
Normal
Low
Not evaluated
Return visit within 30 days

1.4 (0.63.3)

2.8 (1.74.6)

4.2 (2.56.8)

60.3 (55.165.2)

36.8 (32.940.9)

8.3 (5.911.7)

21.1 (16.925.3)

11.2 (8.613.8)

10.3 (7.113.4)

NoteData are percentage (95% CI).

c apture the effect of CT on subsequent clinical outcomes and resource use beyond the
confines of a single ED visit. As the national
focus expands from single visits to episodes
of care, we will need an understanding of the
role of ED management on subsequent care
trajectories [23].
In addition to identifying this relationship
for policy makers, our work also expands
on recent studies showing the ability to feasibly use clinical data obtained for EHRs to
measure the quality and efficiency of acute
care. Most important, we were able to measure the efficiency of imaging use on the basis of the patients chief complaint, not the
discharge diagnosis. This approach has not
only been recommended by the emergency
medicine community [24], but is also likely a
better reflection of the undifferentiated clinical picture by which patients present to the
ED. Using a combination of structured chief
complaintbased EHR data entries and future natural languageprocessing algorithms,
emergency medicine quality measures could
be substantially improved to reflect the value
of acute diagnostic risk stratification. In addition, by incorporating numerous clinical
data elements not available in administrative
claims, we were able not only to effective-

W122

ly balance covariates for propensity score

based risk adjustment but also to show the opportunity for integrating such data into future
measures of provider imaging use.
Mixed results have been shown in previous
studies evaluating imaging and downstream
resource use [25, 26], but those were of outpatient ambulatory care practices. These findings do not necessarily generalize to the ED
setting, where acute presentations with uncertain follow-up create a unique decision-making environment. In the case of atraumatic
headache, a decision must be made to order
a test that may be low yield but could identify an otherwise missed diagnosis that carries substantial risk of morbidity and mortality. Wide variation in use, increasing use, and
a low true-positive rate of imaging all suggest
a general trend toward potential inappropriate overuse; however, this study shows a relationship between CT use and a patients future care trajectory. Similar relationships have
been shown at the hospital level, where longer
hospital stays reduce hospital readmissions
[27]. Finally, the finding that approximately
18% of patients who underwent imaging at
their index visit were reimaged at their repeat
visit is striking. Although our analysis is limited by the lack of outcomes for these patients,

this reimaging more likely represents inappropriate use than index imaging. However,
further study is needed.
Limitations
The associations described in this study
must be interpreted with care: although significant, they reflect complex relationships
among the underlying variables, including
physician and patient knowledge and expectations, and underlying biologic and social
determinants of health. Although this study
raises interesting associations, none of these
should be considered prognostic for an individual patient. The association identified in
this retrospective study should not be interpreted as indicating that CT use decreases revisits in a causative relationship. Future work
should seek to prospectively study patients
care trajectories after initial visits for headache to understand the influence of imaging
on patient and provider behaviors.
Our inclusion criteria, which used a textbased search method, may not have captured
all patients presenting for headaches (e.g., a
chief complaint of migraine would not be included in the study, whereas migraine headache would). The available database provided
information on whether CT was performed but

AJR:207, December 2016

Downloaded from www.ajronline.org by 114.121.237.109 on 01/04/17 from IP address 114.121.237.109. Copyright ARRS. For personal use only; all rights reserved

Brain CT for Atraumatic Headache and 30-Day ED Revisits

no reported results. Although we could assume
that most studies were negative (because most
positive CT findings would result in hospital
admission), this could not be directly assessed.
Furthermore, the database only contains information on CT performed during the ED visit.
Thus, imaging tests ordered in the outpatient
setting before or after the initial ED visit were
not available for inclusion in the analysis and
would render the 16% rate of eventual imaging artificially low. Similarly, we were unable
to estimate the percentage of patients who had
an additional presentation (including possible
imaging) to other EDs, and therefore return 30day visits could also be underreported.
Because this study was a retrospective
analysis, unmeasured confounding variables
may have influenced the analysis. Given the
data structure of our EHR, we were unable to
include patient comorbidities that may have
affected both the decision to order head CT
and risk for revisit. Our use of an electronic
dataset that included rich clinical data from
index visits should minimize this in comparison with prior analyses limited to administrative claims, and the data were matched on
the basis of a propensity score to minimize
the effect of unmeasured confounding variables. Our propensity score analysis created a
group of 360 matched patients selected from a
pool of 562 potential candidates. Our nearest
neighbor match was based on overall propensity score (calculated from the combination of
specified variables) as opposed to a strategy
based on balancing covariates explicitly. As
a result, across some individual variables, the
matched group appears more dissimilar from
the control than the unmatched group. Overall, the result of the matching process was the
exclusion of 202 patients from the group that
underwent CT with propensity scores most
dissimilar to those of patients who did not undergo CT, and thus was able to improve our
ability to control for unmeasured confounding variables. Generalizability is another limitation, because this was a single-center study.
Finally, our abstraction of the patients chief
complaint relied on a free-text field that has
not been validated through formal assessment
of reliability by chart abstractionfuture
work should seek to generate a standardized
nomenclature for chief complaints for the purposes of measuring similar populations between study settings and research efforts [25].
Conclusion
This study identified an association between completion of brain CT examination

and lower rates of all-cause 30-day return

among patients presenting to the ED for
atraumatic headache. The causes for this
association are likely multifactorial, and
this study does not support the conclusion
that increasing CT use would directly improve revisit rates. Rather, this study raises a new consideration: if the relationship
between CT and decreased ED revisitation
is proven in future studies, then determining the appropriateness of imaging may
need to be reevaluated in a context that includes effect on downstream resource use
in addition to safety and quality. In an environment seeking to improve the efficiency of acute care delivery and lower health
care costs, a relationship between these
two high-cost events (imaging and ED revisits) could affect how performance measures are structured going forward across a
wide variety of clinical conditions, including headache, chest pain, and minor traumatic injuries as well. Rather than develop performance measures for emergency
medicine focused on a single ED encounter, consideration of the broader acute care
episode may be needed to more accurately
assess appropriate or inappropriate ED interventions and health care use.
References

1. Pitts SR, Carrier ER, Rich EC, Kellermann AL.

Where Americans get acute care: increasingly, its
not at their doctors office. Health Aff (Millwood)
2010; 29:16201629
2. Berdahl CT, Vermeulen MJ, Larson DB,
Schull MJ. Emergency department computed tomography utilization in the United States and
Canada. Ann Emerg Med 2013; 62:486.e3494.e3
3. Redberg RF, Smith-Bindman R. We are giving ourselves cancer. The New York Times website. www.
nytimes.com/2014/01/31/opinion/we-are-givingourselves-cancer.html. Published January 30, 2014.
Accessed August 8, 2016
4. Gilbert JW, Johnson KM, Larkin GL, Moore CL.
Atraumatic headache in US emergency departments: recent trends in CT/MRI utilisation and
factors associated with severe intracranial pathology. Emerg Med J 2012; 29:576581
5. Prevedello LM, Raja AS, Zane RD, et al. Variation
in use of head computed tomography by emergency physicians. Am J Med 2012; 125:356364
6. Centers for Medicare & Medicaid Services
(CMS). OP-15: use of brain computed tomography in
the emergency department for atraumatic headache
literature review. Baltimore, MD: CMS, 2014
7. Schuur JD, Brown MD, Cheung DS, et al. Assessment of Medicares imaging efficiency measure for

emergency department patients with atraumatic

headache. Ann Emerg Med 2012; 60:280.e4290.e4
8. Raja AS, Walls RM, Schuur JD. Decreasing use of
high-cost imaging: the danger of utilization-based
performance measures. Ann Emerg Med 2010;
56:597599
9. Rising KL, White LF, Fernandez WG, Boutwell
AE. Emergency department visits after hospital
discharge: a missing part of the equation. Ann
Emerg Med 2013; 62:145150
10. Hu KW, Lu YH, Lin HJ, Guo HR, Foo NP. Unscheduled return visits with and without admission post emergency department discharge.
JEmerg Med 2012; 43:11101118
11. White D, Kaplan L, Eddy L. Characteristics of
patients who return to the emergency department
within 72 hours in one community hospital. Adv
Emerg Nurs J 2011; 33:344353
12. Bates DW, Saria S, Ohno-Machado L, Shah A,
Escobar G. Big data in health care: using analytics
to identify and manage high-risk and high-cost patients. Health Aff (Millwood) 2014; 33:11231131
13. Stewart WF, Shah NR, Selna MJ, Paulus RA,
Walker JM. Bridging the inferential gap: the electronic health record and clinical evidence. Health
Aff (Millwood) 2007; 26:w181w191
14. Patterson BW, Venkatesh AK, AlKhawam L,
Pang PS. Abdominal computed tomography utilization and 30-day revisitation in emergency department patients presenting with abdominal
pain. Acad Emerg Med 2015; 22:803810
15. Raven MC, Lowe RA, Maselli J, Hsia RY. Comparison of presenting complaint vs discharge diagnosis for identifying nonemergency emergency
department visits. JAMA 2013; 309:11451153
16. Northwestern University. Northwestern Medicine
Enterprise Data Warehouse (NMEDW). Northwestern University website. www.nucats.northwestern.
edu/resources-services/data-and-informatics-services/
enterprise-data-warehouse/. Accessed January 12,
2015
17. Ho D, Imai K, King G, Stuart EA. MatchIt: nonparametric preprocessing for parametric causal
inference. J Stat Softw 2011; 42:127
18. DAgostino RB. Tutorial in biostatistics: propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat Med 1998; 17:22652281
19. Rosenbaum PR, Rubin DB. The central role of the
propensity score in observational studies for causal effects. Biometrika 1983; 70:4155
20. Rubin DB. Estimating causal effects from large
data sets using propensity scores. Ann Intern Med
1997; 127:757763
21. Ho D, Imai K, King G, Stuart E. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Polit
Anal 2007; 15:199236

AJR:207, December 2016 W123

Downloaded from www.ajronline.org by 114.121.237.109 on 01/04/17 from IP address 114.121.237.109. Copyright ARRS. For personal use only; all rights reserved

Patterson et al.
22. Rusanov A, Weiskopf NG, Wang S, Weng C. Hidden in plain sight: bias towards sick patients when
sampling patients with sufficient electronic health
record data for research. BMC Med Inform Decis
Mak 2014; 14:51
23. Wiler JL, Beck D, Asplin BR, et al. Episodes of
care: is emergency medicine ready? Ann Emerg
Med 2012; 59:351357
24. Griffey RT, Pines JM, Farley HL, et al. Chief
complaint-based performance measures: a new

W124

focus for acute care quality measurement. Ann

Emerg Med 2015; 65:387395
25. Chen J, Fazel R, Ross JS, et al. Do imaging studies
performed in physician offices increase downstream utilization? An empiric analysis of cardiac
stress testing with imaging. JACC Cardiovasc
Imaging 2011; 4:630637
26. Devine PJ, Villines TC, Sullenberger LE, et al. Real-world application of coronary computed tomography angiography and its potential effect on down-

stream resource utilization in evaluating angina.

JCardiovasc Comput Tomogr 2008; 2:214219
27. Bartel AP, Chan CW, Kim S-H. Should hospitals
keep their patients longer? The role of inpatient
and outpatient care in reducing readmissions. National Bureau of Economic Research Working
Paper Series, no. 20499. National Bureau of Economic Research website. nber.org/papers/w20499.
Published September 2014. Revised March 28,
2016. Accessed August 8, 2016

AJR:207, December 2016