You are on page 1of 10

The Stroke Outcomes and Neuroimaging of Intracranial

Atherosclerosis (SONIA) Trial


E. Feldmann, J. L. Wilterdink, A. Kosinski, et al.
Neurology 2007;68;2099; Published online before print April 4, 2007;
DOI 10.1212/01.wnl.0000261488.05906.c1

This information is current as of October 18, 2011

The online version of this article, along with updated information and services, is
located on the World Wide Web at:
http://www.neurology.org/content/68/24/2099.full.html

Neurology ® is the official journal of the American Academy of Neurology. Published continuously
since 1951, it is now a weekly with 48 issues per year. Copyright © 2007 by AAN Enterprises, Inc. All
rights reserved. Print ISSN: 0028-3878. Online ISSN: 1526-632X.

Downloaded from www.neurology.org at University of Oxford (oxu) / England on October 18, 2011
The Stroke Outcomes and Neuroimaging
of Intracranial Atherosclerosis (SONIA)
Trial

E. Feldmann, MD ABSTRACT Background: Transcranial Doppler ultrasound (TCD) and magnetic resonance angiography
J.L. Wilterdink, MD (MRA) can identify intracranial atherosclerosis but have not been rigorously validated against the gold
A. Kosinski, PhD standard, catheter angiography. The WASID trial (Warfarin Aspirin Symptomatic Intracranial Disease)
M. Lynn, MS required performance of angiography to verify the presence of intracranial stenosis, allowing for pro-
M.I. Chimowitz, MB, spective evaluation of TCD and MRA. The aims of Stroke Outcomes and Neuroimaging of Intracranial
ChB Atherosclerosis (SONIA) trial were to define abnormalities on TCD/MRA to see how well they identify
J. Sarafin, RN 50 to 99% intracranial stenosis of large proximal arteries on catheter angiography. Study Design:
H.H. Smith, RN SONIA standardized the performance and interpretation of TCD, MRA, and angiography. Study-wide
F. Nichols, MD cutpoints defining positive TCD/MRA were used. Hard copy TCD/MRA were centrally read, blind to the
J. Rogg, MD results of angiography. Results: SONIA enrolled 407 patients at 46 sites in the United States. For
H.J. Cloft, MD, PhD
prospectively tested noninvasive test cutpoints, positive predictive values (PPVs) and negative predic-
L. Wechsler, MD
tive values (NPVs) were TCD, PPV 36% (95% CI: 27 to 46); NPV, 86% (95% CI: 81 to 89); MRA, PPV
J. Saver, MD
59% (95% CI: 54 to 65); NPV, 91% (95% CI: 89 to 93). For cutpoints modified to maximize PPV,
S.R. Levine, MD
they were TCD, PPV 50% (95% CI: 36 to 64), NPV 85% (95% CI: 81 to 88); MRA PPV 66% (95% CI:
C. Tegeler, MD
58 to 73), NPV 87% (95% CI: 85 to 89). For each test, a characteristic performance curve showing
R. Adams, MD
how the predictive values vary with a changing test cutpoint was obtained. Conclusions: Both trans-
M. Sloan, MD
cranial Doppler ultrasound and magnetic resonance angiography noninvasively identify 50 to 99%
The Stroke Outcomes
and Neuroimaging of intracranial large vessel stenoses with substantial negative predictive value. The Stroke Outcomes
Intracranial and Neuroimaging of Intracranial Atherosclerosis trial methods allow transcranial Doppler ultrasound
Atherosclerosis and magnetic resonance angiography to reliably exclude the presence of intracranial stenosis. Abnor-
(SONIA) Trial mal findings on transcranial Doppler ultrasound or magnetic resonance angiography require a confir-
Investigators* matory test such as angiography to reliably identify stenosis. NEUROLOGY 2007;68:2099–2106

Intracranial atherosclerosis causes an estimated 70,000 ischemic strokes in the United States
Address correspondence and each year.1 In contrast to extracranial atherosclerosis, intracranial atherosclerosis is more
reprint requests to Dr. Edward common in US Hispanics, blacks, Japanese, and Chinese than in US whites.2-5 Currently,
Feldmann Department of
Clinical Neurosciences, Brown catheter angiography is the accepted gold standard in diagnosing intracranial atherosclerosis.
University School of Medicine,
110 Lockwood Street, Suite 324,
The Warfarin-Aspirin Symptomatic Intracranial Disease (WASID) study6 compared aspirin
Providence, RI 02903 and warfarin in patients with more than 50% stenosis of an intracranial artery identified by
EFeldmann@lifespan.org
angiography. However, angiography is a more expensive test and may cause stroke,7 so
clinicians commonly rely on the results of noninvasive tests such as magnetic resonance
angiography (MRA), transcranial Doppler (TCD), and computed tomographic angiography
(CTA)7-10 to evaluate patients suspected of intracranial atherosclerosis.
Despite the widespread use of TCD/MRA, and the increasing use of CTA, their ability to
quantify the severity of intracranial atherosclerosis has not been rigorously characterized.
Editorial, see page
Several studies have compared MRA, TCD, or CTA with angiography for detecting intracra-
2057 nial stenosis.11-21 The studies suffer from small sample sizes, verification bias, selection bias,
Supplemental data at and retrospective bias. In addition, they do not consistently use standardized test perfor-
www.neurology.org

This article was previously published in electronic format as an Expedited E-Pub on April 4, 2007, at www.neurology.org.
*See appendix E-1 for list of investigators.
Funded by a research grant (1 RO1 NS 39131-04) from the US Public Health Service National Institute of Neurological Disorders and Stroke
(NINDS).
Disclosure: The authors report no conflicts of interest.

Copyright
Downloaded from www.neurology.org at University of Oxford (oxu) © on
/ England 2007 by AAN
October Enterprises,
18, 2011 Inc. 2099
mance protocols or well-described, repro- Sample size. The number of patients available for SONIA
was dependent on the number of patients screened for WASID.
ducible measurement methods and often as-
Therefore, a formal sample size estimate was not made for
sess the ability of tests to detect stenoses, but SONIA. Instead, predictions were made about the number of
not quantify them. vessel readings that would be available based on expected
The Stroke Outcomes and Neuroimaging WASID recruitment. Initial sample size assumptions predicted
93 positive TCD tests and 93 positive MRA tests per vessel
of Intracranial Atherosclerosis (SONIA) trial examined by angiography.22 At a PPV of 80%, the 95% CI was
was a prospective, multicenter study aimed expected to be 71 to 88%. Angiographic verification of negative
at validating the ability of TCD and MRA to noninvasive tests occurred when angiography was performed in
diagnose intracranial atherosclerosis com- the symptomatic vascular territory and visualized the unin-
volved arterial vessel proximal to the presumed stenosis. For
pared with catheter angiography. SONIA example, this would include the normal carotid siphon in a pa-
was a companion study to WASID and incor- tient with a diseased middle cerebral artery (MCA) or a normal
porated standardized, reproducible test per- vertebral artery (VA) in a patient with a diseased basilar artery.
If catheter angiography studied other vascular territories for
formance protocols, test measurement
which matching noninvasive data were available, then further
methods, and diagnostic criteria. The trial verification of negative noninvasive tests was possible. The ex-
was designed to avoid or correct for common pected sample size of negative noninvasive tests was 96 negative
forms of test evaluation bias. TCD tests per vessel and 96 negative MRA tests verified by
angiography. At an NPV of 90%, the 95% CI for the true NPV
was expected to be 84 to 96%.
METHODS SONIA collaborated with WASID to study pa-
tients with TIA or ischemic stroke who were suspected of hav- Patient identification and eligibility. SONIA patients had
ing intracranial stenosis. TCD and MRA were used to identify the same inclusion and exclusion criteria as WASID patients
potential candidates for WASID, and angiography was re- with the exception of not requiring a positive angiogram.22 The
quired for entry into WASID, so a large, symptomatic patient flow diagram in figure 1 illustrates how patients were recruited
population with both noninvasive tests and angiograms were in to SONIA. All SONIA patients had to be identified before
identified. Both WASID and SONIA were prospective, multi- their angiogram to be eligible for the study. This requirement
center trials. ensured enrollment into SONIA of patients with positive non-
WASID enrolled patients with 50 to 99% intracranial steno- invasive test results and negative angiogram results, as patients
sis with symptoms in the past 90 days. For patients to be en- with negative angiography were excluded from WASID but
rolled in SONIA, TCD or MRA had to be performed prior to were eligible to be enrolled in SONIA. For the patients who
angiography. Angiography could involve any number of in- underwent angiography as part of routine care, not as part of
jected vessels. If the patient had 50 to 99% stenosis, they were the WASID protocol, SONIA would only enroll those patients
enrolled in WASID and SONIA. If not, they were not enrolled if investigators notified the SONIA coordinating center that an
in WASID, but they could be enrolled in SONIA to study the angiogram was planned prior to its actual performance to
correlation among their noninvasive tests and angiography. avoid the bias of enrolling patients with predominantly positive
The primary aim of SONIA was to define velocity values on angiography results.
TCD and anatomic abnormalities on MRA that identify 50 to
Standardization of TCD and MRA performance. All
99% intracranial stenosis of the M1 segment of the middle cere-
sites participating in SONIA followed standardized guidelines
bral, carotid siphon, and intracranial vertebral and basilar ar-
for TCD/MRA performance.22 Further details regarding stan-
teries on catheter angiography. SONIA was designed to see
dards for test performance may be found in appendix E-2 (on the
whether prespecified vessel-specific TCD velocity abnormali-
Neurology Web site at www.neurology.org). Guidelines for an-
ties and MRA percentage of stenosis and flow gap abnormali-
giography were the same as those used in WASID: contrast media
ties (discontinuity of the blood flow column signal, with distal
were 270 to 300 mg% and only single-vessel injection with biplane
reconstitution) defined in a symptomatic patient population
imaging was required. Adherence to the test performance stan-
could perform with a positive predictive value (PPV) of 80%
dards was ensured in two ways. Prior to entering the trial, each site
when compared against conventional angiography. PPV is the
recruited for SONIA submitted sample TCD/MRA case studies to
probability that a vessel that appears diseased on the noninva-
demonstrate that the site had the capability to meet the standards.
sive test also appears diseased on the confirmatory angiogram.
Throughout the trial, every TCD/MRA was examined for image
The secondary aim of SONIA was to study whether the
quality, and sites were notified when studies were inadequate.
absence of TCD and MRA abnormalities could exclude the
Whenever possible, the quality problems were corrected before the
presence of 50 to 99% stenosis by showing that they perform
study was centrally read. Only studies that met SONIA quality
with a 90% negative predictive value (NPV) when compared
standards were used in calculating PPV and NPV. Persistent fail-
against conventional catheter angiography. NPV is the proba-
ure to meet standards would have resulted in a site being dropped
bility that a vessel that reveals ⬍50% stenosis on the noninva-
from SONIA.
sive test also reveals ⬍50% stenosis on the angiogram. Similar
aims for CTA were added to SONIA after the study was under Collection of films. Angiogram films of all injected arteries
way, recognizing the emergence of CTA as a new imaging mo- were collected. Central angiography readings were performed
dality for intracranial stenosis. The aims were identical for by Harry J. Cloft, MD, PhD, a board-certified neuroradiologist
those specified for MRA. The data for CTA do not represent currently at the Mayo Clinic Department of Radiology, Roch-
the primary aim of SONIA and will be reported separately. ester, MN, blinded to the results of all other testing.

2100 Neurology
Downloaded 68www.neurology.org
from June 12, 2007 at University of Oxford (oxu) / England on October 18, 2011
pretation may be found in appendix E-2. SONIA angiograms
Figure 1 were measured using methods developed specifically for the in-
tracranial arteries.24 Readings were performed with a Mitutoyo
573-225-10 handheld caliper.
There were no published measurement methods for intracra-
nial vessels on MRA, but the one developed for SONIA was se-
lected because it was consistent and reproducible, based on pilot
studies.22 Measurements were performed using the Mitutoyo 573-
225-10 handheld caliper following the same approach used for an-
giograms. Flow gaps were identified visually. Whenever possible,
both source and MIP images were examined, as source images
have been shown to improve the sensitivity of MRA.25 When
source and MIP images were available, MIP images were read if
source images did not adequately define the site of stenosis or visu-
alize a vessel. If both types of images visualized a vessel, the image
of the least severely diseased vessel was measured. Once image
type, source, or MIP had been selected for a given vessel, the single
view with the highest percentage of stenosis was measured.
Local TCD readings were performed at each site automatically
by the TCD equipment. Central measurements were performed by
manually drawing a horizontal midline on the hard copy of the
waveforms to obtain a time-averaged mean of the maximum ve-
locity.26 The horizontal midline separated three cardiac cycles of
the waveform into equal parts, and mean velocity was read where
the midline intersected the vertical axis. This method of measuring
TCD was used in the STOP trial and found to be highly
reproducible.22

Test cutpoints. Pilot data and site surveys performed prior to


SONIA led to the choice of test cutpoints at the inception of this
study. For MRA, ⱖ50% stenosis, without occlusion, or the
presence of a flow gap defined a positive test. Stenosis on TCD
was identified using the time-averaged mean of the maximum
velocity. A positive test consisted of a mean velocity ⬎100 cm/
second in the MCA, ⬎90 cm/second in the intracranial internal
carotid artery (ICA), or ⬎80 cm/second in the basilar artery
(BA) or VAs.

Statistical analysis. To meet the primary and secondary aims


of SONIA, PPVs and NPVs for TCD and MRA were calcu-
lated. For the purposes of this analysis, occlusions on angiogra-
Flow chart for identification TCD waveforms and MRA images were sent to the SONIA phy were considered to reflect an absence of stenosis. Thus, a
and entry of Warfarin-Aspirin coordinating center where they were prepared for central reading. positive noninvasive test followed by occlusion on angiography
Symptomatic Intracranial Whenever possible, both source and maximum intensity projec- was scored as a false positive and a negative noninvasive test
Disease study candidates
tion (MIP) images were obtained for MRA. TCD was centrally followed by occlusion on angiography was scored as a true
into Stroke Outcomes and
read at the Medical College of Georgia by Fenwick Nichols, MD, negative.
Neuroimaging of Intracranial
Atherosclerosis trial.
Director of TCD Reading in the Stroke Prevention Trial in Sickle The PPV and NPV values for the original and modified cut-
*Transcranial Doppler Cell Anemia (STOP).23 The MRA images were centrally read at points were computed as proportions. Namely, for the PPV, the
ultrasound, MRI, or computed Rhode Island Hospital by Jeffrey Rogg, MD, Director of MRI and number of vessels with positive result on both the noninvasive
tomographic angiography. board certified in neuroradiology. Readings on all studies were test and angiography was divided by the number of vessels with
**Result could be positive faxed to the SONIA statistical coordinating center and entered a positive noninvasive test. For the NPV, the number of vessels
based on one artery. Other into the central database. with negative result on both the noninvasive test and angiogra-
arteries would be negative phy was divided by the number of vessels with a negative non-
and also subject to Blinding. Local angiogram readers at the SONIA sites could be-
invasive test. This direct method of computation was possible
angiographic verification. come aware of the results of noninvasive tests and become biased
for the pooled vessel analysis because the pooled approach pro-
***For other arteries, the by that information. Thus, a blinded central reader reread each
noninvasive test could be
vided a reasonable denominator for the computation of a pro-
angiogram to prevent bias. This central reader was blinded to both
either true or false positive or portion. Exact 95% CIs were used.
the noninvasive test results and the local site angiogram readings.
true or false negative The modified cutpoints, which were defined with increased
TCD and MRA central readers were blinded to each other’s re-
depending on the results of TCD velocities and increased MRA percentage of stenosis com-
sults and to the results of angiography. To ensure that readers
the respective tests. pared with the initial cutpoints, were designed to improve PPV,
could not access other noninvasive imaging data, dummy ID num-
but the modification of cutpoints was limited by sparse data for
bers are assigned to the studies before they were centrally read.
higher values of the noninvasive continuous measures. Further
Standardization of readings. Standardized interpretation improvements in PPV could not be estimated with a reasonable
methods were used to read all SONIA studies. These have been precision using the above direct method within the SONIA
described elsewhere22 and further details regarding test inter- data. Accordingly, modified PPV and NPV for each vessel dis-

Neurology
Downloaded from www.neurology.org at University of Oxford (oxu) 68onJune
/ England 12, 2007
October 18, 2011 2101
Figure 2 Table 1 Demographic features of the Stroke
Outcomes and Neuroimaging of
Middle cerebral artery (MCA) Internal carotid artery (ICA) Intracranial Atherosclerosis trial
1.0
NPV
1.0 population (N ⫽ 407 patients)
NPV
0.8 0.8
Age, y, mean 65
0.6 0.6
PPV PPV % Female 40.5
0.4 0.4

0.2 0.2 Race, %


NPV 0.94 0.94 0.93 0.92 0.92 0.91 0.91 NPV 0.91 0.89 0.88 0.87 0.87 0.86 0.86
PPV 0.50 0.52 0.55 0.58 0.62 0.65 0.68 PPV 0.47 0.50 0.54 0.58 0.61 0.65 0.68 American Indian 0.5
0.0 0.0

50 55 60 65 70 75 80 50 55 60 65 70 75 80 Asian 4.3

MRA stenosis cutpoint (%) MRA stenosis cutpoint (%) Black 33.3

Vertebral artery (VA) Basilar artery (BA) Hispanic 6.1


1.0 1.0 White 54.1
NPV NPV
0.8 0.8
Other 1.6
0.6 PPV 0.6 PPV

0.4 0.4

0.2 NPV 0.88 0.88 0.87 0.87 0.87 0.86 0.86 0.2 NPV 0.89 0.87 0.85 0.83 0.82 0.81 0.81
The number of vessels with corresponding an-
0.0
PPV 0.61 0.63 0.65 0.67 0.70 0.72 0.74
0.0
PPV 0.60 0.63 0.67 0.71 0.74 0.77 0.79 giographic data read were TCD 451 vessels and
50 55 60 65 70 75 80 50 55 60 65 70 75 80 MRA 1,310 vessels. Anticipated sample size was
MRA stenosis cutpoint (%) MRA stenosis cutpoint (%)
achieved for NPV for TCD and MRA, as it was
common for normal vessels on TCD/MRA to be
Middle cerebral artery (MCA) Internal carotid artery (ICA)
1.0 1.0
also imaged by catheter angiography in the process
0.8
NPV
0.8
NPV of evaluating the patient as a candidate for WASID.
0.6 0.6
Anticipated sample size for PPV was achieved only for
0.4 0.4
MRA because this test was ordered by clinicians at a
PPV
0.2 0.2
PPV higher rate than initially anticipated, relative to the fre-
NPV 0.86 0.85 0.84 0.83 0.82 0.81 0.81 NPV 0.86 0.85 0.84 0.84 0.84 0.83 0.83
0.0
PPV 0.32 0.37 0.42 0.47 0.52 0.56 0.60
0.0
PPV 0.27 0.31 0.36 0.40 0.44 0.47 0.52 quency of ordering TCD. SONIA anticipated from
80 100 120 140 160 180 200 80 100 120 140 160 180 200
initial investigator surveys that TCD and MRA would
be ordered with equal frequency. The actual ratio of
TCD velocity cutpoint (cm/s) TCD velocity cutpoint (cm/s)
test performance was 1:3 for TCD:MRA. Sample size
Vertebral artery (VA) Basilar artery (BA) was correspondingly not achieved for TCD PPV vali-
1.0 1.0
NPV
NPV dation. The distribution of disease severity on cen-
0.8 0.8
trally read angiograms is presented in table 2.
0.6 0.6
PPV Table 3 illustrates the PPV/NPV and 95% CIs
0.4 0.4 PPV
based on the initial test cutpoints. Note that these
0.2 NPV 0.84 0.82 0.81 0.81 0.80 0.80 0.80 0.2 NPV 0.90 0.89 0.88 0.87 0.85 0.85 0.84
PPV 0.45 0.55 0.66 0.77 0.84 0.86 0.88 PPV 0.36 0.45 0.53 0.61 0.69 0.75 0.80 results represent the pooling of data for individual
0.0 0.0
vessels because of smaller than expected sample
80 100 120 140 160 180 200 80 100 120 140 160 180 200
sizes.
TCD velocity cutpoint (cm/s) TCD velocity cutpoint (cm/s)
The primary aim of SONIA was to see whether
PPVs of 80% were achievable. Adjustment of cut-
Positive predictive value played in figure 2 were derived from a modeling process involv- points during SONIA was prespecified in the study
(PPV) and negative predictive ing a logistic regression fit with the noninvasive test continuous
value (NPV), with changing design but did not occur because of recruitment issues
measure predicting the probability of disease as defined by the
test cutpoints, for magnetic
angiogram. The model fit and smoothed density of the nonin-
in WASID and sample size issues in SONIA. The cut-
resonance angiography points are measures of continuous variables such as
vasive test continuous measure were used to obtain the modi-
(MRA) and transcranial
fied predictive values although approximate numeric velocity on TCD or percentage of stenosis on MRA
Doppler ultrasound. Note
that for MRA, the illustrated integration. and can be adjusted post hoc. Flow gaps on MRA are
logistic regression model
excludes vessels with flow RESULTS SONIA enrolled 407 patients at 46 sites
gap in the graph for PPV. For in the United States over a 4-year period from Feb- Table 2 Vessels with central reading of
each vessel, the PPV for flow angiography
gap (95% CI) was as follows:
ruary 1999 to July 2003. SONIA was successful in
MCA, 67% (95% CI: 57 to achieving widespread compliance with performance Angiography result No. (%) of vessels
76); ICA, 55% (95% CI: 32 standards. The percentages of vessels meeting qual- Normal 1,105 (69.2)
to 77); VA, 81% (95% CI:
ity standards were TCD 94% and MRA 92%. None ⬍50% 141 (8.8)
48 to 98); BA, 64% (95%
CI: 45 to 80). of the sites was dropped from SONIA participation
50–99% 315 (19.7)
because of test quality issues. Features of the patient
Occlusion 35 (2.2)
population are described in table 1.

2102 Neurology
Downloaded 68www.neurology.org
from June 12, 2007 at University of Oxford (oxu) / England on October 18, 2011
points for 70 to 99% catheter angiogram stenosis of
Table 3 PPVs and NPVs and 95% CIs for
TCD/MRA in SONIA based on the initial
the individual arteries as follows: TCD mean veloc-
test cutpoints ity (cm/second) ICA 130, MCA 240, VA 130, BA
130; MRA (percentage of stenosis) all vessels 80%.
PPV NPV

TCD 36% (27–46) 86% (81–89) DISCUSSION SONIA characterizes the perfor-
MRA 59% (54–65)* 91% (89–93) mance of TCD and MRA for the detection of 50 to
99% intracranial stenosis of the MCA, cavernous
Data represent pooled predictive values for all vessels studied.
*PPV (CI) for flow gap on MRA was 66% (58 –73). ICA, intracranial VA, and BA as detected by cathe-
SONIA ⫽ Stroke Outcomes and Neuroimaging of Intracranial ter angiography. SONIA developed minimum qual-
Atherosclerosis trial; PPV ⫽ positive predictive value; NPV ⫽ ity standards for the performance of the diagnostic
negative predictive value; TCD ⫽ transcranial Doppler ultra-
sound; MRA ⫽ magnetic resonance angiography.
tests and standardized the cutpoints for TCD and
MRA across 46 centers nationwide.
an exception as they are not continuous. They are ei- By excluding substandard tests, SONIA did not
ther present or not. The MRA cutpoints retain flexibil- employ an intention to diagnose analysis. However,
ity, however, as it is still possible to adjust the the SONIA standards for test performance reflected
percentage of stenosis required for the test to be de- a general community standard that demands com-
fined as positive. petence levels expected at most community and aca-
The SONIA cutpoints have thus been adjusted demic medical centers rather than unique state-of-
post hoc to see what positive predictive values are the-art research facilities. Thus, the results,
achievable. Cutpoints were adjusted for each vessel. accepting the exclusion of a relatively small percent-
The performance of the tests can be characterized age of substandard tests, are easily generalizable.
by the eight graphs presented in figure 2. The graphs Different predictive values may be achieved at
illustrate how PPVs or NPVs vary as the test cut- unique facilities performing state-of-the-art testing
point is changed. Generally, PPV increases and NPV or more rigorous examination protocols.
decreases as the cutpoint is defined by a more severe SONIA was not designed to address whether MRA
abnormality. We also pooled the results for all ves- or TCD performed better than the other. Such an anal-
sels for each noninvasive test and adjusted the test ysis could only be performed after test validation has
cutpoints to maximize PPV. The resulting overall occurred, cutpoints are fine-tuned to evaluate a spe-
predictive values for catheter angiogram 50 to 99% cific population, and tests are compared on based on
stenosis are presented in table 4. costs and clinical outcomes, not just the sum total of
Cutpoints can also be adjusted to identify more false-positive and false-negative results. As has been
severe disease, such as 70 to 99% stenosis of an in- previously shown, a higher number of test errors may
tracranial artery on catheter angiogram rather than not only be acceptable in certain situations but desir-
50 to 99%. The SONIA data set determined cut- able, where the clinical cost to the patient of a false-
positive and false-negative test result differ markedly.27
Table 4 PPVs and NPVs, pooled for all vessels, SONIA standardized the cutpoints for TCD and
based on test cutpoints adjusted* to MRA across 46 sites because all local investigators
maximize PPV
agreed to the use of the same cutpoints, which were
PPV NPV defined on a per-vessel basis. Such collaboration
TCD 55% (36, 74) 83% (79, 86) among investigators is important for achieving nec-
MRA 66% (58, 73) 87% (85, 89) essary sample sizes, as validation would be impossi-
Adjusted test cutpoints*
ble if every local site used different cutpoints.
TCD Initial (mean velocity) Adjusted (mean velocity)
Standardized cutpoints are not only feasible for a
multicenter clinical trial, but may also be necessary
MCA 100 cm/s 240 cm/s
for the community. Given the currently infrequent
ICA 90 cm/s 120 cm/s
performance of catheter angiography at most insti-
Vertebral 80 cm/s 110 cm/s
tutions, the typical TCD or MRA department can-
Basilar 80 cm/s 130 cm/s
not generate enough comparisons between
MRA ⱖ 50% stenosis ⱖ 80% stenosis noninvasive tests and angiography to accurately de-
or flow gap or flow gap
velop and evaluate its own test cutpoints.
*Adjusted test cutpoints for these predictive values were as Many factors led to the smaller than expected
follows (the initial test cutpoints are given for comparison).
sample size for PPV. The original design of SONIA
PPV ⫽ positive predictive value; NPV ⫽ negative predictive val-
ue; TCD ⫽ transcranial Doppler ultrasound; MRA ⫽ magnetic called for a split-sample validation in which the PPV
resonance angiography. and NPV would be monitored throughout the

Neurology
Downloaded from www.neurology.org at University of Oxford (oxu) 68onJune
/ England 12, 2007
October 18, 2011 2103
course of the trial and the cutpoints for a positive studies may have contributed to overestimates of
noninvasive test would be adjusted to better achieve PPV. The prevalence of disease in SONIA may be
the desired target of 80% PPV. However, this design lower than in a selected group of patients compris-
was changed due to several events that occurred ing a cohort from which pilot data are generated,
throughout the WASID/SONIA collaboration. The and PPV decreases as prevalence decreases. The
original assumptions used to calculate sample size method by which SONIA assessed arteriographic
suggested that approximately 10% of patients occlusions was conservative. Even though an occlu-
would have angiograms performed outside WASID, sion represents severe disease, noninvasive abnormali-
as part of routine care, but the actual number was ties associated with occlusion on angiography were
75%. These patients could still be entered into considered false positives because an occlusion is not
SONIA if a notification call to Emory was placed treated with a stent. This approach results in SONIA
prior to their angiogram, but this option was used increasing NPV but decreasing PPV. Technical and
only 60% of the time. Thus, a larger than predicted performance improvements in the tests, such as using
number of WASID patients were ineligible for gadolinium for MRA, or M-mode for TCD, may im-
SONIA because identifying patients only after an- prove the PPV of these tests. Finally, these tests, TCD
giography would introduce selection bias. In addition, and MRA, when rigorously assessed, may simply per-
the TCD sample size is even lower than expected be- form much better at excluding the presence of stenosis
cause, whereas pilot surveys indicated that TCD and rather than identifying it.
MRA would be performed in a 1:1 ratio, in the trial, SONIA prospectively validated the cutpoint for
MRA was performed much more often than TCD. Be- each test for each vessel. Because of small sample sizes
cause a smaller sample size limits the accuracy of the for some vessels, however, the summary data are pre-
results through widening of the resulting CIs, the sented as pooled for all vessels. The graphs in figure 2
National Institute of Neurological Disorders and illustrate how predictive values change with the cut-
Stroke Data and Safety Monitoring Board (DSMB) for point for each test and for each vessel. In the design of
the trial chose not to sacrifice any of the sample to the future trials employing these tests to evaluate patients,
split-sample validation. The WASID/SONIA DSMB the SONIA data set allows for an informed choice of
also determined that SONIA should not change the test cutpoints on an individual vessel basis.
initial cutpoints as originally specified because of recruit- SONIA determined PPV and NPV for TCD/MRA
ment issues in WASID, our parent trial. SONIA also rather than sensitivity and specificity. Determining
could not complete enrollment of all patients into the sensitivity and specificity directly would require that
study, as WASID and, therefore, SONIA were termi- every patient having noninvasive testing also undergo
nated early because of warfarin-related safety issues in angiography regardless of the result of the noninvasive
WASID. This termination affected the sample size of test. The risks and costs of angiography are too high to
data available in SONIA. A small sample size for a test ethically justify such a design. To calculate sensitivity
validation study does not preclude a result for PPV and and specificity from the SONIA data, corrections must
NPV. The small sample size widens the CI around that be made to account for verification bias. Verification
result. bias occurs when abnormal test results are verified
Thus, SONIA validated the cutpoints chosen at more often than normal test results and commonly
the outset of the trial and achieved NPVs very close causes an overestimation of sensitivity (minimizing
to its targets. Our positive predictive values only ap- false negatives) and an underestimation of specificity
proached our targets using the post hoc adjusted (maximizing false positives).
test cutpoints, but our results compare favorably In SONIA, only patients with positive TCD/
with other studies. For example, a recent study re- MRA tests in a symptomatic vessel had angiography
ported PPV ranging from 55 to 78% and NPV rang- for safety and cost reasons, so verification bias may
ing from 90 to 97% for duplex ultrasound and MRA be present. This problem was partially corrected by
of the extracranial carotid arteries.28 A similar situ- correlating the noninvasive results of the normal
ation of high NPV but lower PPV has been reported vessels with the angiograms of those vessels. Addi-
for CTA of the carotid bifurcation.29 tionally, a mathematical correction for verification
There are several reasons why the PPVs found in bias will be performed as a separate study that re-
SONIA for TCD and MRA were lower than ex- flects an exploratory aim of the SONIA trial.
pected. Target PPVs were determined by pilot data Using the same criteria for patients entering
collected from selected sites that may perform the SONIA and WASID ensured that the noninvasive
noninvasive tests at a higher quality standard than tests examined in SONIA were performed in a
other sites used in the trial. Less rigorous blinding symptomatic and readily identifiable patient pop-
and measurement procedures used during pilot ulation. Bias was avoided by requiring that all

2104 Neurology
Downloaded 68www.neurology.org
from June 12, 2007 at University of Oxford (oxu) / England on October 18, 2011
patients undergoing angiography were included ischemia: report of 159 angiograms and correlation with
in SONIA, regardless of the results of angiogra- surgical risk. Surg Neurol 1987;28:90–92.
10. Qureshi AI, Luft AR, Sharma M, Guterman LR, Hopkins
phy. Without this requirement, patients with neg-
LN. Prevention and treatment of thromboembolic and
ative angiograms who were not enrolled in ischemic complications associated with endovascular pro-
WASID would not be enrolled in SONIA, and the cedures: Part II— clinical aspects and recommendations.
noninvasive tests would have a falsely high PPV. Neurosurgery 2000;46:1360–1375.
SONIA met other important criteria for reliable 11. deBray JM, Missoum A, Dubas F, Emile J, Lhoste P.
assessment of diagnostic tests: blinded measure- Transcranial Doppler sonography versus angiography. J
Ultrasound Med 1997;16:213–218.
ments, prospective enrollment, the use of stan-
12. Demchuk AM, Christou I, Wein TH, et al. Accuracy and
dardized techniques, and the use of a reference
criteria for localizing arterial occlusion with transcranial
standard— catheter angiography. Doppler. J Neuroimaging 2000;10:1–12.
The SONIA data set provides an initial starting 13. Dagirmanjian A, Ross JS, Obuchowski N. High resolution,
point for clinicians and researchers to use these tests magnetization transfer saturation, variable flip angle, time-
because the cutpoints are flexible. Because the cut- of-flight MRA in the detection of intracranial vascular steno-
points generally employed measurements of contin- ses. J Comput Assist Tomogr 1995;19:700–706.
14. Alexandrov AV, Demchuk AM, Wein TH, Grotta JC.
uous variables, velocity on TCD, or percentage of
Yield of transcranial Doppler in acute cerebral ischemia.
stenosis on MRA, the cutpoints can be adjusted as Stroke 1999;30:1604–1609.
needed to target unique cohorts of patients. When 15. Korogi Y, Takahashi M, Mabuchi N, et al. Intracranial
evaluating patients, test cutpoints can be readily ad- vascular stenosis and occlusion: diagnostic accuracy of
justed to take advantage of the relatively strong three-dimensional, Fourier transform, time-of-flight MR
NPV of the tests to exclude severe (⬎70%) intracra- angiography. Radiology 1994;193:187–193.
16. Ley-Pozo J, Ringelstein EB. Noninvasive detection of oc-
nial stenosis in patients, which carries a particularly
clusive disease of the carotid siphon and middle cerebral
dire prognosis.30 Physicians have reliable SONIA artery. Ann Neurol 1990;28:640–647.
data on which to base decisions made after TCD or 17. Oelerich M, Lentschig MG, Zunker P, Reimer P, Rummeny
MRA, including referral to angiography for emerg- EJ, Schuierer G. Intracranial vascular stenosis and occlusion:
ing therapies such as angioplasty and stenting. comparison of 3D time-of-flight and 3D phase-contrast MR
angiography. Neuroradiology 1998;40:567–573.
Received July 21, 2006. Accepted in final form February 15,
2007. 18. Rorick MB, Nichols FT, Adams RJ. Transcranial Dopp-
ler correlation with angiography in detection of intracra-
nial stenosis. Stroke 1994;25:1931–1934.
REFERENCES 19. Stock KW, Radue EW, Jacob AL, Bao X-S. Intracranial ar-
1. Williams GR, Jiang JG, Matchar DB, Samsa GP. Inci- teries. Prospective blinded comparative study of MR angiog-
dence and occurrence of total (first-ever and recurrent) raphy and DSA in 50 patients. Radiology 1995;195:451–456.
stroke. Stroke 1999;30:2523–2528. 20. Wentz KU, Rother J, Schwartz A, Mattle HP, Suchalla R,
2. Sacco R, Kargman D, Gu Q, Zamanillo M. Race-ethnicity Edelman RR. Intracranial vertebrobasilar system: MR an-
and determinants of intracranial atherosclerotic cerebral giography. Radiology 1994;190:105–110.
infarction. The Northern Manhattan Stroke Study. 21. Bash S, Villablanca JP, Jahan R, et al. Intracranial vascular
Stroke 1995;26:14–20. stenosis and occlusive disease: evaluation with CT angiogra-
3. McGarry P, Solberg LA, Guzman MA, Strong JP. Cere- phy, MR angiography, and digital subtraction angiography.
bral atherosclerosis in New Orleans: comparisons of le- AJNR Am J Neuroradiol 2005;26:1012–1021.
sions by age, sex, and race. Lab Invest 1985;52:533–539. 22. The SONIA Investigators. Stroke Outcomes and Neuro-
4. Gorelick PB. Distribution of atherosclerotic cerebrovas- imaging of Intracranial Atherosclerosis (SONIA): design
cular lesions: effects of age, race, and sex. Stroke 1993; of a prospective, multi-center trial of diagnostic tests.
24(suppl I):I-16–I-19. Neuroepidemiology 2004;23:23–32.
5. Feldmann E, Daneault N, Kwan E, Ho KJ, Pessin MS, 23. Adams R, McKie V, Brambilla D, Carl E, Gallagher D,
Langenberg P. Chinese-white differences in the distribu- Nichols F. Stroke prevention trial in sickle cell anemia.
tion of occlusive cerebrovascular disease. Neurology Control Clin Trials 1997;1:110–129.
1990;40:1541–1545. 24. Samuels OB, Joseph GJ, Lynn MJ, Smith HA, Chimowitz
6. Chimowitz M, Lynn M, Howlett-Smith H, et al. Compar- MI. A standardized method for measuring intracranial ar-
ison of warfarin and aspirin for symptomatic intracranial terial stenosis. AJNR Am J Neuroradiol 2000;21:643–646.
arterial stenosis. N Engl J Med 2005;352:1305–1316. 25. Korogi Y, Takahashi M, Nakagawa T. Intracranial vas-
7. Dion J, Gates P, Fox A, Barnett H, Blom R. Clinical cular stenosis and occlusion: MR angiographic findings.
events following neuroangiography: a prospective study. AJNR Am J Neuroradiol 1997;17:832–840.
Stroke 1987;18:997–1004. 26. Bragoni M, Feldmann E. Transcranial Doppler indices of in-
8. Hass WK, Fields WS, North RR, Kricheff I, Chase NE, tracranial hemodynamics. In: Tegeler CH, Babikian VL, Go-
Bauer RB. Joint Study of Extracranial Arterial Occlusion mez CR, eds. Neurosonology. Boston: Mosby; 1996:129–40.
II. Arteriography, techniques, sites, and complications. 27. Wilterdink JL, Feldmann E, Easton JD, Ward R. Performance
JAMA 1968;203:961–968. of carotid ultrasound in evaluating candidates for carotid endar-
9. Theodotou BC, Whaley R, Mahaley MS. Complications terectomy is optimized by an approach based on clinical out-
following transfemoral cerebral angiography for cerebral come rather than accuracy. Stroke 1996;27:1094–1098.

Neurology
Downloaded from www.neurology.org at University of Oxford (oxu) 68onJune
/ England 12, 2007
October 18, 2011 2105
28. U-King-Im J, Trivedi R, Graves M, et al. Contrast-enhanced raphy in the initial evaluation of stroke and TIA. Neurol-
MR angiography for carotid disease; diagnostic and poten- ogy 2004;63:457–460.
tial clinical impact. Neurology 2004;62:1282–1290. 30. Kasner SE, Chiowitz MI, Lynn MJ. Predictors of ischemic
29. Josephson S, Bryant S, Mak H, Johnston S, Dillon W, stroke in the territory of a symptomatic intracranial arte-
Smith W. Evaluation of carotid stenosis using CT angiog- rial stenosis. Circulation 2006;113:555–563.

Check Out New Additions to Rare Books Collection


The AAN Library Collection at the Bernard Becker Medical Library of Washington University
School of Medicine in St. Louis has acquired two significant neurology tomes. The rare 1913
classic on pediatric neurology, Die Erkenntniss und Heilung der Gehirnentzundung, des inneren
Wasserkopfes und der Krampfkrankheiten im kindlichen Alter by Eduard Loebenstein Loebel,
includes a signed presentation letter from American neurologist Hans H. Reese, MD, to Francis
Forster, MD, one of the founders of the AAN. Hughlings Jackson on Aphasia and Kindred Affec-
tions of Speech, by Sir Henry Head (1926), is a book reprint from Brain, Parts I and II (1915), and is
considered the most important work on aphasia in the English language.
Visit www.aan.com/rarebooks for more information on the AAN Library Collection. AAN mem-
bers may use the collection by contacting Lilla Vekerdy, Librarian, at vekerdyl@wustl.edu or (314)
362-4235.

2106 Neurology
Downloaded 68www.neurology.org
from June 12, 2007 at University of Oxford (oxu) / England on October 18, 2011
The Stroke Outcomes and Neuroimaging of Intracranial Atherosclerosis (SONIA)
Trial
E. Feldmann, J. L. Wilterdink, A. Kosinski, et al.
Neurology 2007;68;2099; Published online before print April 4, 2007;
DOI 10.1212/01.wnl.0000261488.05906.c1
This information is current as of October 18, 2011

Updated Information & including high resolution figures, can be found at:
Services http://www.neurology.org/content/68/24/2099.full.html

Supplementary Material Supplementary material can be found at:


http://www.neurology.org/content/suppl/2007/06/12/01.wnl.000
0261488.05906.c1.DC1.html
References This article cites 28 articles, 16 of which can be accessed free
at:
http://www.neurology.org/content/68/24/2099.full.html#ref-list-
1
Citations This article has been cited by 16 HighWire-hosted articles:
http://www.neurology.org/content/68/24/2099.full.html#related-
urls
Subspecialty Collections This article, along with others on similar topics, appears in the
following collection(s):
All Cerebrovascular disease/Stroke
http://www.neurology.org/cgi/collection/all_cerebrovascular_di
sease_stroke
CT
http://www.neurology.org/cgi/collection/ct
MRI
http://www.neurology.org/cgi/collection/mri
Ultrasound
http://www.neurology.org/cgi/collection/ultrasound
Permissions & Licensing Information about reproducing this article in parts (figures,
tables) or in its entirety can be found online at:
http://www.neurology.org/misc/about.xhtml#permissions
Reprints Information about ordering reprints can be found online:
http://www.neurology.org/misc/addir.xhtml#reprintsus

Downloaded from www.neurology.org at University of Oxford (oxu) / England on October 18, 2011

You might also like