Statistics 542

Introduction to Clinical Trials

Meta Analysis

542-11-#1
 Meta-Analysis
Alternatives? Occasionally

Complementary? Yes

Meta-Analysis
Combination of similar studies using
similar subjects and similar treatments
and similar outcomes

542-11-#2
 Figure 2
Odds Ratios and 95% Confidence Limits
for Various Studies and a Pooled Estimate

542-11-#3
 New Method of Analyzing Health Data Stirs Debate
by Lawrence K. Altman
Increasing use of a controversial statistical method to
evaluate medical therapies and surgical procedures is
beginning to affect profoundly the care of pregnant
women and patients with cancer, heart disease and
many other common conditions.
The method, known as meta-analysis promises to plan
an increasingly important role in determining health
risks, environmental hazards and national policy on
payment for medical care.
Backers say technique can draw big, reliable
conclusions from small, inconsistent findings.
Meta-analysis is a term derived from the Greek
meaning an analysis that is more comprehensive. The
larger numbers obtained by combining studies provide a
greater statistical power than any of the individual
studies. Researchers are often able to draw more
reliable inferences or new conclusions from the
combined results than from the smaller studies that may
be inconclusive individually.
In earlier applications of meta-analysis, researchers
evaluated intelligence quotients, government social
welfare programs and many other topics. Meta-analysis
has come to medicine late, but it is now undergoing a
boom in popularity, said Dr. Thomas C. Chalmers, a
distinguished physician of the Department of Veterans
Affairs in Boston and a pioneer in methodology.
The method involves an analysis of previous analyses.
It combines the results of a wide range of existing
smaller studies and then applies one of several
statistical techniques to discover more precisely what is
known from previous research. It may also produce a
unified result from diverse, apparently contradictory
studies.
The technique has already shed new light on the
effectiveness of medical therapies. Although it has not,
in itself, revolutionized any medical treatment it has
helped clear away the confusion caused by studies with
scattered and apparently conflicting findings and has
strengthen and confirmed findings from traditional
clinical trials. NY Times 8/21/90
542-11-#4
Reference: NIH Proceedings
Methodologic Issues in Overviews
of Randomized Clinical Trials

NIH Conference
May 1986

Statistics in Medicine
Vol 6, No. 3, 1987

542-11-#5
 What is the Purpose?
a. Testing for a treatment effect (rejecting the
null hypothesis)

b. Evaluating a safety issue (rare events)

c. Estimating size of treatment effect in

subgroups

d. Design of new studies

e. Develop practice guidelines

542-11-#6
 Ideal Meta Analysis
is
Randomized Multi-center Control Trial

Same protocol
Same treatment
Same type of subjects
Same outcome measure

542-11-#7
 Issues in Meta Analysis
Differences Across Studies in:
a. Treatment
b. Control Group/Population
c. Time Span (Disease, Background Therapy)
d. Outcome Measures

Publication Bias
Completeness/Quality of Data
Access to Data

542-11-#8
What Studies Should Be Included?

All existing studies

All published studies

"Non-flawed" trials

Other selection criteria

542-11-#9
Meta-Analysis: When? (1)
Retrospective Analyses

Test Treatment Effect When:

Definitive answer not yet available
No more studies likely
Need to salvage available results

Develop Practice Guidelines

Design New Studies
542-11-#10
Meta-Analysis: When? (2)
Prospective Analyses

Not recommended

Better to design in advance

proper multi-center trial(s)

542-11-#11
 Meta-Analysis
Methodology Not New

Combining p-values, Fisher (1948)

Analysis of Variance, Fisher (1938)

Combining 2x2 Tables

Mantel-Haenszel (1959)
Cochran (1954)
542-11-#12
 Odds Ratio
OR = ad/bc
T C
S a b a+b
F c d c+d
a+c b+d

more explicitly
a /( a c)
c /( a c) a / c ad
OR
b /( b d ) b / d bc
d /( b d )
542-11-#13
Methods of Meta-Analysis
1.0 Collapse Data

RCT-1 RCT-2 Collapsed

T C T C T C
S 15 5 S 5 15 20 20
F 85 95 F 95 85 180 180
OR = 3.35 OR = 0.30 OR = 1.0

Collapsing can be misleading if there is

qualitative interaction.

542-11-#14
 Methods of Meta Analysis
2. Graphical

See Figure

95% CI for each study

(ad / bc) exp { 1.96 (1/a + 1/b + 1/c + 1/d) }

542-11-#15
Apparent effects of fibrinolytic treatment on morality in the randomised trials of IV treatment of
acute myocardial infarction. Stat in Med 7:890: 1988. 542-11-#16
Comparison of meta-analysis of 12 RCTs of i.v.mixed drugs (double-blind) with i.v. metoprolol
(double-blind) and i.v. atenlol (open study). Stat in Med 6(3): 320, 1987.
542-11-#17
Comparison of meta-analysis of mortality in 11 RCTs and reinfarction rates in 10 RCTs of i.v.
streptokinase with large co-operative study (GISSI). Stat in Med 6(3): 320, 1987.
542-11-#18
Comparison of meta-analysis of 7 small RCTs of phenobarbital in the treatment of neonatal intra-cranial
haemmorrhage with one large co-operative study (3 institutions). Endpoints are total infants with
haemmorrhage and totals with severe haemorrhage (Grades III-IV) only. Stat in Med 6(3): 321, 1987.
542-11-#19
 Odds Ratios and 95% Confidence Limits
for Various Studies and a Pooled Estimate

542-11-#20
 Methods of Meta Analysis
3. Blocking (Peto-MH)

Overall Estimate
Let O = ai
E = Ei Ei = (ai + ci)(ai + bi)
ni
V = Vi Vi = (ai + ci) )(bi + di)(ci + di)(ai + bi
ni2 (ni - 1)
Z=O-E
V

C Pooled OR
OR = exp { (O - E) / V }
95% CI = exp { (O - E) / V 1.96 / V } 542-11-#21
 Methods of Meta Analysis
4. Averaging P-values Fisher (1948)

Pi = P-value for ith trial

Z = -2 log (Pi) ~2 with 2N df

5. Averaging Test Statistics

wi Z i
Z 'i ~ N(0,1)
[ wi2 ]1 / 2

e.g. wi = ni

542-11-#22
Meta-Analysis Examples
Cardiology
Post MI Treatments
(e.g., beta-blockers, aspirin)

Thrombolytic Therapy
(e.g., streptokinase)

Anticoagulants
542-11-#23
 Registries/Databases
Byar (1980) Biometrics

D'Ambrosia, Ellenberg (1980)

Biometrics

Starmer et al. (1980) Biometrics

Mantel (1983) Statistics in Medicine

542-11-#24
 Registries/Databases
Use Clinical Observational Series to:

Describe Clinical Practice

Identify Risk Factors

"Evaluate" Treatment

Historical

Concurrent
542-11-#25
 Databases
Treatment Evaluation

Comparison Requires Risk Factor Comparability

Measured
Not Measured or Unknown

Statistical Models Usually Not Adequate

Association vs. Estimation
Model Only an Approximation
Small Portion of Outcome Explained

542-11-#26
 Potential Biases

Time Trends (Decline in CHD Death)

Ascertainment

Changes in Diagnostic Criteria

Availability of Technology

Selection Bias
542-11-#27
Compliance Adjustment
Coronary Drug Project (NEJM, 1980)
5 Year Mortality

Compliance Clofibrate Placebo

< 80% 24.6% 28.2%
> 80% 15.0% 15.1%
All 18.2% 19.4%

542-11-#28
 Registries
Bias in Treatment Effect

(Peto, Biomedicine, 1978)

Trials of Anticoagulant Therapy

Design Studies Patients Effect

Historical 18 900 50% Reduction
Concurrent 8 3000 50% Reduction
RCT 6 3000 20% Reduction

542-11-#29
 PTCA
PTCA Registry
Tracked and compared usage
Lead to further trials
No PTCA vs. placebo

TIMI-II
Compared immediate vs. delayed PTCA

BARI
Compares PTCA vs. CABG
542-11-#30
 CABG
CASS RCT (Circulation, 1983)
Comparison of immediate vs. delayed
CABG

CASS Registry ( J Clin Inv, 1983)

Prognostic value of Angiography

542-11-#31