You are on page 1of 38

Systematic Reviews

Felix Eduardo R. Punzalan, MD


Dept. of Medicine, Section of Cardiology
Dept. of Clinical Epidemiology
Scenario

• To resolve conflicting evidence


• To answer questions where the answer
is uncertain
• To explain variations in practice
Scenario

• Have there been other good studies of the


same question?
• Is this study the only one to show an
effect? Are there other studies?
Reviews

- done in many different ways

- different strengths and weaknesses


TYPES OF INTEGRATIVE
STUDIES
REVIEW = any 2 or more
articles
SYSTEMATIC REVIEW= comprehensive,
systematic and objective search

META-ANALYSIS=results of trials
are combined statistically
Traditional Reviews
- summary of evidence and recommendations
- usually, broad-gauged questions
“management of the diabetic patients”
- experts know the literature
have the actual practice
- But
lack structure
articles are cited but
personal experience and
conventional wisdom
strength of original researches
not known
Shorthand Indicators of Quality

1. prestige of journal
2. author
3. currency/latest
4. number of articles for and
against a given point
5. design # of RCTs
SYSTEMATIC REVIEWS AND
META-ANALYSES

COMPREHENSIVE

SYSTEMATIC

OBJECTIVE
Systematic Reviews
- rigorous reviews of specific questions
- useful in addressing a single, focused
question
Ex:
Effect Aspirin in prevention of cardiovascular
events
Effect Statin treatment in prevention of
cardiovascular events
Systematic Reviews
- rigorous reviews of specific questions
- useful in addressing a single, focused
question
Ex:
ACE inhibitors of CHF
skin adhesives vs. sutures for superficial
lacerations
Elements of Systematic Reviews
1. Define the clinical question.
2. Identify all completed studies of the question,
published and unpublished.
3. Select the studies that meet high standards for
scientific validity.
4. Look for evidence of bias in the studies
selected.
5. Describe the scientific quality of the studies.
Elements of Systematic Reviews

6. Ask whether the quality is systematically related to


results of the studies.
7. Describe the studies with a figure (forest plot).
8. Decide whether the studies are similar enough to justify
combining them.
9. If they are similar enough to combine, calculate a
summary measure of effect and confidence interval.
Finding all Relevant Studies
• Search MEDLINE.
• Read recent reviews and textbooks.
• Seek the advice of experts in the content area.
• Consider articles cited in the articles already
found by other approaches.
• Consult databases of articles such as the
Cochrane Database of Systematic Reviews.
• Review registries of clinical trials (to detect
publication bias, if present).
Limiting to Scientifically Strong Studies

1. CONSORT Criteria for RCTs


Moher D, et al Annal Intern Med
2001:134:657-662
2. STARD criteria diagnostic test
performance
Bossuyt PM, et al Annal Intern Med
2003:138:40-44
Publication Bias

• Positive vs. negative studies


• editors’ bias
- registration of studies
• industry
- sponsors requirements
• language
Funnel Plots

Way of detecting bias in the selection of


studies for systematic reviews

plot main results vs. sample size


effects size of small trials vs large
trials
Fig. 12.1
How Good are the Best Studies?
A Simple Scale for Measuring Quality of Randomized
Controlled Trials
Questions:
1. Was the study described as randomized?
2. Was the study described as double-blind?
3. Was there a description of withdrawals and dropouts?
Scoring:
Give a score of 1 point for each “yes” and 0 points for each “no”.
There are no in-between marks.

Is Scientific Quality Related to Research Results?


– critically examine each papers in a systematic manner
How Good are the Best Studies?
A Simple Scale for Measuring Quality of Randomized
Controlled Trials
Questions:
1. Was the study described as randomized?
2. Was the study described as blinded?
3. Were the groups comparable?
4. Was follow up adequate?

Is Scientific Quality Related to Research Results?


– critically examine each papers in a systematic manner
Summarizing Results
Forest Plots
- points estimates and confidence interval of each
studies
Six Kinds of Useful Information in a Forest Plot
1. Number of studies
2. References to publications of component studies
3. The pattern of effect sizes
4. The number of studies that are statistically significant
5. Comparison between large vs small studies
6. Chronological listing show how results may change in
time and and when size increases
Meta-Analysis
• Practice of combining results of individual
studies (or patients in these studies)
• Provided the studies are similar enough to
be combined
Summarizing Observational and
Diagnostic Test Studies
Summarizing Diagnostic Test
Are the studies similar enough

• patient
• intervention
• follow-ups
• outcomes
2 General approaches in pooling study
results

1. Statistical test for homogeneity


- The extent to which trials results are similar

2. Make an informed judgment on the


similarities of the judgment
What is Combined-Studies or
Patients?
• usual study results are combined
• more powerful - individual patient data of
component studies
Ex. Anti-thrombotic Trialists’ Collaboration - effect of anti-
platelet therapy on vascular events in patient at high risk for
CVD
195 clinical trials, 200 patients/trial
Overall – decrease vascular events by ¼
Patient-level data – decreased among patients
who had MI, stable angina and fibrillation,
and cardiac valve surgery
How are the Results Summarized

2 Kinds of Mathematical Model

3. Fixed-effect Model
All studies are asking similar questions
4. Random-effects Model
Studies are asking somewhat different but
related questions
Figure 12.5
Summary Effect
Cumulative Meta-Analysis
- another way of looking at same
information
- new survey effect age and confidence
calculated each time results of the study
become available
Fig. 12.7
Advantages and Risk of Combining
Studies
• Advantage:
- can do subgroup analysis
- detecting rare events

• Disadvantage:
- may give misleading impression
Advantages Combining Studies
• Advantage:
- can do subgroup analysis
- detecting rare events
• Ask – Focused Clinical Question

• Acquire – Systematic, comprehensive

• Appraise

• Assess Results – combined result

• Apply – summarize conclusion and apply


to practice
• Among patients at risk of CV events, what
is the effect of statins in preventing CV
outcomes?

• Among patients at risk of CV events, what


is the effect of aspirin in preventing CV
outcomes?
Summary
• Systematic Reviews
• Traditional Review
• Elements of a Systematic Review
• Summarizing Results
• Meta-analysis

You might also like