Evidence-Based Medicine in PRACTICE

I. Introduction II. Appraisal of Original Research III. Appraisal of Qualitative Studies IV. Statistical Measures in EBM

Objectives:
to:
1.

After going through the workshops we are able

2. 3.

4.

Enumerate the steps in applying EBM into clinical practice; Demonstrate efficient literature search; Appraise and use RCTs and other study designs in clinical problem solving; and Identify problems and solutions in the application of EBM.

Steps in EB practice

Identify a problem or area of uncertainty Formulate a relevant, focused, clinically important question that is likely to be answered Find the evidence Appraise the evidence

IDENTIFY FORMULATE SEARCH APPRAISE DECIDE EVALUATE SUMMARIZE

Assess the clinical importance of the evidence Assess the clinical applicability of any recommendations or conclusions

Decide whether or not to act on the evidence Assess the outcomes of its clinical application Summarize and store records for future reference

APPRAISAL
OF

ORIGINAL RESEARCH
Evidence-based Medicine in Clinical Practice

Primary Studies (quantitative methodologies)

Skim read Slowly analyze with these screening questions

Did

the study address a clearly focused issue? Was the assignment of subjects to treatment randomized? Were all the subjects who entered the trial properly accounted for in the conclusion?

Tease out the meaning of the article by looking for answers to these questions:

Are there any differences between the two groups in terms of selection bias or confounding variables that could explain the differences between them? Blinding Were the groups similar at the start of the trial? Excepting the experimental intervention, were the groups treated equally?

Tease out the meaning of the article by looking for answers to these questions:

How large was the difference between groups? How precise was the estimate of the treatment effects? Can the results be applied to your work? Were all the important outcomes considered? Are the benefits worth the harms and costs?

Upper Gastrointestinal Tolerability of Celecoxib, a COX-2 Specific Inhibitor, compared to Naproxen and Placebo Bensen W., et al Journal of Rheumatology, August 2000; 27:8

Population: a total of 5,616 patients ( OA, RA ) were enrolled in the five

trials conducted in the US and Canada

Intervention: Naproxen 500 mgs BID; Placebo; Celecoxib 50mgs BID;

Celecoxib 100mgs BID; Celecoxib 200mgs BID; Celecoxib 400mgs BID

Outcome: Incidence and time until moderate to severe abdominal

pain, nausea and any of the three symptoms were determined using time- intervention analysis

Method: Five 12 week, randomized, double blind, parallel group,

placebo controlled trials were conducted

USERs GUIDE in Critical Appraisal

Primary Validity Guides Secondary Validity Guides Results

Critical Appraisal

Primary Validity Guides

Was

the assignment of patients to the treatment randomized? Were all the patients who entered the trial accounted for and attributed at its conclusion?
Was

the follow-up complete? Were the patients analyzed in the groups to which they are randomized?

Critical Appraisal

Secondary Validity Guides

Were

the patients, their clinicians and study personnel blind to the treatment? Were the groups similar at the start of the trial? Aside from experimental intervention, were the groups treated equally?

Critical Appraisal

Results
How

large was the treatment effect? Can the results be applied to my patients? Were all the important outcomes considered? Was the efficacy in symptomatic relief of the different treatments considered? What was the conclusion of the study?

Results:

Cumulative Indices of GI intolerability

Naproxen 500 mgs BID Placebo Celecoxib 50mgs BID Celecoxib 100mgs BID Celecoxib 200mgs BID Celecoxib 400mgs BID 24.0% 18.3% 17.4% 17.5% 18.6% 15.6%

Results:
RISK REDUCTION using Final COX Proportional Hazards Model

RR to Naproxen 1.00 0.54 0.60 0.63 Naproxen 1.58 0.84 0.94 0.99 Celecoxib 50mgs BID Celecoxib 100mgs BID Celecoxib 200mgs BID

RR to Placebo

Celecoxib 400mgs BID Placebo

0.56 0.63

0.88 1.00

Conclusion:
Compared to taking Naproxen, if a patient takes Celecoxib, there is a 41% reduction in her risk of developing upper GI symptoms

APPRAISAL
OF

QUALITATIVE STUDIES
Evidence-based Medicine in Practice

Parameters to measure in a qualitative work

1.

2. 3.

a clear aim for their project, not just an attempt to gather masses of data. choice of an appropriate methodology. justification for who was and who was not included, and some discussion of the effect on drop-outs and non-responders

In essence, the 3 questions to answer are:

1.

2.
3.

What is the message? Can I believe it? Can I generalize?

it may be more important to use purposive sampling to gather the whole range of possible opinions and experiences, rather than random sampling to produce a representative study group.

Reed J. and Morgan D. (1999) Discharging older people from hospital to care homes. Journal of Advanced Nursing 29:819-825

Screening Questions:
Was there a clear statement of the aims of the research? Yes. There was a clear aim identified.
1.

To explore the experiences of older people discharge from hospital into nursing and residential care, and identify possible forms of support.

Is a qualitative methodology appropriate? Yes. A qualitative methodology is appropriate as the study considers the experiences of the subjects through their own eyes and those of the staff.
2.

Reed J. and Morgan D. (1999) Discharging older people from hospital to care homes. Journal of Advanced Nursing 29:819-825

Detailed Questions:
3.
4. 5.

6.
7. 8.

9.
10.

Sampling Strategy Data Collection Data Analysis Research Partnership Relations Findings Justification of Data Interpretation Transferability Relevance and Usefulness

Sampling Strategy
a) b) c) d) e) From where was the sample selected? Why? Who was selected? Why? How were they selected? Why? Was the sample size justified? Is it clear why some participants chose not to take part? f) Was the sampling strategy appropriate to address the aims?

Sampling Strategy
a)

From where was the sample selected? Why?

The sample was identified from hospital records for an acute trust in the northeast of England, which was the study setting.

b)

Who was selected? Why?

The samples were discharges from hospital, nominated family members and hospital and residential home nursing staff.

c)

How were they selected? Why?

A purposive sample of 20 elder adults discharged from the hospital. From an original sample of 48, nineteen were excluded because of frailty or cognitive impairment; others declined to take part, had left the area or died. The subjects nominated 17 family members. 24 staff were interviewed, plus 6 written responses submitted, although the report states that this amounts to a total of 29 respondents.

Sampling Strategy
d)

Was the sample size justified?

No justification of sample size offered, and there is no indication if there
was data saturation

e)

Is it clear why some participants chose not to take part?

It is not clear why three patients did not participate. Staff responses were reported to be reduced as a consequence of work commitments, and the report indicates a range of efforts that were made to increase staff responses.

f)

Was the sampling strategy appropriate to address the aims?

Overall, the sampling strategy was compromised by problems in sampling, appropriate staff and a high attrition rate among eligible patients.

Data Collection
a) Is the data collection clear? Where the setting of the data collection was,
and why that setting was chosen there is clear description of the setting, but it is unclear why this setting was chosen.

b)

How the data were collected? Why? Patients and family were interviewed
individually, while staff were interviewed in focus groups, with one individual interview and six responses to a questionnaire. It is unclear why focus groups were the method of choice for the staff, and other methods were used in this group on ad hoc basis.

c) d) e)

How the data were recorded? Why? Semi-structured interview schedules

were used: no description of data recording method.

Were the methods modified during the process? Why? The methods
of collection of staff data were modified to increase the sample size.

Were the data collected in a way that addresses the research issue? Overall, data collection methods addressed the research question.

Data Analysis
a) How was the analysis done?
There is no discussion of how data analysis were undertaken

b) Was the data analysis sufficiently rigorous?

It is not possible to tell if the analysis was sufficiently rigorous. There are questions about the validity and reliability of the analysis

Research Partnership Relations

a) Did the researchers critically examined their own role? Were there potential bias and influence? The study claimed to
be an action research approach, working in collaboration with staff.

b) Where were the data collected? Why was the setting chosen? There is no discussion of why the particular research setting was
chosen.

c) How was the research explained to the participants?

There is no description of how the research was explained to the participants, and it was unclear on what basis patients were given an opportunity to refuse to take part. There is no discussion of ethics of interviewing older people about their

experiences.

d) Was relationship between researchers and participants been adequately considered? Overall, there is no adequate
discussion of the research relationships.

Findings

The findings are presented clearly, but the findings from the different groups of respondents address different themes, and there is little effort to bring these together.

Justification of Data Interpretation

a) Is there a sufficient data presented to support the findings?
There is not sufficient data presented to justify the findings. This is particularly the case for the patient group, in which quotations are brief and infrequent

b) Do the researchers explain how the data presented in the paper were selected from the original sample?
The researchers do not explain how the quotations presented were selected.

Transferability

The setting is described adequately to enable judgments of transferability; however this may be compromised because of threats to validity. In particular, the response rate was under 50% and the study is limited to the cognitively intact: i.e. it excludes any dementia or confusion

Relevance and Usefulness

Is the study relevant and useful?

In
In

terms of addressing the research aim?

study is relevant to the aim

The

terms of contributing something new to understanding / new insights / different perspectives?

The

study offers new insights study offers potential for further research are policy and practice implications

In

terms of suggesting further research? terms of impacting on policy / practice?

The

In

There

STATISTICAL MEASURES in EVIDENCEBASED MEDICINE(EBM)

Clinical Benefits of Tests and Treatment expressed in Mathematical Methods

Evidence-based Medicine is..

the paradigm of judicious, explicit, and conscientious utilization current best available information in making high quality

Mathematical Tools in EBM

Decisions on diagnosis

Decisions on interventions/treatment

Likelihood Ratios (LR) Sensitivity Specificity Predictive Values Pre-test Odds Post-test Odds AUC-ROC

Relative Risk Ratio(RR) Absolute Risk Ratio(ARR) Number Needed to Treat (NNT)

CASE SCENARIO A
A 65 year-old retired school principal goes to a family clinic because of occasional dribbling, urgency and hesitancy. The physician suspected BPH or prostatic cancer as the cause of the symptoms, thus he proposed DRE to support his working impression. The patient asks whether there are procedures other than DRE. What must the physician advise? What are the alternate diagnostics should the physician recommends?

Step1- formulate a question or clinical dilemma

Among elderly males, what diagnostic procedure will be used to detect early stage of prostatic cancer using a randomized control trial (RCT)?

Step 2 search for the current best available evidence (CBAE)

Comparison of DRE and PSA in the early detection of prostatic cancer: Results of Multicenter Clinical Trial of 6,630 men.

Step 3 Appraise

Primary Validity Guides

1.

2.

Was there an independent and blind comparison with a reference standard? What was the reference standard? Were they assessed independently? Did the patient sample include an appropriate spectrum of patients to whom the test will be used?
Was the reference standard done regardless of the result of the diagnostic test being evaluated? Were the methods for performing the test described in sufficient detail to permit replication?

Secondary Validity Guides

3.

4.

Step 3 Appraise

Primary Validity Guides

1.

Was there an independent and blind comparison with a reference standard? What was the reference standard? Were they assessed independently?
The reference standard used here is the the quadrant biopsy result done on patients with PSA > 4ugs/L and with suspicious DRE regardless the TRU reveals negative suspicion of cancer; the patients were assessed independently using PPV of both interventions the DRE and PSA

2.

Did the patient sample include an appropriate spectrum of patients to whom the test will be used?
Spectrum of patients: 6,630 males from 6 medical centers Age distribution: 50-96 (mean=62.8)

Reference Standard

..the test that gives nearest the truth

information

Predictive Values

Positive Predictive Value is the proportion of people who score positive on the test who actually have the disorder.
Applying this definition in our case, PPV is the percentage of patients actually having cancer when the detection method is very suspicious.

PPV = a/a+b Negative predictive value is the proportion of people who score negative on the test who actually do not have the disorder. NPV = d/c+d

Prevalence; Sensitivity; Specificity

Prevalence the proportion of people in the sample who have the disorder Sensitivity the proportion of people who have the disorder and who are detected by the test Sn = a/ a+c or TP/ TP+FN Specificity the proportion of people who do not have the disorder and are detected by the test not to have the disorder Sp = d/ d+b or TN/ TN+FP

Likelihood Ratio

LR (+) = the likelihood that a positive test comes from a person with the disorder rather than without LR (+) = Sensitivity/1- Specificity LR (-) = the likelihood that a negative test comes from a person with the disorder rather than the one without the disorder LR (-) = 1- Sensitivity/Specificity

Step 3 Appraise

Secondary Validity Guides

3.

Was the reference standard done regardless of the result of the diagnostic test being evaluated?

The reference standard which was the performance of biopsy was only done to patients with: >4ug/L PSA and suspicious DRE regardless the TRU was not suspicious of cancer
4.

Were the methods for performing the test described in sufficient detail to permit replication?

PSA concentration (HYBRITECH TANDEM-E or PSA immunoenzymatic assay) DRE performers Urosurgeons and Medical Oncologists

Step 4 Decide

What were the likelihood ratios for the different possible test results? Pre-test probability = 60% LR (+) = 0.26/ 1-0.87 = 0.26/0.13 = 2 LR (-) = 1-0.26/ 0.87 = 0.74/0.87 = 0.85

this means, the diagnostic threshold of the test is 85% to rule in the disease, 20% to rule it out

Step 4 Decide

Will the reproducibility of the test and its interpretation be satisfactory in my setting?
PSA

and DRE are available in some parts of the country, hence the results of this study could be reproduced using the same group of patients and the same inclusion criteria in the study

Step 4 Decide

Are the results applicable to the patient?

Yes.

If the user of this info is familiar of how the study was made and is similar to the profile of his patients in his workplace, then this is one of the helpful resources in his clinical decisions.
depends. Generally, if DRE and PSA are not contradictory to the patients health belief system then it will definitely change a generalists perspective in using these diagnostics to confirm impressions or the diagnosis.

Will the results change my management?

It

Step 5 Evaluate

Case Resolution:
PSA

in conjunction with DRE enhances early prostate cancer detection Prostatic biopsy should be considered if:
PSA >

4mgs/L DRE result is suspicious of cancer (even in the absence of trans-rectal ultrasound)

CASE SCENARIO B
A 60 year old male businessman consults because his younger brother diagnosed to have CHD 3 years ago, died of Acute MI, 2 weeks ago. He is generally healthy and asymptomatic. His BP range is 110-120/70-80. Smoking history of 10 pack years. He asks..
Doc, will taking aspirin prevent me from developing symptoms? What are my chances of developing heart attack like my brother?

Step 1- formulate a question or clinical dilemma

Among persons 60years and above, will prophylactic aspirin prevent angina attacks using RCT?

Step 2 search for the current best available evidence

From the article:

Aspirin in the Primary Prevention of Angina Pectoris in a Randomized Trial of United States Physicians
JoAnn E. Manson, Diederick E. Grobbee, Meir J. Stampfer, James O. Taylor, Samuel Z. Goldhaber, Michael Gaziano, Paul M. Ridker, Julie E. Buring, Charles H. Hennekens

Step 3 appraise

Primary Validity Guides

Was the assignment of patients to treatment randomized? Were all patients who entered the trial properly accounted for and attributed at its conclusion?

Was follow-up complete? Were patients analyzed in the groups to which they were randomized?

Secondary Validity Guides

Were patients, their clinicians, and study personnel blind to treatment? Were the groups similar at the start of the trial? Aside from the experimental intervention, were the groups treated equally?

Step 3 appraise

Primary Validity Guides

Was

the assignment of patients to treatment randomized?

Randomization versus random selection

Were

all patients who entered the trial properly accounted for and attributed at its conclusion?

Was follow-up complete?

Drop-outs; Withdrawals

Were patients analyzed in the groups to which they were randomized?

Intention-to-treat analysis

Step 3 appraise

Secondary Validity Guides

Were

patients, their clinicians, and study personnel blind to treatment? the groups similar at the start of the

Were

trial?
Aside

from the experimental intervention, were the groups treated equally?

Co-interventions

Step 4 decide

Overall, is the study valid? What are the results?

How

large was the treatment effects? Risk in Control, Risk in Treatment, Relative Risk (RR), Relative Risk Reduction (RRR), Absolute Risk Reduction (ARR) precise was the estimate of treatment effect? 95% confidence interval, p value

How

Step 4 decide

Can the results help me in caring for my patients?

Can the results be applied to my patient care? Inclusion criteria Exclusion criteria Were all clinically important outcomes considered? Outcome Results Are the likely treatment benefits worth the potential harm and costs? Side effects NNT Costs

Step 4 decide

Can the results help me in caring for my patients?

Can the results be applied to my patient care?

Inclusion criteria: 40-80 years old Male Free from previous myocardial infarction, stroke, and transient cerebral ischemia Exclusion criteria Angina History of coronary revascularization

Step 4 decide

Can the results help me in caring for my patients?

Were all clinically important outcomes considered?

Outcome and Results

The baseline characteristics of the analyzed participants were virtually identical in the aspirin and placebo groups During an average of 60.2 months of follow-up, there were 331 subjects with confirmed angina pectoris, 194 of whom underwent coronary artery revascularization procedures (bypass graft surgery or percutaneous transluminal coronary angioplasty). When compared to those assigned placebo, the RR of confirmed angina pectoris among participants randomized to receive aspirin was 1.10 (95%CI, 0.88-1.38). For coronary revascularization procedures, the relative risk was 1.19 (95%CI, 0.88-1.59). After simultaneous control for other coronary risk factors, the RR were not materially altered; 1.07(0.84-1.36) and 1.11(0.81-1.52) respectively.

Step 4 decide

Can the results help me in caring for my patients?

Are

the likely treatment benefits worth the potential harm and costs?
effects abdominal discomforts, nausea, vomiting, or GI bleeding NNT 1/ARR = 1/Rc-Rt =1/0.003 = 1/0.023-0.026 = 333
Side

Number needed to treat is the number of patients who need to be treated to achieve one favorable outcome
Aspirin is a relatively inexpensive drug

Costs

Clinical Decision about Therapy or Prevention

The advantage of randomization is that if sample size is sufficiently large, it assures that both known and unknown determinants of outcome are evenly distributed between the treatment and control groups. If substantial numbers are lost to follow-up, the validity of the conclusions are open to question. A dropout rate of 20% or more is usually substantial The numbers lost in the treatment group are assumed to have bad outcomes and the numbers lost in the control group are assumed to have been cured.

Clinical Decision about Therapy or Prevention

The principle of attributing all patients to the group to which they were randomized results in an intention to treat analysis. This strategy preserves the value of randomization: prognostic factors are equally distributed in the 2 groups, and the effect will be just that due to the treatment assigned. Blinding is the process by which the intervention being given is concealed from the patient, the clinicians, and the one who analyzes the data. Opinions of patients, clinicians or data analysts, whether optimistic or pessimistic, can systematically distort treatment outcomes, called the reporter and observer bias.

Clinical Decision about Therapy or Prevention

Risk in Control (Rc) = Death in the control/N patients in the control group Risk in Treatment (Rt) = Death in the treatment/N patients in the treatment

Absolute Risk Reduction (ARR) = the absolute difference between the proportion who died in the placebo/control group compared with the treatment group = Rc Rt
Relative Risk (RR) = the risk of events in the treatment group relative to the placebo/control group = Rt/Rc Relative Risk Reduction (RRR) = 1- RR is the most useful measure to use in explaining the benefit of treatment to patients.

Step 5 evaluate

Resolution of the clinical problem:

This

randomized trial data indicate that chronic platelet inhibition with low-dose aspirin for an average duration of 60.2 months does not reduce the incidence of angina pectoris Among elderly asymptomatic patients, low-dose aspirin is not recommended because it has no proven effect on the prophylaxis of clinical angina

Thank You
..Thank you for