Professional Documents
Culture Documents
Lesson 6
Section 1.5
Experiments
1
Experiments
Studies where the researchers assign treatments
to cases are called experiments. When this
assignment includes randomization (such as coin
flips) to decide which treatment a patient receives,
it is called a randomized experiment.
2
Experiment
For example, consider a medical study designed to
test whether large doses of Vitamin C can help
prevent colds.
3
Experiment
It is difficult to determine whether an experimental
treatment works unless you compare groups that
receive the treatment to groups that don't.
5
Principles of Experimental Design
7
Randomizing
Researchers randomize patients into treatment
groups to account for variables that cannot be
controlled.
For example, some patients may be more
susceptible to a disease than others due to their
dietary habits. Randomizing patients into the
treatment and control groups helps even out such
differences, and it also prevents accidental bias
from entering the study.
8
Replicating
The more cases researchers observe, the more
accurately they can estimate the effect of the
explanatory variable on the response.
In a single study, we replicate by collecting a
sufficiently large sample. Additionally, a group of
scientists may replicate an entire study to verify an
earlier finding.
9
Blocking
Researchers sometimes know or suspect that
variables, other than the treatment, might influence
the response. Under these circumstances, they
may first group individuals based on this variable
into blocks and then randomize cases within each
block to the treatment groups.
This strategy is often referred to as blocking.
10
Blocking
For instance, if we are
looking at the effect of a
drug on heart attacks, we
might first split patients in
the study into low-risk and
high-risk blocks, then
randomly assign half the
patients from each block
to the control group and
the other half to the
treatment group.
11
Principles of Experimental Design
12
Reducing Bias in Human
Experiments
Randomized experiments are the gold standard for
data collection, but they do not ensure an
unbiased perspective into the cause and effect
relationships in all cases.
Human studies are perfect examples where bias
can unintentionally arise. The problem is that
people can be affected by their beliefs as well as
by real treatments.
13
Reducing Bias in Human
Experiments
For example, stress and other psychological
factors have been shown to affect resistance to
colds. If people taking Vitamin C get fewer colds
than people who don't, we can't necessarily
conclude that the Vitamin C was responsible.
It might be that people stayed healthier because
they believed that Vitamin C works.
14
Reducing Bias in Human
Experiments
Put yourself in the place of a person in the study.
If you are in the treatment group, you anticipate
that the Vitamin C will help you.
On the other hand, a person in the control group
sits idly, hoping their participation doesn't increase
her risk of a cold.
15
Reducing Bias in Human
Experiments
These perspectives suggest there are actually two
effects: the one of interest is the effectiveness of
the drug, and the second is an emotional effect
that is difficult to quantify.
Researchers aren't usually interested in the
emotional effect, which might bias the study. To
circumvent this problem, researchers do not want
patients to know which group they are in.
16
Reducing Bias in Human
Experiments
As long as the participants don't know whether
they are in the treatment or control group (that is,
whether they got the real pills or the placebo), any
effect arising from psychological factorsknown
as a placebo effectshould affect both groups
equally.
Then, if people in the Vitamin C group get fewer
colds than people in the control group, we have
evidence that Vitamin C really works.
17
Blinding
In statistical terminology, the practice of keeping
people in the dark about who is in the treatment
group and who is in the control group is called
blinding.
A single-blind experiment is one in which the
participants don't know which group they belong
to, but the experimenters (the people administering
the treatment) do know. Using a placebo is one
way to create a single-blind experiment.
18
Blinding
However, sometimes a single-blind experiment can
still be unreliable if the experimenters can
influence outcomes.
For example, in an experiment that involves
interviews, the experimenters might speak
differently to people who received the real
treatment than to those who received the placebo.
19
Blinding
This type of problem can be avoided by making
the experiment double-blind, which means
neither the participants nor the experimenters
know who belong to each group.
(Of course, someone must keep track of the two
groups in order to evaluate the results at the end.
In typical double-blind experiments, researchers
hire experimenters to make any necessary contact
with the participants.)
20
Blinding
An experiment is single-blind if the participants do
not know whether they are members of the
treatment group or the control group, but the
experimenters do know.
An experiment is double-blind if neither the
participants nor the experimenters (people
administering the treatment) know who belongs to
the treatment group and who belongs to the
control group.
21
Example 1
For the experiment below, identify any problems
and explain how the problems could have been
avoided.
22
Example 2
For the experiment below, identify any problems and
explain how the problems could have been avoided.
24
Scope of Inference
Scope of inference refers to the type of
inferences (conclusions) that can be drawn from a
study.
The types of inferences we can make (inferences
about the population and inferences about cause-
and-effect) are determined by two factors in the
design of a study: how the subjects were selected
from the population and how the subjects were
assigned to groups.
25
Scope of Inference
If the subjects of a study are randomly selected
from a population, we can assume our results can
be extended to the overall population (assuming
the sample is large enough).
Once selected, if the subjects are then randomly
assigned to the treatment and control groups, then
we can claim causal inference. Differences
between the treatment and control group will even
out, except for the treatment itself.
26
Scope of Inference
Observational Studies Experiments
Can have population Can have population
inference, and well-designed inference, but often don't.
observational studies will Ethically, human
have it. experiments can only be
conducted on volunteers.
27
Example 3
A dentist convinces some his dental patients to be
subjects in an experiment. Participants are split into
two groups, those who are taking at least 500 mg of
Vitamin C each day and those who don't. He then
asks them how often they have canker sores in their
mouth. He finds that the proportion of those who take
Vitamin C daily and complain of canker sores is
significantly smaller than the proportion of those who
don't.
Can this study be extended to the overall population?
Can this study be used to show Vitamin C reduces
canker sores?
28
Example 4
A dentist convinces some his dental patients to be
subjects in an experiment. He randomly assigns half
of them to take 500 mg of Vitamin C each day and the
other half to take a placebo for three months. He finds
that the proportion of those who take Vitamin C daily
and complain of canker sores is significantly smaller
than the proportion of those who don't.
Can this study be extended to the overall population?
Can this study be used to show Vitamin C reduces
canker sores?
29
Example 5
A dentist selects a random sample of dental patients in
town and convinces them to be subjects in an
experiment. Participants are split into two groups,
those who are taking at least 500 mg of Vitamin C
each day and those who don't. He finds that the
proportion of those who take Vitamin C daily and
complain of canker sores is significantly smaller than
the proportion of those who don't.
Can this study be extended to the overall population?
Can this study be used to show Vitamin C reduces
canker sores?
30
Example 6
A dentist selects a random sample of dental patients in
town and convinces them to be subjects in an
experiment. He randomly assigns half of them to take
500 mg of Vitamin C each day and the other half to
take a placebo for three months. He finds that the
proportion of those who take Vitamin C daily and
complain of canker sores is significantly smaller than
the proportion of those who don't.
Can this study be extended to the overall population?
Can this study be used to show Vitamin C reduces
canker sores?
31