You are on page 1of 31

MATH& 146

Lesson 6
Section 1.5
Experiments

1
Experiments
Studies where the researchers assign treatments
to cases are called experiments. When this
assignment includes randomization (such as coin
flips) to decide which treatment a patient receives,
it is called a randomized experiment.

Randomized experiments are fundamentally


important when trying to show a causal connection
between two variables.

2
Experiment
For example, consider a medical study designed to
test whether large doses of Vitamin C can help
prevent colds.

To conduct this study, researchers must ask some


people in the sample to take large doses of
Vitamin C. This is an experiment because some
participants receive a treatment that they would
not otherwise receive.

3
Experiment
It is difficult to determine whether an experimental
treatment works unless you compare groups that
receive the treatment to groups that don't.

In the Vitamin C study, for example, researchers


might create two groups of people: a treatment
group that takes large doses of Vitamin C and a
control group that does not take Vitamin C.
Having a control group is usually crucial to
interpreting the results of experiments.
4
Treatment and Control Groups

The treatment group in an experiment is the


group of sample members who receive the
treatment being tested.
The control group in an experiment is the group
of sample members who do not receive the
treatment being tested.
It is important for the treatment and control groups
to be selected randomly and to be alike in all
respects except for the treatment.

5
Principles of Experimental Design

Randomized experiments are generally built on


four principles.
Controlling so that subjects receive identical
treatments
Randomizing subjects between groups
Replicating the number of subjects between
groups
Blocking for other variables that might influence
the results
6
Controlling
Researchers assign treatments to cases, and they
do their best to control any other differences in
the groups.
For example, when patients take a drug in pill
form, some patients take the pill with only a sip of
water while others may have it with an entire glass
of water. To control for water consumption, a
doctor may ask all patients to drink a 12 ounce
glass of water with the pill.

7
Randomizing
Researchers randomize patients into treatment
groups to account for variables that cannot be
controlled.
For example, some patients may be more
susceptible to a disease than others due to their
dietary habits. Randomizing patients into the
treatment and control groups helps even out such
differences, and it also prevents accidental bias
from entering the study.

8
Replicating
The more cases researchers observe, the more
accurately they can estimate the effect of the
explanatory variable on the response.
In a single study, we replicate by collecting a
sufficiently large sample. Additionally, a group of
scientists may replicate an entire study to verify an
earlier finding.

9
Blocking
Researchers sometimes know or suspect that
variables, other than the treatment, might influence
the response. Under these circumstances, they
may first group individuals based on this variable
into blocks and then randomize cases within each
block to the treatment groups.
This strategy is often referred to as blocking.

10
Blocking
For instance, if we are
looking at the effect of a
drug on heart attacks, we
might first split patients in
the study into low-risk and
high-risk blocks, then
randomly assign half the
patients from each block
to the control group and
the other half to the
treatment group.

11
Principles of Experimental Design

It is important to incorporate the first three


experimental design principles (controlling,
randomization, replication) into any experiment,
and the material we will learn in this course will
describe methods for analyzing data from such
experiments.
Blocking is a slightly more advanced technique,
but the statistical methods we will learn may be
extended to analyze data collected using blocking.

12
Reducing Bias in Human
Experiments
Randomized experiments are the gold standard for
data collection, but they do not ensure an
unbiased perspective into the cause and effect
relationships in all cases.
Human studies are perfect examples where bias
can unintentionally arise. The problem is that
people can be affected by their beliefs as well as
by real treatments.

13
Reducing Bias in Human
Experiments
For example, stress and other psychological
factors have been shown to affect resistance to
colds. If people taking Vitamin C get fewer colds
than people who don't, we can't necessarily
conclude that the Vitamin C was responsible.
It might be that people stayed healthier because
they believed that Vitamin C works.

14
Reducing Bias in Human
Experiments
Put yourself in the place of a person in the study.
If you are in the treatment group, you anticipate
that the Vitamin C will help you.
On the other hand, a person in the control group
sits idly, hoping their participation doesn't increase
her risk of a cold.

15
Reducing Bias in Human
Experiments
These perspectives suggest there are actually two
effects: the one of interest is the effectiveness of
the drug, and the second is an emotional effect
that is difficult to quantify.
Researchers aren't usually interested in the
emotional effect, which might bias the study. To
circumvent this problem, researchers do not want
patients to know which group they are in.

16
Reducing Bias in Human
Experiments
As long as the participants don't know whether
they are in the treatment or control group (that is,
whether they got the real pills or the placebo), any
effect arising from psychological factorsknown
as a placebo effectshould affect both groups
equally.
Then, if people in the Vitamin C group get fewer
colds than people in the control group, we have
evidence that Vitamin C really works.

17
Blinding
In statistical terminology, the practice of keeping
people in the dark about who is in the treatment
group and who is in the control group is called
blinding.
A single-blind experiment is one in which the
participants don't know which group they belong
to, but the experimenters (the people administering
the treatment) do know. Using a placebo is one
way to create a single-blind experiment.

18
Blinding
However, sometimes a single-blind experiment can
still be unreliable if the experimenters can
influence outcomes.
For example, in an experiment that involves
interviews, the experimenters might speak
differently to people who received the real
treatment than to those who received the placebo.

19
Blinding
This type of problem can be avoided by making
the experiment double-blind, which means
neither the participants nor the experimenters
know who belong to each group.
(Of course, someone must keep track of the two
groups in order to evaluate the results at the end.
In typical double-blind experiments, researchers
hire experimenters to make any necessary contact
with the participants.)

20
Blinding
An experiment is single-blind if the participants do
not know whether they are members of the
treatment group or the control group, but the
experimenters do know.
An experiment is double-blind if neither the
participants nor the experimenters (people
administering the treatment) know who belongs to
the treatment group and who belongs to the
control group.

21
Example 1
For the experiment below, identify any problems
and explain how the problems could have been
avoided.

A chiropractor wants to know if his adjustments


relieve back pain. He performs adjustments on 25
patients with back pain. After interviewing all 25
patients, 18 say they feel better. He concludes
that the adjustments are an effective treatment.

22
Example 2
For the experiment below, identify any problems and
explain how the problems could have been avoided.

A new drug for attention deficit disorder (ADD) is


supposed to make the affected children more polite.
Randomly selected children suffering from ADD are
divided into treatment and control groups. Those in
the control group receive a placebo that looks like the
real drug. The experiment is single-blind.
Experimenters interview the children one-on-one to
decide whether they became more polite.
23
Inference
Statistical inference means drawing conclusions
about a study population based on data from a
sample. There are two main types of inferences:
Population Inference, or generalizability: The
results seen in the data can be extended to a
wider population.
Causal Inference: The differences seen in the
data were due to the group assignment. Cause
and effect relationships can be drawn.

24
Scope of Inference
Scope of inference refers to the type of
inferences (conclusions) that can be drawn from a
study.
The types of inferences we can make (inferences
about the population and inferences about cause-
and-effect) are determined by two factors in the
design of a study: how the subjects were selected
from the population and how the subjects were
assigned to groups.

25
Scope of Inference
If the subjects of a study are randomly selected
from a population, we can assume our results can
be extended to the overall population (assuming
the sample is large enough).
Once selected, if the subjects are then randomly
assigned to the treatment and control groups, then
we can claim causal inference. Differences
between the treatment and control group will even
out, except for the treatment itself.

26
Scope of Inference
Observational Studies Experiments
Can have population Can have population
inference, and well-designed inference, but often don't.
observational studies will Ethically, human
have it. experiments can only be
conducted on volunteers.

Cannot show causal Can have causal inference,


inference, since confounding and well-designed
variables cannot be ruled experiments will have it.
out.

27
Example 3
A dentist convinces some his dental patients to be
subjects in an experiment. Participants are split into
two groups, those who are taking at least 500 mg of
Vitamin C each day and those who don't. He then
asks them how often they have canker sores in their
mouth. He finds that the proportion of those who take
Vitamin C daily and complain of canker sores is
significantly smaller than the proportion of those who
don't.
Can this study be extended to the overall population?
Can this study be used to show Vitamin C reduces
canker sores?
28
Example 4
A dentist convinces some his dental patients to be
subjects in an experiment. He randomly assigns half
of them to take 500 mg of Vitamin C each day and the
other half to take a placebo for three months. He finds
that the proportion of those who take Vitamin C daily
and complain of canker sores is significantly smaller
than the proportion of those who don't.
Can this study be extended to the overall population?
Can this study be used to show Vitamin C reduces
canker sores?

29
Example 5
A dentist selects a random sample of dental patients in
town and convinces them to be subjects in an
experiment. Participants are split into two groups,
those who are taking at least 500 mg of Vitamin C
each day and those who don't. He finds that the
proportion of those who take Vitamin C daily and
complain of canker sores is significantly smaller than
the proportion of those who don't.
Can this study be extended to the overall population?
Can this study be used to show Vitamin C reduces
canker sores?
30
Example 6
A dentist selects a random sample of dental patients in
town and convinces them to be subjects in an
experiment. He randomly assigns half of them to take
500 mg of Vitamin C each day and the other half to
take a placebo for three months. He finds that the
proportion of those who take Vitamin C daily and
complain of canker sores is significantly smaller than
the proportion of those who don't.
Can this study be extended to the overall population?
Can this study be used to show Vitamin C reduces
canker sores?
31

You might also like