FDN 5000

Research in Education
Dr. George H. Olson, Instructor

Research Designs
Descriptive Research Designs
Correlation Research Designs
Causal-comparative Designs
Experimental Designs
Non-experimental Designs
Single-subject designs
Purposes of Research
Designs

 Provide guidance in conducting research

 Provide guidance in interpreting research
 Guidance in recognizing types of research
 Guidance in recognizing potential threats to
the validity of research conclusions
 Help guard against threats to internal and
external validity
Descriptive research
designs

 Survey research
 Longitudinal research designs
 Cohort designs
 Quasi-cohort designs
 Cross-sectional research
 Case studies
 Ethnographic studies
Correlational research
designs

 Objective: study the relationship between

variables
 Examine scatter plots (e.g., pages 223 and
225 in our text)
 Compute correlation coefficients
 range from -1 through 0 to +1
 do not imply causation
 lack of correlation does NOT imply no causation
Interpreting correlations
 Rules of thumb:
Correlation coefficient Strength of relationship
.00 to .20 negligible
.24 to .40 low
.40 to .60 moderate
.60 to .80 substantial
.80 to 1.00 high, very high
Causal comparative
research designs

 A.k.a. ex-post-facto designs

 Aimed at discovering cause and effect
relationships
 Defined groups are compared after they
have been formed
 Theory plays an important role
 In arguing for the cause-effect relationship
 In eliminating rival explanations
Causal comparative
designs: An example
Of students receiving the This proportion is known to
following grades…. drive a car to school

A 8%
B 23%
C 43%
D 77%
F 96%
What should the superintendent conclude?
Diagramming Experimental
Research Designs

 Symbols used:
 T = treatment intervention
 C = control or comparison condition (often
simply no treatment)
 O = observation (often some test score)
 R = designates random assignment
 M = designates matching
Examples of Research
Design Diagrams

(1) T O

(2) O1 T O2

O1 T O2
(3) ----------------------- (No Random Assignment)
O1 C O2
Three pre-experimental
designs

 Three designs frequently used in education

research that…
 are not sufficient for permitting strong tests of
causal hypotheses
 often due suggest new ideas
 One-group posttest-only design
 One-group pretest-posttest design
 Comparison-group posttest-only design
One-group, posttest-only
research design

T O
 A treatment followed by an observation
 Should not be confused with the one-shot case study
 Threats to internal validity:
 ALL (except regression and mortality)
 Threats to external validity:
 ALL
One-group pretest-posttest
research design
Opre T Opost
 One of the most frequently used research designs in
education
 Threats to internal validity:
 extraneous events (history and maturation)
 statistical regression
 testing
 experimenter and subject effects
 Threats to external validity
 selection and settings interactions with treatment
Comparison-group,
posttest-only design

T O
--------------
C O
--------------
C2 O
--------------
C3 O
Comparison-group,
posttest-only design

 Threats to internal validity

 Since the comparison groups are non-
equivalent, the major threat is selection
 Other threats include mortality, and subject
and experimenter reactive effects
 Threats to external validity
 Selection and settings by treatment
interaction
True experiment research
designs

 Randomized experiments
 Result in probabilistic equivalence
 Not a panacea that rules out all threats to
internal validity
 Does not control for experimenter and subject
reactive effects.
 Does not guarantee group equivalency
(especially in small samples).
Randomized posttest
comparison group design

T O
R: ---------------
C O

Note: R: means RANDOMIZATION

Randomized pretest-posttest
control group design

Opre T Opost
R: ---------------------------
Opre C Opost
R: ---------------------------
Opre C2 Opost
Randomized matched-
group design

T O
M: R: ---------------
C O
Randomized factorial
designs

TA1,B1 O
---------------
TA1,B2 O
R: ---------------
TA2,B1 O
---------------
TA2,B2 O
Factorial Design: Example

Method (B)____
Word Type (A) Computer Handwriting
B1 B2
Easy A1 2026
Hard A2 1620
____________________________
Quasi-experiments: Time
series designs

O1 O2 O3 O4 T O5 O6 O7 O8

 Pre-observations to establish a baseline

 A treatment intervention
 Post-observations to establish new
baseline
Quasi-experiments: Non-
equivalent control groups

 In these designs, randomization is either

not possible or not feasible.
 Characterized by ...
 using intact groups for treatment and
comparison
 manipulated independent variable
 Often, the best we can expect from
education research
Non-equivalent, control group,
pretest-posttest design

Opre T Opost
-------------------------
Opre C Opost

 Except for reactive effects, most threats to

internal validity are controlled
 Again settings and selection by treatment
interactions pose threats to external validity
Matched comparison
group, posttest design

T O
M: ---------
C O
 Validity depends upon how well matching
is achieved
 Potential threats to internal validity are
same as those for posttest-only designs
Single-subject designs

 Similar to time-series designs, only with a

single individual
 Repeated measurements over time
(baselines)
 Subjects serve as their own controls
 Involve a manipulated independent
variable (the intervention)
Basic single-subject
designs
 Reversal: A-B-A
 Double reversal: A - B - A - B
 Multiple baseline:
A-B-A
------------------------
A-B-A
------------------------
A-B-A
------------------------
A-B-A
A is a period of no treatment
B is a period of treatment
Example of a stable
baseline

Stable Baseline Pattern

100
90
80
70
60
50
40
30
20
10
0
Example of an increasing
baseline
Increasing Baseline Pattern
100

80

60

40

20

0
Non-experimental research
designs

 Characterized by the lack of manipulation

of an independent variable
 Three types of non-experimental research
designs:
 Causal comparative research designs
 Correlational research designs
 Descriptive research designs