STA 5305 Exam 1 Review I. Chapter 1: Research Strategies and the Control of Nuisance Variables A. Introduction 1.

. Experimental design a plan for assigning subjects to experimental conditions and the statistical analysis associated with the plan. ED involves a number of interrelated activities: a) Formulation of statistical hypotheses statements about (1) One or more parameters of a population, or (2) The functional form of a population (3) Statistical hypotheses are rarely identical to scientific hypotheses b) Determination of the experimental conditions (independent variable) to be used, the measurement (dependent variable) to be recorded, and the extraneous conditions (nuisance variables) that must be controlled c) Specification of the number of subjects (experimental units) required and the population from which they will be sampled. (1) Experimental unit that entity that is assigned to an experimental condition independently of other such entities; may contain several observational units (e.g. classroom experimental unit; each student observational unit) d) Specification of the procedure for assigning the subjects to experimental conditions e) Determination of the statistical analysis that will be performed 2. Two kinds of computational algorithms a) Classical sum of squares approach uses scalar algebra and is suitable for hand-held calculators b) Cell means model approach uses matrix algebra and is more suitable for computers B. Formulation of Plans for the Collection and Analysis of Data 1. Acceptable Research Hypotheses a) Some questions cannot be subjected to scientific investigation because there is no procedure for observing the experiment b) An experiment involves the manipulation of one or more variables by a researcher in order to determine the effect of this manipulation on another variable. (1) Questions should be reducible to if A, then B. c) We cannot always do this due to natures manipulation of the independent variable. 2. Distinction Between Independent and Dependent Variables a) Independent variable any suspected causal event that is under investigation (a.k.a. treatment) b) Dependent variable the measurement that is used to assess the effects of manipulating the independent variable 3. Selection of the Independent Variable a) Quantitative Independent Variable different treatment levels are different amounts of the independent variable

-1-

(1) Selection of the appropriate levels of the independent variable can be based on the results of previous experiments or on theoretical considerations (2) May be beneficial to carry out a small pilot experiment to identify the most appropriate treatment levels. b) Qualitative Independent Variable different treatment levels represent different kinds rather than different amounts of the independent variable 4. Selection of the Dependent Variable a) May be based on theoretical considerations, although often determined by practical considerations b) Consider all of the following (1) Whether the population distributions for the various treatment levels are approximately normal (2) Whether the populations have equal variances c) If theoretical considerations do not dictate the selection of a dependent variable and if several alternative variables are equally sensitive and reliable, in addition to being normally distributed with equal variances, a researcher should select the variable that is most easily measured. 5. Nuisance Variables a) Defined undesired sources of variation in an experiment that affect the dependent variable b) Sources of nuisance variables are many (1) Calibration of equipment may change during the course of an experiment (2) Presentation of instructions may vary slightly from subject to subject (3) Errors may occur in measuring or recording a subjects response (4) Environmental factors may not be constant for all subjects c) If not controlled, nuisance variables can affect the outcome of an experiment d) Bias occurs when a nuisance variable systematically distorts results in a particular direction e) Nuisance variable can also increase the variability of the phenomenon being measured and, thereby, increase the error variance f) Worst case scenario a nuisance variable systematically distorts results in a particular direction and increases the error variance C. Research Strategies 1. Research is performed for four purposes a) To explore b) To describe or classify c) To establish relationships, or d) To establish causality 2. Research strategies to accomplish these purposes include a) The experiment b) The quasi experiment c) The survey

-2-

d) The case study e) Naturalistic observation 3. Experiment a) Enables the researcher to test a hypothesized relationship between an independent variable and a dependent variable by manipulating the independent variable b) Usually performed in an environment that permits a high degree of control of nuisance variables c) Characterized by (1) The manipulation by the researcher of one or more independent variables (2) The use of controls such as the random assignment of subjects to experimental conditions in order to minimized the effects of nuisance variables, and (3) The careful observation or measurement of one or more dependent variables d) The first and second characteristics above distinguish experiments from other research strategies. e) Manipulation of one or more independent variables also is necessary for inferring causality f) It is inferred that A causes Y if (1) A precedes Y (temporal precedence of A) (2) Whenever A is present, Y occurs (sufficiency of A), and (3) A must be present for Y to occur (necessity of A) 4. Quasi-Experiments a) Similar to experiments except that the subjects are not randomly assigned to the independent variable b) Used when random assignment is not possible or when it is necessary to use preexisting or naturally occurring groups (e.g. subjects with a particular illness or subjects who have been sexually abused) c) Random assignment is the best safeguard against undetected nuisance variables 5. Surveys a) Rely on the technique of self-report to obtain information about such variables as peoples attitudes, opinions, behaviors, and demographic characteristics b) Data are collected by means of an interview or questionnaire c) Cannot establish causality, but can explore, describe, classify, and establish relationships among variables d) Enables a researcher to collect a considerable amount of information about a large number of people e) Survey respondents often not representative (1) Mail surveys typically have between 10% and 45% response rate (2) People who do not cooperate differ in significant ways from those who do (3) Some people give socially acceptable responses (4) People may have incomplete or inaccurate memories for past events

-3-

f) In spite of all this, surveys can be efficient, useful sources of information 6. Case Studies a) A researcher observes selected aspects of a subjects behavior over a period of time b) Subject is usually a person, but it may be a setting such as a business, school, or neighborhood 7. Naturalistic Observation a) Involves observing individuals or events in their natural setting without using manipulative interventions or measuring techniques that might intrude on the setting b) A passive form of research in the sense that the individual being observed determines the events that are available to be recorded c) One of the oldest methods for studying individuals and events d) In some instances, the strategy has led to extremely accurate predictions e) Two advantages over more controlled strategies (1) Findings from naturalistic observation generalize readily to other real-life situations (2) Avoids the reactive arrangements problem (described shortly) avoided because subjects are unaware that their behavior is being studied and do not, therefore, react in an unnatural way as they might if they were aware they were being studied 8. The above five research strategies are listed in order of decreasing control of decreasing control of the independent and dependent variable D. Other Research Strategies 1. Ex Post Facto Studies a) Any non-experimental research strategy in which subjects are singled out because they have already been exposed to a particular condition or because they exhibit a particular characteristic b) Researcher does not manipulate the independent variable or assign the subjects to the experimental conditions c) Examples include (1) Retrospective cohort study (2) Case-control study 2. Retrospective and Prospective Studies a) Non-experimental research studies in which the independent and dependent variables occur before or after, respectively, the beginning of the study b) Retrospective studies use historical records to look backward in time; prospective studies look forward in time c) Retrospective study is particularly useful for studying the relationship between variables that occur infrequently or variables whose occurrence is difficult to predict; also useful when there is a long time interval between a presumed cause and effect d) Two types of retrospective studies

-4-

(1) Retrospective cohort study historical cohort study records are used to identify two groups of subjects: those who have and those who have not been exposed to the independent variable. Once identified, each is compared in terms of the frequency of the occurrence of the dependent variable; cannot establish causality (2) Case-control study case-referent study records are used to identify two groups of subjects: those who exhibit evidence of the dependent variable, called cases, and those who do not, called controls; cannot establish causality (3) Prospective study follow-up study longitudinal study cohort study non-experimental research strategy in which the independent and dependent variables are observed after the onset of the investigation; subjects are classified as exposed or nonexposed based on whether they have been exposed to a naturally occurring independent variable; the groups are followed for a period of time, and the incidence of the dependent variable is recorded 3. Longitudinal and Cross-Sectional Studies a) Longitudinal study refers to any research strategy in which the same individuals are observed at two or more times (1) Involves studying the same individuals over time b) Cross-sectional study any research strategy in which two or more cohorts are observed at the same time (1) Cohort a person or group of people who have experienced a significant life event such as birth, marriage, or illness during a given time interval (2) Tend to be less expensive than longitudinal studies and provide more immediate results 4. Longitudinal-Overlapping and Time-Lag Studies a) Longitudinal-Overlapping study sequential study can be used to compress the time required to perform a longitudinal study b) Time-lag study observations are made at two or more times, but different subjects (or cohorts) are measured each time (as in SAT scores) 5. Time-Series and Single-Subject Studies a) Time-series study involves making multiple observations of one or more subjects or cohorts before and after the introduction of an independent variable b) Single-subject study multiple observations of a single subject are made before and after the introduction of an independent variable, which the researcher controls E. Threats to Valid Inference Making 1. Statistical Conclusion Validity concerned with threats to valid inference making that result from random error and the ill-advised selection of statistical procedures 2. Internal Validity concerned with correctly concluding that an independent variable is, in fact, responsible for variation in the dependent variable

-5-

3. Construct Validity of Causes and Effects concerned with the possibility that operations that are meant to represent the manipulation of a particular variable can be construed in terms of other variables 4. External Validity concerned with the generalizability of research findings to and across populations of subjects and settings 5. Threats to Statistical Conclusion Validity a) Low statistical power (1) Sample size is inadequate (2) Irrelevant sources of variation are not controlled or isolated (3) Inefficient test statistics are used b) Violated assumptions of statistical tests c) Fishing for significant results and the error rate problem (1) With certain test statistics, the probability of drawing one or more erroneous conclusions increases as a function of the number of tests performed d) Reliability of measures (1) Use of a dependent variable that has low reliability may inflate the estimate of the error variance and result in not rejecting a false null hypothesis e) Reliability of treatment implementation (1) Failure to standardize the administration of treatment levels may inflate the estimate of the error variance and result in not rejecting a false null hypothesis f) Random irrelevancies in the experimental setting (1) Variation in environment in which a treatment level is administered may inflate the estimate of the error variance and result in not rejecting a false null hypothesis g) Random heterogeneity of respondents (1) Idiosyncratic characteristics of the subjects may inflate the estimate of the error variance and result in not rejecting a false null hypothesis 6. Threats to Internal Validity a) History (1) Events other than the administration of a treatment level that occur between the time the treatment level is assigned to subjects and the time the dependent variable is measured may affect the dependent variable b) Maturation (1) Processes not related to the administration of a treatment level that occur within subjects simply as a function of the passage of time may affect the dependent variable c) Testing (1) Repeated testing of subjects may result in familiarity with the testing situation or acquisition of information that can affect the dependent variable d) Instrumentation (1) Changes in the calibration of a measuring instrument, shifts in the criteria used by observers and

-6-

scorers, or unequal intervals in different ranges of a measuring instrument can affect the measurement of the dependent variable e) Statistical regression (1) When the measurement of the dependent variable is not perfectly reliable, there is a tendency for extreme scores to regress or move towards the mean (2) The amount of statistical regression is inversely related to the reliability of the test f) Selection (1) Differences among the dependent-variable means may reflect prior differences among the subjects assigned to the various levels of the independent variable g) Mortality (1) The loss of subjects in various treatment conditions may alter the distribution of subject characteristics across the treatment groups h) Interactions with selection (1) Some of the foregoing threats to internal validity may interact with selection to produce effects that are confounded with or indistiniguishable from treatment effects; include selection-history effects and selectionmaturation effects (e.g. when subjects with different maturation schedules are assigned to different treatment levels) i) Ambiguity about the direction of causal influence (1) May be difficult to determine whether X is responsible for the change in Y or vice versa; not present when X is known to occur before Y j) Diffusion or imitation of treatments (1) Sometimes the independent variable involves information that is selectively presented to subjects in the various treatment levels (2) If subjects in different levels can communicate with one another, differences among the treatment levels may be compromised k) Compensatory rivalry by respondents receiving less desirable treatments (1) Social competition may motivate control subjects to attempt to reverse or reduce the anticipated effects of the desirable treatment levels (the John Henry effect) l) Resentful demoralization of respondents receiving less desirable treatments (1) Responses may be to perform at an abnormally low level 7. Threats to External Validity a) Interaction of testing and treatment (1) Results obtained under conditions of repeated testing may not generalize to situations that do not involve repeated testing

-7-

F. Other 1.

2.

3.

(2) A pretest may sensitize subjects to a topic, thereby enhancing the effectiveness of a treatment; the opposite may also occur b) Interaction of selection and treatment (1) Constellation of factors that affect the availability of subjects to participate in an experiment may restrict the generalizability of results to populations that share the same constellation of factors (e.g. if volunteers were used in an experiment, the results may generalize only to volunteer participants) c) Interaction of setting and treatment (1) The unique characteristics of the setting in which results are obtained may restrict the generalizability of the results to settings that share the same characteristics d) Interaction of history and treatment (1) Results are obtained on the same day as a particularly noteworthy event, which may be different from those that would have been obtained in the absence of the noteworthy event e) Reactive arrangements (1) Subjects whoa re aware that they are being observed may behave differently than the subjects who are not aware that they are being observed f) Multiple-treatment interference (1) When subjects are exposed to more than one treatment, the results may generalize only to populations that have been exposed to the same combination of treatments Threats to Valid Inference Making Experimenter-Expectancy Effect a) The effects of a researchers expectations and the desires on the outcomes of an experiment Demand Characteristics a) Any aspect of the experimental environment or procedure that leads a subject to make inferences about the purpose of an experiment and to respond in accordance with or in some cases contrary to the perceived purpose Subject-Predisposition Effects a) Subjects can come to an experiment with a predisposition to respond in a particular way b) Four kinds (1) Cooperative-subject effect Subjects try to provide data that support the researchers hypothesis (2) Screw you effect subjects tending to be uncooperative and may even try to sabotage the experiment (3) Evaluation apprehension subjects are apprehensive about being evaluated; those with this apprehension arent interested in the experimenters hypothesis; primary concern is to gain a positive evaluation from the researcher

-8-

(4) Faithful subjects try to put aside their own hypotheses about the purpose of an experiment and follow the researchers instructions ot the letter 4. Placebo effect a) Placebo an inert substance or neutral stimulus that is administered to subjects as if it was the actual treatment condition b) Placebo effect - any change in the dependent variable attributable to receiving a placebo G. Controlling Nuisance Variables and Minimizing Threats to Valid Inference Making 1. General Approaches to Control a) Experimental control (1) Hold the nuisance variable constant for all subjects (2) Assign subjects randomly to experimental conditions (3) Include the nuisance variable as a factor in the experimental design b) Statistical control (1) Use of statistical control is referred to as the Analysis of Covariance 2. Some Specific Approaches to Control a) Single-blind procedure (1) Subjects are not informed about the nature of their treatment or the purpose of the experiment (when feasible) (2) Helps to minimize the effects of demand characteristics b) Double-blind procedure (1) Neither the subjects nor the researcher is informed about the nature of the treatment (2) Partial-blind procedure researcher does not know until just before administering the treatment level which level will be administered c) Deception (1) When subjects are not told the relevant details of an experiment or when they are told that an experiment has one purpose when in fact the purpose is really something else d) Disguised-experiment technique (unobtrusive experimentation) (1) The subjects are not aware that they are participating in an experiment e) Multiple researchers (1) Characteristics of a researcher can affect the results that are obtained (2) Can seriously limit the generalizability of results f) Debriefing (1) Hold a post-experimental meeting with subjects at which time details of the experiment are shared g) Experimenter-epectancy control groups (1) The magnitude of the experimenter-expectancy can be determined by using several groups of researchers: one group is led to expect one experimental outcome,

-9-

the second group is led to expect the opposite outcome, and a the third group is led to believe that the treatment will have no effect on the dependent variable (2) Requires numerous researchers and subjects h) Unrelated-experiment technique (1) Designed to disguise the purpose of an experiment and minimize subject demand characteristics by separating presentation of the independent variable from the measurement of dependent variable (2) Requires subjects who to participiate in two experiments i) Quasi-control group (1) Uses a second control group to assess the effects of demand characteristics j) Yoked control procedure (1) Allows a researcher to match two subjects on some important aspects of the experience they have had in an experiment

- 10 -

II. Chapter 2: Experimental Designs: An Overview A. Overview of some Basic Experimental Designs 1. t-test for Independent Samples a) One of the simplest experimental designs b) Often used to test the null hypothesis that the difference between two population means is equal to some value, usually zero c) Involves two levels of treatment 2. Completely Randomized Design a) Like a t-test for independent samples, but involves three or more levels of a treatment b) Used to test null hypotheses about the equality of the p means c) Denoted by CR-p (1) CR Completely Randomized (2) p number of levels in the treatment d) Experimental Design Model Equation

Yij ! Q  E j  I i  j (1) Yij is the observation for subject i in treatment level j. (2) Q is the population grand mean of Q1 , Q 2 , Q 3 (3) E j is the treatment effect for population j and is equal to Q j  Q , the deviation of the grand mean from
the j
th

population mean

(4) I i  j is the error affect associated with Yij and is equal to Yij  Q  E j ; the error effect represents effects unique to subject i , effects attributable to chance fluctuation in subject i s behavior, and any other effects that have not been controlled. The notation i j
th

indicates that the i subject appears only in treatment level j (i.e., subject i is said to be nested in treatment level j . 3. t-test for Dependent-Samples Design a) Dependent samples can be obtained by (1) Repeated measures observing each subject under each treatment level in the experiment (2) Subject matching forming sets of subjects who are similar with respect to a variable that is correlated with the dependent variable (3) Obtaining sets of identical twins or littermates and assigning one member of the pair randomly to one treatment level and the other member to the other treatment level (4) Obtaining pairs of subjects whoa re matched by mutual selection for example, husband and wife pairs or business partners.

- 11 -

b) Uses a more complex randomization and analysis plan than that for independent samples (1) Usually accompanied by greater power c) Block subjects in each pair (1) Example: Block 1: those who smoke less than ten cigarettes a day; block 2: those who smoke between ten and twenty cigarettes a day; etc. 4. Randomized Block Design a) Denoted by RB-p (1) RB Randomized Block (2) p number of treatment levels b) Akin to the t-test for dependent samples but with more than two treatment levels c) Experimental Design Model Equation

Yij ! Q  E j  T i  I ij
(1) Yij is the observation for subject i in treatment level

j.
(2) Q is the population grand mean of Q11 , Q 21 ,..., Q15, 3 (when there are 15 blocks and three treatment levels) (3) E j is the treatment effect for population j and is equal to Q j  Q , the deviation of the grand mean from the j
th

population mean

(4) T i is the block effect for population i and is equal to

Q i  Q , the deviation of the grand mean from the i th

population block mean (5) I ij is the error affect associated with Yij and is equal to Yij  Q  E j ; the error effect represents effects unique to subject i , effects attributable to chance fluctuation in subject i s behavior, and any other effects that have not been controlled. d) The sum of squared error terms in the randomized block

, will be lower than Y that for the completely randomized design,   Q  E

design,

2 ij

! Yij  Q  E j  T i

ij

, which shows it is more powerful e) The test statistic used to test the null hypothesis in analysis of variance can be thought of as the ratio of error and treatment effects:

F!
f)

f ErrorEffects  f TreatmentEffects f ErrorEffects

f  denotes a function of the effects in parentheses

5. Latin Square Design a) Denoted by LS-p (1) LS Latin Square (2) p number of levels in the treatment

- 12 -

b) Enables a researcher to isolate the effects of two nuisance variables c) The design allows for testing of the following null hypotheses:

H 0 : Q1 ! Q2 ! Q3 H 0 : Q1 ! Q2 ! Q3 H 0 : Q1 ! Q2 ! Q3
(1) The first tests that the three treatment levels are equal (2) The second and third tests the equality of the three means given the two nuisance variables. For example, the average is equal for each block of participants when blocked into # of cigarettes smoked per day, and the average is equal for each block of participants when blocked into # of years smoking d) Experimental Design Model Equation

Yij ! Q  E j  Fk  K l  I pooled
(1) Yijkl is the observation for subject i in treatment level j , row k , and column l . (2) Q is the population grand mean of Q111 , Q123 ,..., Q 331 (3) E j is the treatment effect for population j and is equal to Q j  Q , the deviation of the grand mean from the j
th

population mean

(4) F k is the row effect for population k and is equal to

Q k  Q , the deviation of the grand mean from the k th

population mean (5) K l is the column effect for population l and is equal to Q l  Q , the deviation of the grand mean from the l population column mean (6) I pooled is the pooled error effect associated with Yijkl and is equal to Yijkl  Q  E j  F k  K l e) The sum of squared error terms for the Latin Square design will be lower than that of the randomized block design if the combined effects of
th

2 k

and

2 l

are greater than

2 i

6. Building Block Designs a) The three mentioned above are the simplest analysis of variance designs by which all complex ANOVA designs are constructed 7. Completely Randomized Factorial Design a) Denoted by CRF-pq (1) CRF Completely Randomized Factorial (2) p number of levels in treatment A (3) q number of levels in treatment B b) Completely crossed

- 13 -

(1) Every treatment level from each treatment is paired with every treatment level from the other treatment level c) Experimental Design Model Equation

Yijk ! Q  E j  F k   jk  I i  jk EF (1) Yijk is the observation for subject i in treatment combination a j bk (2) Q is the population grand mean of Q11 , Q12 ,..., Q pq (3) E j is the treatment effect for population a j and is equal to Q j  Q , the deviation of the grand mean from
the j
th

population treatment mean

(4) F k is the treatment effect for population bk and is equal to Q k  Q , the deviation of the grand mean from the k
th

(5) EF

population mean
jk

is the interaction effect for populations a j

and bk and is equal to Q jk  Q j  Qk  Q (6) I i  jk is the error affect associated with Yijk and is given by Yijk  Q  E j  F k  EF

jk

; represents all effects

not attributable to either treatment level or the interaction between the two treatment levels d) Null hypotheses

H0 : Q1 ! Q2 !... ! Q p H0 : Q1 ! Q2 !... ! Qq H0 : Q jk  Q j d  Q jk d Q j dd! 0 k k

(1) The last hypothesis is unique to factorial designs and tests that all AB interaction effects equal zero (2) Two treatments are said to interact if differences in performance under the levels of one treatment are different at two or more levels of another treatment e) Comparison of CR-p and CRF-pq designs (1) Very similar (2) Both use random assignment (3) Layouts are similar in appearance B. Classification of Analysis of Variance Designs 1. Randomized Designs a) Experimental designs that use the randomization principle b) Randomization Principle the random assignment of experimental units to treatment levels c) Designs are subdivided into distinct categories based on (1) Whether the subjects are subdivided into homogeneous blocks or groups prior to assigning them to treatment levels, and

- 14 -

(2) The number of restrictions placed on the random assignment of the subjects to treatment levels d) Analysis of Variance Designs (1) Systematic Designs (2) Randomized designs with one treatment (a) Experimental Units not subdivided on any basis other than randomization prior to assignment to treatment levels; no restriction on random assignment other than the option of assigning the same number of subjects to each treatment level (i) Completely Randomized Design (CR-p) (b) Experimental units subdivided on some nonrandom basis or one or m ore restrictions on random assignment other than assigning the same number of subjects to each treatment level (i) Balanced Incomplete Block Design (BIB-p) (ii) Cross-Over Design (CO-p) (iii) Generalized Randomized Block Design (GRB-p) (iv)Graeco-Latin square design (GLS-p) (v) Hyper-Graeco-Latin square design (HGLS-p) (vi)Latin square design (LS-p) (vii) Partially balanced incomplete block design (PBIB-p) (viii) Randomized block design (RB-p) (ix)Youden square design (YBIB-p) (3) Randomized designs with two or more treatments (a) Factorial experiments: designs in which all treatment levels are crossed (without confounding) (i) Completely randomized factorial design (CRF-pq) (ii) Generalized randomized block factorial design (GRBF-pq) (iii) Randomized block factorial design (RBF-pq) (b) Factorial experiments with group-treatment confounding (i) Split-plot factorial design (SPF-p y q) (c) Factorial experiments with group-interaction confounding (i) Latin square confounded factorial design (LSCF-pk) (ii) Randomized block completely confounded factorial design (RBCF- pk) (iii) Randomized block partially confounded factorial design (RBPF- pk) (d) Factorial experiments with treatmentinteraction confounding

- 15 -

(i) Completely randomized fractional factorial design (CRFF- pk-i) (ii) Graeco-Latin square fractional factorial design (GLSFF- pk) (iii) Latin square fractional factorial design (LSFF- pk) (iv)Randomized block fractional factorial design (RBFF- pk-i) (e) Heirchical experiments: designs in which one or more treatments are nested (complete nesting) (i) Completely randomized hierarchical design (CRH-pq(A)) (ii) Randomized block hierarchical design (RBH-pq(A)) (f) Hierarchical experiments with partial nesting (i) Completely randomized partial hierarchical design (CRPH-pq(A)r) (ii) Randomized block partial hierarchical design (RBPH-pq(A)r) (4) Randomized designs with one or more covariates (a) A covariate can be used with all designs listed above (when done, the letters AC are added to the nomenclature) (i) Completely randomized analysis of covariance design (CRAC-p) (ii) Completely randomized factorial analysis of covariance design (CRFAC-pq) (iii) Latin square analysis of covariance design (LSAC-p) (iv)Randomized block analysis of covariance design (RBAC-p) (v) Split-plot factorial analysis of covariance design (SPFAC-p y q) e) Each of the above differs by one or more of: (1) Randomization (2) Experimental design model equation (3) Number of treatments (4) Inclusion of a nuisance variable as a factor in the experiment (5) Power f) Other ways designs may differ include (1) Use of crossed or nested treatments or a combination of the two (2) Presence or absence of confounding (3) Use of a covariate g) Common threads that run through the various designs (1) All complex designs can be constructed from three building block designs: completely randomized design, randomized block design, and Latin square design (2) There are only four kinds of variation in ANOVA: total variation, between-groups variation, within-groups variation, and interaction variation

- 16 -

(3) All error terms involve either within-groups variation or interaction variation (4) The numerator of an F statistic should always estimate one more source of variation than the denominator, and that source of variation should be the one that is being tested C. Selecting an Appropriate Design 1. Questions to Cinsider a) Does the design permit the calculation of a valid estimate of the experimental effects and the error effects? b) Does the data-collection procedure produce reliable results? c) Does the design possess sufficient power to permit an adequate test of the statistical hypotheses? d) Does the design provide maximum efficiency within the constraints imposed by the experimental situation e) Does the experimental procedure conform to accepted practices and procedures used in research area? Other things being equal, a researcher should use procedures that offer an opportunity for comparing the findings with the results of other investigations

- 17 -

III. Chapter Three: Fundamental Assumptions in Analysis of Variance A. Sampling Distributions in Analysis of Variance 1. Chi-square a) The square of a standard normal variable is a chi-square with one degree of freedom b) The sum of independent chi square variables is a chi square variables with degrees of freedom equal to the sum of the degrees of freedom being added c) The expected value of a chi-square random variable is its degrees of freedom; the variance of a chi-square random variable is twice its degrees of freedom d) Be able to show that for random samples of n observations
n

from a normally distributed population,

i !1

Y  Q
i

W2

is a chi

square with n degrees of freedom 2. The F distribution a) An F random variable is the ratio of two independent chisquare variables, each divided by its degrees of freedom b) The most common use of an F is in testing hypotheses regarding the equality of three or more population means c) Distribution depends on two parameters, R 1 and R 2 . d) If values of F that cut off the lower 1  E region are needed, they can be readily computed from

F1 E ;R

1 ,R 2

1 FE ;R
2

,R 1

e) The expected value of an F is

E F !

R2 for R 2 " 2 R2  2 2R 22  1  R 2  2 R
2

f) The variance of an F is

V F !

R 1  2  2  2  4 R R

for R 2 " 4

g) The mean is always greater than one, and the mode is always less than one 3. Relationship between the t, the F, and the chi-square distributions a) A t with R is, when squared, an F with R 1 ! 1 and R 2 ! R 4. Summary of Basic Assumptions Associated with the F Distribution a) Observations are drawn from normally distributed populations b) Observations are random samples from the populations, or the experimental units (subjects) are randomly assigned to treatment levels c) The numerator and the denominator of the F statistic are independent d) The numerator and denominator of the F statistic are 2 estimates of the same population variance, W I B. Partition of the Total Sum of Squares 1. Partition of SSTO for a CR-p design into SSBG and SSWG

- 18 -

2. Recall the experimental design model:

Yij ! Q  E j  I i  j
3. The above can be estimated from sample data as follows:

Yij ! Y  Y j  Y  Yij  Y j

4. The partition of SSTO into SSWG and SSBG is obtained by rearranging terms in the above equation into:

Yij  Y ! Y j  Y  Yij  Y j
5. Squaring both sides gives:

Y
n

ij

 Y

! ?Y 
2

 Y  Yij  Y j
n

A
2

6. Summing both gives:

Y 
i !1 p n

ij

 Y

! ?Y 
2 i !1 2 p

 Y  Yij  Y j
n 2

A
2
j

7. Simplifying gives:

Y 
j !1 i !1 p n

ij

 Y

! Y
j !1 i ! 1 p n

 Y

 2Y
2 p p

 Y Yij  Y j  Yij  Y j


j !1 i !1 p n j !1 i !1 p n

Yij  Y  Y  Y

! Y j  Y
j !1 i ! 1 p 2
j

 2 Y j  Y Yij  Y j  Yij  Y j
j !1 i !1 j !1 i !1 p
j


ij

ij

! n Y
2 j !1 p

 Y  Y

 2 Y
j !1 2 p n j !1 i !1

 Y

Y
n i !1

 Y j  Yij  Y j
j !1 i !1

Y 
j !1 i !1

ij

! n Y
2 j !1

 Y
2

ij

 Y j

SSTO=SSBG+SSWG 8. Computational Formulae a) SSTO

SSTO ! Yij2 
j !1 i !1

p n Yij j !1 i !1 N

b) SSWG
p n 2 ij p

SSWG !
c) SSBG

Y
j !1 i !1


j !1

n Yij i !1 nj
2

SSB

!
j !1

n Yij i !1 nj

p n Yij j !1 i !1 N

9. Degrees of Freedom a) The number of observations whose values are free to vary (1) SSTO np-1

- 19 -

(2) SSBG p-1 (3) SSWG p(n-1) C. Expectation of Mean Squares 1. Be able to derive the expected values of Mean Square for the both the fixed-effect model and the random-effects model 2. Fixed-effect model rules

p E E j ! 0 j !1 E Iij ! 0

? A

2 p E E j ! 0 j !1 n E I i  j ! 0 i !1

p p 2 E E j ! E 2 j j ! 1 j !1

n E I i2 j ! nW I2 i !1

2 n p n E I i  j ! nW I2 E I i2 j ! npW I2 i !1 j !1 i !1 2 p n E I i  j ! npW I2 j !1 i !1

3. Random-effects model rules

p E E j ! 0 j !1 E Iij ! 0

? A

2 p p 2 2 E j ! pW E E E 2 ! pW E E j j !1 j !1 n 2 n E I i  j ! 0 E I i  j ! nW I2 i !1 i !1

2 n p n E I i  j ! nW I2 E I i2 j ! npW I2 i !1 j !1 i !1

a) Note the rules for epsilon are the same for both fixed and random effects model D. The F statistic in Analysis of variance 1. Fixed-Effects Model a) Hypotheses as stated in terms of p population means

H 0 : Q1 ! Q 2 !... ! Q p
b) Hypotheses as stated in terms of p population treatment effects

H 0 :E 1 ! E 2 !... ! E p ! 0
c) Why analyze variance to tell about means? (1) Consider the expected value of MSBG and MSWG such that

E  MSB ! E  MSW

W  n
2 I j !1 2 I

E2 j p 1

- 20 -

(2) A high F statistic indicates

E
j !1

2 j

is greater than

zero 2. Random-Effects Model a) Hypotheses as stated in terms of p population means

H 0 : Q1 ! Q 2 !... ! Q p
b) Hypotheses as stated in terms of the variance of the population treatment effects
2 H 0 :W E ! 0

(1) This is appropriate because the expectation of MSBG 2 2 is W I  nW E and an F statistic appreciably greater than zero indicates W E " 0 E. Effects of Failure to Meet Assumptions in Analysis of Variance 1. F Assumptions a) Observations are drawn from normally distributed populations b) Observations are random samples from the populations, or the experimental units are randomly assigned to the treatment levels c) The numerator and denominator of the F statistic are independent d) The numerator and denominator of the F statistic are 2 estimates of the same population variance, W I . 2. Model Assumptions a) The model equation Yij ! Q  E j  I i  j contains all of the sources of variation that affect Yij b) The experiment contains all of the treatment levels of interest c) The error effect, I i  j , is (1) Independent of all other I i  j s and (2) Normally distributed within each treatment population with mean equal to zero and variance equal 2 to W I 3. Robustness a statistic is said to be robust if violations of its assumptions have little effect on the sampling distribution of the statistic and thus on type I and type II errors 4. Evaluating the Assumptions a) Assumption that Yij ! Q  E j  I i  j (1) Appropriate for designs with one treatment in which the experimental units are randomly assigned to the p treatment levels (2) If an experiment contains two treatments or if the experimenter wants to observe the experimental units under more than one treatment level, a different design
2

- 21 -

must be used; otherwise, the probability of a type I error is affected, as is the power of the test b) Assumption that the experiment contains all treatment levels of interest (1) MSBG will be different for fixed-effects and for random-effects models c) Assumptions of independence of error effects and random sampling (1) F is not robust with respect to the violation of the assumption of the independence of error effects (2) The assumption is likely to be violated if two or more observations are obtained on each subject (3) Random sampling / assignment helps ensure the error terms are independent d) Assumptions of normality of error terms and populations, and independence of MSBG and MSWG (1) F is quite robust with respect to violation of the normality assumption, particularly when the populations are symmetrical, but not normal, and the sample sizes are equal and greater than 12. e) Assumption that the expected value of the error term is zero for each treatment population f) Assumption of homogeneity of variance (1) F is robust with respect to violation of the homogeneity of variance assumption if (a) There is an equal number of observations in each treatment level (b) The populations are normal, and (c) The ratio of the largest variance to the smallest variance does not exceed three F. Transformations 1. Any systematic alteration in a set of scores whereby certain characteristitcs of the set are changed and other characteristics remain unchanged 2. Three reasons to transform: a) To achieve homogeneity of error variance b) To achieve normality of error effects c) To obtain additivity of effects 3. Square Root Transformation a) Use when mean and variance is proportional 4. Logarithmic Transformation a) Use when the treatment means and standard deviations are equal 5. Reciprocal Transformation a) Use when the squares of treatment means are proportional to standard deviations 6. Angular or inverse sine transformation a) Use when the means and variances are proportional and the distribution has a binomial form

- 22 -