Professional Documents
Culture Documents
Statistical Inference:
Significance Tests
Goal: Use statistical methods to test hypotheses such
as
• Hypotheses:
Null hypothesis (H0): A statement that parameter(s) take
specific value(s) (Usually: “no effect”)
Alternative hypothesis (Ha): states that parameter
value(s) falls in some alternative range of values
(an “effect”)
• Test Statistic: Compares data to what null hypo. H0
predicts, often by finding the number of standard
errors between sample point estimate and H0 value
of parameter
• P-value (P): A probability measure of evidence
about H0. The probability (under presumption that H0
true) the test statistic equals observed value or value
even more extreme in direction predicted by Ha.
– The smaller the P-value, the stronger the
evidence against H0.
• Conclusion:
– If no decision needed, report and interpret P-value
– If decision needed, select a cutoff point (such as
0.05 or 0.01) and reject H0 if P-value ≤ that value
– The most widely accepted cutoff point is 0.05, and
the test is said to be “significant at the .05 level” if
the P-value ≤ 0.05.
y = 11.4, 11.0, 5.5, 9.4, 13.6, -2.9, -0.1, 7.4, 21.5, -5.3,
-3.8, 13.4, 13.1, 9.0, 3.9, 5.7, 10.7
Is there evidence that family therapy
has an effect?
• Let µ = population mean weight change
• Test H0: µ = 0 (no effect) against Ha: µ 0.
• Data have
---------------------------------------------------------------------------------------
Variable N Mean Std.Dev. Std. Error Mean
weight_change 17 7.265 7.157 1.736
----------------------------------------------------------------------------------------
se s / n 7.157 / 17 1.736
y 0 7.265 0
• Test Statistic (df = 16): t 4.18
se 1.736
One-Sample Statistics
N Mean Std. Deviation Std. Error Mean
weight_change 17 7.265 7.1574 1.7359
One-Sample Test
Test Value = 0
t df Sig. (2-tailed) Mean Difference 95% Confidence
Interval of the Difference
Lower Upper
weight_change 4.185 16 .001 7.2647 3.58 10.945
Then,
se s / n 7.16 / 4 3.58
and t (7.265 0) / 3.58 2.0
Now, df = 3. This t statistic has two-sided P-value = 0.14.
Note: We say “Do not reject H0” rather than “Accept H0”
because H0 value is only one of many plausible values.
• Assumptions:
– Categorical variable
– Randomization
– Large sample (but two-sided ok for nearly all n)
• Hypotheses:
– Null hypothesis: H0: 0
– Alternative hypothesis: Ha: 0 (2-sided)
– Ha: 0 Ha: 0 (1-sided)
– Set up hypotheses before getting the data
• Test statistic:
ˆ 0 ˆ 0
z
0 (1 0 ) / n
Note ˆ
ˆ se0 0 (1 0 ) / n , not se ˆ (1 ˆ ) / n as in a CI
• P-value:
Ha: 0 P = 2-tail prob. from standard normal dist.
Ha: 0 P = right-tail prob. from standard normal dist.
Ha: 0 P = left-tail prob. from standard normal dist.
• The farther the true parameter value falls from the null
value, the easier it is to reject null, and P(Type II error)
goes down. (see graph of null and alternative dist’s)
• So, we’ll get t < 1.729 and P-value > 0.05 (and make a
Type II error) if the sample mean < 3.87.
True Decision
effect? Reject null?
Yes (40)
Yes (80) ---------------|
| No (40)
1000 studies---|
| Yes (46 = .05 x 920)
No (920) ----------------|
No (874)
n!
P( x) x (1 ) n x , x 0,1,..., n
x !(n x)!
n! 5!
P(0) x (1 ) n x (0.50) 0 (0.50)5 (0.50) 5 1/ 32
x !(n x)! 0!5!
n! 5!
P (1) x (1 ) n x (0.50)1 (0.50) 4 5(0.50) 5 5 / 32
x !(n x )! 1!4!
n! 5!
P(2) x (1 ) n x (0.50) 2 (0.50)3 10(0.50) 5 10 / 32
x !(n x )! 2!3!
5!
P(3) 0.53 (1 0.5) 2 10 / 32
3!2!
5!
P(4) 0.54 (1 0.5)1 5 / 32
4!1!
5!
P(5) 0.55 (1 0.5)0 1/ 32
5!0!
• For Ha : 0.50,
E ( x) n , n (1 )
whereas sample proportion ˆ = x/n has
E (ˆ ) , ˆ (1 ) / n
Example: Poll results for proportion
with n = 1000, = 0.50
• x = number in category of interest has
a. Ha : 0.50
b. Ha : ˆ 0.50
c. Ha : 0.00
d. Ha : 0.50
e. Ha : 0.50
What happens to P(Type II error)
1. when you decrease the P(Type I error) from 0.05 to
0.01 for making the decision?
2. when the actual population proportion gets closer
to the null hypothesis value?
a. Decreases
b. Increases
c. Stays the same
Let’s practice with one more problem (optional
HW exercise 6.21)
(answer: P = 0.002)