Probability

deductive reasoning

Statistical Inference
population

inductive reasoning

population
Pr (H ) = Pr (T ) = 0.5

Pr (H ) = Pr (T ) = ?

ten times sample

Pr (all heads ) = (0.5)
10

ten times sample All heads. Is it a fair coin?

Beginning of Probability
C. de Mre

Classical School
Pascal
N equally probable possible outcomes

Head Tail A get 1 point B get 1 point win $100 4 points Interrupted at 3:2

Pr (E ) = M N
3 = 0.5 6

Fermat A B $75 $25

M satisfy event E

13 Pr (heart ) = = 0.25 52

Pr (odd number from dice toss ) =

$100
?

Pr (different birthdays) =

Pr(B win ) = Pr (both tails) = 0.25

365 364  (365 n + 1) 365 Pn = 365 365  365 (365)n

Frequency School
Results of coin toss experiment
No. of Tosses No. of Heads Relative Frequency of Heads 5 3 0.6 100 47 0.47 1000 508 0.508 1000000 499987 0.499987

Subjective School
Probabilities are quantitative expressions of uncertain about a persons knowledge of the occurrence of some event. I thought there is 30% chance that tomorrow will rain. Arsenal will have more than 80% chance to win the championship this year. I wished I had worked hard in last semester. I think I should have 80% chance to get better grade if I did so.

relative frequency of Heads 0.5

as

f Pr (E ) = lim n n
define

Counting Procedures
Example: Possible outcomes of tossing three coins
HH H HT TH T TT
st

Permutation
A specific arrangement of objects in a definite order. Example: possible arrangements of three persons in a row
A AB AC B C BA BC CA CB ABC ACB BAC BCA CAB CBA

HHH HHT HTH HTT THH THT TTH TTT

nd

N = 222 =8

N = 3 2 1 = 6

n1 ways in 1 step n2 ways in 2 step

nk ways in k step

th

n1n2 nk ways in k steps

No. of permutations of n distinct objects = n(n 1)(2 )(1) = n !

Permutation
6 objects: A B C D E F

Permutation
n distinct objects pick r objects and arrange

a row

round table
A F E D B C

bracelet
A F E D B C B C D A F E

No. of ordered subset :

Pr = n(n 1)(n r + 1) =

ABCDEF

(n r )!
N =14 P3 = 14! = 2184 11!

n!

Example: Tierce outcomes in horse race with 14 horses 5 6 14 7 1 2 1 2 7

6! = 720

(6-1)! = 120

(6-1)!/2 = 60

Combination
A subset of r objects from n distinct objects, without regard to order.
Example: Choose three letters from {A, B, C, D}
Combination {A, B, C} {A, B, D} {A, C, D} {B, C, D} Permutations (A,B,C),(A,C,B),(B,A,C),(B,C,A),(C,A,B),(C,B,A) (A,B,D),(A,D,B),(D,A,B),(D,B,A),(B,A,D),(B,D,A) (A,C,D),(A,D,C),(C,A,D),(C,D,A),(D,A,C),(D,C,A) (B,C,D),(B,D,C),(C,B,D),(C,D,B),(D,B,C),(D,C,B)

Combination
Example: 6 numbers chosen from {1, 2, .., 49}

N = 49 C 6 =

49 ! = 13983816 6 ! 43 !

Example: draw five poker cards, Pr(full house) = ?

Full house:
no. of possible outcomes : N = 52 C5 = 2598960 no. of possible full houses : M =13 C 2 2 4 C3 4 C 2 = 3744

24 4P 3 = =4 4 C3 = 3! 6

4! 24 = = 24 4P = 3 (4 3)! 1

Cr =

Pr n! = r ! r!(n r )!

Pr (full house ) =

3744 = 0.00144 2598960

Set
Set Element Venn Diagram
A B A B

Set operations
collections of objects object in the set E = {main characters in Doraemon } }
E

Union
A B

x A B
equivalent to

x A or x B
A B

Intersection
A B

x A B
equivalent to

x A and x B
A
A

Complement
B A B is a subset of A
A

x A
equivalent to

x A

Set operations
A (B C ) =
A

Set operations
(A B) (A C )
B

A (B C ) =
A

(A B) (A C )
B

B C
C C

B C

Event
Sample Space all possible outcomes Event subset of sample space

Event
Disjoint two events have no common element
A B =

B
AC

C
= { outcomes of a die toss } E = { even number }

A = { even number } B = { odd number }

C = { multiple of 3}

Event
Mutually exclusive
A1 A1 A2 A4

Occurrence
Exhaustive

E occurs if the outcome belongs to E.

B
occur

outcome

+
A3 A4 A2

A3

not occur

A BC

AB C

Partition
A1 A3 A3 A4 A5 A2

C A = { even number } B = { odd number } C = { multiple of 3}

A C BC

Komogorovs Axiom System

Pr (E ) 0 for any event E

Example
M M ) = 0marijuana Pr (: using .1

Pr ( ) = 1
if A B =

Pr (M C ) = 0.05
Pr (M C A) = 0.01
C
0.03 0.04 0.01 0.02 0.08 0.09 0.11

Pr ( A B ) = Pr ( A) + Pr (B )

Pr ( A) = 0.2 cigarettes A) = 0.03 C : smoking Pr (M

Pr(:C ) = 0.25 alcohol A consuming

Pr ( ) = 0

Pr (C A) = 0.1
M

If A B, then Pr ( A) Pr (B )
Pr (E ) 1 for any event E

Pr ( A ) = 1 Pr ( A)

Pr(none of these behaviors) = ? 0.62

Pr (M C A) = ? .38 0

Pr ( A B ) = Pr ( A) + Pr (B ) Pr ( A B )

Conditional Probability
randomly pick one

Conditional Probability
For any two events A and B, the conditional probability of A given that B has occurred is written as Pr(A|B) and defined as

Pr(red circle) = 5/10 = 0.5 Pr(blue triangle) = 3/10 = 0.3 Pr(red) = 6/10 = 0.6 Pr(triangle) = 4/10 = 0.4
You are told that it is blue. Pr(triangle) = 3/4 = 0.75
3 10 Pr (blue triangle ) Pr (triangle | blue ) = = 4 10 Pr (blue )

Pr ( A | B ) =
Multiplication theorem

Pr ( A B ) Pr (B )

Pr ( A B ) = Pr (B )Pr ( A | B )
You are told that it is red. Pr(triangle) = 1/6 = 0.17
1 10 Pr (red triangle ) Pr (triangle | red ) = = 6 10 Pr (red )

Pr ( A B C ) = Pr (C ) CBPr ( )PrBA | B C ) B Pr ( ) | C A | ( C )

Independence
Two events A and B are called independent if and only if

Independence vs Mutually Exclusive

Disjoint two events have no common element
A B =

Pr ( A B ) = Pr ( A)Pr (B )
Example : opinion poll on abortion Favor (F) White (W) Non-white (N) 0.459 0.051

Pr (B ) = Pr(B | B ) if Pr (B ) > 0 A A A A
Opposed (O) 0.441 0.049

Pr (W ) = 0.459 + 0.441 = 0.9

Pr (W F ) = 0.459

Pr (F ) = 0.459 + 0.051 = 0.51 = Pr (W )Pr (F ) = 0.459 W and F are independent.

Pr ( A B ) = 0

AAre A and not independent ! and B are B independent?

Pr ( A)Pr (B ) if Pr ( A) > 0, Pr (B ) > 0

Independence vs Mutually Exclusive

Independence events have no addition information to each other
Pr ( A B ) = Pr ( A )Pr (B )

Law of Total Probability

B1 , B2 ,..., Bk
partition of the sample space cannot occur at the same time at least one of them will occur
A
A A
Pr ( A) = ?

A = { odd } B = {multi of 3}
A B = {3}

mutually exclusion exhaustive

Pr( A | B1 ) Pr(B1 )

B1 B2

A A
A

A and B areB independent? Are A and independent !


Bk

Pr ( A B ) = 1 6 = 1 2 1 3 = Pr ( A)Pr (B )

Pr ( A) = Pr (B1 )Pr ( A | B1 ) +  + Pr (Bk )Pr ( A | Bk )

Law of Total Probability

Example: Morse code transmission
dot : dash = 3 : 4
Pr (dot ) = 3 7 Pr (dash

Bayes Rule
B1 , B2 ,..., Bk
)= 1 )= 1
8 8

-... .- -.-- . ... Bayes

Pr (receive Pr (receive

partition of the sample space

error rate = 1/8

dot | dash dash | dot

)= 4

Pr(Bi A) Pr (Bi )Pr( A | Bi ) Pr(Bi | A) = = Pr( A) Pr ( A )

= Pr (Bi )Pr( A | Bi ) Pr (B1 )Pr ( A | B1 ) + Pr (B2 )Pr ( A | B2 ) + h + Pr (Bk )Pr ( A | Bk )

7/8 3/7 dot sent 1/8 1/8 dash sent 7/8

dot received dash received dot received dash received dot received

4/7

Example: Morse code transmission

3 7 4 1 25 Pr (receive dot )) = + = receive dot = ? 7 8 7 8 56

Pr (dot sent | dot received) =

Pr (dot sent )Pr (dot received | dot sent ) Pr (dot received)

Pr (dot sent | dot received) = ?

3 77 8 = 0.84 25 56

Bayes Rule
Example: virus test Test shows positive result as carrier A B Infected by the virus Pr (B ) = 0.003 0.3% infected 2% false positive Pr ( A | B ) = 0.02 1% false negative Pr (A | B ) = 0.01
Pr (B )Pr ( A | B ) Pr ( B | A) = ? Pr (B )Pr ( A | B ) + Pr (B )Pr ( A | B ) = 0.003 0.99 = 0.1296 0.003 0.99 + 0.997 0.02

prior

posterior