You are on page 1of 152

Probability

A Lecture in ENGSTAT
Introduction
• In statistics, we are concerned with
chance outcomes that occur in a planned
study or scientific investigation.
• For example:
– Number of accidents in an intersection
– Number of defects in a production line
– Tree height when fertilizer is varied
Introduction
• The data involved in statistics can either
be numerical (e.g., number of rejects) or
categorical (e.g., defective, non-defective)

• Any recording of information, whether


numerical or categorical, is referred to as
an observation.
Introduction
• The word experiment describes any
process that generates a set of data.
• We are interested in observations
obtained by repeating an experiment
several times. In most cases, the
outcomes will depend on chance, and thus
cannot be predicted with certainty.
Introduction
Whether a statistical study deals with a
designed experiment, an observational
study, or a retrospective study, the end
result will be a set of data that is subject to
uncertainty.
Sample Space
• The set of all possible outcomes of a
statistical experiment is called the sample
space and is represented by the symbol S.
• Each outcome in a sample space is called an
element or a member of the sample space,
or simply a sample point.
Sample Space
• If the sample space has a finite number of
elements, we may list the members
separated by commas and enclosed in
braces.
• In some experiments, it is helpful to list the
elements of the sample space
systematically by means of a tree diagram.
• Sample spaces with a large or infinite
number of sample points are best de-
scribed by a statement or rule method.
Example
Solution
(a) S = {8, 16, 24, 32, 40, 48}
(b) S = {-5, 1}
(c) S = {T, HT, HHT, HHH}
(d) S = {Asia, Africa, North America, South
America, Europe, Australia, Antartica}
(e) S =  (null set)
Example
Solution

S = {(x, y) | x2 + y2 < 9, x ≥ 0, y ≥ 0}

“S is the set of all ordered pairs (x, y) such


that ….”
Example
A section of an exam contains four True-
False questions. A completed exam paper
is selected at random, and the four
answers are recorded. List all 16
outcomes in the sample space.
Solution

T F Question 1

T F T F Question 2

T F T F T F T F Question 3

T F T F T F T FT F T F T F T F Question 4

Sample Space = {TTTT, TTTF, TTFT, TTFF,


….FFFT, FFFF}
Example
A box contains four bolts. Two of them,
labeled #1 and #2, are 5 mm in diameter,
and two of them, labeled #3 and #4, are 7
mm in diameter. Bolts are randomly
selected until a 5 mm bolt is obtained. The
outcomes are the sequences of bolts that
can be selected. So, one outcome is 1,
and another is 3-4-2. List all the possible
outcomes.
Solution:
S = {1, 2, 3-1, 3-2, 3-4-1, 3-4-2,
4-3-1, 4-3-2, 4-1,4-2}
Event
• For any given experiment, we may be
interested in the occurrence of certain
events rather than in the occurrence of a
specific element in the sample space.
• For example, when we toss a die three
times, we may be interested in the events
wherein the outcome is divisible by three.
Hence, an event is a subset of a
sample space.
Example
Solution

F N River 1

F N F N River 2

F N F N F N F N River 3

(a) S = {FFF, FFN, FNF, FNN, NFF, NFN,


NNF, NNN}
Solution

F N River 1

F N F N River 2

F N F N F N F N River 3

(b) E = {FFF, FFN, FNF, NFF}


(c) {FFF, NFF, FFN, NFN}  River 2 is safe
Complement of a Set
• The complement of an event A with
respect to a set S is the subset of all
elements of S that are not in A.
• We denote the complement of event A by
either A’, Ac, or A.
Intersection of a Set
• The intersection of two events A and B,
denoted by the symbol A ∩ B, is the event
containing all elements that are common
to A and B.
• In some books, A ∩ B is written as AB.
The symbol ∩ is omitted.
Mutually Exclusive Events
• For certain statistical experiments it is by
no means unusual to define two events, A
and B, that cannot both occur
simultaneously.
• Two events A and B are mutually
exclusive, or disjoint, if A ∩ B = , that is,
if A and B have no elements in common.
Union of Events
• Often one is interested in the occurrence of at
least one of two events associated with an
experiment.
• For example, in drawing a card from a stack,
if A = {ace, king, queen} and B = {ace, 2, 3},
we might be interested in either A or B
occurring, or both A and B occurring.
• Such an event, called the union of A and B,
will occur if the outcome is an element of the
subset {ace, 2, 3, king, queen}.
Union of Events
The union of the two events A and B,
denoted by the symbol A  B, is the event
containing all the elements that belong to
A or B or both.
Venn Diagram
• A pictorial representation of events and
manipulations with events is obtained by
using Venn diagrams.
• To construct a Venn diagram, draw a
rectangle whose interior will represent the
sample space S. Then any event A is
represented as the interior of a closed
curve (often a circle) contained in S.
Venn Diagrams

Two events A and B


Venn Diagrams

The complement of A
(orange part)
Venn Diagrams

AB
Venn Diagrams

AB
Venn Diagrams

AB=
(mutually exclusive events)
Relationship among events
• A  A’ = S for any set A
• A  B =  if events A and B are mutually
exclusive.
• Distributive Laws
A  (B  C) = (A  B)  (A  C)
A  (B  C) = (A  B)  (A  C)
• De Morgan’s Laws
(A  B )’ = A’  B’
(A  B )’ = A’  B’
Relationship among events
Example: #4.2
Example: #4.4
Counting Sample Points
• In many cases, one can solve a probability
problem by counting the number of points
in the sample space without actually listing
each element.
• The fundamental principle of counting,
often referred to as the multiplication rule,
is applied.
The Multiplication Rule
Example
Permutations
• Frequently, we are interested in a sample
space that contains as elements all
possible orders or arrangements of a
group of objects.
• The different arrangements are called
permutations.
Permutations
• A permutation is an arrangement of all or
part of a set of objects.
• The number of permutations of n objects
is n!
Permutations
The number of permutations of n distinct
objects taken r at a time is

n Pr  n
Pr
n!
 n(n  1)...(n  1  r ) 
(n  r )!
Example
DNA molecules consist of chemically linked
sequences of the bases adenine (A), guanine
(G), cytosine (C), and thymine (T). A sequence
of three bases is called a codon. A base may
appear more than once in a codon.
a) How many different codons are there?
b) The bases A and G are purines, while C and T are
pyrimidines. How many codons are there whose first
and third bases are purines and whose second base is
a pyrimidine?
c) How many codons consist of three different bases?
Circular Permutations
Permutations that occur by arranging
objects in a circle are called circular
permutations.
– Two circular permutations are not considered
different unless corresponding objects in the
two arrangements are preceded or followed
by a different object as we proceed in a
clockwise direction.
Circular Permutations
• For example, if 4 people are playing
bridge, we do not have a new permutation
if they all move one position in a clockwise
direction.
• By considering one person in a fixed
position and arranging the other three in 3!
ways, we find that there are 6 distinct
arrangements for the bridge game.
Circular Permutation

Hence, the number of permutations of n


objects arranged in a circle is (n−1)!.
Example:

In how many ways can 5 different trees be


planted in a circle?

Solution:
Let n = number of arrangements
Then,
n = (5 – 1)! = 4! = 4(3)(2)(1) = 24
Another type of permutation
The number of distinct permutations of n
things of which n1 are of one kind, n2 of a
second kind,..., nk of a kth kind is

n!
n1! n 2 ! ... nk !
Example
How many distinct permutations can be
done from the letters of the word
INFINITY?
Solution
There are three I’s
two N’s
one F, one T, one Y.

8!
n  3360
3! 2! 1! 1! 1!
There are 3,360 ways.
Combinations
• In many problems, we are interested in the
number of ways of selecting r objects from
n without regard to order.
– These selections are called combinations.
• A combination is actually a partition with
two cells, one cell containing the r objects
selected and the other cell containing the
(n − r) objects that are left.
Combinations
The number of combinations of n distinct
objects taken r at a time is

n  n 
      Cr
n
 r   r, n  r 
n!
 n Cr 
r! (n  r )!
Example
Solution
a) The number of ways to adopt five of the
seven rules,
n = 7C5 = 21
b) Since the person already observes two
of the seven rules, only three more must
be adopted. The number of ways of
doing so is
n = 5C3 = 10
Partitioning
Often we are concerned with the number
of ways of partitioning a set of n objects
into r subsets called cells.
– A partition has been achieved if the
intersection of every possible pair of the r
subsets is the empty set  and if the union of
all subsets gives the original set.
– The order of the elements within a cell is of no
importance.
Partitioning
The number of ways of partitioning a set of
n objects into r cells with n1 elements in the
first cell, n2 elements in the second, and so
forth, is

 n  n!
  
 n1 , n 2 , ..., nr  n1! n 2 ! ... nr !
where n1 + n2 + n3 + … + nr = n
Example
A college team plays 12 football games
during a season. In how many ways can
the team end the season with 7 wins, 3
losses, and 2 ties?
Solution:
12!
n  7920
7! 3! 2!

Alternative Solution:
n  ways to win 7 times x ways to lose 3 times
xways to tie 2 times
 12   5  2
n         12C7 5 C 3 2 C 2  7,920
7   3  2
Permutation & Combination
• Permutations of {M, N, R} taken two at a
time = 3P2 = 6
{M, N} {M, R} {N, M} {N, R}
{R, M} {R, N}

• Combinations of {M, N, R} taken two at a


time = 3C2 = 3
{M, N} or {N, M} {N, R} or {R, N}
{R, M} or {M, R}
Example: #4.18
Solution to #4.18:
a) The first person (job) can be chosen from
among the 7 applicants, and the second
person (job) can be selected from the
remaining 6 applicants:
# of ways = (7)(6) = 42
b) If there are no differences between the
two jobs, then
# of ways = [(7)(6)]/2 = 21
Alternate Solution to #4.18:

7! 7(6)(5)(4)( 3)( 2)(1)


a) 7 P2    42
5! (5)(4)( 3)( 2)(1)
7! 7(6)(5)(4)( 3)( 2)(1)
b) 7 C2    21
5! 2! (5)(4)( 3)( 2)(1)  ( 2)(1)
Probability
• Perhaps it was humankind’s unquenchable
thirst for gambling that led to the early
development of probability theory.
• In an effort to increase their winnings,
gamblers called upon mathematicians to
provide optimum strategies for various
games of chance.
– Some of the mathematicians providing these
strategies were Pascal, Leibniz, Fermat, and
James Bernoulli.
Probability
As a result of this development of
probability theory, statistical inference,
with all its predictions and generalizations,
has branched out far beyond games of
chance to encompass many other fields
associated with chance occurrences, such
as politics, business, weather forecasting,
and scientific research.
Probability
• The likelihood of the occurrence of an
event resulting from such a statistical
experiment is evaluated by means of a set
of real numbers, called weights or
probabilities, ranging from 0 to 1.
• To every point in the sample space we
assign a probability such that the sum of
all probabilities is 1.
Probability
Probability
• If the sample space for an experiment
contains N elements, all of which are
equally likely to occur, we assign a
probability equal to 1/N to each of the N
points.
• The probability of any event A containing n
of these N sample points is then the ratio
of the number of elements in A to the
number of elements in S.
Probability
If an experiment can result in any one of N
different equally likely outcomes, and if
exactly n of these outcomes correspond to
event A, then the probability of event A is

n
P( A ) 
N
Example: #4.14
Solution
a) There is a total of (3)(3) = 9 possible
outcomes:
SL, SR, SS,
LL, LR, LS
RL, RR, RS
b) Number of ways that neither vehicle
turns left is (2) (2) = 4.
P(at least one turns left) = (9 – 4) / 9
=5/9
Solution
c) Number of ways both vehicles make a
turn = (2) (2) = 4
Number of ways that at most one
vehicle makes a turn = 9 – 4 = 5
P(at most one vehicle turns) = 5/9
Example: #4.17
Solution to #4.17
Let A  event that both generators are non  defective
Then,
 ways of choo sin g 2 non  defects 
 
 from 3 non  defects 
P( A )   
ways of choo sin g two generators
 
 from five generators 
 
3 C2 3
P( A )  
5 C2 10
Example: #4.19
Solution to #4.19
Let B  event that all bulbs are non  defective
Then,
 ways of choo sin g 3 non  defective 
 
 bulbs from 4 non  defective bulbs 
P( B )  
ways of choo sin g three bulbs 
 
 from six bulbs 
 
4 C3 4 1
P( B )   
6 C3 20 5
Example: #4.21
Solution to #4.21

8!
a) # of ways   168
2! 5! 1!
alternate solution:
# of ways  8 C2 6 C5 1 C1  168
Solution to #4.21
7!
b ) # of ways   21
2! 5!
alternate solution:
# of ways  7 C 2 5 C5 1 C1  21

21 1
P( Jones assigned to C)  
168 8
Example: #4.23
Solution to #4.23
a) Let A  the event that the employee
ranked first is selected

4 C2 6 3
P( A )   
5 C3 10 5
Solution to #4.23
b ) Let B  the event that the highest
ranked employee is 2nd or lower

4 C3 4 2
P( B )   
5 C3 10 5
Solution to #4.23
c) Let C  the event that the employees ranked
4th and 5th are selected

3 C1 3
P(C)  
5 C3 10
Example: #4.27
Solution to 4.27
9!
a) # of ways   1,680
3! 3! 3!
alternate solution:
# of ways  9 C3 6 C3 3 C3  1,680
Solution to 4.27
Let D  event that a line gets both used wrenches
7!
1! 3! 3! 140 1
b ) P( D)   
9! 1,680 12
3! 3! 3!
alternate solution:
7 C1 6 C 3 3 C 3 140 1
P( D)   
9 C3 6 C3 3 C3 1,680 12
Example:
#4.65
Solution to 4.65
Let A  event that an ace and a face card
is drawn (in any order )
4 C1 12C1 48
P( A )    0.0362
52 C 2 1326
Solution to 4.65

Let B  event that five cards drawn


are all spades
13 C5 1287 4
P( B )    4.95x10
52 C5 2,598,960
Solution to 4.65

Let C  event that five cards drawn


are of the same suit
13C5 ( 4) 5148 3
P ( C)    1.98x10
52 C5 2,598,960
#4.79
Solution to 4.79

Let A  event that the best tire of the


four selected is ranked 3rd
5 C3 10 1
P( A )   
8 C4 70 7
Probability
• If the outcomes of an experiment are not
equally likely to occur, the probabilities must
be assigned on the basis of prior knowledge
or experimental evidence.
• For example, if a coin is not balanced, we
could estimate the probabilities of heads
and tails by tossing the coin a large number
of times and recording the outcomes.
Additive Rules of Probability
If A and B are two events, then
P(AB) = P(A) + P(B) – P(AB)
Additive Rules of Probability
Probability Rules
• Often it is more difficult to calculate the
probability that an event occurs than it is to
calculate the probability that the event
does not occur.
• Should this be the case for some event A,
we simply find P(A’) first and then use the
following theorem:
Example
The probability that an American industry
will locate in Shanghai, China is 0.7, the
probability that it will locate in Beijing,
China is 0.4, and the probability that it will
locate in either Shanghai or Beijing or both
is 0.8. What is the probability that the
industry will locate
(a) in both cities?
(b) in neither city?
Solution
Let
S  event that the industry will locate in S hanghai
B  event that the industry will locate in Beijing
Then,
P(S  B )  P(S )  P( B )  P(S  B )
0.8  0.7  0.4  P(S  B )
P(S  B )  0.3
(a) P(S  B )  0.3
(b ) P(S  B )  1  P(S  B )  1  0.8  0.2
Example: #4.10
Solution to #4.10
0.10
A
0.30 0.50
0.10

a) P( A)  0.30  0.10  0.40


b) P( A  B)  0.30  0.10  0.50  0.90
Solution to #4.10
0.10
A
0.30 0.50
0.10

c) P( B )  1  (0.50  0.10)  0.40


d) P( A  B)  0.10
Solution to #4.10
0.10
A
0.30 0.50
0.10

e) P( A  B )  0.30  0.10  0.10  0.50


f ) P( A  B)  1  0.10  0.90
Solution to #4.10
0.10
A
0.30 0.50
0.10

g) P( A  B)  0.10
Example: #4.13
Solution to #4.13
0.84
A
(Bushing) 0.08
0.06 0.02
B
(shaft)

a) P( A)  0.06  0.02  0.08


b) P( A  B)  0.06  0.02  0.08  0.16
Solution to #4.13
0.84
A
(Bushing) 0.08
0.06 0.02
B
(shaft)

c) P( A only or B only)  0.06  0.08  0.14


d) P(no defects)  0.84
Conditional Probability
• In some applications, the practitioner is interested
in the probability structure under certain
restrictions.
• For instance, in epidemiology, rather than studying
the chance that a person from the general
population has diabetes, it might be of more
interest to know this probability for a distinct group
such as Asian women in the age range of 35 to 50.
• This type of probability is called a conditional
probability.
Conditional Probability
• The probability of an event B occurring
when it is known that some event A has
occurred is called a conditional probability
and is denoted by P(B|A).
• The symbol P(B|A) is usually read “the
probability that B occurs given that A
occurs” or simply “the probability of B,
given A.”
Conditional Probability

Note that
P(AB) = P(B|A)P(A) is equivalent to
P(AB) = P(A|B)P(B)
Conditional Probability
• The notion of conditional probability
provides the capability of reevaluating the
idea of probability of an event in light of
additional information, that is, when it is
known that another event has occurred.
• Hence, the probability P(A|B) is an
updating of P(A) based on the knowledge
that event B has occurred.
Independence
• Sometimes the occurrence of one event alters
the probability of occurrence of another event.
Such events are called dependent events.
• At other times, the probability of an event (say
A) does not change when another event occurs.
– If the extra information derived from knowing that an
event B, for instance, has occurred does not
change the probability of A — that is, if P(A|B) =
P(A) — then events A and B are said to be
independent.
Independence
For independent events,
P( A  B )
P( A | B ) 
P( B )
becomes
P( A  B )
P( A | B )  P( A )  or
P( B )
P( A  B )  P( A )P ( B )
Independence
The Product or Multiplicative Rule
The product rule (or multiplicative rule)
enables us to calculate that two events will
both occur:

Note that since AB is equivalent to BA, it


does not matter which event is referred to as
A and which event is referred to as B.
The Product Rule & Independence
The General Product Rule
Example: #4.33
Solution to #4.33
P( A  B )
(a ) P( A | B ) 
P( B )
0.15
P( A | B )   0.5
0.30
(b ) P( A )P(B )  (0.6)(0.3)  0.18
P( A  B )  0.15
 P ( A  B )  P ( A )P ( B )
 A and B are NOT independent of each other.
Example: #4.36
Solution to 4.36
Left

Straight

Left Left
Right

Straight Straight

Left
Right Right
Straight

Right
Solution to 4.36
Let
A  event that at least one of the two vehicles turn
B  event that at least one of the two vehicles turn left
Then,
P( B  A ) P( B ) 5/9 5
P( B | A )    
P( A ) P( A ) 8/9 8

Assumption: the action of the first vehicle does


NOT affect the action of the second vehicle.
Example: #4.38
Example: #4.38
Accidental Death Data in the US
Solution to #4.38
Let
A  event that it was a motor vehicle accident
46,263
P( A )   0.4979
92,911
Let
B  event that it was a motor vehicle accident
if you knew the victim was male
32,949
P( B )   0.5144
64,053
Solution to #4.38
Let
C  event that it was a motor accident
if you knew the victim to be 15  24 y .o.
14,738
P (C )   0.7443
19,801
Let
D  event that it was a fall if you
knew the victim to be over 75 y .o.
7,067
P( D)   0.4399
16,065
Solution to #4.38

Let
E  an accidental death given that
the person was male
64,053
P( E )   0.6894
92,911
Example: #4.43
Solution to #4.43

85%
5% 9.5  to 10.5 
10%
< 9.5 
> 10.5 

Assume that the lot of resistors is very large


such that sampling a few units does not affect
the indicated percentages.
Solution to #4.43
a) Let A = the event that the first randomly
selected resistor has a resistance
between 9.5 and 10.5 ohms
Let B = the event that the second
randomly selected resistor has a
resistance between 9.5 and 10.5 ohms
P(A  B) = P(A|B)P(B) = P(A)P(B|A)
= P(A)P(B)
= (0.85)(0.85)
= 0.7225
Solution to #4.43
b) Let C = the event that the two randomly
selected resistors both have resistances
 10.5.
Let D = the event that at least one of
two randomly selected resistors has a
resistance > 10.5

P(D) = 1 – P(C)
= 1 – (0.9)(0.9)
= 0.19
Example:
#4.44
Solution to #4.44 (a)
P(current flows )  3 [ P(only one path works ) ]
 3 [ P(two paths works ) ]
 P(all paths work )
 3[(0.9)(0.1)(0.1)]
 3[(0.9)(0.9)(0.1)]
 (0.9)(0.9)(0.9)
 0.999
Alternate Solution to #4.44 (a)

P(current flows )  1  P(no current flows )


 1  (0.1)(0.1)(0.1)
 0.999
Solution to #4.44 (b)

P(relay 1 is closed | current flows )


P(relay 1 is closed  current flows )

P(current flows )
P(relay 1 is closed)

0.999
 P(all closed)  2  P(#1 closed, #2 or #3 closed) 
 
  P(#1 closed, #2 and #3 open ) 

0.999
Solution to #4.44 (b)

P(relay 1 is closed | current flows )


(0.9)(0.9)(0.9)  2[(0.9)(0.1)(0.9)]
 (0.9)(0.1)(0.1)

0.999
 0.9009
Example: #4.45

Note: According to #4.44, the probability of a relay


working is 0.9. The relays work independently.
Solution to #4.45

S eries S ystem:
P(current flow from a to b )  (0.9)(0.9)  0.81
(B)
Parallel S ystem:
 (0.9)(0.9)  (0.9)(0.1)  (0.1)(0.9)
 0.99
The numbers in the boxes represent the probability
that the component represented by the boxes are
functioning. The diagram is that of an electrical
system. For the system to work, current must be
able to flow from point A to D. What is the
probability that the system works? Assume the
components fail independently.
Solution:
P(system works )  P( A  B  C  D)
 P( A  B  C' D)
 P( A  B'C  D)
 (0.95)(0.7 )(0.8)(0.9)
 (0.95)(0.7 )(0.2)(0.9)
 (0.95)(0.3)(0.8)(0.9)
 0.8037
Theorem of Total Probability

(See figure on the next slide)


Partitioning of the sample space S.
P(B) = P(BA) + P(BA’)
P(B) = P(A)P(B|A) + P(A’)P(B|A’)
P(B) = P(BE1) + P(BE2) + P(BE3) + P(BE4)
P(B) = P(E1)P(B|E1)+P(E2)P(B|E2)+P(E3)P(B|E3)+P(E4)P(B|E4)
Example
Adult over 40
Adult over 40
years old
years old with
without
cancer
cancer
(0.05)
(0.95)
Positive
diagnosis for 0.78 0.06
cancer

Negative
diagnosis for ? ?
cancer
Adult over 40
Adult over 40
years old
years old with
without
cancer, A
cancer, B
(0.05)
(0.95)
Positive
diagnosis for 0.78 0.06
cancer, C

Negative
diagnosis for 0.22 0.94
cancer, D
Solution
P(C)  P( C  A )  P(C  B )
 P( C | A )P ( A )  P (C | B )P( B )
 (0.78)(0.05)  (0.06)(0.95)
 0.096
This implies that people over 40 years of age will
get diagnosed as having cancer REGARDLESS of
whether or not they really have the disease.
Bayes’ Rule
• Suppose that the DLSU bookstore sources
its office supplies from three suppliers:
Power Books, Fully Booked, and National
Bookstore.
• Suppose also that 10% of the supplies
delivered to DLSU are defective.
• What is the probability that a particular
defective item came from Power Books?
• The answer lies in Bayes’ Rule.
Example: #4.29
Person does
Person has
not have the
the disease
disease
(0.01)
(0.99)
Positive
diagnosis for 0.90 ?
disease

Negative
diagnosis for ? 0.90
disease
Person does
Person has
not have the
the disease
disease
(0.01)
(0.99)
Positive
diagnosis for 0.90 0.10
disease

Negative
diagnosis for 0.10 0.90
disease
Solution
Let A  event that a person has the disease
B  event that a person has a (  ) diagnosis
C  event that a person does not have the disease
D  event that a person has a (  ) diagnosis
P( A  B )
Then, P( A | B ) 
P( B )
P( A )P( B | A )

P ( A ) P ( B | A )  P (C ) P ( B | C )
(0.01)(0.90) 1
 
(0.01)(0.90)  (0.99)(0.10) 12
Schematic of the sample space

1% has
99% does not
the
have the disease
disease

Positive
diagnosis

(Not drawn to scale)

You might also like