Foundations of Probability: Part I

Cyr Emile MLAN, Ph.D.

mlan@stat.uconn.edu

Foundations of Probability: Part I

p. 1/35

Introduction
Text Reference: Introduction to Probability and Its
Application, Chapter 2.
Reading Assignment: Sections 2.1-2.3, January
20-January 26

Uncertainty or chance variation can be found in nearly

every aspect of our lives.
For example,
Will it rain tomorrow?
Will the patient survive?
How long will his illness last?
Will the economy expand next year?
How large a rain will I receive next year?
Foundations of Probability: Part I

p. 2/35

Introduction
Even the best deterministic theory cannot give an
accurate answer to these questions and an explanation
for the variations that are observed.
How can an inexact situation be studied? To aid,
probability or chance is used to report and explain such
variation.
The method of probability makes precise the degree of
predictability and further indicates the likelihood of
occurrence of each possibility.
Probability can be defined as a scientific discipline to
study uncertainty in a systematic fashion.

Foundations of Probability: Part I

p. 3/35

Random Experiments
Many experiments such as tossing a coin, rolling a dice, drawing a
card, spinning a roulette wheel, counting the number of arrivals at
emergency room, guessing tomorrow weather, measuring the lifetime
of an bulb, etc... have unpredictable outcomes. We cannot say with
absolute certainty which outcome will show up. Such experiment

are called random experiment.

A random experiment or a probability experiment is an
action or process that leads to one of several possible
outcomes and before it is performed, one cannot guess
which outcome will come out.
An outcome is a result of a random experiment.
Examples
Experiment: Record marks on a statistics test (out of 100).
Outcomes: Numbers between 0 and 100
Experiment: Record student evaluations of a course.

Foundations of Probability: Part I

Outcomes: Poor, fair, good, very good, and excellent

p. 4/35

Sample Space and Events

Sample space

A sample space, S , is the set that consists of all

possible outcomes of a random experiment listed in
a mutually exclusive and exhaustive way.
A probability experiment consists in determining
the gender of a new-born child. The sample space
is S = {girl, boy}.
A probability experiment consists of tossing a coin.
The sample space is S = {H, T }.
A probability experiment consists of tossing two
coins simultaneously. The sample space is
S = {(H, H), (H, T ), (T, H), (T, T )}.
A probability experiment consists of rolling a
regular six-sided die onto a table and recording
the number on the upper face. The sample space
is S = {1, 2, 3, 4, 5, 6}.Foundations of Probability: Part I
p. 5/35

Sample Space and Events

A probability experiment consists of selecting a
faculty member at Uconn and recording her income.
The sample space is S = [0, ).
A probability experiment consists of selecting 5
faculty members at Uconn and recording their
incomes. The sample space is S = [0, )5 .

A sample space is said to be discrete if it contains

either a finite or a countable number of distinct
sample points.
A probability experiment consists of tossing a coin
repeatedly until a head is obtained. The sample
space is S = {H, T H, T T H, T T T H, ...}.
A probability experiment consists of counting the
number of bacteria on a computer keyboard. The
sample space is S =Foundations
{1, 2, 3, 4,of5,Probability:
...}.
Part I
p. 6/35

Sample Space and Events

Event
An event is a subcollection of outcomes of a
probability experiment.
The events, two heads, A = {(H, H)}, one head and
one tail B = {(H, T )}, one tail and one
head C = {(T, H)}, and two tails D = {(T, T )} are
each called simple events. Only one simple event
occur in a single trial of an experiment.
The event, at least one tail,
E = {(H, T ), (T, H), (T, T )} is called composite event
or compound event (can always be expressed using
two or more simple events).
Many composite events can occur simultaneously
in an experiment.
Foundations of Probability: Part I

p. 7/35

Sample Space and Events

Approaches to Define Events
We can define an event A by
(a) Making an exhaustive list of its simple outcomes if
possible; or
(b) Imposing some condition a condition satisfied by
the outcomes in A.

Foundations of Probability: Part I

p. 8/35

Sample Space and Events

Occurrence of an Event
An event A occurs if the outcome of the experiment
is contained in the event A.
For example, suppose we toss a die once and
observe the number on the upper face. In this case,
the sample space S = {1, 2, 3, 4, 5, 6}. Let A = the
event that the observed number is even = {2, 4, 6}.
If a die is tossed and 6 comes up, we say that A
occurs.
if a die is tossed and 5 comes up, we say that A
does not occur.

Special Events
Certain event or sure event: the sample space S .
Impossible event or null event: the empty set .
Foundations of Probability: Part I

p. 9/35

Sample Space and Events

More complicated events can be constructed from
simpler events using basic concepts of set theory.

Relationship Between Events

Let A and B denote two events. Then,
(a) A B means that if A occurs, B must occur.
(b) A = B means that (i) if A occurs, B must occur, and
(ii) if B occurs, A must occur.
(c) B is the complement of A, denoted by B = A, if B
contains all the outcomes in S that are not contained
in A.
(d) A B is an event that contains all outcomes in either
A or B .
(e) A B or AB is an event that contains all outcomes in
both A and B .
Foundations of Probability: Part I

p. 10/35

Sample Space and Events

Relationship Between Events
(f) A and B are mutually exclusive or disjoint if the
occurrence of A precludes the occurrence of B , and
vice versa. Mathematically, we have A B = .
(g) B A denotes an event that contains all outcomes in
B but not in A. In other words, it is B A
(h) DeMorgans laws
AB =AB
AB =AB
or more generally
n

Ai = Ai

i=1

i=1

and

i=1

i=1

Ai = Ai

(i) Venn Diagram

Foundations of Probability: Part I

p. 11/35

Sample Space and Events

Example 2.1:
Flip a coin three times and define by A, B, and C the
event more than one head, more than one tail, and at least
one head and one tail, respectively. Find
A, C, A B, A B, A C , and C A = C A

Solution:
S = {T T T, T T H, T HT, T HH, HT T, HT H, HHT, HHH}.
A = {HHH, HHT, HT H, T HH}
B = {T T T, T T H, T HT, HT T }.
C = {T T H, T HT, T HH, HT T, HT H, HHT }.
A = B and C = {HHH, T T T }.
A B = S and A B = (disjoint)
A C = {T HH, HT H, HHT
}.
Foundations of Probability: Part I
C A = {T T H, T HT, HHT }

p. 12/35

Probability
Roughly speaking, probability of an event A is a
measure of the likelihood of the occurrence of A on a
single run of a experiment whose outcome cannot be
predicted.
At the start of a football game, a balanced coin is
flipped in the air to decide which team will receive the
ball first. What is the chance that the coin will land
heads up? Most would say .5.
What is the meaning of the numbers 0.5?

.5 is regarded as a long-term or limiting relative

frequency as the number of flips gets large.

This idea of a stabilizing relative frequency after many

trials lies at the heart of random behavior.
Foundations of Probability: Part I

p. 13/35

Probability
Most gamblers are very aware of the idea of chance in
the long-run. Indeed, history tends to show that
gambling was at the heart of the mathematical
formulation of chance or probability.
Most famous example of that is the French gambler de
Mr glorious history and his falling down (around
1650).

Foundations of Probability: Part I

p. 14/35

Definition of Probability
Empirical Definition:

Probability can be regarded as the ratio of successes to

the total number of trials in the long run. Indeed, we are
counting occurrences in successive runs.
The French naturalist Count Buffon (1707-1788) tossed a coin
4040 times resulting in 2048 heads, i.e. 2048/4040 = 0.5069.
While imprisoned by the Germans during World War II, the
Australian mathematician John Kerrich tossed a coin 10000
times, resulting in 5067 heads, i.e. 5067/10000 = 0.5067.
Around 1900, the English statistician Karl Pearson heroically
tossed a coin 24000, resulting in 12012 heads, i.e. 12012/24000
= 0.5005.
When we say that McGuire batting average was .333, we really
mean that the probability of a hit during his career is about 1 in 3.
Foundations of Probability: Part I

p. 15/35

Definition of Probability
More precisely, suppose a very large set of N identical
systems are available to be run together. Suppose we
are interested in the occurrence of an event E . Let nE
be the number of occurrences of the event E . The
probability of the occurrence of E , denoted Pr(E), is
defined as:
nE
P (E) = lim
.
N N
Note that this definition requires infinite number of trials
which is not practical. In real world, we would monitor
the limit for few repeated trials and guess the limit.

P (E) represents the probability of the event E in a

single trial.
Foundations of Probability: Part I

p. 16/35

Interpretation of Probability
W e always interpret probability as the relative
frequency or fraction or proportion an outcome occurs
in a infinite number of repeat of an experiment.

Foundations of Probability: Part I

p. 17/35

Joke
A statistics major was completely hung over the day of his final
exam. It was a True/False test, so he decided to flip a coin for the
answers. The stats professor watched the student the entire two
hours as he was flipping the coin...writing the answer...flipping
the coin...writing the answer. At the end of the two hours,
everyone else had left the final except for the one student. The
professor walks up to his desk and interrupts the student,
saying: Listen, I have seen that you did not study for this
statistics test, you didnt even open the exam. If you are just
flipping a coin for your answer, what is taking you so long?
The student replies bitterly, as he is still flipping the coin: Shhh!
I am checking my answers!
Foundations of Probability: Part I

p. 18/35

Definition of Probability
It is clear that this intuitive definition has clear
limitations. It has no application for unique event. E.g.:
What does the phrase "the probability of re-election for
the current president is 60%" really mean given the
empirical definition?
Subjective Definition:

Probability can be thought of as the degree of belief, not

necessarily the same from one person to the next.
E.g: There is a high probability (.99) of another form
of life in the universe. Note that here we are not
counting occurrences or non-occurrences. Nor did
we sample the universe to ascertain that where we
found a new form of life. We mean our degree of
belief in other form of life elsewhere in the universe
is .99.
Foundations of Probability: Part I

p. 19/35

Definition of Probability
When assuming that some form of symmetry exist
among the outcomes, one does not need to conduct a
probability experiment to compute probabilities.
Classical Definition:

If a probability experiment has n possible outcomes

1 , 2 , , n , all equally likely, then
each individual outcome, i , has probability 1/n, i.e.,

1
P {i } = .
n

the probability of an event A is

m
P (A) =
n

where m denote the number of ways that an event A

Foundations of Probability: Part I
p. 20/35
can occur.

Definition of Probability
In other words, if a sample space is defined so that all
simple outcomes are equally likely, this is a convenient
way to compute probabilities of event by just counting.

Example 2.1: Guessing Yet Passing a Pop

Quiz
For a three-question multiple choice pop quiz, a student
is totally unprepared and randomly guesses the answer
to each question. Each question has two options.
Find the probabilities of each possible student
outcomes for the quiz, in terms of whether each
response is correct (C) or incorrect (I).
(b) Find the probability the student passes, answering at
least three question correctly.
(a)

Foundations of Probability: Part I

p. 21/35

Frequency Table and Relative Frequency Table

Solution:
The sample space is
S = {CCC, CCI, CIC, CII, ICC, ICI, IIC, III}. Each
question has two options and with guessing, the
responses are expected to be equally likely. Hence,
the probability of selecting any of these 8 simple
outcomes is 1/8.
(b) The probability of at least two passes is
(a)

4
1
P ({CCC, CCI, CIC, ICC}) = =
8
2

In general, one needs counting rules to compute

probability using the classical definition.
Foundations of Probability: Part I

p. 22/35

Kolmogorovs Probability Axioms

In any case, we need a better definition of probability.
Definition of Probability

Definition 2.1:
Let S be a sample space associated with a probability experiment. To every event A in S (A is a subset of S), we
assign a number, P (A), called the probability of A, so that
the following three axioms hold:
Axiom 1: (Nonnegativity)
Axiom 2: (Norming)

For every event A S, P (A) 0.

P (S) = 1.

If A1 , A2 , . . . is a sequence of
pairwise mutually exclusive events in S, i.e., Ai Aj =
for i 6= j, then

 X

P Ai =Foundations
P (Ai ).of Probability: Part I
p. 23/35

Axiom 3: (Countable Additivity)

i=1

i=1

Kolmogorovs Probability Axioms

These axioms were proposed by Andrey Nikolaevich
Kolmogorov in 1933 as the first mathematical
foundation to the calculus of probabilities.
Kolmogorovs definition of probability states the conditions an
assignment probabilities must satisfy.

However, it comes short in telling us how to assign

specific probabilities number to events in the first place.
Probability assignment in practice are based on
empirical rule or a careful thought about the selection
process (classical rule) or are just subjective.
Foundations of Probability: Part I

p. 24/35

Kolmogorovs Probability Axiom

Steps for Calculating Probabilities
Step 1: Define the experiment, i.e, describe the

Step 2:
Step 3:
Step 4:
Step 5:

process leading to the observation and the

type of observation that will be recorded.
List all simple events and the sample space.
Assign probabilities to the simple events.
Determine the collection of simple events
that make up an event of interest.
Sum the simple event probabilities to get the
probability of the event of interest.

Probabilities of composite events can be calculated using

the probabilities of simple events. Note that this process
works only well for experiment with few simple sample
points. In other cases, we need rules for counting.
Foundations of Probability: Part I

p. 25/35

Kolmogorovs Probability Axioms

Example 2.2:
All human blood can be typed as one of O, A, B or AB, but the
distribution of the types varies a bit with race. Here is how
blood type is distributed in any African American.
blood type

AB

probability

.49

.27

.20

a.

What is the probability of type AB blood? Justify.

b.

Interpret these number.

c.

Maria has type B blood. She can safely receive blood

transfusions from people with either blood type O or B,
what is the probability that a randomly chosen African
American can donate blood to Maria?
Foundations of Probability: Part I

p. 26/35

Kolmogorovs Probability Axioms

Solution:


a. P (AB) = 1 P (O) + P (A) + P (B) = .04

b.

These numbers represents the proportion of African

American with these blood types. Another way to think of it
is if I were to select randomly an African American, the
probability that it would have a blood type A will be .27.

c.

Let D denote the event of a successful donation.

P (D) = P (O) + P (B) = .49 + .20 = .69

Foundations of Probability: Part I

p. 27/35

Kolmogorovs Probability Axioms

Properties
(i) P () = 0.
(ii) For every event A, P (A) 1.
(iii) Let A1 , A2 , . . . , An be a finite collection of mutually
exclusive events. Then,
n

P ( Ai ) =
i=1

n
X

P (Ai ).

i=1

(iv) Complement Rule of Probability: P (A) = 1 P (A).

(v) If A B , then P (A) P (B) and
P (B A) = P (B) P (A).
More generally, we have
P (B A) = P (B A) = P (B) P (B A) for any
set of event A and B . Foundations of Probability: Part I
p. 28/35

Kolmogorovs Probability Axioms

(vi) Additive Rule of Probability: Let A and B be any
two subsets of S . Then
P (A B) = P (A) + P (B) P (A B) .

(vii) P (A B) P (A) + P (B).

(viii) P (A B) 1 P (A) P (B).
(ix) Theorem 2.1: Inclusion-Exclusion Principle:
P

Ei

i=1

n
X

P (Ei )

i=1

(1)

P (Ei1 Ei2 ) + +

i1 <i2
r+1

P (Ei1 Ei2 Eir ) + +

i1 <i2 <<ir

(1)n+1 P (E1 E2 En )
Foundations of Probability: Part I

p. 29/35

Kolmogorovs Probability Axioms

Example 2.3:
Suppose that 60% of all customers of a large insurance agency have
automobile (A) policies with the agency, 40% have homeowners (H)
policies, and 25% have both types of policies.
a.

What is the probability that a randomly selected customer has at

least one of these two types of policies?

b.

What is the probability that a customer has neither policy?

Solution:
a.

Define E = customer has A policy, F = customer has H policy.

Then,
P (at least one of E or F) = P (E F )
= P (E) + P (F ) P (E F )
= .6 + .4 .25 = 0.75

b.

P (Not in E or F) = P (E F ) = 1 P (E F ) = .25
Foundations of Probability: Part I

p. 30/35

Kolmogorovs Probability Axioms

Example 2.4:
In a certain residential suburb, 60% of all households subscribe to
the metropolitan newspaper published in a nearby city, 80%
subscribe to the local paper, and 50% of all households subscribe to
both papers. If a household is selected at random, what is the
probability that it subscribes to
a.

at least one of the two newspapers?

b.

exactly one the two newspapers?

Solution:
a.

Define M = {subscribes to the metropolitan paper},

L = {subscribes to the local paper}. Then,
P (at least one of M or L) = P (M L)
= P (M ) + P (L) P (M L)
= .6 + .8 .5 = 0.9
Foundations of Probability: Part I

b.

p. 31/35

P (exactly one) = P (M L) + P (M L) = (.6 .5) + (.8 .5) = .4

Kolmogorovs Probability Axioms

Example 2.5:
After a hurricane, 50% of the residents in a particular Florida county
were without electricity, 47% were without water, and 38% were
without telephone service. Denote these events by A, B and C,
respectively. Although 20% of the residents had all three utilities,
10% were without all three, 12% were without electricity and water
but still had a working telephone, and 4% were without electricity and
a working telephone but still had water. A county resident is randomly
selected and whether or not the person was without water, electricity,
or telephone service after the hurricane is recorded. Express each of
these following event in a set notation and find the percent of county
residents represented by each
a.

The selected county resident was without electricity only but had
water and telephone service.

b.

The selected county resident was without only one of the utilities
Foundations of Probability: Part I
p. 32/35
but had the other two.

Kolmogorovs Probability Axioms

c.

The selected county resident was without exactly two of the

utilities.

d.

The selected county resident was without at least one of the

utilities.

Solution:
Draw the venn diagram and fill it out with numbers.
a.

This event is ABC and P (ABC) = 24%.

b.

This event is (ABC) (ABC) (ABC) and

P (ABC) (ABC) (ABC) = 35%.

This event is (ABC) (ABC) (ABC) and

P (ABC) (ABC) (ABC) = 35%.
d. This event is A B C and P (A B C) = 80%.
c.

Foundations of Probability: Part I

p. 33/35

Probability and Unusual Events

One goal of this course is to learn how probabilities can
be used to identify unusual events.

Unusual Events
An unusual event is an event that has a low
probability of occurring.
Typically, an event with a probability less than 5% is
considered unusual.
This cutoff point that separate unusual event from not
so unusual events is not set in stone. It is determined
by the researcher and the context of the problem.
The cutoff point can be subjective. The most commonly
used cutoff points are 1%, 5%, and 10%.
Foundations of Probability: Part I

p. 34/35

Probability and Unusual Events

Example 2.6: Should We Be Surprised by
Finding No Women on a Jury?
A jury of 12 people is chosen for a trial. The defense attorney claims
it must have been chosen in a biased manner, because 50% of the
citys adult residents are female yet the jury contains no women.
If the jury were randomly chosen from the population,
(a)

what is the probability the jury would have no female?

(b)

what is the probability the jury would have at least one female?

Solution:
a.

Only one of the 212 = 4096 possible outcomes corresponds to a

no-female jury. So the probability is 1/4096=.00024. This is
extremely unlikely if a jury is truly chosen by random sampling.

b.

P (at least one female) = 1 P (no females) = 1 .00024 = .99976

Foundations of Probability: Part I

p. 35/35