Probability Distributions

1. Probability distributions
2. Discrete probability distributions
3. Continuous probability distributions
4. Multidimensional probability distributions


Probability Distributions
Random (stochastic) variables
Experimental measurements are not reproducible in a
deterministic fashion
Each measurement can be viewed a random variable X
Defined on sample space S of an experiment
Probability distribution
Determines probability of particular events
Discrete distributions: random variables are discrete
quantities
Continuous distributions: random variables are
continuous quantities
Cumulative probability distribution function F(x)
) ( ) ( ) ( ) ( ) ( a F b F b X a P x X P x F = s < s =
Discrete Probability Distributions
Random variable X can only assume countably many
discrete values: x
1
, x
2
, x
3
,
Probability density function f(x)



Cumulative distribution function



Properties

= =
=
otherwise 0
if ) (
) (
j j j
x x x X P p
x f

s s
= =
x x x x
j j
j j
p x f x F ) ( ) (

=
= = s <
s <
j
j
b x a
j
p
p a F b F b X a P
j
1
) ( ) ( ) (
Discrete Distribution Example

Matlab function: unicdf(x,n),
>> x = (0:6);
>> y = unidcdf(x,6);
>> stairs(x,y)

Continuous Probability Distributions
Random variable X can assume infinitely many real values
Cumulative distribution function


Probability density function


Properties
}

=
x
dv v f x F ) ( ) (
dx
x dF
x f
) (
) ( =
}
}

=
= = s <
1 ) (
) ( ) ( ) ( ) (
dv v f
dv v f a F b F b X a P
b
a
Continuous Distribution Example
Probability density function


Cumulative distribution function



Probability of events
1 ) 1 ( 75 . 0 ) (
otherwise 0
1 1 ) 1 ( 75 . 0
) (
1
1
2
2
} }

= =

s s
= dv v dv v f
x x
x f

>
s < +
s
=
+ = = =
} }

1 1
1 1 25 . 0 75 . 0 5 . 0
1 0
) (
25 . 0 75 . 0 5 . 0 ) 1 ( 75 . 0 ) ( ) (
3
3
1
2
x
x x x
x
x F
x x dv v dv v f x F
x x
73 . 0 25 . 0 75 . 0 5 . 0 95 . 0 ) ( ) (
% 75 . 68 ) 1 ( 75 . 0 ) 5 . 0 ( ) 5 . 0 ( ) 5 . 0 5 . 0 (
3
5 . 0
5 . 0
2
= + = = = s
= = = s s
}

x x x x F x X P
dv v F F x P
Mean and Variance
Discrete distribution



Continuous distribution



Symmetric distribution
If f(c-x) = f(c+x), then f(x) is symmetric with respect to = c
Transformation of mean and variance
Given random variable X with mean and variance o
2

The standardized random variable Z has zero mean and unity variance

=
=
j
j j
j
j j
x f x
x f x
) ( ) ( Variance
) ( Mean
2 2
o

}
}

=
=
dx x f x
dx x xf
) ( ) ( Variance
) ( Mean
2 2
o

o

X
Z
Expectations and Moments
Moments for continuous distributions





Continuous distribution example
}
}
} }

=
=
= = = =
dx x f x X E k
dx x f x X E k
X E dx x f x X E dx x xf
k k
k k
) ( ) ( ) ] ([ moment central th
) ( ) ( moment th
) ] ([ ) ( ) ( ) ( ) (
2 2 2

o
| |
| |
}
}

= = = =
= = = =

s s
=
1
1
1
1
5
5
1
3
3
1
2 2 2 2
1
1
1
1
4
4
1
2
2
1
2
2
2 . 0 ) 75 . 0 ( ) 75 . 0 ( ) 1 ( 75 . 0 ) 0 ( ) ] ([
0 ) 75 . 0 ( ) 75 . 0 ( ) 1 ( 75 . 0 ) (
otherwise 0
1 1 ) 1 ( 75 . 0
) (
x x dx x x X E
x x dx x x X E
x x
x f
o

Binomial Distribution
Governs randomness in games of chance, quality
inspections, opinion polls, etc.
X = {0,1,2,,n} = number of times event A occurs in n
independent trials


Probability of obtaining A exactly x times in n trials



Mean and variance:
>> binopdf(x,n,p) binomial probability
>> binopdf(0,200,0.02) ans = 0.0176
>> binocdf(x,n,p) binomial cumulative probability
>> 1 - binocdf(100,162,0.5) ans = 0.0010
p q A X P p A X P = = = = = 1 ) ( ) (
n
n
x n x
x n x
n
x
n
x f q p q p
x
n
x f
|
.
|

\
|

=
|
.
|

\
|
|
|
.
|

\
|
= = =
|
|
.
|

\
|
=

2
1
)! ( !
!
2
1
) ( ) (
2
1
npq np = =
2
o
Poisson Distribution
Infinitely many possible events
Probability distribution function


Limit of binomial distribution as


Mean and variance: o
2
=
Example
Probability of a defective screw p = 0.01
Probability of more than 2 defects in a lot of 100 screws?
Binomial distribution: = np = (100)(0.01) = 1
Since p <<1, can use Poisson distribution to approximate solution




Matlab functions: poisspdf(x,mu), poisscdf(x,mu)
, 2 , 1 , 0
!
) ( = =

x e
x
x f
x

constant 0 = np n p
( )
% 03 . 8 ) 2 ( 1 ) 2 (
9197 . 0
!
) ( ) 2 ( ) 2 (
2
0
! 2
1
! 1
1
! 0
1
1
2
0
2 1 0
= s = >
= + + = = = = s

=

=
X P X P
e e
x
x f F X P
j
j
x
j
j
j

Matlab: Poisson-Binomial Comparison

Screw example (p=0.01, n=100)
for less than 05 defective screws
>> x = (0:5);
>> y = poisscdf(x,1);
>> z = binocdf(x,100,0.01)

Compare Poisson to Binomial with equal , different p and n

Number Binomial Poisson
0 0.366032 0.367879
1 0.735762 0.735759
2 0.920627 0.919699
3 0.981626 0.981012
4 0.996568 0.99634
5 0.999465 0.999406
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
0.4
0.5
0.6
0.7
0.8
0.9
1


Binomial n = 50, p = .02
Binomial n = 20, p = .05
Binomial n = 10, p = .1
Binomial n = 5, p = .2
Poisson mu=1
Normal (Gaussian) Distribution
Probability density function



Cumulative distribution function



Standardized normal distribution ( = 0, o
2
= 1)
}

=
(
(

|
.
|

\
|

= 1 ) (
2
1
exp
2
1
) (
2
dx x f
x
x f
o

t o
dv
v
x F
x
}

(
(

|
.
|

\
|

=
2
2
1
exp
2
1
) (
o

t o
|
.
|

\
|

u = = u
}


o

t
x
x F du e z
z
u
) (
2
1
) (
2
2
Computing Probabilities
Interval probabilities


Sigma limits


Example
X is a random variable with = 0.8 and o
2
= 4
Use Table A7 in text
|
.
|

\
|

u
|
.
|

\
|

u = = s <
o

o
a b
a F b F b X a P ) ( ) ( ) (
% 7 . 99 ) 3 3 (
% 5 . 95 ) 2 2 (
% 68 ) (
~ + s <
~ + s <
~ + s <
o o
o o
o o
X P
X P
X P
% 79 7939 . 0 ) 82 . 0 (
2
8 . 0 44 . 2
) 44 . 2 ( ) 44 . 2 ( ~ = u =
|
.
|

\
|

u = = s F X P
Matlab: Normal Distribution
Normal distribution: normpdf(x,mu,sigma)
normpdf(8,10,2) ans = 0.1210
normpdf(9,10,2) ans = 0.1760
normpdf(8,10,4) ans = 0.0880
Normal cumulative distribution:
normcdf(x,mu,sigma)
normcdf(8,10,2) ans = 0.1587
normcdf(12,10,2) ans = 0.8413
Inverse normal cumulative distribution:
norminv(p,mu,sigma)
norminv([0.025 0.975],10,2) ans = 6.0801 13.9199
Random number from normal distribution:
normrnd(mu,sigma,v)
normrnd(10,2,[1 5]) ans = 9.1349 6.6688
10.2507 10.5754 7.7071
Matlab: Normal Distribution Example
The temperature of a bioreactor follows a
normal distribution with an average temperature
of 30
o
C and a standard deviation of 1
o
C. What
percentage of the reactor operating time will the
temperature be within +/-0.5
o
C of the average?
Calculate probability at 29.5
o
C and 30.5
o
C, then
calculate the difference:
p=normcdf([29.5 30.5],30,1)
p = [0.3085 0.6915]
p(2) p(1)
0.3829

The reactor temperature will be within +/- 0.5
o
C
of the average ~68% of the operating time
25 26 27 28 29 30 31 32 33 34 35
0
0.1
0.2
0.3
0.4
Temperature
Multidimensional Probability Distributions
Consider two random variable X and Y
Two-dimensional cumulative distribution function

Discrete distributions

Continuous distribution


Marginal distributions
) , ( ) , ( y Y x X P y x F s s =

= =
s s i j
j i
x x y y
j i
y x f y x f y x F
i j
1 ) , ( ) , ( ) , (
} }
} } } }
= s < s <
= =

2
2
1
1
) , ( ) , (
1 * * *) *, ( * * *) *, ( ) , (
2 2 1 1
b
a
b
a
y x
dxdy y x f b Y a b X a P
dy dx y x f dy dx y x f y x F
}

=
= = =
dy y x f x f
y x f Y x X P x f
j
j
) , ( ) ( Continuous
) , ( ) arbitrary , ( ) ( Discrete
1
1
Independent Random Variables
Basic property

Addition and multiplication of means


Addition of variance



Covariance
) ( ) ( ) , ( ) ( ) ( ) , (
2 1 2 1
y f x f y x f y F x F y x F = =
) ( ) ( ) ( ) ( only s t variable Independen
) ( ) ( ) ( ) ( General
2 1 2 1
2 1 2 1
n n
n n
X E X E X E X X X E
X E X E X E X X X E


=
+ + + = + + +
XY
Y E X E Y X E
Z Y X Y X
o o o o

o o o
2
) ( ) ( ) (
} , { : } , { : } , { :
2
` 2
2
` 1
2
2 1
2 2 1 1
+ + =
+ = + = + =
= +
2
2
2
1
2
) ( ) ( ) (
s t variable Independen
) ( ) ( ) (
o o o
o
+ =
=

=
Y E X E XY E
Y E X E XY E
XY