Probability Distribution

Statistical Experiment
All statistical experiments have three things in common:
1. The experiment can have more than one possible outcome. 2. Each possible outcome can be specified in advance. 3. The outcome of the experiment depends on chance.

 Example: A coin toss has all the attributes of a statistical experiment.

1. There is more than one possible outcome. 2. We can specify each possible outcome in advance - heads or tails. And there is an element of chance. 3. We cannot know the outcome until we actually flip the coin.

 Variable
A variable is a symbol (A, B, x, y, etc.) that can take on any of a specified set of values.

Random Variable
When the value of a variable is the outcome of a statistical experiment, that variable is a random variable

 Generally, statisticians use a capital letter to represent a random variable and a lower-case letter, to represent one of its values. For example,
X represents the random variable X. P(X) represents the probability of X. P(X = x) refers to the probability that the random variable X is equal to a particular value, denoted by x.
As an example, P(X = 1) refers to the probability that the random variable X is equal to 1.

 probability distribution is a table or an equation that links each outcome of a statistical experiment with its probability of occurrence.

 Suppose you flip a coin two times. This simple statistical experiment can have four possible outcomes:
1. 2. 3. 4. HH HT TH TT

Now, let the variable X represent the number of Heads that result from this experiment.

 The variable X can take on the values 0, 1, or 2. In this example, X is a random variable; because its value is determined by the outcome of a statistical experiment. The following table, which associates each outcome with its probability, is an example of a probability distribution.

Probability Theory
A random experiment is one whose outcome is not predictable with certainty in advance Sample Space (S): The set of all possible outcomes A sample space is discrete if it consists of a finite (or countably infinite) set of outcome; otherwise it is continuous

10

Probability Theory
Any subset (say A) of S is an event.

Events are sets so we can consider their complements, intersection, union and so forth

11

 Example: Consider the experiment where two coins are simultaneously tossed. The various elementary events are S HH , HT , TH , TT The subset A HH , HT , TH is the same as Head occurred at least once and qualifies as an event The subset B HH is the same as Both heads occur simultaneously and qualifies as an event

12

 Union: Does an outcome belong to A or B A B A B HH , HT , TH Intersection: Does an outcome belong to A and B A B AB HH

Complement:B HH an outcome falls Does outside A A TT

13

Probability

Probability: Definition
The Probability of an event E is defined apriori without actual experimentation as
Number of outcomes favorable to E P( E ) Total number of possible outcomes

Provided all these outcomes are equally likely.

15

 Two coin example:

Probability that at least one heads occurs = P(A) = 3/4 Probability that two heads occurs simultaneously = P(B) =1/4 Probability that two tails occur simultaneously = 1/4 Note that P(S) = 1

Consider a box with n white and m red balls. In this case, there are two elementary outcomes: white ball or red ball. Probability of
n selecting a white ball = nm
16

 Probability of a Sample Point By convention, statisticians have agreed on the following rules.
The probability of any sample point can range from 0 to 1. The sum of probabilities of all sample points in a sample space is equal to 1.

 Example 1
Suppose we conduct a simple statistical experiment. We flip a coin one time. The coin flip can have one of two outcomes - heads or tails. Together, these outcomes represent the sample space of our experiment. Individually, each outcome represents a sample point in the sample space. What is the probability of each sample point?

 Solution:
The sum of probabilities of all the sample points must equal 1. And the probability of getting a head is equal to the probability of getting a tail. Therefore, the probability of each sample point (heads or tails) must be equal to 1/2.

 Example 2: Let's repeat the experiment of Example 1, with a die instead of a coin. If we toss a fair die, what is the probability of each sample point? Solution: For this experiment, the sample space consists of six sample points: {1, 2, 3, 4, 5, 6}. Each sample point has equal probability. And the sum of probabilities of all the sample points must equal 1. Therefore, the probability of each sample point must be equal to 1/6.

 Probability of an Event
The probability of an event is a measure of the likelihood that the event will occur.

Convention, statisticians have agreed on the following rules.

The probability of any event can range from 0 to 1. The probability of event A is the sum of the probabilities of all the sample points in event A. The probability of event A is denoted by P(A). Thus, if event A were very unlikely to occur, then P(A) would be close to 0. And if event A were very likely to occur, then P(A) would be close to 1.

 Example 1: Suppose we draw a card from a deck of playing cards. What is the probability that we draw a spade? Solution: The sample space of this experiment consists of 52 cards, and the probability of each sample point is 1/52. Since there are 13 spades in the deck, the probability of drawing a spade is
P(Spade) = (13)(1/52) = 1/4.

 Example 2: Suppose a coin is flipped 3 times. What is the probability of getting two tails and one head? Solution: For this experiment, the sample space consists of 8 sample points.
S = {TTT, TTH, THT, THH, HTT, HTH, HHT, HHH} Each sample point is equally likely to occur, so the probability of getting any particular sample point is 1/8.

 The event "getting two tails and one head" consists of the following subset of the sample space. A = {TTH, THT, HTT} The probability of Event A is the sum of the probabilities of the sample points in A. Therefore, P(A) = 1/8 + 1/8 + 1/8 = 3/8

Cumulative Probability Distributions

A cumulative probability refers to the probability that the value of a random variable falls within a specified range. If we flip a coin two times, we might ask:
What is the probability that the coin flips would result in one or fewer heads? The answer would be a cumulative probability.

 It would be the probability that the coin flip experiment results in zero heads plus the probability that the experiment results in one head. P(X < 1) = P(X = 0) + P(X = 1) = 0.25 + 0.50 = 0.75 Like a probability distribution, a cumulative probability distribution can be represented by a table or an equation.
In the table below, the cumulative probability refers to the probability than the random variable X is less than or equal to x.

P(x)

0.5

0.25

0.25

Heads

The probability distribution for discrete random variable is called Probability Mass Function (PMF)

Example

Random Variable: Grades of the students

Student ID

10

Grade

Probability Mass Function 2 p 1 P X 1 0.2

10 4 p 2 P X 2 0.4 10 4 p 3 P X 3 0.4 10

PDF

Grade

Uniform Probability Distribution

The simplest probability distribution occurs when all of the values of a random variable occur with equal probability.
This probability distribution is called the uniform distribution.

 Example 1: Suppose a die is tossed. What is the probability that the die will land on 6 ? Solution: When a die is tossed, there are 6 possible outcomes represented by:
S = { 1, 2, 3, 4, 5, 6 }.

Each possible outcome is a random variable (X), and each outcome is equally likely to occur.
Thus, we have a uniform distribution. Therefore, the P(X = 6) = 1/6.

 Example 2: Suppose we repeat the dice tossing experiment described in Example 1. This time, we ask what is the probability that the die will land on a number that is smaller than 5 ? Solution: When a die is tossed, there are 6 possible outcomes represented by:
S = { 1, 2, 3, 4, 5, 6 }. Each possible outcome is equally likely to occur. Thus, we have a uniform distribution.
Conti

 This problem involves a cumulative probability. The probability that the die will land on a number smaller than 5 is equal to:
P( X < 5 ) = P(X = 1) + P(X = 2) + P(X = 3) + P(X = 4) = 1/6 + 1/6 + 1/6 + 1/6 = 2/3

Discrete and Continuous Probability Distributions

If a variable can take on any value between two specified values, it is called a continuous variable; otherwise, it is called a discrete variable.

 Some examples will clarify the difference between discrete and continuous variables.
1. Suppose the fire department mandates that all fire fighters must weigh between 150 and 250 pounds. The weight of a fire fighter would be an example of a continuous variable; since a fire fighter's weight could take on any value between 150 and 250 pounds. 2. Suppose we flip a coin and count the number of heads. The number of heads could be any integer value between 0 and plus infinity. However, it could not be any number between 0 and plus infinity. We could not, for example, get 2.5 heads. Therefore, the number of heads must be a discrete variable.

Just like variables, Probability Distributions can be classified as Discrete or Continuous.

Discrete Probability Distribution

Discrete Probability Distributions

If a random variable is a discrete variable, its probability distribution is called a discrete probability distribution.

 Example: Suppose you flip a coin two times. This simple statistical experiment can have four possible outcomes:
HH, HT, TH, and TT. Now,

let the random variable X represent the number of Heads that result from this experiment.
The random variable X can only take on the values 0, 1, or 2, so it is a discrete random variable.

The probability distribution for this statistical experiment appears as.

The table represents a discrete probability distribution because it relates each value of a discrete random variable with its probability of occurrence.

 Discrete Probability Distributions:

Binomial Probability Distribution

HyperGeometric Probability Distribution

Multinomial Probability Distribution Poisson Probability Distribution

 In many practical situations we are interested in measuring how many times a certain event occurs in a specific time interval or in a specific length or area. For instance:
The number of phone calls received at an exchange or call center in an hour; The number of customers arriving at a toll booth per day; The number of flaws on a length of cable; The number of cars passing using a stretch of road during a day.

The Poisson distribution plays a key role in modeling such problems.

 A Poisson experiment is a statistical experiment that has the following properties:

The experiment results in outcomes that can be classified as successes or failures. The average number of successes () that occurs in a specified region is known. The probability that a success will occur is proportional to the size of the region. The probability that a success will occur in an extremely small region is virtually zero.

 Suppose we are given an interval (this could be time, length, area or volume) and we are interested in the number of successes" in that interval. Assume that the interval can be divided into very small subintervals such that:
The probability of more than one success in any subinterval is zero; The probability of one success in a subinterval is constant for All subintervals and is proportional to its length;
Subintervals are independent of each other.

 The following notation is helpful, when we talk about the Poisson distribution.
e: A constant equal to approximately 2.71828. (Actually, e is the base of the natural logarithm system.) : The mean number of successes that occur in a specified region. x: The actual number of successes that occur in a specified region. P(x; ): The Poisson probability that exactly x successes occur in a Poisson experiment, when the mean number of successes is .

 A Poisson random variable is the number of successes that result from a Poisson experiment. The probability distribution of a Poisson random variable is called a Poisson distribution. The Poisson distribution has the following properties:
The mean of the distribution is equal to . The variance is also equal to

 Given the mean number of successes () that occur in a specified region, we can compute the Poisson probability based on the following formula:

 The major difference between Poisson and Binomial distributions is that the
Poisson does not have a fixed number of trials. Instead, it uses the fixed interval of time or space in which the number of successes is recorded

Cumulative Poisson Probability

A cumulative Poisson probability refers to the probability that the Poisson random variable is greater than some specified lower limit and less than some specified upper limit.

 Example 1: Suppose the average number of lions seen on a 1-day safari is 5. What is the probability that tourists will see fewer than four lions on the next 1-day safari? Solution: This is a Poisson experiment in which we know the following:
= 5; since 5 lions are seen per safari, on average. x = 0, 1, 2, or 3; since we want to find the likelihood that tourists will see fewer than 4 lions; that is, we want the probability that they will see 0, 1, 2, or 3 lions. e = 2.71828; since e is a constant equal to approximately 2.71828.

 To solve this problem, we need to find the probability that tourists will see 0, 1, 2, or 3 lions. Thus, we need to calculate the sum of four probabilities: P(0; 5) + P(1; 5) + P(2; 5) + P(3; 5). To compute this sum, we use the Poisson formula:

 P(x < 3, 5) = P(0; 5) + P(1; 5) + P(2; 5) + P(3; 5) P(x < 3, 5) = [ (e-5)(50) / 0! ] + [ (e-5)(51) / 1! ] + [ (e-5)(52) / 2! ] + [ (e-5)(53) / 3! ] P(x < 3, 5) = [ (0.006738)(1) / 1 ] + [ (0.006738)(5) / 1 ] + [ (0.006738)(25) / 2 ] + [ (0.006738)(125) / 6 ] P(x < 3, 5) = [ 0.0067 ] + [ 0.03369 ] + [ 0.084224 ] + [ 0.140375 ] P(x < 3, 5) = 0.2650

Thus, the probability of seeing at no more than 3 lions is 0.2650.

 Example 2: The average number of homes sold by the Acme Realty company is 2 homes per day. What is the probability that exactly 3 homes will be sold tomorrow? Solution: This is a Poisson experiment in which we know the following:
= 2; since 2 homes are sold per day, on average. x = 3; since we want to find the likelihood that 3 homes will be sold tomorrow. e = 2.71828; since e is a constant equal to approximately 2.71828.

 We plug these values into the Poisson formula as follows:

P(x; ) = (e-) (x) / x! P(3; 2) = (2.71828-2) (23) / 3! P(3; 2) = (0.13534) (8) / 6 P(3; 2) = 0.180

Thus, the probability of selling 3 homes tomorrow is 0.180 .

 Example 3: The number of flaws in a fiber optic cable follows a Poisson distribution. The average number of flaws in 50m of cable is 1.2.
i. What is the probability of exactly three flaws in 150m of cable? ii. What is the probability of at least two flaws in 100m of cable? iii.What is the probability of exactly one flaw in the first 50m of cable and exactly one flaw in the second 50m of cable?`

 Example 4: The number of visitors to a webserver per minute follows a Poisson distribution. If the average number of visitors per minute is 4, What is the probability that:
i. There are two or fewer visitors in one minute?; ii. There are exactly two visitors in 30 seconds?.

 The previous example is a standard example of a queuing process. These are very important in many applications in contemporary communications engineering .Other examples of this type include
The number of calls to an exchange The arrival of customers at a service desk or The arrival of cars at a toll booth.

Continuous Probability Distribution

Continuous Probability Distributions

If a random variable is a continuous variable, its probability distribution is called a continuous probability distribution. Most often, the equation used to describe a continuous probability distribution is called a probability density function. Sometimes, it is referred to as a density function, a PDF, or a pdf.

 A continuous probability distribution differs from a discrete probability distribution in several ways.
The probability that a continuous random variable will assume a particular value is zero. As a result, a continuous probability distribution cannot be expressed in tabular form. Instead, an equation or formula is used to describe a continuous probability distribution.

 For a continuous probability distribution, the density function has the following properties:
Since the continuous random variable is defined over a continuous range of values (called the domain of the variable), the graph of the density function will also be continuous over that range. The area bounded by the curve of the density function and the x-axis is equal to 1, when computed over the domain of the variable. The probability that a random variable assumes a value between a and b is equal to the area under the density function bounded by a and b.

 For Example,
consider the probability density function shown in the graph above. Suppose we wanted to know the probability that the random variable X was less than or equal to a. The probability that X is less than or equal to a is equal to the area under the curve bounded by a and minus infinity - as indicated by the shaded area.

 Note:
The shaded area in the graph represents the probability that the random variable X is less than or equal to a. This is a cumulative probability. However, the probability that X is exactly equal to a would be zero. A continuous random variable can take on an infinite number of values. The probability that it will equal a specific value (such as a) is always zero.

 Continuous Probability Distributions:

Normal Probability Distribution Exponential Probability Distribution Student's t Distribution Chi-square Distribution F Distribution

Normal Probability Distribution

Normal probability distribution, also called Gaussian distribution refers to a family of distributions that are bell shaped. These are symmetric in nature and peak at the mean, with the probability distribution decreasing away before and after this mean smoothly, as shown in the figure below.

 The figure also shows a family of curves with different peaks centered about the same mean, which differ in their spread and height.

 Normal distribution occurs very frequently in statistics, economics, natural and social sciences and can be used to approximate many distributions occurring in nature and in the manmade world.
For example, the height of all people of a particular race, the length of all dogs of a particular breed, IQ, memory and reading skills of people in a general population and income distribution in an economy all approximately follow the normal probability distribution shaped like a bell curve.

 The theory of normal distribution also finds use in advances sciences like astronomy, photonics and quantum mechanics. The normal distribution can be characterized by the mean and standard deviation. The mean determines where the peak occurs, which is at 0 in our figure for all the curves. The standard deviation is a measure of the spread of the normal probability distribution, which can be seen as differing widths of the bell curves in our figure.

 THE FORMULA
The mean is generally represented by and the standard deviation by . For a perfect normal distribution, the mean, median and mode are all equal. The normal distribution function can be written in terms of the mean and standard deviation as follows:

 From the above formula for normal distribution, it can be inferred that about 68% of all values lie within one standard deviation from the mean; 95.4% of all values lie within two standard deviations from the mean and 99.7% of all values lie within three standard deviations from the mean. From the basic bell curve, there can be many special cases derived that become meaningful under different situations.

 A basic study of the normal distribution therefore is necessary before a meaningful study can be made into these special cases. This concept can be extended to 3-D normal distributions as well, which are used for more advanced applications.

Generally the exponential distribution describes waiting time between Poisson occurrences

1. You have observed that the number of hits to your web site follow a Poisson distribution at a rate of 2 per day.
Let T be the time (in days) between hits.

2. You observe the number of calls that arrive each day over a period of a year, and note that the arrivals follow a Poisson distribution with an average of 3 per day.
Let T be the waiting time between calls.

3. Records show that job submissions to a busy computer center have a Poisson distribution with an average of 4 per minute.
Let T be the time in minutes between submissions.

4. Records indicate that messages arrive to a computer server following a Poisson distribution at the rate of 6 per hour.
Let T be the time in hours that elapses between messages.

 Let

T = time

that elapses after a Poisson event.

P(T > t) = probability that no event occurred in the time interval of length t. The probability that no Poisson event occurred in the time interval [0, t]:
where is the average Poisson occurrence rate in a unit time interval.

 Examples:
Time between telephone calls Time between machine breakdowns. Time between successive job arrivals at a computing center. Time between customer arrival at some shop