MTH 263 Probability and Random Variables

Lecture 4, Chapter 2 Dr. Sobia Baig Electrical Engineering Department COMSATS Institute of Information Technology, Lahore

Contents
Conditional probability Bayes theorem Independent Events

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

Basic Concepts of Probability Theory

set theory is used to specify the sample space and the events of a random experiment the axioms of probability specify rules for computing the probabilities of events the notion of conditional probability allows us to determine how partial information about the outcome of an experiment affects the probabilities of events Conditional probability also allows us to formulate the notion of independence of events and of experiments. We consider sequential random experiments that consist of performing a sequence of simple random subexperiments.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

Conditional Probability
we are interested in determining whether two events A and B, are related in the sense that knowledge about the occurrence of one, say B, alters the likelihood of occurrence of the other A. This requires that we find the conditional probability, of event A given that event B has occurred. The conditional probability is defined by

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

Computing Conditional Probability

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

Conditional Probability
For instance, one picks a card at random from a 52-card deck. One knows that the card is black. What is the probability that it is the ace of clubs? The sensible answer is that if one only knows that the card is black, then that card is equally likely to be any one of the 26 black cards. Therefore, the probability that it is the ace of clubs is 1/26.

Similarly, given that the card is black, the probability that it is an ace is 2/26, because there are 2 black aces (spades and clubs)

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

Examples of Conditional Probability

A digital communication channel has an error rate of one bit per every thousand transmitted. Errors are rare, but when they occur, they tend to occur in bursts that affect many consecutive bits. If a single bit is transmitted, we might model the probability of an error as 1/1000. However, if the previous bit was in error, because of the bursts, we might believe that the probability that the next bit is in error is greater than 1/1000.

In a thin film manufacturing process, the proportion of parts that are not acceptable is 2%. However, the process is sensitive to contamination problems that can increase the rate of parts that are not acceptable. If we knew that during a particular shift there were problems with the filters used to control contamination, we would assess the probability of a part being unacceptable as higher than 2%.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

10

Example
An urn contains two black balls and three white balls. Two balls are selected at random from the urn without replacement and the sequence of colors is noted. Find the probability that both balls are black.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

11

 Let B1 and B2 be the events that the outcome is a black ball in the first and second draw, respectively.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

12

Binary Communication System

Many communication systems can be modeled in the following way. First, the user inputs a 0 or a 1 into the system, and a corresponding signal is transmitted. Second, the receiver makes a decision about what was the input to the system, based on the signal it received. Suppose that the user sends 0s with probability (1-p)and 1s with probability p,

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

13

Total Probability Rule

For any event B, we can write B as the union of the part of B in A and the part of B in A.

Because A and A are mutually exclusive, and A intersection B and A intersection B are mutually exclusive. Therefore, from the probability of the union of mutually exclusive events and the Multiplication Rule total probability rule is obtained.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

14

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

15

Total Probability Rule- Generalized to n Partitions

Let B1, B2, Bn be mutually exclusive events whose union equals the sample space S We refer to these sets as a partition of S. Any event A can be represented as the union of mutually exclusive events

By Corollary 4,

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

16

Example-Total Probability Rule

in Example 2.25, find the probability of the event that the second ball is white.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

17

Example
A manufacturing process produces a mix of good memory chips and bad memory chips. The lifetime of good chips follows the exponential law introduced in Example 2.13, with a rate of failure The lifetime of bad chips also follows the exponential law, but the rate of failure is 1000 Suppose that the fraction of good chips is 1-p and of bad chips, p. Find the probability that a randomly selected chip is still functioning after t seconds.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

18

Example Solution

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

19

Bayes Rule
We might know one conditional probability but would like to calculate a different one. From the definition of conditional probability, Therefore, it can be stated,

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

20

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

21

When to Apply Bayes Rule

We have some random experiment in which the events of interest form a partition. The a priori probabilities of these events P[Bj], are the probabilities of the events before the experiment is performed. Now suppose that the experiment is performed, and we are informed that event A occurred; the a posteriori probabilities are the probabilities of the events in the partition P[Bj ], given this additional information.
Probability and Random Variables, Lecture 4, by Dr. Sobia Baig 22

Binary Communication System

In the binary communication system in Example 2.26, find which input is more probable given that the receiver has output a 1. Assume that, a priori, the input is equally likely to be 0 or 1

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

23

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

24

Example- Bayes Rule

Let C be the event chip still functioning after t seconds, and let G be the event chip is good, and B be the event chip is bad. The problem requires that we find the value of t for which

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

25

Example Quality Control

Consider the memory chips discussed in Example 2.28. a fraction p of the chips are bad and tend to fail much more quickly than good chips. Suppose that in order to weed out the bad chips, every chip is tested for t seconds prior to leaving the factory. The chips that fail are discarded and the remaining chips are sent out to customers. Find the value of t for which 99% of the chips sent out to customers are good.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

26

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

27

INDEPENDENCE OF EVENTS
If knowledge of the occurrence of an event B does not alter the probability of some other event A, then it would be natural to say that event A is independent of B. In terms of probabilities this situation occurs when

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

28

 Two events, A and B are said to be independent,

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

29

Example-Independent Events
A ball is selected from an urn containing two black balls, numbered 1 and 2, and two white balls, numbered 3 and 4. Let the events A, B, and C be defined as follows:

Are events A and B independent? Are events A and C independent?

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

30

 First, consider events A and B. The probabilities are

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

31

Example Solution
First, consider events A and B. The probabilities are

and the events A and B are independent.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig 32

 These two equations imply that because the proportion of outcomes in S that lead to the occurrence of A is equal to the proportion of outcomes in B that lead to A. Thus knowledge of the occurrence of B does not alter the probability of the occurrence of A.
Probability and Random Variables, Lecture 4, by Dr. Sobia Baig 33

Mutually Exclusive & Independence

In general if two events have nonzero probability and are mutually exclusive, then they cannot be independent. For example, suppose they were independent and mutually exclusive; then which implies that at least one of the events must have zero probability.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

34

Mutually Exclusive & Independence

If two events A and B are independent, then P[A and B] = Pr[A]Pr[B]; that is, the probability that both A and B occur is equal to the probability that A occurs times the probability that B occurs. If A and B are mutually exclusive, then P[A and B] = 0; that is, the probability that both A and B occur is zero. Clearly, if A and B are nontrivial events (Pr[A] and Pr[B] are nonzero), then they cannot be both independent and mutually exclusive.
Probability and Random Variables, Lecture 4, by Dr. Sobia Baig 35

Mutually Exclusive & Independence

Consider a fair coin and a fair six-sided die. Let event A be obtaining heads, and event B be rolling a 6. Then we can reasonably assume that events A and B are independent, because the outcome of one does not affect the outcome of the other The probability that both A and B occur is P[A and B] = P[A]P[B] = (1/2)(1/6) = 1/12. Since this value is not zero, then events A and B cannot be mutually exclusive.
Probability and Random Variables, Lecture 4, by Dr. Sobia Baig 36

Mutually Exclusive & Independence

Consider a fair six-sided die as before, only in addition to the numbers 1 through 6 on each face, we have the property that the even-numbered faces are colored red, and the odd-numbered faces are colored green. Let event A be rolling a green face, and event B be rolling a 6. Then P[A] = 1/2 P[B] = 1/6 as in our previous example. But it is obvious that events A and B cannot simultaneously occur, since rolling a 6 means the face is red, and rolling a green face means the number showing is odd. Therefore Pr[A and B] = 0. Therefore, we see that a mutually exclusive pair of nontrivial events are also necessarily dependent events. if A and B are mutually exclusive, then if A occurs, then B cannot also occur; and vice versa. This stands in contrast to saying the outcome of A does not affect the outcome of B, which is independence of events.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

37

Example
Consider the experiment discussed in Example 2.32 where two numbers are selected at random from the unit interval. Let the events B, D, and F be defined as follows:

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

38

 It can be easily verified that any pair of these events is independent:

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

39

n Independent Events
In order for a set of n events to be independent, the probability of an event should be unchanged when we are given the joint occurrence of any subset of the other events. This requirement naturally leads to the following definition of independence.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

40

Example 2.35 System Reliability

A system consists of a controller and three peripheral units. The system is said to be up if the controller and at least two of the peripherals are functioning. Find the probability that the system is up, assuming that all components fail independently.
Probability and Random Variables, Lecture 4, by Dr. Sobia Baig 41

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

42

Sequential Experiments
Many random experiments can be viewed as sequential experiments that consist of a sequence of simpler subexperiments These subexperiments may or may not be independent.

Sequences of Independent Experiments

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

43

 Let be events such that concerns only the outcome of the kth subexperiment. If the subexperiments are independent, then it is reasonable to assume that the above events are independent. Thus

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

44

Example

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

45

Example

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

46

Summary
A conditional probability quantifies the effect of partial knowledge about the outcome of an experiment on the probabilities of events. It is particularly useful in sequential experiments where the outcomes of subexperiments constitute the partial knowledge.

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

47