Department Of Electronics & Communication Engineering

Unit 1. Probability Theory The basic probability theory were developed in the context of practical or conceptual experiments such as tossing of coins, games of chance, throwing of dice, choosing objects, distribution of cards, shooting a target, etc. The outcome of these experiments is not known for certain. For example when a dice is thrown, we can get any one of the numbers, from 1 to 6, and not a particular number always. All activities based on prediction such as planning for the future, weather forecasting, stock investments, estimating the risks of medical treatments, etc.,has certain amount of uncertainty associated with it. Decision making under certainty does not carry its full credibility unless the amount of uncertainty is specified in a meaningful way. The numerical quantity that we assign to a given situation of uncertainty is called probability.

Definitions and basic concepts: The following definitions and terms used in studying the theory of probability. Random experiment: Random experiment is one whose results depend on chance, that is the result cannot be predicted. Tossing of coins, throwing of dice are some example of random experiments. Trail: Performing a random experiment is called a trail. Outcomes: The results of a random experiment are called its outcomes. When two coins are tossed the possible outcomes are HH, TT, HT and TH. Event: An outcome or a combination of outcomes of random experiment is called an event. For example tossing of a coin is a random experiment and getting a head or tail is an event. Sample space: Each conceivable outcome of an experiment is called a sample point. The totality of all sample points is called a sample space and is denoted by S. For example when a coin is tossed, the sample space is S={H, T}. H and T are the sample points of the sample space. Another example is when a die is rolled, the sample space is S= {1,2,3,4,5,6}

Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

Discrete sample space: If the elements of the sample space are discrete in nature, For example when a coin is tossed, the sample space is S= {H, T}. H and T are discrete sample points of the sample space. Another example is When a die is rolled, the sample space is S= {1,2,3,4,5,6} Here 1,2,3,4,5,6 are discrete sample points. Continuous sample space: If the elements of the sample space are continuous in nature, For example the set of numbers from 0 to 10. The elements are infinite and continuous. Discrete events: Any outcome of discrete sample space is discrete event Continuous event: Any outcome of continuous sample space is continuous event. An event is certain, if it always appears as an outcome. For example, Sun rises in the morning. An event which never occurs is an impossible event. For example, a null set is an event with no elements. Equally likely events: Two or more events are said to be equally likely if each one of them has an equal chance of occurring. For example in tossing of a coin, the event of getting head and event of getting tail are equally likely events. Another example, when a die is thrown, all the faces are shown are equally likely events. Mutually exclusive events: When two events are said to be mutually exclusive, when the occurrence of any one event excludes the occurrence of another event. That is mutually exclusive events cannot occur simultaneously. For example, when a coin is tossed, either head or tail will come up. Therefore the occurrence of head completely excludes the occurrence of the tail. Thus getting head or tail in tossing of a coin is a mutually exclusive event. Exhaustive events: Events are said to be exhaustive when their totality includes all the possible outcomes of a random experiment. For example while throwing a die, the possible outcomes are {1,2,3,4,5,6} and hence the number of cases are 6. Complementary events: The event A occurs and the event A does not occur are complementary events to each other.

Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

The event A which does not occur is denoted by A or Ac or . for example in throwing a die the event of getting odd numbers is {1,3,5} and getting even numbers is {2,4,6}. These two events are mutually exclusive and complement to each other. Set operations: Union: The union of two sets A and B is the set of all elements belonging to A or B or both. It is denoted by A U B. Intersection: The intersection of two sets A and B is the set of all elements common to both A and B. It is denoted by A B.

Probability definitions: Classical definition of probability: If the probability of an event can be calculated even before the actual happening of the event, that is , even before conducting the experiment, it is called mathematical probability or classical probability or priori probability ( because we can state the answer in advance, without tossing of coins or without rolling the dice). The probability P(A) of an event is the ratio of number of favorable outcomes of event A to the total number of possible outcomes, provided that all the outcomes are equally likely(equally probable).
Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

This is widely used but it cannot be applied under the following situations. 1. If the total number of outcomes are infinity 2. If each and every outcome is not equally likely. Some of the drawbacks are removed in the definition given below. Probability introduced through Relative frequency: Here the probability of an event is determined only after the actual happening of the event. If an event occurs m times out n, its relative frequency is m/n. In the limiting case, when n becomes sufficiently large it corresponds to a number which is called the probability of the event. In symbol P(A) = Limit n tends to infinity (m/n).

Axiomatic approach of probability: The modern approach of probability is purely axiomatic and it is based on set theory. Axioms of probability: Let S be the sample space and A be an event in S and P(A) is the probability of satisfying the following axioms.
1. The probability of any event is always a non negative real number ie P(A) 0. 2. The probability of sample space is 1 ie P(S) = 1. Because S includes all possible events of an experiment. 3. If A1, A2, are a sequence of mutually exclusive events in S the P(A1 U A2 U A3..) = P(A1) + P(A2) +P(A3)+

Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

Properties of Probability: 1. 0 P(A) 1. 2. P( ) = 1- P(A) 3. P(A) P(B) if A is proper subset of B. 4. P (A U B) = P (A) + P(B) P(AB ) 5. P( B) = P(B) P(AB) 6. P(A1 U A2 U A3..) = P(A1) + P(A2) +P(A3)+ If A1, A2 are mutually exclusive events in S 7. For null set P() =0 and for universal set in S, P(S) =1 8. For equally likely of n events in S, the probability of each event is 1/n.

Proofs: 1. If two events A and B are not mutually exclusive, the probability of event that either A or B or both occur is given as P (A U B) = P (A) + P(B) P(AB )

Let us take an experiment with a sample space S of N sample points. Then by the definition of probability and from venn diagram P(AUB) =

=
=

= P (A) + P(B) P(AB ) Note: If A and B are mutually exclusive events, P(AB ) =0 then P (A U B) = P (A) + P(B). Which is called Addition theorem.
Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

Independent Events: Events are said to be independent if the occurrence of one does not affect (influence) the others. In the experiment of tossing a coin the occurrence of the event head in the first toss is independent of the occurrence of the event head in the second toss, third toss and subsequent tosses. Similarly if a bag contains 5 white and 7 red balls and then two balls are drawn one by one in such a way that the first ball is replaced before the second one is drawn. In this situation, two events the first ball is white and second ball is red will be independent since the composition of the balls in the bag remains unchanged before a second draw is made.

Dependent events: If the occurrence of one event influences the occurrence of other, then the second event is said to be dependent on the first. In the above example if we dont replace the first ball drawn this will change the composition of balls in the bag while making the second draw and therefore the event of drawing a red ball in the second will depend on event occurring in the first draw. Similarly, if a person draw a card from a full pack and does not replace it, the result of draw made afterwards will be dependent on the first draw.

Multiplication Theorem for independent events: If two events A and B are independent, the probability that both of them occur is equal to product of their individual probabilities. ie P(

) = P(A) P(B)
) = P(A)P(B)P(C)..

Note1: If A B C.are independent events then P(

Note2: If A and B are independent, then the complements of A and B are also independent ie

P(

) = P(A) P(B)

Multiplication Theorem for dependent events: If A and B are two dependent events, i.e the occurrence of one event is affected by the occurrence of the other event, then the probability that both A and B will occur is

P(

) = P(A) P(B/A)

Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

Note: In the case of three events A, B, C, P(A

) = P(A) P(B/A)P(C/AB) ie the

probability of occurrence of A, B and C is equal to the probability of A times the probability of B given that A has occurred, times the probability of C given that both A and B have occurred.

Conditional Probability
Let A and B be any two events. The probability of the event A given that the event B has already occured is known as conditional probability of A given B, denoted by P(A|B), and is defined as P(A|B) = provided P(B) 0 )/P(A)

Note: Similarly P(B|A) = P(B

Probability of any event must satisfy all the three axioms defined earlier. i) Since P(A) and P(B) are non negative real numbers, P(A real number. P(A|B) = ii) iii) P(S|B) = B) is also non negative

0 =1 lly P(S|A) = = =1

If A, B and C are mutually exclusive events, then A C and B C are also mutually exclusive events. P(A U B |C) =

=
= P(A|C) + P(B|C)
Hence P(A|B) is satisfied all axioms of probability.

Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

Total Probability Consider a sample space S that has N mutually exclusive events Bi i=1 2 3N such that Bm Bn ={} for m n=1 2...N. The probability of any event A, defined on this sample space can be expressed in terms of the conditional probabilities of events Bi . This probability is known as the total probability of event A. Mathematically P(A) = |

Bayes theorem
The concept of conditional probability discussed earlier takes into account information about the occurrence of one event to predict the probability of another event. This concept can be extended to revise probabilities based on new information and to determine the probability that a particular effect was due to specific cause. The procedure for revising these probabilities is known as Bayes theorem. Statement: Let B1, B2 Bn be a set of n mutually exclusive and collectively exhaustive events and P(B1), P(B2)..P(Bn) are their corresponding probabilities. If A is another event such that P(A) is not zero and the priori probabilities P(A|Bi) i=1 2 3 4 are also known. Then

P(Bi | A) Proof:

| |

The proof can be derived from the total probability and the definition of conditional probabilities. We know that the conditional probability P(Bi | A) = Also =
| | | | | | |

if P(A) 0
| |

From total probability theorem P(A) = =

P(Bi | A) = =

Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

Bernoulli (Binomial) trails

Consider any experiment in which there are only two possible outcomes. Assume that these events are statistically independent for every trial. Let an event occur on any given trial with probability P(A) = p. If the experiment is repeated for N trials, called Bernoulli trials, then the probability of event A occurring exactly K times in N independent trials (K N) is given by P(A occurs exactly K times) = NCK pK (1-p)N-K Proof: It is given that the probability of event A is P(A) = p Since there are only two events in the sample space, P( ) = 1- p Assume that the trials are statistically independent. In N trials, let event A occurs only K times Using the multiplication theorem for the combined sample space, the resultant probability for one of the sequence of occurrence is = P(A)*P(A)*..P(A)* P( )* P( )* .. *P( ) [ P(A) occurs K times and P( ) occurs N-K times] = p*p*..*p*(1-p)*(1-p)**(1-p) = pK (1-p)N-K We know that the number of combinations of a sequence in any order can be determined from the binomial coefficients. For N trials, the number of combinations, where the event A occurs exactly K times is NCK (similar to selecting K things from N things) Therefore the resultant probability is P(A occurs exactly K times) = NCK pK (1-p)N-K
1) Determine the probability of the card being either red or a king when one card is drawn from a regular deck of 52 cards. Answer: Given a deck of 52 cards. Let the events A = Draw a red card, B= Draw a King Card

P(either Red or a king card) = P(AUB)= P(A)+P(B)-P(AB) Since there 26 red cards P(A)= 26/52 {Out of 52 cards, 26 are red colored and 26 are black colored cards. Out of 26 red colored cards, 13 are diamond cards and 13 are heart cards. Out of 26 black colored cards, 13 are spade cards and 13 are club cards.} Since there are 4 king cards ( 1 from diamond set, 1 from heart set, 1 from spade set and 1 from club set), P(B)=4/52

Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

Since there are two red king cards (one is the king of diamonds and the other is the king of hearts) P(AB)= 2/52 P(either Red or a king card) = P(AUB)= P(A)+P(B)-P(AB) = 26/52+4/52-2/52 = 28/52 2) In a box there are 100 resistors having resistance and tolerance as shown in Table. Let a resistor be

selected from the box and assume each resistor has the same likelihood of being chosen. For the three Events; A as draw a 470 resistor, B as draw a resistor with 5 % tolerance and C as draw a 100 resistor . Calculate individual , joint and conditional probabilities. Number of resistors in a box having resistance and tolerance. Resistance() 22 47 100 Total Tolerance 5% 10 28 28 62 10% 14 16 8 38 Total 24 44 32 100

Answer: Event A Drawing a 47 ohm resistor Event B Drawing a resistor with 5% tolerance Event C Drawing a 100 resistor Total number of resistors =100 P(A) = 44/100, P(B)= 62/100, P(C) = 32/100 Joint Probabilities: The joint event AB is Drawing a 47 ohm resistor with 5% tolerance. There are 28 such resistors P(AB) = 28/100 The joint event AC is 47 ohm resistor with a 100 ohm resistor This is a disjoint event, P(AC) =0 The joint event BC is Drawing a 100 Ohm resistor with 5% tolerance. There are 24 such resistors P(BC) =24/100 The joint event ABC is also disjoint P(ABC) =0 Conditional Probabilities: P(A|B) = P(B|A) = P(B|C) =

P(C|B) = P(A|C)= 0

Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

2.

A pack contains 4 white and 2 green pencils; another contains 3 white and 5 green pencils. If one pencil is drawn from each pack, find the probability that i) both are white. Ii) One is white and another is green

Answer: Let A is the event of drawing white pencil from 1st pack and B is the event of drawing a white pencil from 2 nd pack Since drawing a white pencil from one pack does not affect the other one. Hence A and B are independents. P(A )= P(A)P(B)

= *
P(one white and another is green) = P(one white)* P(another green) = 4/6* 5/8 4. a) Urn I has 2 white and 3 black balls. Urn II has 4 white and 1 black balls. Urn III has 3 white and 4 black balls. An Urn is selected at random and ball drawn at random is found to be white. Find the probability that Urn I was selected? Answer:

P(Bi | A) =

| | | | |

Assume that Selecting Urn1, Urn2 and Urn3 are equally likely events. P(Urn1) =P(Urn2)= P(Urn3)=1/3 | | | = 2/5 ) = 4/5 = 3/7
| |

P(Urn1 |White) =

5. a) When two dice are thrown determine the probabilities for the following events. (i)A = {Sum = 7} Answer: (i)Outcomes favorable to A= {Sum=7} are { (1,6),(6,1),(2,5), (5,2), (3,4),(4,3)} Total number of outcomes are =36 (ii) B = {8< Sum 11} (iii) C= {10 < Sum} and determine (IV) P (B

Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

P(A = {Sum = 7}) = 6/36 (ii)Outcomes favorable to B= {8<Sum11} are { (4,4),(5,3),(3,5), (6,2), (2,6),(4,5),(5,4),(6,3),(3,6),(5,5),(4,6),(6,4), (6,5),(5,6)} Total number of outcomes are =36 P( B= {8<Sum11}) = 14/36 iii) Outcomes favorable to C= {Sum>10} are { (5,6),(6,5),(6,6)} Total number of outcomes are =36 P(C = {Sum >10}) = 3/36 iv)Outcomes favorable to B and C are { (6,5),(5,6)} Total number of outcomes are =36 P( B and C) = 2/36 6. a)One card is selected from an ordinary 52 card deck with an event A as select a king, B as select a Jack or Queen and C as select a Heart. Determine whether A, B and C are independent by pairs. Answer: Given a deck of 52 cards Let the event A is drawing a king card B is drawing a jack or queen card C is drawing a heart card P(A) = 4/52 since there 4 king cards P(B)= 8/52 since there are 4 jack and 4 queen cards P(C)=13/52 since there are 13 heart cards a) Now P(A)P(B)=4/52*8/52 But P(A P(A b) ) = 0 since A and B are not having cards in common. ) P(A)P(B) hence A and B are not independent.

Now P(B)P(C)=2/13*13/52 =2/52 But P(B P(B ) = 2/52 ) P(B)P(C) hence B and C are independent by pair.

c)

Now P(A)P(C)=1/13*13/52 =1/52 But P(A P(A ) = 1/52 )= P(A)P(C) hence Aand C are statistically independent.

Combinations: A combination is a selection of objects without considering the order of arrangements. For example, out of three things A, B and C, we have to select two things. This can be selected in three different ways as follows: AB Here the number of combinations from 3 different things taken 2 at a time is 3. This is written symbolically as 3C2 = 3 ways In a given n events, the number of ways to get r events, where r N, is called combinations. BC CA. Here the selection of the object AB and BA are one and the same. Here the order of arrangement is not considered in combination.

Prepared by Venkata Satish.N

Department Of Electronics & Communication Engineering

It is given by N C r = Here the sequence order of r events is not considered.

Here the order of sequence is not considered. 7a. A letter is Known to have come from either TATANAGAR or CALCUTTA. On the envelope, just two consecutive lettersTA s are visible. Find the Probability that the letter has come from CALCUTTA. Solution:Let the events A be letter from TATANAGAR, B be letter from CALCUTTA. Assume that both events are equally likely events P(A)= P(B)=1/2 Let the event C be two consecutive letters TA The probability of the letters TA being selected from TATANAGAR is TA TA N A G A R The Probability of the letter TA being selected from CALCUTTA is C A L C U T TA Using Bayess Theorem The probability of the letter coming from CALCUTTA when the event C selected is
| | | |

P(C|A) = 2/7 P(C|B) = 1/7

P(B|C) =

Prepared by Venkata Satish.N