Markov Analysis

Overview
• A probabilistic decision analysis
• Does not provide a recommended decision
• Provides probabilistic information about a
decision situation that can aid the DM
• Applicable to systems that exhibit probabilistic
movement from one state to another, over time
– Probability that a machine will be running one day and
broken down the next
– Probability that a customer will change her
department store to the next, called brand switching
 Brand Switching Example
• Customers are usually royal to a particular brand or store, or
supplier
• Two gas stations in a community , P and N
• Study indicates customers are not royal to either one
• Willing to change based on advertisement factors
• If a customer bought gas from P in any given month, there was 0.6
probability that the customer would buy from P and 0.4 probability
from N the next month
• If a customer traded with N in any given month, there was 0.8
probability that the customer would buy from N and 0.2 probability
from N the next month
Next Month
This month P N
P 0.6 0.4
N 0.8 0.2
 Terminology
• Gas station that a customer trades at a
given month is called state of the system
(two states of system)
• Probabilities of various states are called
transition probabilities
– Transition probability sum to one
– Probabilities apply to all participants
– Probabilities are constant over time
– States are independent over time
What Information MA Provides?
• Answers the probability of being in a state at some future time period
• Determining the probability that a customer would trade with them in
month 3 given that the customer trades with them this month
• Use the following decision tree 1
– The probability of a customer’s purchasing gas from P in month 3 given
that the customer traded with P in month1 =0.36+0.08=0.44
– The probability of a customer’s purchasing gas from N in month 3 given
that the customer traded with N in month1 =0.24+0.32=0.56
• Use the following decision tree 1
– Given that N is the starting state in month1, the probability of a
customer’s purchasing gas from N in month3: 0.08+0.64=0.72
– Given that N is the starting state in month1, the probability of a
customer’s purchasing gas from P in month3: 0.12+0.16=0.28
 Month 3-Result
Month 3
This month P N
P 0.44 0.56
N 0.28 0.72
• Easy for month 3, but not for month 10 or 15
• Follow the notes in class