You are on page 1of 7

Markov Chains Applied to Marketing

Author(s): George P. H. Styan, Harry Smith and Jr.


Source: Journal of Marketing Research, Vol. 1, No. 1 (Feb., 1964), pp. 50-55
Published by: American Marketing Association
Stable URL: https://www.jstor.org/stable/3150320
Accessed: 12-10-2018 09:55 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

American Marketing Association is collaborating with JSTOR to digitize, preserve and


extend access to Journal of Marketing Research

This content downloaded from 14.139.157.21 on Fri, 12 Oct 2018 09:55:48 UTC
All use subject to https://about.jstor.org/terms
50

Markov Chains Applied to Marketing*


GEORGE P. H. STYAN
and
HARRY SMITH, JR.t

The classical approach to market behavioral analysis rarely uses data provided by the transitional, or
switching, habits of the consumer. In this article, the authors have taken types of laundry powders pur-
chased by a housewife to define the state space of a Markov chain. Using this model future purchase
behavior is predicted, and statistical inferences on the switching habits are made.

In recent years there has been a great deal of interest of four mutually exclusive and wholly exhaustiv
in the applications of stochastic processes in industry.gories of buying habits. These were
In particular, Markov chain models have been tried in (1) family buying detergent only,
quite a few areas. The number of papers appearing in (2) family buying soap powder only,
both advertising and market research journals on this (3) family buying both detergent and soap po
subject has been on the increase in the past few years together,
[3]. This paper is an illustration of certain concepts ofand (4) family buying no laundry powder at all.
discrete time parameter finite Markov chains, a Markov
chain being a stochastic process in which the future These four categories will be referred to as "states" in
state of the system is dependent only on the present the Markov analysis. A summary of purchase informa-
tion is shown in Table 1. One can see that over the 26-
state and is independent of past history.
Naturally, one of the main concerns of any producer week period, the purchases were divided up on the
of consumer goods is to get people to use his product.average as follows:
Table I
Also important, however, is to get people repurchasing
DISTRIBUTION OF HOUSEWIVES' PURCHASESa
his product once they have used it. That is, the producer
wants his customers to be loyal customers. However, State
complete loyalty is seldom found, and thus it is useful Week Period
for the producer to have information on the switching Ending (k) 1 2 3 4
habits of buyers in the commodity market. 1957

This paper restricts itself to a discussion of laundryJanuary 5 I 22 44 9 25


January12 2 18 44 8 30
soap powder and detergent buying by a small panel of
January 19 3 18 38 11 33
housewives over a six-month period of time. The loyalty January 26 4 22 44 10 24
and switching probabilities of these women are ana- February 2 5 24 39 II 26
lyzed, and the limiting distribution of the Markov chain February 9 6 25 35 9 31
for this particular study is discussed. February 16 7 28 37 9 26
February 23 8 28 39 9 24
March 2 9 20 41 9 30
THE SURVEY March 9 10 19 34 5 42
March 16 II 23 40 2 35
A survey1 of a hundred households or families
March in 23 12 21 37 5 37
Leeds, Yorkshire, England, was made between January 30 13 23 40 5 32
March
April
and June, 1957. The housewives in this survey reported 6 14 16 45 3 36
April 13 15 24 41 5 30
their purchases of laundry cleaning products
April in a20 16 19 39 4 38
weekly diary. This information was tabulated and con-27 17 18 44 6 32
April
May 4 18 21 50 6 23
densed for the purpose of this paper into a consideration
May 11 19 22 48 4 26
May 18 20 17 49 4 30
* Originally presented at the 16th MayAnnual 25 ASQC Conven-
21 22 45 7 26
tion, Cincinnati, Ohio, in May 1962. June I 22 16 43 5 36
t George P. H. Styan is a graduate student
June in Columbia
8 23 17 41 7 35
University's Department of Mathematical
June 15 Statistics.
24 20 41 Harry
4 35
Smith, Jr., is Head of the Department
Juneof Mathematics
22 25 22 37 and5 36
Statistics in the Procter and Gamble June
Company.
29 26 22 37 5 36
xMade by Attwood Statistics Ltd.,Average
London, England,
21.04 for 31.31
41.23 6.42
Procter and Gamble, Ltd. (then Thos. Hedley and Co. Ltd.),
Newcastle-upon-Tyne, England. ' Total 100 housewives per week.

This content downloaded from 14.139.157.21 on Fri, 12 Oct 2018 09:55:48 UTC
All use subject to https://about.jstor.org/terms
MARKOV CHAINS APPLIED TO MARKETING 51

(1) Detergents 21.04 percent Table 3


(2) Soap powders 41.23 percent TRANSITION FREQUENCY MATRICES*
(3) Both powders 6.42 percent
(4) No powder 31.31 percent F, = 12,2, 7 F2 = 1, 3,1, 3 F= 13, 2,0, 3
1,35,1 , 7 2,28,3, I 3,27,4, 5
The question which immediately arises is this: 1, 2,6, 0 0, 0,7, I 3, 2,6, O
4, 5,0,161 5, 7,0,18L 44, 13,0, 16
"Are these percentages the result of loyal stable buyers,
F4 = 15,2,2,3 Fs= 14217 F= 16, 2,1, 6
or is there a lot of switching taking place?" 2,28,3,1 13,25,3, 8 1,23,4,7
To answer this question, one should consider informa- 2, 2,6,0 1I, 3,5, 2 2, 3,3,1
tion on successive purchases and formulate estimates L , 7,0,12 7, 5,0,14 9, 9,1,1
of transition probability matrices. The first transition
F7= 18, 3,1, 6 F8= 16, 2,2, 8 F9= 13.,1.0,6
probability matrix estimate is shown in Table 2. For 2,26,5, 4 2,28,2, 7 2,29,1, 9
3, 3,2, I 0, 4,4, 0, 3,3, 3
Table 2 5, 7,1,13 2, 7,1,14 4, 1,1,24j
FIRST TRANSITION PROBABILITY MATRIX Fo= 11,3,0,5 F=- 14, 0,1, 8 F12= 16, 3,0, 2
1,27,0, 6 1,28,2, 9 1,29,0, 7
Second Week
I, 1,2, I o0, 0,2, 0 I, 2,2, 0
Soap Both No 10, 9,0,23_j 6, 9,0,201 5, 6,3,231
F13=
Detergent powder powders 14, 1,0, 8 F14= 15, 0,0, I F15 = 16, 3,1, 4
powder
S1,31,2, 6 2,30,2,11 I 1,29,1,10
Detergent 0.5454 0.0909 0.0455 0.3182 1, 2,0, 2 1, 0,2, 0 0, 0,2, 3
First Soap powder 0.0227 0.7955 0.0227 0.1591
Week Both powders 0.1111 0.2222 0.6667 0 0,11,I,20J 6,11,1,18 J 2, 7,0,21 J
No powder 0.1600 0.2000 0 0.6400
F16= 14, 1,0, 4 F17,
1,32,0, = 14, 2,0,
6 1,35,1, 7 21,36,1,12
Fa18 = 17, 2,0, 2-
0, 0,4, 0 0, 2,4, 0 0, 1,3, 2
example, of the 22 households which purchased only
detergents in the first week, 54.54 percent or L3,11,2,22J
12 house- L 6,11,1,14 4, 9,0,10J
holds purchased only detergents in the Fi second
= 15, 2,0, 5 F20= 15, o0,1, I F21 = 13, 3,0, 6
week.
This is an estimate of detergent loyalty. 9.090,37,
percent I, I of
1,38,2, 8 1,31,3,10
1, 2,1, 0 1, 0,3, 0 0, 3,2,2
the 22 households switched to soap powder only, and
so on. Similar two-period matrices were formed to
1, 8,2,15 [5, 7,,17J 2, 6,0,18
cover the 26 weeks completely. SomeF22=
2512,transition
1,0, 3 F23= 12, 0,0, 5 F24 = 13, 2,0, 5
matrices were therefore constructed. These matrices are 1,29,4, 9 1,28,0,12 4,26,0,11
I, 1,3, 0 1, 3,3, 0, 0,4,
shown in frequency form in Table 3. They demonstrate
that the buying habits of women are dependent upon L3,0,0,23_j L 6,10,1
their previous purchases. However, it is necessary to F25 = 14, 3,0, 5-1
3,28,1, 5
test such an implication statistically, and the test to be
0,0,4, 1
employed examines the "order of the Markov chain."
L5, 6,0,25j
ORDER OF THE CHAIN
* Fk denotes transition from period k to period k + I, k = 1,2,. ... ,25
If a purchase of a particular type of laundry powder
is dependent not only on the last purchase, but on the
previous r (r > 1) purchases made, then the model Table 4

being applied here is said to be an rth order X2 TEST RESULTS FOR ZERO VERSUS
Markov
chain. FIRST ORDER MARKOV CHAINS

Consider the null hypothesis:


k X9 X29 k
Ho: A zero order Markov chain (or independent trials 9 89.55 18 96.98
I 91.08
sequence); against the alternative hypothesis:
2 96.08 10 82.49 19 77.70
3 79.74
H1: A first order Markov chain (or dependence being II 91.57 20 111.23
4 68.99 12 89.53 21 68.29
only on the state occupied immediately previously).
5 58.49 13 69.88 22 90.85
6 44.83 14 89.68 23 78.61
Following Anderson and Goodman [1], a7x251.76
test of15 87.96 24 118.77
homogeneity is formulated in the appendix8at68.09
the end16 139.53 25 124.22
of this article to test these hypotheses. 17 100.11
This test was carried out on the 25 matrices shown in
" k denotes the transition from week k to we
Table 3, with the results as shown in Table 4. Each ... 25], and x29 the calculated value of x2 with
transition matrix yields a highly significant value of x29. freedom as given by (I) in the appendix.

This content downloaded from 14.139.157.21 on Fri, 12 Oct 2018 09:55:48 UTC
All use subject to https://about.jstor.org/terms
52 JOURNAL OF MARKETING RESEARCH, FEBRUARY 1964

The null hypothesis of a zero order Markov Tablechain


5 is
rejected at the 0.1 percent level in MAXIMUM
all 25LIKELIHOOD
cases. That
ESTIMATE is,
OF THE
STATIONARY
the hypothesis that the observations at TRANSITION
successivePROBABILITY MATRIX P
peri-
ods are statistically independent is rejected. Second Week
With diary information, sequential tests for higher
order Markov chains can be made. These tests Soap Both No
require
a very large number of observed transitions, Defergent
and powder
such powders po
data were unavailable in this study. Only 0.6724
Detergent a first
0.0857order
0.0229 0.2190
Markov chain is to be considered in this article. First Soap powder 0.0367 0.7179 0.0444 0.2010
Week Both powders 0.1235 0.2407 0.5123 0.1235
No powder 0.1465 0.2584 0.0219 0.5732
STATIONARITY

As is indicated by the matrices in Table 3, the loyalty impossible. Such a matrix defining a Mar
therefore
probability for each state is much higher than chain
the switch-
has the property that
ing probability between two different states. To con-
Lim P" = E
sider the possibility of regularities throughout the 26-
n-->o
week period, one may graph the loyalty probabilities2
for each state, as in Figure 1. One can see relatively
where E is an idempotent matrix with all i
stable loyalties, with the exception of Figure 3 where
same, and each row adding to unity. A matr
the wild fluctuations are clearly due to the small fre-
be idempotent when its square equals itself,
quencies observed for this state. E = E2. The row defining E will be a vecto
This leads to the possibility that the transition prob- and also the left-hand characterist
abilities,
abilities are "stationary," that is, independent of time
P corresponding to its characteristic root of
or purchase period. A system with stationarydenotes
transitiona column vector of unities, then on
probabilities is called a homogeneous MarkovEchain.
= e 1', and 1' P = -', where 1' is the row
Denote the transition probability2 from stateThus
i to state
if the Markov chain starts with probabil
j at periods k and k + 1 respectively as pij(k),
by the i,j elements
= in 1', it will always have
1,2,3,4; k = 1,2,...,25. Then to test the hypothesis of One says that 1' defines the limit
abilities.
stationary transition probabilities, considertionary
the null distribution of the Markov chain. T
hypothesis: case the elements of 1' will contain the shares of the
market attained by the various states if the switching
Ho: pij(k) = pi for all k = 1,2,...,25
pattern defined by P were to persist for a long period of
against the composite alternative hypothesis: time. It thus gives an indication of where the market is
heading, which is a useful piece of information for the
Hi: p1j(k) dependent on the period k.
market strategist. Comparison of 1' with the current mar-
A likelihood ratio test can be used to test these hypoth- ket shares will indicate how far from stationarity the
eses and is formulated in the appendix. Asymptotically current market distribution is.
one finds an equivalent standard normal variate as given In this case, one finds that
by (3) in the appendix, which for the data under analy- i' = (21.14, 40.85, 6.14, 31.87) percent
sis proves to be 0.69. This is clearly not significant,
since the corresponding significance level is over 24 per- while observed share vectors (from Table 1) are found
cent. to be

Hence one has insufficient evidence to reject the null Week 1: (22.00, 44.00, 9.00, 25.00) percent
hypothesis and hence may consider a homogeneous Week 26: (22.00, 37.00, 5.00, 36.00) percent
Markov chain model. Thus one represents the system by Average: (21.04, 41.23, 6.42, 31.31) percent
a single "stationary" transition probability matrix P, the
maximum likelihood estimate of which is shown in The closeness of 1' to the average market share indicates
Table 5. The switching pattern of the system is there- the market distribution is approximately stationary
that
fore taken to be independent of time. throughout the 26 weeks considered.
It has thus been found that the data from this survey
LIMITING DISTRIBUTION fit fairly well to the model of a first order Markov chain
with stationary transition probabilities and with, on the
Inspection of P shows that it is possible to move from
average, a stationary market share distribution.
every state to every other state. No particular switch is
PREDICTION
' Maximum likelihood estimates of these are found by divid-
One of the possible further applications of M
ing the transition frequency (as shown in Table 3) by the in-
chains
itial period frequency (as shown in Table 1), and is prediction of future market positions.
it is these
that are used. assumes the next position primarily dependent o

This content downloaded from 14.139.157.21 on Fri, 12 Oct 2018 09:55:48 UTC
All use subject to https://about.jstor.org/terms
MARKOV CHAINS APPLIED TO MARKETING 53

FIGURE 1
LOYALTY PROBABILITIES BY STATE

PERCENTAGE PERCENTAGE
(state loyalty) (state loyalty)
100 100

90 90

80 80

70 70

60 60

50 50

40 40

30 30

20 20

10 10

0 0
5 10 15 20 25 5 10 15 20 25
TRANSITIONS TRANSITIONS

A. DETERGENT LOYALTY B. SOAP POWDER LOYALTY

PERCENTAGE PERCENTAGE
(state loyalty) (state loyalty)
100 100

90 90
80 80

70 70

60 60
50 50

40 40
30 30
20 20
10 10

5 10 15 20 25 5 10 15 20 25
TRANSITIONS TRANSITIONS
C. BOTH POWDER LOYALTY D. NO POWDER LOYALTY

This content downloaded from 14.139.157.21 on Fri, 12 Oct 2018 09:55:48 UTC
All use subject to https://about.jstor.org/terms
54 JOURNAL OF MARKETING RESEARCH, FEBRUARY 1964

present position, then the following method


m of predic-
m
tion can be used. Let f,1(k) = f.j(k); X f1j(k) = ft.(k)
i=1
Let P'k denote the row vector of market j=1
positions for
period k. Then an estimate of P'k+1m
is found
m mby forming
the product p'k P, where P is the transition probability
C Cnumber
fj(k)= C of
fi.(k)=
i=lj=
matrix, estimated over the largest 1 i=1 previous
m
transitions for which the x2-test confirmed stationarity
I f.j(k) = f..(k)
of the transition probability elements of P. For example,
the first prediction can be made for period 3, usingj-1
p'2,
with P estimated by F1 in Table 3. That is p'2 P esti-
Anderson and Goodman [1] show that
mates p'3. Further estimates follow similarly.
However, because of the stationary nature of the data
In nft.( fi.(k) k) _ fj(k) 2
at hand, it is unnecessary to use this procedure for pre-
diction in this case. Prediction m can
here m f.(k) f(k.)f(
be made k)
by the
i-= which
stationary distribution (1') alone, is in factX2(m-1)...
1 j = 1 f.j(k)/f..(k) the (1)
same as the naive persistence model with these data.

where ~ indicates that the ratio of the distribution fun


SUMMARY
tions of the left-hand side to that of the right-hand si
converges to 1 as f.. (k) tends to infinity.
Consecutive purchases of laundry cleaning powders
In this case m = 4, f..(k) = 100. Hence after som
by a small panel of housewives over a six-month period
simplification,
have been considered. First, a classification was made it can be seen that
of the various powders, into four mutually exclusive and
wholly exhaustive states. Frequency data for these four 100 f2(k)X29
states were examined and found to be fairly stable.
Transition probability matrices were estimated, giving i[f 1 j - lf. (k)f,-(k) 1 X2
the probability of occurrence of state j in week k + 1,
Likelihood Ratio Test for Sationarity
immediately after state i in week k. The duration of the
stage of the system is approximately equal to the period t t
[2] of use of a laundry powder. Each such matrix was Let F
considered to define a separate Markov chain, and withk = 1 k = 1
the aid of a statistical test, evidence was found that it
Then the maximum likelih
was of first-order rather than of zero-order, the inde-
transition probability2 bet
pendent trials sequence.
All the transition matrices were then tested to see if m

they were simply random deviates from some underlying


matrix. Again using a statistical test, evidence was pij-= fi/ft., where j=1
fi. = fij,
found to support this. The unique matrix thus found and the maximum likelihood estimate of the transi
was considered to define a homogeneous Markov chain.
probability between states i and j at periods k and k
Inspection showed that the average distribution of the is
market during the period concerned was very nearly
equal to the stationary distribution of the Markov chain. pij(k) = fij(k)/ft.(k) (k = 1,2,...,t)
Finally a method of using the transition probabilities Following Anderson and Goodman [1], one defines the
as a predictive device to estimate future market posi- likelihood ratio criterion X as
tions was described. Markov chains are thus seen to be
useful in marketing as a predictive device for short and m t /(, Pjfij(k)
long range purposes. They can also be used to indicate
which brand in the market place gains (or loses) its cus-
in/ ,...(2)
tomers from (or to) which other brand. For large f.. (k) one has

MATHEMATICAL APPENDIX -2 loge X X2m(m-1) (t-1

Homogeneity Test for Order In this case m = 4, t = 25, so w


Let Fk = [fi(k)], denote a transition frequency ma- finds that approximately
(2) one
trix, where fi(k) denotes the frequency of occurrence of
state j in period k + 1 and state i in period k (i,j = 4.6052 1 4 fij(k) l
1,2,...,m; k= 1,2,...,t). k--1 i-=1 j-=1

This content downloaded from 14.139.157.21 on Fri, 12 Oct 2018 09:55:48 UTC
All use subject to https://about.jstor.org/terms
MARKOV CHAINS APPLIED TO MARKETING 55

4 4 4 where v is the number of degrees of freedom. In this


Sf. loglo fi. - . fj loglo fj - case one finds
i=1 i=lj= 1
V2x2288 , N(23.98, 1).
25 4 REFERENCES
kE E fi.(k)
1. T. W. Anderson and L. A. Goodman,loglo
"Statistical Inference
about Markov Chains," Annals of Mathematical Statistics,
28 (1957), 89-110.
When m(m 2. R. A.-Howard, 1) (t
"Stochastic Process -
Models of 1)
Consumer >
of x2 are not Behavior," readily
Journal of Advertising Research, 3 avai
(September
proximation 1963). to an equiv
used, i.e., 3. B. Lipstein, "The Dynamics of Brand Loyalty and Brand
Switching," Annual Conference of the Advertising Research
Foundation, September 1959.
/2x,, -~ N(V'2,,- 1, 1) .0.0. (3)

This content downloaded from 14.139.157.21 on Fri, 12 Oct 2018 09:55:48 UTC
All use subject to https://about.jstor.org/terms

You might also like