ERP System Selection Using a Simulation-Based AHP Approach: A Case of Korean

Homeshopping Company
Author(s): B. S. Ahn and S. H. Choi
Source: The Journal of the Operational Research Society, Vol. 59, No. 3 (Mar., 2008), pp.
322-330
Published by: Palgrave Macmillan Journals on behalf of the Operational Research Society
Stable URL: https://www.jstor.org/stable/30132753
Accessed: 28-08-2018 08:38 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

Operational Research Society, Palgrave Macmillan Journals are collaborating with JSTOR
to digitize, preserve and extend access to The Journal of the Operational Research Society

This content downloaded from 14.139.45.244 on Tue, 28 Aug 2018 08:38:25 UTC
All use subject to https://about.jstor.org/terms
Journal of the Operational Research Society (2008) 59, 322-330 C 2008 Operational Research Society Ltd. All rights reserved. 0160-5682/08 $30.00

www.palgravejou rnals.com/jors

ERP system selection using a simulation-based AHP

approach: a case of Korean homeshopping company
BS Ahn' and SH Choi2*
'ChungAng University, Seoul, Korea; and 2Gyeongsang National University, Gyeongnam, Korea

An enterprise resource planning (ERP) system is a critical investment that can significantly affect future
competitiveness and performance of a company. The analytic hierarchy process (AHP) method is often applied
to select an ERP system since it is well suited to multiple criteria decision-making problems. This study
presents a simulation-based AHP (SiAHP) method for group decision making and is applied to the real-world
problem of selecting a suitable ERP system for a Korean homeshopping company. To enhance the fitness of
a group AHP method and to facilitate the ERP system selection process, this paper proposes a simulation-
based approach for building a group consensus instead of forming point estimates that are aggregated from
individual preference judgments. To be specific, the proposed method is based on observations from empirically
observed frequency distributions and does not use aggregation procedures, compared to typical group AHP for
obtaining a group solution. This approach, reflecting the diversification of group members' opinions as they
are, is conceived to be useful as a tool for obtaining insights into agreements and disagreements with respect
to the alternatives among the individuals of a group. The real-world example demonstrates the feasibility of
our proposed approach.
Journal of the Operational Research Society (2008) 59, 322- 330. doi: 10.1057/palgrave.jors.2602365
Published online 10 January 2007

Keywords: information systems; enterprise resource planning; simulation; analytic hierarchy process; group
decision making

Introduction and background unified enterprise view of the business that encompasses all
functions and departments; and (2) an enterprise database
In these days of mature markets, increasing competition, and
where all business transactions are entered, recorded, pro-
sophisticated, wary consumers, marketers must work harder
cessed, monitored, and reported. This unified view increases
than ever. A clear, consistent brand image and a long-term
the requirement for, and the extent of, interdepartmental
relationship with the customer are critical to a marketer's
cooperation and coordination (Umble et al, 2003).
success in today's marketplace. In essence, what is required To cope with ever-changing customer needs and com-
is a synergistic approach, in which all marketing efforts
petitive environment surrounding the Korean homeshopping
work in concert to deliver a coherent message to consumers
industry, many companies consider the adoption of an ERP
(Teltumbde, 2000). The concept of enterprise resource plan-
system. Owing to the complexity of the criteria to be con-
ning (ERP) has recently emerged to meet the challenges
sidered, it is imperative that the ERP selection decision be
of an environment that the traditional selling approach of carefully conducted by a group of experts. In the paper,
retail companies cannot accommodate. ERP is a business we consider the important decision criteria such as business
management system that integrates all facets of the busi-
functionality and coverage, supporting services, technology,
ness, including planning, manufacturing, sales and finance total costs and vendor credentials. The selection of the best
so they can become more closely coordinated by sharing
ERP system is a very critical multiple criteria decision-
information. The ERP system models and automates many
making process which requires the input of multiple experts.
processes with the goal of integrating information across the
The increasing complexity of socio-economic environments
enterprise and eliminating complex, expensive links between
makes it less and less possible for a single decision maker to
computer systems in different areas of the business (Reuther
consider all relevant aspects of the problem. Therefore, many
and Chattopadhyay, 2004). ERP provides two major benefits
organizations employ groups in decision-making problems,
that do not exist in non-integrated departmental systems: (1) a
and decisions are made collectively, regardless of whether
the organization is public or private, national or international.
The group analytic hierarchy process (AHP) is widely used
*Correspondence: SH Choi, Department of Industrial and Systems
for dealing with multiple criteria decision-making problems
Engineering, Engineering Research Institute, Gyeongsang National
University, 900 Gazwadong Jinju, Gyeongnam 660-701, South Korea. including multiple experts in real situations. In spite of its
E-mail: chois@gsnu.ac.kr popularity and simplicity in concept, this method has often

This content downloaded from 14.139.45.244 on Tue, 28 Aug 2018 08:38:25 UTC
All use subject to https://about.jstor.org/terms
 BS Ahn and SH Choi-ERP system selection using simulation-based AHP approach 323

had difficulty in adequately handling the inherent imprecision framework for selecting a suitable ERP system. The frame-
associated with the mapping of the experts' perception to ex- work could systematically construct the objectives of ERP
act numbers (Chan and Kumar, 2005). To enhance the fitness selection to support the business goals and strategies of an
of a group AHP method and to facilitate the ERP selection enterprise, identify the appropriate criteria, and set up a con-
process, we propose a simulation-based AHP (SiAHP) as a sistent evaluation standard for facilitating a group decision
preference aggregation method which is different from other process. Finally, Reuther and Chattopadhyay (2004) ana-
approaches to use point estimates that are aggregated from lysed critical factors for ERP selection and implementation
individual preference judgments. In applying the proposed with the use of surveys and interviews. They suggested six
method, it is prerequisite to have a multitude of decision mak- major categories of critical selection factors which include
ers (at least the number of scales used in the AHP) since functional requirements, financial benefit, cost drivers, flexi-
their paired judgments are used to form the frequency dis- bility, scalability and others. The need for alignment between
tributions and then random observations are generated from information technology application and strategy is well es-
the frequency distributions to conduct a simulation analysis tablished in the literature. In the manufacturing sector, which
from which a preferred alternative of a group is eventually has extensive investments in information technologies, an
identified (Ahn, 2000). information technology application should be aligned with
a company's strategy in terms of competitive priorities and
process structure (Kathuria and Igbaria, 1997; Kathuria et al,
Prior researches of selecting the ERP system
1999; Somers and Nelson, 2003).
ERP is increasingly important in modern business because A number of methods, including scoring, ranking, mathe-
of its ability to integrate the flow of material, finance, and matical optimization, and multiple criteria decision analysis,
information and to support organizational strategies (Yusuf have been applied to ERP or other IS selection problems
et al, 2004). Owing to the complexity of the business en- (Wei et al, 2005). Among others, the AHP method, intro-
vironment, the limitations in available resources, and the duced by Saaty (1980), directs how to determine the priority
diversity of ERP alternatives, the ERP system selection of a set of alternatives and the relative importance of crite-
is tedious and time consuming. However, given the con- ria in a multiple criteria decision-making problem, and has
siderable financial investment and the potential risks and been widely discussed in various aspects (Schniederjans and
benefits, the importance of a pertinent ERP system selection Wilson, 1991; Lai et al, 1999; Teltumbde, 2000). The AHP
cannot be overemphasized (Teltumbde, 2000). Academic has been widely applied to group decision-making problems
literature on the issue of criteria structure for evaluating as well (Saaty, 1989). There are largely two ways to anal-
ERP systems is still scarce. Shankarnarayanan recom- yse a group decision problem in the classical literature on
mended the following criteria for evaluating ERP systems AHP: (i) aggregation of individual judgments (AIJ) where a
(http://www.angelfire.com/co/troyc/advant.html, accessed 27 new pairwise comparison matrix for the group is constructed
December 2005): the functional fit with the company's busi- aggregating the individual judgments by means of consen-
ness processes, the degree of integration between the various sus, voting, or statistical procedures such as, for instance,
components of the ERP systems, flexibility and scalability, the weighted geometric mean. From this matrix, the prior-
user friendliness, quick implementation, ability to support ity vector is then calculated following any of the existing
multi-site planning and control, technology, availability of prioritization procedures. (ii) Aggregation of individual pri-
regular upgrades, amount of customization required, local orities (AIP) where the individual priorities are aggregated
support infrastructure, availability of reference sites and total in order to obtain the priority of the group (Altuzarra et al,
costs. Teltumbde (2000) proposed a structured framework 2006, http://ima.udg.es/Activitats/coDawork05/CD/Session3/
to select an ERP software. The framework included some Altuzarra-MorenoJimenez-Salvador.pdf). In the course of ap-
of the important requirements such as strategy considera-
plying any one of two ways for aggregating information, both
tion and comprehensive vendor credentials. The following the geometric mean method (GMM) and the arithmetic mean
10 criteria were proposed on these considerations: strategic
method (AMM) are appropriate procedures for ratio scales.
Ramanathan and Ganesh (1994) showed that the GMM in the
fit, technology, change management, risk, implementability,
business functionality, vendor credentials, flexibility, cost
context of AIJ is not appropriate to find applications in group
and benefit. Kumar et al (2002) investigated critical ERP
decision problems since it violates at least one of the estab-
lished axioms of group preference aggregation. Specifically
adoption issues and criteria for the selection of product ven-
dor and implementation partners. The criteria were collected
they argued that the GMM violates the Pareto optimality ax-
iom among others which is almost universally accepted in
through questionnaires and structured interviews. The cri-
social choice theory. This finding is significant as the GMM
teria included functionality, reliability, fit with parent/allied
organization systems, available business best practices in has
the been the most commonly used method in AHP for com-
system, cross module integration, system using latest tech-
bining individual opinions to form a group opinion. They bol-
ster up the use of (weighted) AMM in the context of AIP
nology, vendor reputation, better fit with company's business
processes, etc. Wei et al (2005) presented a comprehensive
accordingly, which has been found to satisfy all the axioms

This content downloaded from 14.139.45.244 on Tue, 28 Aug 2018 08:38:25 UTC
All use subject to https://about.jstor.org/terms
324 Journal of the Operational Research Society Vol. 59, No. 3
except the independence of irrelevant alternatives that has re-
ceived a number of criticisms from several researchers (Luce
and Raiffa, 1957). These arguments are refuted by Forman
and Peniwati (1998) who insist that there are two basic ways
to aggregate individual preference into a group preference,
depending on whether the group wants to act together as a
unit or as separate individuals. More specifically, with AIJ
the individuals first work together to agree on a common hi-
erarchy before they can work on aggregating their judgments
and then to agree on the relative importance of the criteria.
Once this process is done, the previous individual judgments
with respect to the relative importance of the criteria become
irrelevant, the same way their original hierarchies do as soon
as a common hierarchy is agreed upon. Consequently, there
is no synthesis for each individual and the Pareto principle is
inapplicable. It is evident from the previous statements that
some controversy exists not in single AHP but in group AHP
when selecting a method of aggregating individual opinions.
Selection criteria for the ERP system in homeshopping
industry
Through the survey of selection criteria for ERP system, we
have identified some of the important requirements such as
strategic consideration, comprehensive vendor credentials and
the timing of system implementation. The strategic consider-
ation can be largely evaluated by the coverage and function-
ality of the system. The vendor consideration can be specified
by the characteristics of vendor and supporting services after
implementing the system. Finally, we identified such evalua-
tion criteria as functionality and coverage, supporting service,
technology, total costs and vendor credentials. The timing
consideration was used to determine the schedule to imple-
ment the requiring functions. The characteristics of processes
performed in the homeshopping industry require the rapid ex-
ecution of order receipt, and payment and delivery with the
help of real-time systems responding to a customer request
for a product which is broadcasting at real time. We adopted
a two-phased approach in which the phase is determined by
the ease of implementation and the degree of urgency criteria
since we have to take account of the risks of implementing
all the functions for all processes in the company at the same
time.
Functionality and coverage
ERP systems are expected to have a functional fit with the
company's business processes. Although all the major ERP
systems claim process support across industries, there is none
yet to do so in totality. The extent of lack of support and
the consequences thereof needs to be carefully ascertained
while considering a particular ERP system (Teltumbde,
2000). The functionality and coverage criterion considered
in this study was evaluated by the fitness of the requirements
for the following major processes: financial administration,
management planning, broadcasting, product management,
This content downloaded from 14.139.45.244 on Tue, 28 Aug 2018 08:38:25 UTC
All use subject to https://about.jstor.org/terms
purchasing and logistics, sales and service and marketing
processes. The requirements are specified by the degree of
functional support, the expected values in the future and the
ease of use with respect to each process.
Supporting service
The vendor's ability to provide advanced supporting services
is of prime concern in the ERP system selection. The ser-
vice basically relates to the degree of structurization training
courses for system utilization education and the degree of
support for system maintenance and upgrade.
Technology
The ERP system being essentially an IT application, its tech-
nological dimensions assume importance particularly in the
environment of rapid technological changes. Foremost,
the technology determines the longevity of the product. If the
ERP system is built on a proprietary technology, however ad-
vanced, it would subject the organization to the potential risk
of obsolescence. In order to fortify the products from these
eventualities, the ERP system design should be independent
of the crucial technologies of, for example, databases, oper-
ating systems, hardware, networks, etc (Teltumbde, 2000).
Technology also determines the performance and flexibil-
ity (in terms of functionality expansion) dimensions of the
project and thus has a direct impact on the total project cost.
The other technological considerations are the degree of ease
of implementation, security and stability.
Total costs
The company always requires the minimum price of the prod-
uct to increase the profitability. Profit maximization cannot be
achieved without cost minimization. The cost here relates to
the total costs. They consist of the ERP system cost, imple-
mentation cost and ongoing licence upgrade costs (Reuther
and Chattopadhyay, 2004). The licence upgrade costs as a
percentage of the implementation cost were also analysed.
Vendor credentials
The performance and past history of the vendors help in mak-
ing decisions for the selection of ERP system. The value of
partnership should be analysed based on its track record to
decide its superiority over other alternative vendors. Some of
the important characteristics of vendors are summarized as
the capability of human experts, their experiences and appro-
priateness of the vendor's implementation strategy in order to
minimize implementation risks.
After the survey of prior researches, structured interviews
and discussions were used to elicit the structure of crite-
ria from the members of the project team during several
meetings. A total of 20 different ERP selection criteria have
been identified through the meetings. Figure 1 displays the
fundamental criteria hierarchy. In the viewpoint of business
 BS Ahn and SH Choi-ERP system selection using simulation-based AHP approach 325

Financial administration Mathematical representation of the method

Management planning In order to evaluate the ERP systems by using a SiAHP

method, we shall develop the method in a detailed way for
Broadcasting

C1: Business functionality dealing with a general case. Let A1, A2 ... An be a set of n
Product management
& coverage
alternatives. We define a square matrix Ak = (a0j), Vi, j E
Purchasing & logistics
[1, n], k E K to be a reciprocal matrix in which a = 1/a i
Sales & service and akj indicates typical AHP interpretation of values in pair-
wise comparison matrix of alternatives in view of kth indi-
Marketing

vidual. Similarly, let CI, C2 .... Cm be a set of m criteria.

Goal: Choosing the
C2: Supporting service
Education & training
We define a square matrix Ck = (Ckpq), Vp, q E [1, m], k E
most appropriate
Maintenance support
ERP system
K toindicates
also be a reciprocal
typical AHPmatrix in which
interpretation cp,qin=pair-
of values 1/cz, and cp, q
Ease of integration with
other IS wise comparison matrix of criteria in view of kth individual.

Ease of implementation Then, let ai,j and cp,q, be the variables ranging in [ai1, a~]
Performance
C3: Technology and [cLq, value
(maximum) Cpjq]ofrespectively
K judgments of in which
paired aL (aUf')
alternatives (i, is a minimum
Extensibility

Stability j) andofcpq
ments (cUq)
paired is a(p,minimum
criteria (maximum)
q). Further, let value
f(ai,j) and f(cp,q,) of Kjudg-
be the empirically observed relative frequency distributions,
Security

and let F(ai,j) and F(cp,q) be the cumulative frequency dis-

Product price
tributions on aij and cp,q, respectively.
C4: Total costs
Maintenance costs
Let a) and cr r = 1, 2,., R, i, j =1, 2.. ,n,p,q
Human resource expertise
1, 2,..., m be empirically observed judgments generated
from the cumulative frequency distributions F(ai,j) and
C5: Vendor credentials Experience
F(cp,q) in the rth simulation run. Let \Y(r) be an m x n ma-
Implementation strategy trix of which the ith row is an eigenvector calculated from
a pairwise comparison matrix generated with respect to the
Figure 1 Criteria for evaluating the ERP systems. ith criterion, and C(r) be an 1 x m eigenvector associated
with a pairwise comparison matrix between criteria in the rth
simulation run. Then the final priorities of alternatives can
be determined by displaying descending order of magnitude
functionality and coverage, the systems are evaluated by the
extracted requirements for each of target processes. of C(r) x ,(r). The simulated final priorities are sometimes
obtained in case the simulated pairwise judgment matrices
result in a high IR. To avoid this case, we consider the pair-
wise comparison matrices generated with IR less than or
The SiAHP method for evaluating the ERP systems
equal to 0.1.
We will briefly explain the overall flow of the method. First,
K individuals of a group specify their pairwise judgments be-
Explanatory example
tween criteria, and between alternatives with respect to each
of criteria. The scale used for pairwise judgments lies be- To illustrate the simulation procedure, we consider a simple
tween 1/9 and 9. Next, the number of times that each judg- example with three alternatives (A1, A2 and A3) evaluated on
ment repeats is counted to obtain the frequency distributions three criteria (C1, C2 and C3). The preference judgments of
and then their cumulative relative frequency distributions are individuals of a group (K = 25) are collected and shown in
constructed. In the third step, random numbers are generated Table 1.
to obtain random observations from the cumulative relative Then a random observation is generated from the equation,
frequency distributions. The matrices which are comprised of rn = F(.) in which rn denotes a random number. After all
the random observations correspond to empirically observedmatrices (ie, one pairwise comparison matrix between crite-
ria and three comparison matrices between alternatives) were
judgments between the criteria, and between the alternatives
with respect to each of criteria. The final step is to checkdetermined, the overall synthesized priorities were calculated
the inconsistency ratio (IR) of the pairwise comparison ma-and thus the final rank was recorded according to their magni-
trices and to determine the final priorities of alternatives if ittudes. This process was repeated 500 times. Of the 500 runs,
is within the boundary allowed. If not, the third step is re-64% resulted in the first alternative with the first rank, 53%
peated. The entire simulation steps are repeated until we getin the third alternative with the second rank and 77% in the
a stable result. second alternative with the third rank (See Table 2).

This content downloaded from 14.139.45.244 on Tue, 28 Aug 2018 08:38:25 UTC
All use subject to https://about.jstor.org/terms
326 Journal of the Operational Research Society Vol. 59, No. 3

Table 1 The frequency of preference judgments from individuals of a group

Between criteria Between alternatives with respect to criteria

C12 C13 C23 C1 C2 C3

Scale A12 A13 A23 A12 A13 A23 A12 A13 A23
9 3 1
7 1 4 5 3 1 2 5 6

5 6* 5 7 4 1 1 1 3 3 7 1
3 7 7 1 3 5 3 5 4 3 6 3 3
1 5 3 4 2 8 5 7 6 2 4 4 2
1/3 4 2 3 4 3 7 7 3 6 3 4 5
1/5 2 3 10 1 2 7 5 5 4 2 1 7
1/7 1 7 2 4 5 1 5
1/9 1 2

*Six of 2

Table 2
Table 4 Expected weight and rank using the formula (1) and (2)
distribu
Alternative Expected rank Expected weight
Compo
Al 1 0.4708
Rank A1 A2 A3 Al (%) A2 (%) A3 (%) A2 3 0.1538
A3 2 0.3754
1 320 25 155 64 5 31
2 142 91 267 28 18 53
3 38 384 78 8 77 16
consider expected
gested by Hauser
The expected sc
Table 3 Composite
of 500 runs with IR
add together the
empirically observed distribution functio
of each rank)occurred and n + 1 minus the rank itself instead of the
rank itself because the rank and the fraction of the time each
Composite results
rank occurred is inversely correlated.
Rank A1 A2 A3 A1 (%) A2 (%) A3 (%) n

1 405 5 90 81 1 18 ESi =(pi,)(n 1 - k) Vi e [1, n] (1)

2 84 50 366 17 10 73 k=l

3 11 445 44 2 89 9
where ESi is the expected score of the ith alternative and p
is the proportion of the trials that the ith alternative had r
k. Next, let the expected weight be the normalized expec
Confidence measure
scores. When alternatives are placed in descending order
the expected weights, the results reveal the expected rank
In the AHP with a single decision-mak
alternatives. Hence, we define
ties with an IR greater than 0.10 are
ments which are too random-like ESi (Va
decision context, each
EWi
of
= Ei
En , ESk
Vi e [1, n] (2)
the preferen
inconsistency can be combined finall
The expected weights of (2) are determined from the fre-
sensus which
ends with a large inco
quency that each rank occurred for each alternative (Table 4).
wise comparison judgments with an
Hence, these weights are statistical weights indicating a com-
0.10 for simulated criteria matrices are considered in Table
posite frequency or a mean of feasible weights around which
3. Although each alternative has seen each possible rank, it
we expect the actual weight to be scattered.
is clear that alternative one is inclined to be positioned in the
first rank with 81% of the time. Likewise, alternative two is
Application of the SiAHP
inclined to be positioned in the third rank with 89% of the
time, and alternative three in the second rank with 73% of The problem discussed here is concerned with a Korean home
the time. However, how much confidence can we have in first shopping company, searching the best ERP system for sup-
two rankings and others? In order to address this question, weporting seven major processes as shown in the sub-criteria of

This content downloaded from 14.139.45.244 on Tue, 28 Aug 2018 08:38:25 UTC
All use subject to https://about.jstor.org/terms
 BS Ahn and SH Choi-ERP system selection using simulation-based AHP approach 327

Table 5 Frequency of preference judgments about pairs of criteria

Between criteria

Scale C12 C13 C14 C15 C23 C24 C25 C34 C35 C45
9 1 1 1 1
7 3 1 2 1 1 1 2 3 2
5 6 4 4 3 1 2 1 3 3 4
3 7 5 3 4 3 4 5 3 6 5
1 4 7 10 12 6 9 10 9 7 6
1/3 2 6 5 5 6 4 3 5 5 6
1/5 2 1 1 7 5 5 2 1 1
1/7 1 1
1/9

to each of criteria C2, C3, C4 and C5 from the preference fre-

Goal
quencies in Table 6 although there is some extent of disagree-
ment. Finally, if we aggregate these conjectures obtained from
(Bit. ( C C, C, C
functionality rough analysis
(Supporting of preference judgments to(Total
(Technology) identify a group's
costs
& coverage) service) Credentials)
final decision, a partial rank of alternatives such as Al >- A3 >-
A2, A4, considering all the criteria, can be obtained and a re-
lation, A2 >- A4 will hold if the sum of criteria weights of C2,
Al A, A3 A4
C3, C4 and Cs is larger than the weight of C1. Consequently,
we are strongly confident that A1 is most preferred, A3 is sec-
Figure 2 The hierarchy for the selection of the best ERP system. ondly, A2 is thirdly and A4 is the least preferred as a group
opinion, which is the result we want to show with the SiAHP
method.

business functionality and coverage criterion of Figure 1. The
company wants to take into account all the possible important
criteria which can affect the selection of the ERP systems.
A task force team, consisting of 25 experts having at least 5
years experience working at target processes and ERP con-
sultants in order to evaluate the ERP system, was organized
to formulate a decision problem including the identification
of alternative options and criteria structures. They discussed
about criteria and eligible ERP systems included in the anal-
ysis and finally identified five critical criteria and four candi-
date ERP systems as several rounds of discussions evolved.
The whole hierarchy for the selection of a best ERP system
Consider the results in Table 7. Although each alternative
has seen each possible rank, it is clear that alternative Al is
inclined to be positioned in the first rank. Likewise, alternative
A2 is inclined to be positioned in the third rank, alternative
A3 in the second rank, and alternative A4 in the fourth rank.
To concrete this conjecture, we use expected score, expected
weight, and expected rank in (1) and (2) (See Table 8).
To distinguish the results of the method with others, we
compare the results from the SiAHP method with the point
estimate approaches including the GMM and the AMM by
using the same input data as shown in Table 9.
In a situation where it is not clear which to choose between

is depicted in Figure 2. the GMM and the (weighted) AMM, the two methods some-
After construction of the hierarchy, the experts filled in times produce conflicting results (See Table 9). After we have
the questionnaires which were prepared for eliciting their finished the evaluation processes to select a best ERP system
pairwise comparison judgments both on criteria and on al- using the SiAHP method, we need to determine the imple-
ternatives with respect to each of criteria. The frequencies mentation phase of each of the target processes. To do this,
of preference judgments on the pairs of criteria are denoted we evaluate the target processes by ease of implementation
in Table 5 and the frequencies of preference judgments on and degree of urgency criteria. We determine the phase for
the pairs of alternative ERP systems with respect to each each process by the evaluation score with respect to each cri-
criterion in Table 6. terion rather than aggregating the scores into one value. The
Before performing the SiAHP method, let us scrutinize theprocesses whose score with respect to each criterion is larger
than threshold will be implemented at the first phase and the
preference frequencies in Tables 5 and 6. At first, we can infer
others at the second or third phase. Figure 3 shows the imple-
a group's preference tendency on criteria: criterion C1 is most
mentation phases for target processes according to the scores
preferred, C3 is the second, C4 is the third, C5 is the fourth, and
then finally C2, that is C1 >- C3 >- C4 >- C5 > C2. Roughly,evaluated by the group members.
we can infer the rank of alternatives A1 >- A3 > A4 >- A2 Finally, the task force team selects the first alternative (ie,
A1) with the use of the SiAHP method and determines the
with respect to criterion C1, Al >- A3 >- A2 >- A4 with respect

This content downloaded from 14.139.45.244 on Tue, 28 Aug 2018 08:38:25 UTC
All use subject to https://about.jstor.org/terms
328 Journal of the Operational Research Society Vol. 59, No. 3

Table 6 Frequency of preference judgments between alternatives with respect to criteria

Between alternatives on criteria

Business functionality & coverage (C1) Supporting service (C2) Technology (C3)

Scale A12 A13 A14 A23 A24 A34 A12 A13 A14 A23 A24 A34 A12 A13 A14 A23 A24 A34
9 1 1 1 1 1 1
7 3 2 5 1 2 3 2 5 1 3 3 4 4 4 4
5 5 3 7 2 3 4 3 4 5 2 5 5 5 7 4 3 6 5
3 4 5 5 3 5 5 5 5 6 5 6 4 3 3 5 3 4 2
1 6 7 4 4 5 10 5 6 2 4 3 6 7 7 5 6 5 8
1/3 4 4 2 6 5 3 4 4 3 7 6 3 4 3 6 4 4 4
1/5 3 2 2 5 4 1 2 3 3 4 4 2 1 1 4 2 2
1/7 1 5 2 2 1 2 1 5
1/9 1

Between alternatives on criteria

Total costs (C4) Vendor credentials (C5)

Scale A12 A13 A14 A23 A24 A34 A12 A13 A14 A23 A24 A34
9 1 1 1 1
7 4 3 4 2 3 4 1 4 2 3
5 6 2 8 1 3 5 3 5 6 3 5 5
3 3 6 5 4 5 4 6 4 5 3 5 4
1 7 6 4 5 6 9 4 6 3 4 4 6
1/3 3 4 3 7 4 4 5 5 4 6 5 3
1/5 1 3 1 4 3 3 2 3 4 3 2
1/7 1 3 2 1 5 1
1/9 1

Table 7 Compos

Composit

Rank A1 A2 A3 A4 A1 (%) A2 (%) A3 (%) A3 (%)

1 329 21 141 9 66 4 28 2
2 129 95 232 44 26 19 46 9
3 35 194 111 160 7 39 22 32
4 7 190 16 287 1 38 3 57

5
Table 8 Expected
Phase 1
wei
Sales
Pui chasing &
Alternative Expected rank Expected weight logistics
0

Phase 2 Product
Financial
Al 1 0.356 management
administfiation
A2 3 0.1894 Service

A3 2 0.2996
A4 4 0.155 Marketing
Broaddasting
3

Phase 3
Management
planning
EC,

Table 9 Comparison with tw

Human
Alternative GMM AMM SiAHP resource

Al 1 1 1
A2 3 2 3 1 3 5
A3 2 3 2 Degr
A4 4 4 4
Figu

This content downloaded from 14.139.45.244 on Tue, 28 Aug 2018 08:38:25 UTC
All use subject to https://about.jstor.org/terms
 BS Ahn and SH Choi--ERP system selection using simulation-based AHP approach 329

0.4
kA1 by AMM

y SiAHP
0.35
Al byGMM

A3 by HP
0.3

A2 by AMM A3 byGMM

0.25
~A3 byAMM

(1)"0

A2 by SiAHP A2byGMM
0.2

.A4 by GMM

0.15 A4 by SiAHP

A4 by AMM

0.1
10 50 100 150 200 250 300 350 400 450 500
Number of Simulation

Figure 4 Comparison result of the SiAHP method with two group aggregation methods.

select an ERP system. A case example is presented wher

implementation of functions for the processes of sales and
service, product management, purchasing and logistics and
this framework was used successfully.
financial administration at the first phase. In a situation where exact solutions are sometimes more im-

portant than probabilistic ones, and thus combining judgments

Discussions of a group is so important and cannot be replaced by a statis-
tical approach, prior aggregation method is recommended for
We compared the result of the SiAHP method with two group
deriving a group solution. Even in that case, a simulation ap-
aggregation methods with respect to the number of simulation
proach which reflects the diversification of group members'
runs as shown in Figure 4.
preferences as they are, is useful as a complementary tool to
In view of the number of simulation runs, the expected
get some insights into preference judgments of a group.
weight that each of alternatives attained converges into a spe-
cific value at more than 200 times and thus it is necessary to
run the SiAHP algorithm at least 200 times to obtain a stable
References
simulation result. The rank by the SiAHP happens to coincide
with the rank by the GMM, but it shows some discrepancies
Ahn BS (2000). The analytic hierarchy process in an uncertain
with the AMM in the ERP systems A2 and A3. The general environment: A simulation approach by Hauser and Tadikamalla
comparisons of the SiAHP method with the other two ag-
(1996). Eur J Opl Res 124: 217-218.
gregation methods, as the number of criteria and alternatives
Altuzarra A, Moreno-Jimenez JM and Salvador M (2006). Searching
varies, remain for future research work. for consensus in AHP-group decision making: A Bayesian
perspective, Unpublished manuscript. Available online.
The SiAHP method helps decision makers to use random
Chan FTS and Kumar N (2005). Global supplier development
observations from empirically observed frequency distribu-
considering risk factors using fuzzy extended AHP-based approach.
tion which is determined from the frequency of responses.
OmegaThe35: 417-431.
proposed method can be applied to the real-world group
Forman Ede-
and Peniwati K (1998). Aggregating individual judgments
andfunc-
cision problems without any assumption of distribution priorities with the analytic hierarchy process. Eur J Opl Res
108: 165-169.
tion about experts' judgments.
Hauser D and Tadikamalla P (1996). The Analytic Hierarchy Proce
in an uncertain environment: A simulation approach. Eur J Op
Res 91: 27-37.
Concluding remarks
Kathuria R, Anandarajan M and Igbaria M (1999). Selecting IT
Considering the huge amount of risk exposure and the applications
virtual in manufacturing: A KBS approach. Omega 27
605-616.
irreversibility of the decisions, the necessity of proper evalu-
ation of ERP projects cannot be overemphasized. This Kathuria
studyR and Igbaria M (1997). Aligning IT applications with
manufacturing strategy: An integrated framework. Int J Opns Mngt
seeks to propose a comprehensive framework for evaluation
17: 611-629.
of the ERP systems. The framework includes the structured
Kumar V, Maheshwari B and Kumar U (2002). Enterprise resource
criteria hierarchy which contains the fitness with major busi- systems adoption process: A survey of Canadian
planning
ness functions and a newly proposed SiAHP approach to
organizations. Int J Prod Res 40: 509-523.

This content downloaded from 14.139.45.244 on Tue, 28 Aug 2018 08:38:25 UTC
All use subject to https://about.jstor.org/terms
330 Journal of the Operational Research Society Vol. 59, No. 3

Lai VS, Trueblood RP and Wong BK (1999). Software selection: A Somers TM and Nelson KG (2003). The impact of strategy and
case study of the application of the analytical hierarchical process integration mechanisms on enterprise system value: Empirical
to the selection of a multimedia authoring system. Inform Mngt evidence from manufacturing firms. Eur J Opl Res 146:
36:221-232. 315-338.
Luce RD and Raiffa H (1957). Game and Decisions. Wiley:Teltumbde
New York. A (2000). A framework for evaluating ERP projects. Int
J Prod Res 38: 4507-4520.
Ramanathan R and Ganesh LS (1994). Group preference aggregation
methods employed in AHP: An evaluation and an intrinsicUmble EJ, Haft RR and Umble MM (2003). Enterprise resource
process
planning: Implementation procedures and critical success factors
for deriving members' weightages. Eur J Opl Res 79: 249-265.
Reuther D and Chattopadhyay G (2004). Critical factors for Eur J Opl Res 146: 241-257.
enterprise
resources planning system selection and implementation Vargas LG (1982). Reciprocal matrices with random coefficients
projects
within small to medium enterprises. Eng Mngt Conf IEEE MathIntComput
2: Model 3: 69-81.
851-855. Wei CC, Chien CF and Wang MJJ (2005). An AHP-based approac
Saaty TL (1980). The Analytic Hierarchy Process. McGraw-Hill:
to ERP system selection. Int J Prod Econ 96: 47-62.
New York. Yusuf Y, Gunasekaran A and Abthorpe MS (2004). Enterpris
Saaty TL (1989). Group Decision Making and the AHP. In: Golden information systems project implementation: A case study of ERP
B, Wasil E and Harker P (eds). The Analytic Hierarchy Process: in Rolls-Royce. J Prod Econ 87: 251-266.
Application and Studies. Springer-Verlag: Berlin.
Schniederjans MJ and Wilson RL (1991). Using the analytic hierarchy
process and goal programming for information system project Received January 2006;
selection. Inform Mngt 20: 333-342. accepted September 2006 after four revisions

This content downloaded from 14.139.45.244 on Tue, 28 Aug 2018 08:38:25 UTC
All use subject to https://about.jstor.org/terms