JOURNAL OF MULTI-CRITERIA DECISION ANALYSIS J. Multi-Crit. Decis. Anal.

20: 141155 (2013) Published online 22 August 2012 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/mcda.1479

A Comprehensive Literature Review of the Rank Reversal Phenomenon in the Analytic Hierarchy Process
HAMED MALEKIa* and SAJJAD ZAHIRb
a b

Young Researchers Club, South Tehran Branch, Islamic Azad University, Tehran, Iran Faculty of Management, University of Lethbridge, Lethbridge, AB T1K 3M4, Canada

ABSTRACT During the last few decades, several multi-criteria decision analysis methods have been proposed to help in selecting the best compromise alternatives. Among them, analytic hierarchy process (AHP) and its applications have attracted much attention from academics and practitioners. However, since the early 1980s, critics have raised questions regarding its proper use. One of them concerns the unacceptable changes in the ranks of the alternatives, called rank reversal, upon changing the structure of the decision. Several modications were suggested to preserve ranks. In this paper, a classication scheme and a comprehensive literature review are presented in order to uncover, classify and interpret the current research on AHP methodologies and rank reversals. On the basis of the scheme, 61 scholarly papers from 18 journals are categorized into specic areas. The specic areas include the papers on the topics of adding/deleting alternatives and the papers published in adding/deleting criteria. The scholarly papers are also classied by (1) year of publication, (2) journal of publication, (3) authors geographic location and (4) using the AHP in association with other methods. It is hoped that the paper can meet the needs of researchers and practitioners for convenient references of AHP methodologies and rank reversals and hence promote the future of rank reversal research. Copyright 2012 John Wiley & Sons, Ltd.
KEY WORDS:

analytic hierarchy process; rank reversal; analytic network process; literature review; classication; normalization; aggregation

1. INTRODUCTION The multi-criteria decision analysis (MCDA) has been one of the fastest growing areas of operational research during the last two decades. The MCDA often deals with the ranking of decision alternatives being judged with respect to conicting criteria. It concerns theories and methodologies for treating complex problems in management, business, engineering, science and other areas of academic and practical applications. Recently, several MCDA methods have been proposed to help in selecting the best compromise alternatives. The development of MCDA methods has been motivated not only by a variety of real-life problems requiring the consideration of multiple criteria, but also by practitioners desire to propose enhanced decision-making techniques using recent advancements in mathematical optimization, scientic computing and computer technologies (Wiecek et al., 2008).

*Correspondence to: Young Researchers Club, South Tehran Branch, Islamic Azad University, Tehran, Iran. E-mail: St_h_maleki@azad.ac.ir

The analytic hierarchy process (AHP), developed by Saaty (1980, 1994b), is one of the most popular MCDA methods. The importance of the AHP and its variants is best illustrated by its use in more than 1000 articles (Saaty, 1994b), in Special Issues (e.g. Socio-Economic Planning Sciences 10(6), 1986; Mathematical Modelling 9(35), 1987; European Journal of Operational Research 48(1), 1990; and Mathematical and Computer Modelling 17(4/5), 1993), the International Symposium on the Analytic Hierarchy Process and the development of the Expert Choice (www.expertchoice.com), Super Decisions (www.superdecisions.com) and Decision Lens (www.decisionlens.com) software. Emphasizing the rapid spread of academic research and practical applications of the AHP, this paper conducts a comprehensive literature review on the AHP methodologies and rank reversals. For this purpose, a reference bank has been established on the basis of a classication scheme that includes 61 papers already published in 18 scholarly journals since 1983. Scholarly papers in the scheme are categorized into specic areas and are further classied by the year of publication, journal of publication, authors
Received 12 December 2011 Accepted 29 June 2012

Copyright 2012 John Wiley & Sons, Ltd.

142

H. MALEKI AND S. ZAHIR

geographic location and association of the AHP with other methods for avoiding rank reversals. The rest of the paper is organized as follows: Section 2 briey describes the history of the rank reversal and rank preservation in AHP and software packages. Section 3 outlines the research methodology under a scheme for classication of scholarly papers. Analysis of rank reversal papers is presented in Section 4. Section 5 distributes the reviewed papers to several categories. Finally, the conclusion of the paper is presented in Section 6. 2. A BRIEF REVIEW OF RANK REVERSAL AND RANK PRESERVATION IN THE AHP 2.1. Chronological summary The AHP family of methods, including the absolute mode, the relative mode with the distributed or the ideal normalization mode and the supermatrix method (analytic network process (ANP)), was developed by Saaty (1980, 2001). In the absolute mode, alternatives are rated one at a time without any normalization. The distributed mode normalizes alternative priorities under each criterion so that they sum up to one. The ideal mode idealizes alternative priorities under each criterion by dividing each alternative priority by the priority of

the best alternative under each criterion. The supermatrix method allows one to consider dependencies between different levels of a feedback network. However, from the early days, it was observed that certain ranking irregularities may occur. Belton and Gear (1983) rst noticed that when copies (or near copies) of existing alternatives were introduced in a decisionmaking problem, the aggregate ranking (i.e. when criteria weights are taken into account) of the alternatives could be altered. They considered an example with three consistent comparison matrices over four alternatives A, B, C and D with respect to three criteria a, b and c, where D was a copy of B and the three criteria were assumed to be of equal importance. They rst considered alternatives A, B and C and derived a ranking for them, and then considered the four alternatives together and obtained a new ranking for them, only to nd that the ranking between A and B was reversed after the addition of D. Tables I and II show the comparison matrices, the local and the global (aggregate) priorities of the four decision alternatives. As can be seen from Table II, where the symbol > means is superior to, the ranking between A and B is B > A before D is introduced, but becomes A > B after D is added; the ranking is reversed after the addition of alternative D. They attributed this phenomenon to the fact that in the AHP, the values of relative perfor-

Table I. Pairwise comparison matrices of alternatives A, B, C and D with respect to three criteria and their local weights (Belton and Gear, 1983)
Criterion Criterion a Alternatives A B C A B C A B C A B C D A B C D A B C D A 1 9 1 1 1/9 1/9 1 9/8 1/8 1 9 1 9 1 1/9 1/9 1/9 1 9/8 1/8 9/8 B 1/9 1 1/9 9 1 1 8/9 1 1/9 1/9 1 1/9 1 9 1 1 1 8/9 1 1/9 1 C 1 9 1 9 1 1 8 9 1 1 9 1 9 9 1 1 1 8 9 1 9 D 1/9 1 1/9 1 9 1 1 1 8/9 1 1/9 1 Local weights 1/11 9/11 1/11 9/11 1/11 1/11 8/18 9/18 1/18 1/20 9/20 1/20 9/20 9/12 1/12 1/12 1/12 8/27 9/27 1/27 9/27

Criterion b

Criterion c

Criterion a

Criterion b

Criterion c

Copyright 2012 John Wiley & Sons, Ltd.

J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

LITERATURE REVIEW OF THE RANK REVERSAL PHENOMENON IN THE AHP

143

Table II. Global weights of the four alternatives A, B, C and D and their ranks
Alternatives Criterion a 1/3 A B C A B C D 1/11 9/11 1/11 1/20 9/20 1/20 9/20 Local priorities Criterion b 1/3 9/11 1/11 1/11 9/12 1/12 1/12 1/12 Criterion c 1/3 8/18 9/18 1/18 8/27 9/27 1/27 9/27 Global priorities 0.4512 0.4697 0.0791 0.3654 0.2889 0.0568 0.2889 Rank 2 1 3 1 2 4 2

mance of the alternatives, in terms of each decision criterion in the decision matrix, are normalized so they sum up to unity. Using a set of recursive relations, Zahir (2009) has recently shown how normalization ignores the unit of scale in the aggregation process and proposed a new aggregation method that preserves ranks when it should. Dyer (1990a) provided an example when a new alternative was added, and Saaty (1990) and Harker and Vargas (1990) replied to Dyers remarks. Dyer (1990b) counter-argued that his procedure was misunderstood. Troutt (1988) provided another example of rank reversal when an alternative was removed. To avoid rank reversals, Belton and Gear (1983) proposed a normalization procedure by dividing the alternative priorities by the largest entry of each column of the decision matrix. This variant of the AHP was called the revised AHP. Later, Saaty (1994b) accepted this variant approach and called it the ideal mode of the AHP. According to Belton (1986) and Belton and Gear (1997), a key issue for the AHP ranking reversals is the interpretation of the criteria weights. Besides the revised AHP, other authors also introduced other variants of the original AHP (see, for instance, Lootsma, 1991, 1993). However, the original AHP and some of its variants are considered by many as the most reliable MCDA methods (Triantaphyllou and Mann, 1995). Dyer and Ravinder (1983) and Dyer and Wendell (1985) have also studied the fact that rank reversals also occur in the AHP when near copies are considered. Saaty (1986, 1987a) provided some axioms and guidelines on how close a near copy can be to an original alternative without causing a rank reversal. He suggested that the decision-maker (DM) has to eliminate alternatives from consideration that score within 10% of another alternative. This recommendation was later sharply criticized by Dyer (1990a, 1990b). Clearly, these problems are still controversial in decision analysis. Some additional discussion on these truly important issues can be found in Triantaphyllou and Mann (1989,
Copyright 2012 John Wiley & Sons, Ltd.

1994a, 1994b), Harker and Vargas (1990), Saaty (1990), Winkler (1990) and Buede and Maxwell (1995). Finan and Hurley (2002) proved that for a twolevel AHP model when the comparison matrix is perfectly consistent, ignoring a wash (non-discriminating) criterion would not change the rank order for the alternatives. They stated that they could not prove or disprove for the case of an imperfectly consistent DM. They then constructed a three-level example to illustrate that rank reversal does occur when a wash criterion is ignored. By demonstrating the occurrence of rank reversal with a counter-example, they tried to lay out a contradiction, and thus a aw, in the AHP methodology. In a later study, Liberatore and Nydick (2004) claimed that after removing wash criteria, the relative weights should be re-evaluated and hence there are no rank reversal problems. Furthermore, Saaty and Vargas (2006) asserted that wash criteria could not be blindly deleted. Lin et al. (2008) showed how the case of an imperfectly consistent DM mentioned by Finan and Hurley (2002) could be solved, leading to a more general approach. Wijnmalen and Wedley (2009b) showed why rank reversal occurred when synthesizing the two-level hierarchy and how it should have been avoided. They also suggested that there was a necessary link between the normalization and weighting processes. They argued that Finan and Hurley (2002) might have misunderstood the meaning of a weight in a multiple-level hierarchy in relation with the unit of normalization of the criteria weights. They showed that proper scaling of the appropriate weights avoided the rank reversal problem of Finan and Hurley (2002). Wijnmalen and Wedley (2009b) further asserted that it was not necessary to assume a perfectly consistent DM in order to prove that rank reversal would never occur when removing a nondiscriminating criterion from a single-level hierarchy. Perez et al. (2006) showed that the addition of different criteria (for which all alternatives performed
J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

144

H. MALEKI AND S. ZAHIR

equally) caused a signicant alteration of the aggregated priorities of alternatives, with important consequences. Although not in three-level hierarchies, rank reversal might happen in more complex hierarchies. 2.2. Software tools Analytic hierarchy process software packages, including Expert Choice, Criterium and HIPRE, have been developed to facilitate the AHP process. These packages are not just research tools, but very capable and very usable software packages for analysing decisions with multiple and often conicting criteria. There are a number of critical analytical capabilities that one would want in a package that implements the AHP, as well as some other features commonly associated with multi-attribute utility analysis packages. One key feature is the exibility to build and analyse various sizes and types of hierarchies of criteria. All three of these packages enable the user to build large hierarchies within certain limitations. The elicitation of criteria weights and option scores is a second key feature of any multi-criteria package. Each of these packages offers the user signicant choices that are critical in obtaining valid subjective value judgments. Expert Choice also allows the user to provide nonnormalized ratio-scaled scores for options on each criterion, thus eliminating the possibility of rank reversals in this mode. A third key analytical feature is the display of the analysis results and the provision of sensitivity analysis to the user. Sensitivity analysis provides the effect of changing a given top-level weight on the nal result. HIPRE allows the user to activate a value conversion function that eliminates the normalization of AHP judgments to check for rank reversals (Buede, 1992). Documentation is one of the weakest aspects of Expert Choice. Documentation does a good job of working through prepared demonstrations step-bystep, but does not present well, in a single location, the steps involved in creating a new decision model of ones own. Nonetheless, social scientists may appreciate the separate chapters on cost benet analysis, group decision-making and sensitivity analysis using this package (1985). The Super Decisions software is used for decisionmaking with dependencies and feedback (it implements the ANP with many additions). Such problems often occur in real life. Super Decisions extends the AHP that uses the same fundamental prioritization process based on deriving priorities through judgments on pairs of elements or from direct measurements. In the AHP, the elements are arranged in a hierarchic decision structure, whereas the ANP uses one or more at networks
Copyright 2012 John Wiley & Sons, Ltd.

of clusters that contain the elements. Most decisionmaking methods assume independence between the criteria of a decision and the alternatives of that decision, or simply among the criteria or among the alternatives themselves. The ANP is not limited by such assumptions. It allows for all possible and potential dependencies (www.superdecisions.com). Decision Lens (www.decisionlens.com) is a new software package. It is a prioritization software based on Saatys (1980, 1994b) AHP engaging key decision stakeholders in an organized decision framework. It helps create multiple scenarios integrating existing data with stakeholder judgments for compelling visualizations and performs sensitivity analysis that is extremely valuable for testing the impact of changing priorities on alternative business decision choices. Every decision-making project starts with an account creation with a client decision manager to work directly with client sponsors and advocates to understand the needs and challenges of a complex decision. 3. RESEARCH METHODOLOGY A literature review, based on a study of scholarly journals, was conducted as a research methodology to build a framework for AHP rank reversal research. To identify those journal papers that describe methodologies and rank reversals in AHP, an extensive search using library databases was carried out. The literature review was undertaken to identify articles in high-ranking journals, which provide the most valuable information to researchers and practitioners studying the AHP methods. Hence, conference proceeding papers, masters theses, doctoral dissertations, textbooks and unpublished working papers were excluded from the literature review. For the purpose of this review, 1983 was chosen as a starting date for the search, as it was the time when one of the rst papers on rank reversal and preservation was published by the journal Omega (Belton and Gear, 1983). We believe, therefore, that this year may be considered the starting point of the review. Based on the search in the library databases, 61 papers from 18 scholarly journals appeared on the subject of rank reversal and preservation. The rank reversal and preservation papers in scholarly journals were identied, analysed, classied, coded and recorded under a classication scheme, which is shown in Table III. As each paper was reviewed, it was classied by several categories: year of publication, authors geographical location, specic areas, other tools/methodologies, journal of publication
J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

Table III. The classication scheme for the literature review on analytic hierarchy process (AHP) rank reversals
Authors geographic location Specic area Journal of publication Omega Omega Journal of Mathematical Psychology Omega Management Science Management Science Decision Sciences Mathematical Modelling Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Supermatrix Absolute measurement Absolute measurement Eigenvector method BG revised AHP Other tools/ methodologies used Cause of rank reversal UK USA USA UK USA USA USA USA

S.N

Year of publication

Authors

1 2

1983 1984a

Belton and Gear Saaty and Vargas

1984b

Saaty and Vargas

Improper normalization Relevance or irrelevance of the new alternative Inconsistency

Copyright 2012 John Wiley & Sons, Ltd. USA USA Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Interval judgments Alternative Absolute measurement Mathematical Modelling European Journal of Operational Research Omega Decision Sciences Operations Research Letters Management Science Management Science Decision Sciences Management Science MAV function Management Science Omega Decision Sciences Unchanged criteria weights Structural dependence of the criteria on the alternatives (relative measurement) Inconsistency Structural dependence of criteria on alternatives The new alternative brings additional information on the number of alternatives and on their measurement with respect to the criteria, and the weights of the criteria change accordingly. Structural dependence of criteria on alternatives Uncertainty Aggregation rule Incorrect criteria assessments Arithmetic mean aggregation rule USA USA USA USA USA USA Canada MAV function Referenced AHP Geometric mean aggregation rule Inappropriate criteria weights Inappropriate criteria weights Non-linearity Relative measurement (alternatives depend on other alternatives) The scale used to measure the intensity of preference Aggregation rule Process of normalizing local priorities (Continues)

4 5

1985 1986

Belton and Gear Saaty

6 7

1987 1987a

Harker and Vargas Saaty

1987b

Saaty

1987c

Saaty

10

1987

Saaty and Vargas

LITERATURE REVIEW OF THE RANK REVERSAL PHENOMENON IN THE AHP

11 12 13

1988 1989 1990

Troutt USA Schoner and Wedley Canada Barzilai and Golany Canada and Israel

14 15 16 17

1990a 1990b 1990 1990

Dyer Dyer Forman Harker and Vargas

18

1990

Saaty

19 20

1990 1992

Trout and Tadisina

145

J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

Table III. (Continued)

146

S.N

Year of publication Specic area Journal of publication Cause of rank reversal

Authors

Authors geographic location Other tools/ methodologies used

21 The Netherlands Canada USA Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Alternative Absolute measurement Weight-ratio matrices Alternative Alternative Alternative Multiplicative AHP

1993

Schoner, Wedley and Choo Forman USA Alternative Substitution effect

22

1993

Lootsma

23

1993

Having one scale only and ignores scale dependence Incorrect criteria assessments

Copyright 2012 John Wiley & Sons, Ltd. Minimum and linking pins Relevance or irrelevance of the new alternative Axiom 3 Normalization Measurement Eigenvector normalization Relative measurement Criteria weights Arithmetic mean aggregation rule USA USA
H. MALEKI AND S. ZAHIR

24

1993

Schoner, Wedley and Choo Saaty and Vargas

25

1993

26 27

1994 1994a

Wedley, Schoner Canada and Choo Barzilai and Golany Canada and Israel Saaty USA

28

1994

Schenkerman

29

1994

Vargas

30

1995

Buede and Maxwell USA

31 32

1995 1997

33

1997

Perez Spain Barzilai and Lootsma Canada and the Netherlands Belton and Gear UK Multiplicative AHP Multiplicative AHP

Geometric mean aggregation rule

34

1997

Relative importance of the average performance on each criterion A sequence of inappropriate operations

35 Canada USA

1997

Lootsma and Barzilai The Netherlands and Canada Salo and Hmlinen Finland

36

1997

Normalization at the lowest level of the hierarchy -

37

1997

Schoner, Choo and Wedley Stam

Mathematical and Computer Modelling Journal of Multi-Criteria Decision Analysis European Journal of Operational Research Mathematical & Computer Modelling Journal of Multi-Criteria Decision Analysis INFOR European Journal of Operational Research European Journal of Operational Research European Journal of Operational Research Journal of Multi-Criteria Decision Analysis Management Science Journal of Multi-Criteria Decision Analysis Journal of Multi-Criteria Decision Analysis Journal of Multi-Criteria Decision Analysis Journal of Multi-Criteria Decision Analysis Journal of Multi-Criteria Decision Analysis Journal of Multi-Criteria Decision Analysis Decision Sciences

38 USA South Africa

1997

Stam and Silva

USA and Portugal

Alternative Alternative Alternative

39

1997

Vargas

Normalization and aggregation method used is incompatible with the ratio scale stochastic nature of the pairwise comparisons Axiom 3 Supermatrix-ideal mode Journal of Multi-Criteria Decision Analysis Interval judgments Journal of Multi-Criteria Decision Analysis Uncertainty in the decision-makers preference judgments

J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

40

1998

Honert

41 Alternative Alternative Alternative Criteria Alternative Alternative Criteria Alternative Alternative Criteria Alternative Alternative Alternative Criteria DEAHP TOP Multiple inputs or outputs

2000

Millet and Saaty

USA

Alternative

Ideal mode

Dependency on other alternatives

42

2000

43

2001

44 Canada USA and Portugal

2001

Sinuany-Stern, Israel Mehrez and Hadad Leung and Cao Hong Kong and China Triantaphyllou USA Relative comparisons in normalized ratios Normalization step and the use of additive function The number of hierarchy levels

45

2002

Finan and Hurley

46

2003

Stam and Silva

Copyright 2012 John Wiley & Sons, Ltd. Incorrect normalization and Inconsistency of pairwise comparisons Not to re-evaluate criteria weights European Journal of Operational Research International Transaction in Operational Research Sinarchy European Journal of Operational Research Multiplicative AHP Journal of Multi-Criteria Decision Analysis Computers & Operations Research Multiplicative AHP European Journal of Operational Research Linear Algebra and its Applications Computers & Operations Research Ratio-scale measurement Journal of the Operational Research Society Quality Engineering Consistency ratio and the number of alternatives Consistency ratio and the number of alternatives Dependent and normalization Singapore and Indonesia Spain Oman USA UK and China Taiwan Computers & Operations Research European Journal of Operational Research Decision Support Systems European Journal of Operational Research Criteria Alternative Alternative Change of local priorities before and after adding or deleting an alternative USA and Turkey China Alternative Alternative Alternative Measurement and independence Independence axiom Not to re-evaluate criteria weights Aggregation rule Aggregation rule The Netherlands and Canada The Netherlands and Canada Canada UK Mathematical and Computer Modelling Mathematical and Computer Modelling Journal of Multi-Criteria Decision Analysis Journal of Multi-Criteria Decision Analysis International Journal of Operational Research International Journal of Operational Research

47

2004

48

2004

49

2005

Farkas, Gyorgy and Hungary Rozsa Liberatore and USA Nydick Leskinen and Kangas Finland

50

2005

Raharjo and Endah

51

2006

52

2006

Perez, Jimeno and Mokotoff Ramanathan

53

2006

Saaty

54

2006

Wang and Elhag

55

2008

56

2009

Lin, Chou, Chouhuang and Hsu Saaty and Sagir

LITERATURE REVIEW OF THE RANK REVERSAL PHENOMENON IN THE AHP

57

2009

Wang and Luo

58

2009a

59

2009b

60

2009

Wijnmalen and Wedley Wijnmalen and Wedley Zahir

61

2011

Ramanathan and Ramanathan

147

J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

MAV, multi-attribute value; DEAHP, data envelopment analytic hierarchy process.

148

H. MALEKI AND S. ZAHIR

and the association of the AHP with other methods for avoiding rank reversal. Although this review cannot claim to be exhaustive, it covers a large portion of AHP publications on methodologies and rank reversals; hence, it is a useful source for AHP researchers and practitioners. 4. ANALYSIS OF RANK REVERSALS The phenomenon of rank reversal has been discussed and debated by AHP researchers in various contexts in reference to various types of causes. In order to remain focused in our literature review, we refer to cases where aggregate rank of a decision alternative is reversed with respect to one or more other alternatives in the decision problem when the decision model is modied in either of the two ways: (a) when new alternatives are added or old ones deleted and (b) when new criteria are added or old ones deleted. In the latter case, the caveat is that the priorities of the alternatives would be tied under these criteria and hence argued that the criteria should be irrelevant when ranking the alternatives. Reversals occur in aggregate ranks of the alternatives that are obtained by folding the criteria weights with the local normalized alternative priorities. The normalized criteria weights and the local alternative priorities are obtained through pairwise comparisons and ratio measurements. Rank reversals that followed such structural changes were attributed to the use of ratio measurements and normalization. Often such reversals of alternative ranks are argued as problematic by AHP researchers because intuitively such reversals are not expected to occur; however frequently, counter-arguments are advanced by other AHP researchers in the literature as well. The following sections review 61 scholarly papers on the basis of the two main specic areas. The rst area includes rank reversals that occurred by using (1) scale, (2) aggregation rule for deriving local weights and (3) aggregation rule for deriving nal weights. The last area covers the papers published in two elds: indifferent criteria and wash criteria. First, a small number of the papers are briey mentioned in each section. Then, in each topic, the entire papers are summarized in the specic tables. In order to offer a brief overview on AHP rank reversals published in each topic, the papers are arranged in alphabetical order by author. In the vast majority of rank reversals papers, a number of AHP extensions or modications have been proposed to prevent rank reversals. Those are included in the Other tools/methodologies used column of each table.
Copyright 2012 John Wiley & Sons, Ltd.

4.1. Change in structure by adding/deleting alternatives Out of all the papers reviewed, 56 of them (91.8%) have been related to adding/deleting alternatives. Table III presents rank reversal papers addressed on the topic of adding/deleting alternatives. When a scholarly paper might have belonged to more than one topic, the best possible topic was chosen to categorize it without duplication. In order to avoid the rank reversal, Belton and Gear (1983) suggested normalizing the eigenvector priorities of alternatives using their maximum value rather than their sum (in the literature, this methodology often is called BG modied AHP). Saaty and Vargas (1984a) provided a counterexample to show that BG modied AHP was also subject to rank reversal. Belton and Gear (1985) argued that their procedure was misunderstood and insisted that their approach would not result in any rank reversal if criteria weights were changed accordingly. Schoner and Wedley (1989) presented a referenced AHP to avoid a rank reversal phenomenon, which requires the modication of criteria weights when an alternative is added or deleted. Schoner et al. (1993) also suggested a method of normalization to the minimum value and a linking pin AHP (see also Schoner et al., 1997). In this method, one of the alternatives under each criterion is chosen as the link for criteria comparisons, and the values in the linking cells are assigned a value of 1, with proportional values in the other cells. Barzilai and Golany (1994) showed that no normalization could prevent rank reversal and suggested a multiplicative aggregation rule, which replaces normalized weight vectors with weightratio matrices, to avoid rank reversal. Lootsma (1993) and Barzilai and Lootsma (1997) suggested a multiplicative AHP for rank preservation. Vargas (1997) provided a practical counterexample to show the invalidity of the multiplicative AHP. Triantaphyllou (2001) offered two new cases to demonstrate that the rank reversals do not occur with the multiplicative AHP but do occur with the AHP and some of its additive variants. He concluded that the addition step of the synthesis process was the cause of certain ranking irregularities and suggested that the multiplicative variant should be used instead. Wijnmalen and Wedley (2009a) demonstrated that the cause was completely different and that additive synthesis, if used correctly, should not be abandoned. The supermatrix technique (Saaty, 1980) has been suggested as a remedy to rank reversals in the AHP (Harker and Vargas, 1987). Harker and Vargas (1987) and Vargas (1994) argued that criticisms of AHP were based on invalid applications; thus, AHP should not be used when criterion weights depended
J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

LITERATURE REVIEW OF THE RANK REVERSAL PHENOMENON IN THE AHP

149

on alternatives. When dependencies exist, ANP should be used instead. ANP is the general form of AHP, where dependencies among levels are allowed. Unfortunately, there are no clear-cut rules for determining when supermatrices should be used (Dyer, 1990a); moreover, the supermatrix technique requires that the DM answers a much larger number of questions. Hmlinen and Sepplinen (1986) were the rst to call the method the analytic network process and were indeed able to apply it in a policy problem with interrelationships between two planning horizons. Despite claims to the contrary, the supermatrix technique does not eliminate rank reversals (Salo and Hmlinen, 1997). Saaty (1997) discussed three ideas in the paper: (1) that rank reversal is legitimate and should be allowed for in a mathematically precise way; (2) Salo and Hmlinens example of the supermatrix is the wrong one for rank preservation; and (3) the supermatrix allows for both rank preservation and reversal just as the AHP itself does. Nevertheless, one should keep in mind that the mathematics underlying the aggregation procedure of the original additive AHP (A-AHP) may be suspect, with or without the supermatrix approach (Stam, 1997; Zahir, 2009). Leung and Cao (2001) showed that Sinarchy, a particular form of ANP, could prevent rank reversal. More recently, Ramanathan (2006) suggested data envelopment AHP, which is claimed to have no rank reversal phenomenon, but in fact, it still suffers from rank reversal (Wang and Elhag, 2006). In relative measurement, unlike measurement on a scale with an arbitrary unit where alternatives are assigned a value independently of other alternatives, nal ranks of alternatives (compared with several criteria) derived from their aggregate scores can change when alternatives are added or deleted (Watson and Freeling, 1982; Belton and Gear, 1983; Dyer and Ravinder, 1983; Dyer, 1990a). The question is not whether rank should be preserved, because it is widely believed that it cannot and should not always be preserved (Tversky et al., 1990), but whether or not the assumption of independence applies, an assumption used by most multicriteria methods. Utilitarian philosophers of the 18th century believed that people ought to desire those things that will maximize their utility. However, this utilitarian viewpoint was abandoned because it was deemed that utility was impossible to measure. Instead, structural accounts of rationality and formal denitions of utility such as rational choice theory were favoured. In rational choice theory, the criteria are assumed to be independent of utility and the condition empirically tested. But because the criteria
Copyright 2012 John Wiley & Sons, Ltd.

cannot be separated from the alternatives, the resulting criteria weights are not really important; rather, they are scaling constants. Consequently, according to strong advocates of this theory, independence of the criteria among themselves must be assumed (Keeney and Raiffa, 1976; Kamenetzky, 1982). Contrary to this assumption, in the AHP, everything can depend on everything else, including itself. Additionally, in the AHP, rank is always allowed to change. It is preserved only when the criteria are conditions imposed on the alternatives and possibly attributes that have had long standing acquired importance of their own apart from any particular alternative (Saaty, 1991). For example, we all have the habit of ascribing to human beings a kind of rationality to judge how the universe operates and then assign rationality a high priority (Saaty, 2010). 4.2. Change in structure by adding/deleting criteria Out of all reviewed papers, ve (8.2%) were related to adding/deleting criteria. Table III shows scholarly papers on the topic of adding/deleting criteria. The following paragraph presents a review of these ve papers. In general, it is known in decision-making that if one alters criteria or criteria weights, then the outcome of a decision will change, thus possibly leading to rank reversal. This is precisely what some authors use to criticize the AHP. There are two main criticisms. The rst is called wash criteria, which are assumed irrelevant because the alternatives have equal or nearly equal priorities under them. It has been pointed out (Finan and Hurley, 2002) that if such a criterion occurs in a lower level of hierarchy, its deletion may cause rank reversals. However, other researchers have emphasized the need for caution while deleting such wash criterion as discussed subsequently. The second one is called indifferent criteria, which involve the addition of a new criterion that is assumed irrelevant for the same reason as wash criteria (Perez et al., 2006). In the rst case, the authors made the error of renormalizing the weights of the remaining criteria that gave rise to rank reversal because the weights of the criteria were changed (Saaty and Vargas, 2006). In the second case, the addition of a new, yet irrelevant, criterion also led to rank reversal for exactly the same reason of changing the weights of the criteria. It is surprising that anyone would want to add irrelevant criteria and use it to make an important decision. This approach treats the weights of the criteria not as representative of their importance, but as scaling constants
J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

150

H. MALEKI AND S. ZAHIR

like in multi-attribute utility theory (Keeney and Raiffa, 1976). The correct approach to deal with wash and indifferent criteria is not to delete them or add them, but simply, in the former case, to assign zero priorities to the alternatives and keep that criterion. In the latter case, it is best not to add them. But, if added, it is better to consider this a new decision model/scenario, respecting the inuence of added criteria on the nal outcome, which could lead to different priorities and ranks (Saaty, 2010). 5. OTHER CLASSIFICATION RESULTS In the two previous sections, 61 scholarly papers were classied into rank reversal type categories according to the proposed scheme. This section of the paper analyses the classication of 61 papers according to the following attributes: (1) year of publication; (2) journal of publication; (3) authors geographic location; and (4) using the AHP in association with other methods. Attribute (1) is chosen because in this paper we are performing a literature review and it is always interesting to see how a topic of interest evolves with time by looking at the number of research contributions in successive years. The second attribute will help us understand the importance of the topic (i.e. rank reversals in the AHP) by looking at the journals that have published articles on the subject. The subject of multi-criteria decision-making has been enriched by intellectual contributions made by different North American and European schools of thoughts. Many of these institutions involved are located in the USA, Canada, France, the Netherlands and the Nordic countries. In addition, other institutions from across the globe made valuable contributions as well. Although it is understood that researchers often relocate from one country to another, the main centres of activities pertaining to a particular school of thought remain the dominant source of ideas. Therefore, by looking at the geographical dispersion of researchers, we can have an idea about the contributions made by the various schools of thought to this debate on rank reversal. In the previous sections, we attempted to identify how various articles compare with each other or are related to each other. Such a task is too strenuous, complex and time consuming. However, in our research, we have partially compensated this by adding the fourth attribute in the classication as we can indirectly measure, to some extent, how one article is related to another by looking at other methods that are used in association with the AHP for investigating the rank
Copyright 2012 John Wiley & Sons, Ltd.

reversal phenomenon. A better approach can be a network diagram showing a starting theme (i.e. addition/ deletion of alternative or criteria, normalization method, aggregation approach, consistency etc.) and then linking all articles contributing to the rank reversal debate pertaining to a particular theme and identifying the suggested solutions or conceptual supports. We leave such works for future research. 5.1. Distribution by year of publication Table IV gives the frequency distribution by year for rank reversal and preservation papers published in scholarly journals since 1983. As shown in the table, there has been a decreasing growth over time in the number and percentage of papers published on rank reversals. Until 1990, 19 papers were published. The number of papers published decreased during the period 19911993. Moreover, six papers (9.8%) were published during 19941996, nine (14.8%) during 19971999 and ve (8.2%) during 20002002. In recent years, the number of papers published on rank reversal topics has revived. More than 34.4% (21 papers) of all papers have been published since 2000. 5.2. Distribution by journal of publication Sixty-one papers are classied by journal of publication to develop a percentage of total papers. In this review, there have been 18 journals that have published at least one paper on the rank reversal topics. Nine out of these 18 journals just contributed to one paper, and nine journals contributed to more than one paper. Table V provides a list of journals arranged by the number and percentage. The table indicates that the Journal of Multi-Criteria Decision Analysis has published by far the largest number of papers (14 papers, i.e. 23%) on rank reversals, because it has always had a special focus on the AHP methodologies and rank reversals. There is a signicant difference in the volume of the papers published between this journal and the other journals. The European Journal of
Table IV. Frequency of papers by year of publication
Year Prior to 1990 19911993 19941996 19971999 20002002 20032005 20062008 Since 2009 Total N 19 6 6 9 5 5 5 6 61 % 31.2 9.8 9.8 14.8 8.2 8.2 8.2 9.8 100

J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

LITERATURE REVIEW OF THE RANK REVERSAL PHENOMENON IN THE AHP

151

Table V. Frequency of papers by journals

Journal name Journal of Multi-Criteria Decision Analysis European Journal of Operational Research Management Science Omega Decision Sciences Mathematical & Computer Modelling Computers & Operations Research Mathematical Modelling International Journal of Operational Research TOP INFOR Quality Engineering Journal of the Operational Research Society Linear Algebra and its Applications Decision Support Systems Journal of Mathematical Psychology Operations Research Letter International Transaction in Operational Research Total N 14 10 7 5 5 4 3 2 2 1 1 1 1 1 1 1 1 1 61 % 23 16.4 11.5 8.2 8.2 6.6 4.9 3.3 3.3 1.6 1.6 1.6 1.6 1.6 1.6 1.6 1.6 1.6

Table VI. Frequency of papers by authors geographic location

Country USA Canada The Netherlands UK Israel China Spain Finland Portugal Taiwan Hungary Hong Kong Oman Singapore Indonesia South Africa Turkey Total N 30 13 5 5 3 3 2 2 2 1 1 1 1 1 1 1 1 73 % 41.1 17.8 6.8 6.8 4.1 4.1 2.7 2.7 2.7 1.4 1.4 1.4 1.4 1.4 1.4 1.4 1.4 100

Operational Research, Management Science and Omega are the second, third and fourth ones, respectively, regarding the number and percentage of the total papers published. 5.3. Distribution by authors geographic location Having developed for more than 20 years in North America, many countries in Europe, Asia and Africa participated in the rank reversal and preservation publications. Table VI shows a geography distribution of the rank reversal and preservation papers, in number and percentage, published in different countries worldwide. Although most prolic authors are from the USA, Canada, the Netherlands and the UK, there have been 17 countries in the world that have contributed to at least one paper. The value N in Table VI stands for the total number of times authors from a country published a paper/ papers on the rank reversal and preservation topics. For instance, the US authors contributed to 30 papers. In addition, the Canadian, Dutch and British researchers contributed to 13, 5 and 5 papers, respectively. In this review, 48 out of 61 papers belong to one country, and 12 papers to two different countries; therefore, the total value of N was calculated as 73 (49 1 + 12 2). Although the number of papers published in many countries is small, as shown in Table VI, it can clearly highlight the importance of the European countries (N = 17), the Asian countries (N = 12) and the African countries (N = 1), compared with the North American
Copyright 2012 John Wiley & Sons, Ltd.

countries (N = 43), in which rank reversal and preservation have had a worldwide concern. 5.4. AHP used in association with other methods for avoiding rank reversal Ten out of 61 papers are reviewed, integrated into or compared with the performance of AHP methods with the other methods, such as data envelopment analysis (DEA, ANP and ratio estimation in magnitudes or decibells to rate alternatives which are non-dominated Ratio Estimation in Magnitudes or deci-Bells to Rate Alternatives which are Non-DominaTed (REMBRANDT). Table VII shows an alphabetical order by authors. The purpose of the comparative papers (eight papers) has been to dene the differences of ranking between the AHP methods and other MCDA methods and to avoid rank reversal in AHP. A small number of the comparative papers are briey described subsequently. Lootsma (1993) made a comparison between AHP and REMBRANDT, which showed REMBRANDT prevents rank reversal. Triantaphyllou (2001) revealed that multiplicative AHP is slightly preferable to AHP, on the basis of ranking irregularities. In comparison with original AHP, Stam and Silva (2003) indicated that the multiplicative AHP precludes certain types of rank reversals as the composite priority ratings continue to follow a ratio scale, even after normalization. The purpose of the hybrid integration papers has been to utilize the AHP methods together with other MCDA methods to avoid rank reversals. From the viewpoint of practical applications, developing hybrid
J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

152

H. MALEKI AND S. ZAHIR

Table VII. Authors use of other multi-criteria decision analysis (MCDA) techniques with the analytic hierarchy process (AHP)
Author(s) Leung and Cao (2001) Lootsma (1993) Ramanathan (2006) Ramanathan and Ramanathan (2011) Schoner and Wedley (1989) Schoner, Wedley and Choo (1993) Sinuany-Stern, Mehrez and Hadad (2000) Stam (1997) Stam and Silva (2003) Triantaphyllou (2001) Using the AHP in association with another MCDA technique A comparison of AHP and a particular form of ANPthe Sinarchy A comparison of REMBRANDT and original AHP A hybrid integration of DEA and AHP A comparison of multiplicative AHP and original AHP A comparison of referenced AHP and BG revised AHP A comparison of linking pins and supermatrix A hybrid integration of DEA and AHP A comparison of multiplicative AHP and original additive AHP A comparison of multiplicative AHP and additive AHP A comparison of multiplicative AHP and original AHP

REMBRANDT, ratio estimation in magnitudes or deci-bells to rate alternatives which are non-dominated; DEA, data envelopment analysis.

methods has made a more realistic and promising decision than the stand-alone AHP. A number of papers in this regard combined AHP with DEA (Sinuany-Stern et al., 2000; Ramanathan, 2006). The local priorities of alternatives, in this integration, were generated by the AHP and then aggregated by DEA. 6. FUTURE RESEARCH DIRECTIONS In this paper, a classication scheme and a comprehensive literature review are presented based on four criteria (publication year, journal, authors geographical location and association of the AHP with other methods for dealing with rank reversal) and two specic areas (addition/deletion of alternatives and addition/deletion of criteria). The ndings will help us understand the evolution of the subject of the research over time. Researchers will be able to map out the inuence of leading schools of thoughts located in specic geographic locations on the issue of rank reversal in multi-criteria decision analysis. In addition, researchers may sort out suggested methodologies for understanding and tackling rank reversals, compare each other and discover relations among them. Future research could be improved by including other causes of rank reversals as noted by researchers and considering conference proceeding papers, masters theses, doctoral dissertations and textbooks for more comprehensive review than as it is. 7. CONCLUSION This paper has presented an extensive review of the literature on AHP methodologies and rank reversals,
Copyright 2012 John Wiley & Sons, Ltd.

consisting of 61 papers from 18 scholarly journals. For this purpose, a classication scheme was developed to organize each paper into several categories. The rank reversal and preservation papers in the proposed scheme were categorized into specic areas and then were classied by year of publication, journal of publication and authors geographical location, while also using the AHP in association with other MCDA methods for avoiding rank reversals. Moreover, the papers under specic areas were further classied into two sub-topics related to addition or deletion of alternatives. The methodology used in this review has some limitations. First, the ndings are based on the data collected from scholarly journals, which do not include conference proceeding papers, masters theses, doctoral dissertations, textbooks and unpublished working papers in the rank reversal literature. Second, most of the ndings were based on English journals; hence, the journals in other languages were not included. Although this means that the review is not exhaustive, the authors believe that it provides a comprehensive review and includes the majority of papers that were published by scholarly journals. Therefore, this paper provides useful insights into the anatomy of the preventing rank reversal methods and offers academic researchers and practitioners a roadmap and framework for future investigations. ACKNOWLEDGEMENTS We sincerely thank the Associate Editor Luis G. Vargas and two anonymous referees for useful comments that were very helpful in preparing the improved revised version of the paper.
J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

LITERATURE REVIEW OF THE RANK REVERSAL PHENOMENON IN THE AHP

153

REFERENCES
Barzilai J, Golany B. 1990. Deriving weights from pairwise comparison matrices: the additive case. Operations Research Letters 9(6): 407410. Barzilai J, Golany B. 1994. AHP rank reversal, normalization and aggregation rules. INFOR 32(2): 5763. Barzilai B, Lootsma FA. 1997. Power relations and group aggregation in the multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis 6: 155165. Belton V. 1986. A comparison of the analytic hierarchy process and a simple multi-attribute value function. European Journal of Operational Research 26: 721. Belton V, Gear T. 1983. On a short-coming of Saatys method of analytic hierarchies. Omega 11: 228230. Belton V, Gear T. 1985. The legitimacy of rank reversala comment. Omega 13(3): 143144. Belton V, Gear T. 1997. On the meaning of relative importance (discussion paper). Journal of Multi-Criteria Decision Analysis 6: 335337. Buede DM. 1992. Software review. Three packages for AHP: Criterium, Expert Choice and HIPRE 3+. Journal of Multi-Criteria Decision Analysis 1: 119121. Buede DM, Maxwell DT. 1995. Rank disagreement: a comparison of multi-criteria methodologies. Journal of Multi-Criteria Decision Analysis 4: 121. Dyer JS. 1990a. Remarks on the analytic hierarchy process. Management Science 36: 249258. Dyer JS. 1990b. A clarication of Remarks on the analytic hierarchy process. Management Sciences 36: 274275. Dyer JS, Ravinder HV. 1983. Irrelevant alternatives and the analytic hierarchy process. Technical Report, Department of Management, The University of Texas at Austin. Dyer JS, Wendell RE. 1985. A critique of the analytic hierarchy process. Technical Report 84/85-4-24, Department of Management, The University of Texas at Austin. Expert Choice. 1985. Decision Support Software, Inc.: McLean, VA. Farkas A, Gyorgy A, Rozsa P. 2004. On the spectrum of pairwise comparison matrices. Linear Algebra and its Applications 385: 443462. Finan JS, Hurley WJ. 2002. The analytic hierarchy process: can wash criteria be ignored. Computers and Operation Research 29(8): 10251030. Forman EH. 1990. AHP is intended for more than expected value calculations. Decision Sciences 36: 671673. Forman EH. 1993. Facts and ctions about the analytic hierarchy process. Mathematical and Computer Modelling 17(45): 1926. Hmlinen RP, Sepplinen TO. 1986. The analytic network process in energy planning. Socio-Economic Planning Sciences 20: 399405. Harker PT, Vargas LG. 1987. The theory of ratio scale estimation: Saatys analytic hierarchy process. Management Science 33(11): 13831403. Harker PT, Vargas LG. 1990. Reply to Remarks on the analytic hierarchy process. Management Science 36: 269273.
Copyright 2012 John Wiley & Sons, Ltd.

Honert RC. 1998. Stochastic pairwise comparative judgments and direct ratings of alternatives in the REMBRANDT system. Journal of Multi-Criteria Decision Analysis 7: 8797. Kamenetzky RD. 1982. The relationship between the analytic hierarchy process and the additive value function. Decision Sciences 13(4): 702713. Keeney RL, Raiffa H. 1976. Decision with Multiple Objectives: Preferences and Value Tradeoffs. John Wiley & Sons: New York. Leskinen P, Kangas J. 2005. Rank reversals in multi-criteria decision analysis with statistical modeling of ratio-scale pairwise comparisons. Journal of the Operational Research Society 56(7): 855861. Leung LC, Cao D. 2001. On the efcacy of modeling multiattribute decision problems using AHP and Sinarchy. European Journal of Operational Research 132: 3949. Liberatore MJ, Nydick RL. 2004. Wash criteria and the analytic hierarchy process. Computers & Operations Research 31: 889892. Lin JS, Chou S, Chouhuang WT, Hsu CP. 2008. Note on Wash criterion in analytic hierarchy process. European Journal of Operational Research 185: 444447. Lootsma FA. 1991. Scale sensitivity and rank preservation in a multiplicative variant of the AHP and SMART. Report 9167, Faculty of Technical Mathematics and Informatics, Delft University of Technology, Delft, The Netherlands. Lootsma FA. 1993. Scale sensitivity in the multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis 2: 87110. Lootsma FA, Barzilai J. 1997. Response to the comments by Larichev, Korhonen and Vargas on Power relations and group aggregation in the multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis 6: 171174. Millet I, Saaty TL. 2000. On the relativity of relative measures accommodating both rank preservation and rank reversals in the AHP. European Journal of Operational Research 121: 205212. Perez J. 1995. Some comments on Saatys AHP. Management Science 41(6): 10911095. Perez J, Jimeno JL, Mokotoff E. 2006. Another potential shortcoming of AHP. TOP 14(1): 99111. Raharjo H, Endah D. 2005. Evaluating relationship of consistency ratio and number of alternatives on rank reversal in the AHP. Quality Engineering 18(1): 3946. Ramanathan R. 2006. Data envelopment analysis for weight derivation and aggregation in the analytic hierarchy process. Computers and Operations Research 33: 12891307. Ramanathan U, Ramanathan R. 2011. An investigation into rank reversal properties of the multiplicative AHP. International Journal of Operational Research 11(1): 5477. Saaty TL. 1980. The Analytic Hierarchy Process. McGrawHill International: New York. Saaty TL. 1986. Axiomatic foundations of the analytic hierarchy process. Management Science 32: 841855. Saaty TL. 1987a. Rank generation, preservation, and reversal in the analytic hierarchy process. Decision Science 18: 157177.
J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

154

H. MALEKI AND S. ZAHIR

Saaty TL. 1987b. Decision making, new information, ranking and structure. Mathematical Modelling 8: 125132. Saaty TL. 1987c. The analytic hierarchy process: what it is and how it is used. Mathematical Modelling 9: 161176. Saaty TL. 1990. An exposition of the AHP in reply to the paper Remarks on the analytic hierarchy process. Management Science 36: 259268. Saaty TL. 1991. Rank and the controversy about the axioms of utility theorya comparison of AHP and MAUT. The 2nd International Symposium on the Analytic Hierarchy Process, Pittsburgh, PA. Saaty TL. 1994a. Highlights and critical points in the theory and application of the analytic hierarchy process. European Journal of Operational Research 74: 426447. Saaty TL. 1994b. Fundamentals of Decision Making and Priority Theory with the AHP. RWS Publications: Pittsburgh, PA. Saaty TL. 1997. That is not the analytic hierarchy process: what the AHP is and what it is not. Journal of Multi-Criteria Decision Analysis 6: 320339. Saaty TL. 2001. Decision Making with Dependence and Feedback: The Analytic Network Process. RWS Publications, Pittsburgh, USA. Saaty TL. 2006. Rank from comparisons and from ratings in the analytic hierarchy/network processes. European Journal of Operational Research 168: 557570. Saaty TL. 2010. Principia Mathematica Decenendi. RWS Publications: Pittsburgh, PA. Saaty TL, Sagir M. 2009. An essay on rank preservation and reversal. Mathematical and Computer Modelling 49: 12301243. Saaty TL, Vargas LG. 1984a. The legitimacy of rank reversal. Omega 12(5): 513516. Saaty TL, Vargas LG. 1984b. Inconsistency and rank preservation. Journal of Mathematical Psychology 28: 205214. Saaty TL, Vargas LG. 1987. Uncertainty and rank order in the analytic hierarchy process. European Journal of Operational Research 32(13): 107117. Saaty TL, Vargas LG. 1993. Experiments on rank preservation and reversal in relative measurement. Mathematical and Computer Modelling 17(45): 1318. Saaty TL, Vargas LG. 2006. The analytic hierarchy process: wash criteria should not be ignored. International Journal of Management and Decision Making 7: 180188. Salo AA, Hmlinen RP. 1997. On measurement of preferences in the analytic hierarchy process. Journal of MultiCriteria Decision Analysis 6(6): 309319. Schenkerman S. 1994. Avoiding rank reversal in AHP decision-support models. European Journal of Operational Research 74: 407419. Schoner B, Wedley WC. 1989. Ambiguous criteria weights in AHP: consequences and solutions. Decision Sciences 20: 462475. Schoner B, Wedley WC, Choo EU. 1992. A rejoinder to Forman on AHP, with emphasis on the requirements of composite ratio scales. Decision Sciences 23: 509517. Schoner B, Wedley WC, Choo EU. 1993. A unied approach to AHP with linking pins. European Journal of Operational Research 64: 384392.
Copyright 2012 John Wiley & Sons, Ltd.

Schoner B, Choo EU, Wedley WC. 1997. A comment on Rank disagreement: a comparison of multi-criteria methodologies. Journal of Multi-Criteria Decision Analysis 6: 197200. Sinuany-Stern Z, Mehrez A, Hadad Y. 2000. An AHP/DEA methodology for ranking decision making units. International Transactions in Operational Research 7: 109124. Stam A. 1997. Short note on On the measurement of preference in the analytic hierarchy process by Salo AA, and Hmlinen RP. Journal of Multi-Criteria Decision Analysis 6(6): 320339. Stam A, Silva APD. 1997. Stochastic judgments in the AHP: the measurement of rank reversal probabilities. Decision Sciences 28(3): 655688. Stam A, Silva APD. 2003. On multiplicative priority methods for the AHP. European Journal of Operational Research 145(1): 92108. Triantaphyllou E. 2001. Two new cases of rank reversals when the AHP and some of its additive variants are used that do not occur with the multiplicative AHP. Journal of Multi-Criteria Decision Analysis 10: 1125. Triantaphyllou E, Mann SH. 1989. An examination of the effectiveness of four multi-dimensional decision-making methods: a decision-making paradox. International Journal of Decision Support Systems 5: 303312. Triantaphyllou E, Mann SH. 1994a. Some critical issues in making decisions with pairwise comparisons. In Proceedings of the Third International Symposium on the AHP, Forman EH (ed.). George Washington University: Washington, DC; 225235. Triantaphyllou E, Mann SH. 1994b. An evaluation of the AHP and the revised AHP when the eigenvalue method is used under a continuity assumption. Computers and Industrial Engineering 26: 609618. Triantaphyllou E, Mann SH. 1995. Using the analytic hierarchy process for decision making in engineering applications: some challenges. International Journal of Industrial Engineering: Applications and Practice 2(1): 3544. Trout M, Tadisina S. 1990. Corrigendum and further results: rank reversal and the dependence of priorities on the underlying MAV function. Omega 18(6): 655658. Troutt M. 1988. Rank reversal and the dependence of priorities on the underlying MAV function. Omega 16: 365367. Tversky A, Slovic P, Kahneman D. 1990. The causes of preference reversal. The American Economic Review 80(1): 204215. Vargas LG. 1994. Reply to Schenkermans avoiding rank reversal in AHP decision support models. European Journal of Operational Research 74: 420425. Vargas LG. 1997. Why the multiplicative AHP is invalid: a practical example. Journal of Multi-Criteria Decision Analysis 6(3): 169170. Wang Y, Elhag TMS. 2006. An approach to avoiding rank reversal in AHP. Decision Support Systems 42(3): 14741480. Wang Y, Luo Y. 2009. On rank reversal in decision analysis. Mathematical and Computer Modelling 49: 12211229. Watson SR, Freeling ANS. 1982. Assessing attribute weights. Omega 10(6): 582583.
J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda

LITERATURE REVIEW OF THE RANK REVERSAL PHENOMENON IN THE AHP

155

Wedley WC, Schoner B, Choo EU. 1993. Clustering dependence and ratio scales in AHP: rank reversals and incorrect priorities with a single criterion. Journal of Multi-Criteria Decision Analysis 2(3): 145158. Wiecek MM, Ehrgott M, Fadel G, Figueira JR. 2008. Editorial: multiple criteria decision making for engineering. Omega 36: 337339. Wijnmalen DJD, Wedley WC. 2009a. Correcting illegitimate rank reversals: proper adjustment of criteria weights

prevent alleged AHP intransitivity. Journal of MultiCriteria Decision 15: 135141. Wijnmalen DJD, Wedley WC. 2009b. Non-discriminating criteria in the AHP: removal and rank reversal. Journal of Multi-Criteria Decision 15: 143149. Winkler RL. 1990. Decision modeling and rational choice: AHP and utility theory. Management Science 36: 247248. Zahir S. 2009. Normalisation and rank reversals in the additive analytic hierarchy process: a new analysis. International Journal of Operational Research 4(4): 446467.

Copyright 2012 John Wiley & Sons, Ltd.

J. Multi-Crit. Decis. Anal. 20: 141155 (2013) DOI: 10.1002/mcda