Electronic Commerce Research and Applications 12 (2013) 124135

Contents lists available at SciVerse ScienceDirect

Electronic Commerce Research and Applications

journal homepage: www.elsevier.com/locate/ecra

A quality evaluation model for the design quality of online shopping websites
Selcuk Cebi
Department of Industrial Engineering, Karadeniz Technical University, Trabzon, Turkey

a r t i c l e

i n f o

Article history:
Received 31 January 2012
Received in revised form 1 December 2012
Accepted 1 December 2012
Available online 28 December 2012
Keywords:
Choquet integral
DEMATEL
Design characteristics
Fuzzy sets
Website design
Quality

a b s t r a c t
Design quality of websites is known to have a positive impact on user attention. Although there are lots of
papers that investigate design qualities of websites in the literature, none of them takes into account
interactions among design characteristics. The main objective of this study is to evaluate the perceived
design quality of websites while considering the interactions among design characteristics. For this purpose, a new approach named the quality evaluation model (QuEM), which includes fuzzy set theory, the
decision-making trial and evaluation laboratory method (DEMATEL), and generalized Choquet integral techniques, is proposed. The critical design characteristics of websites are determined based on interactions
among them by using DEMATEL. Then, the degrees of importance are utilized in the generalized Choquet
integral to evaluate the perceived design quality of website designs. Fuzzy set theory is utilized to cope
with ambiguity in the linguistic evaluation of the different design characteristics and supports the analysis of the collected data. Finally, the proposed method is applied for the evaluation of the quality of the
most used shopping websites in Turkey.
2012 Elsevier B.V. All rights reserved.

1. Introduction
In parallel with the development of technology on hardware
and software facilities in the computer world, the Internet has become an indispensable part of our daily lives. This leads to new
opportunities and new perspective on all aspects of business for
reaching a wider customer population (Hasan and Abuelrub
2011). In particular, electronic commerce (e-commerce) has become essential to our daily lives and represents a growing market
day by day. Firms and organizations, from those that are newlyfounded to the most well-known, try to reach a wide customer
base to maximize their benets by introducing their products or
services through their websites. A well-designed user interface
has a positive effect on customers to use the shopping website
again and again (Fan and Tsai 2010).
The design quality of websites has a considerable effect on customers willingness to shop (Gregg and Walczak 2010). The perceived design quality of websites and its effects on customers are
primary concerns for both online businesses and researchers
(McCoy et al. 2009) since website design quality deals with customers initial online purchase behavior (Zhou et al. 2009). The design quality of a website is based on design characteristics and it is
not an easy task to improve the design quality because of its complex nature, which depends on human expectations (Lee and Koubek 2010). Furthermore, design quality assessments of websites

Tel.: +90 462 3774135; fax: +90 462 3256482.

E-mail addresses: cebiselcuk@gmail.com, scebi@ktu.edu.tr
1567-4223/$ - see front matter 2012 Elsevier B.V. All rights reserved.
http://dx.doi.org/10.1016/j.elerap.2012.12.001

include a combination of quantitative and qualitative factors that

affect the perceived design quality from the customers viewpoint
(Cocquebert et al. 2010). Therefore, design quality assessments of
websites deals with multiple attribute problems. So multiple
attribute decision-making methods should be used to evaluate
the design quality of websites (Kaya and Kahraman 2011; Huang
and Huang 2010).
The perceived design quality of the websites has considered by
researchers over the last decade (Bai et al. 2008; Chiu et al. 2005;
Gregg and Walczak 2010; Jones and Kim 2010; Kaya 2010; Kaya
and Kahraman 2011; Kim and Niehm 2009; Kim and Stoel 2004;
Lee and Kozar 2006; McCoy et al. 2009; Wells et al. 2011; Cebi
2012). Most of these studies examine one or two design characteristics using experimental or survey techniques during their evaluation of perceived design quality of websites, however, a few of
them use multiple attribute techniques. It is necessary to consider
all of the design characteristics at once (Huang and Huang 2010).
Moreover, none of the research considered the interactions among
design characteristics.
To ll the gap in the literature, this article addresses the concern
for perceived design quality of online shopping websites with the
use of multiple criteria decision making methods, and by taking
into account the interactions among design characteristics. For this
purpose, an integrated method named the Quality Evaluation Model
(QuEM) is proposed to evaluate the design quality of online shopping websites. QuEM is based on fuzzy set theory, DEMATEL and
generalized Choquet integral techniques. The main reason to use
fuzzy logic is that the evaluation procedure is based on linguistic
terms and fuzzy set theory is generally used to dene the linguistic

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135

terms in the decision-making process and to describe the vagueness, ambiguity and subjectivity of human judgment (Kaya and
Kahraman 2011). The DEMATEL technique is used to assess the
importance of design characteristics and their interactions. The
generalized Choquet integral is also utilized to evaluate the perceived design quality of shopping websites. The main reason why
the generalized Choquet integral is used is to consider the existing
interactions between design characteristics. It is an appropriate
representation of fuzzy measures, which capture the importance
of individual criteria or their combination. Also, it is used as an
aggregation operator in multi-criteria decision making under
uncertainty (Tsai and Lu 2006).
This article is organized as follows. Section 2 reviews the literature review on design characteristics used for the evaluation of
website design quality and presents the design characteristics used
for the evaluation. Section 3 describes QuEM. In Section 4, the proposed methodology is applied to popular online shopping websites
in Turkey. Section 5 presents the results, and Section 6 concludes.

125

The design characteristics used in this research have been compiled from the prior literature. They are shown in hierarchical
structure in Table 2 (Cebi 2013). The website design characteristics
consist of four main criteria: usability, visual aspects, technical
adequacy, security, communication, and prestige. Usability includes ease of use, ease of learning, and memorability. Visual aspects involve layout, graphics, and text. Technical adequacy
includes system availability speed accessibility and navigation.
Security consists of reliability, accuracy, and privacy. Communication includes contact info, online help, and responsiveness. And
prestige involves reputation, sustainability, and currency.
3. Methodology
In this article, I propose an integrated methodology, QuEM, to
evaluate the perceived design quality of shopping websites. The
structure of the proposed methodology is given in Fig. 1.
3.1. DEMATEL method

2. Design characteristics
The most used design characteristics and methodologies in the
literature have been presented in Table 1 (Cebi 2013).
Survey, experimental, and multiple criteria decision making
(MCDM) techniques have been widely used for the evaluation of
website design quality (Cebi 2013). In the survey method, data
are collected from participants who answer a questionnaire. Experimental evaluation requires participants to accomplish a specied
task by following a detailed set of instructions (Chiou et al. 2011).
In the MCDM method, the evaluation is based on expert evaluations by taking multi-dimensional factors into consideration. Many
studies evaluate design quality of websites using a wide variety of
criteria. However, taking the overall criteria into consideration is a
required procedure. MCDM methods are the best option to handle
a large number of criteria during the evaluation of website design
quality (Huang and Huang 2010). Research utilizing MCDM techniques has been increasing. None of the studies analyzes the interactions among the design characteristics though.

The decision-making trial and evaluation laboratory method

(DEMATEL) was introduced to the literature between 1972 and
1976 in order to represent the relationship between the causes
and effects of some criteria into an intelligible structure (Fontela
and Gabus 1974). DEMATEL has been commonly used to extract
the interrelationship between multiple design characteristics. It
is accepted as an effective approach to analyze the structure and
relationships among criteria. DEMATEL denes the design characteristics of a system as cause and effect groups that are relevant
in decisions (Chang and Cheng 2011). The method ranks the criteria with respect to type and importance of interrelationships
among the criteria. If any criterion has more effect on another, it
will have a higher priority and will be assigned as a cause criterion.
Otherwise, if the criterion is more affected by another criteria, it
will have a lower priority and be an effect criteria (Tseng and Lin
2009, Tzeng et al. 2011a). Many studies have used DEMATEL to
determine interactions among different criteria and importance
of them (Huang et al. 2011; Lin et al. 2011; Shen et al. 2010,

Table 1
Design characteristics used in academic studies between 2006 and 2011. (See above-mentioned references for further information.)

126

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135

Table 2
Hierarchy of website design characteristics (Cebi, 2013).
Website design parameters

Main design parameters

Sub-design parameters

Explanation

C1

Usability
C11
C12
C13

Ease of use
Ease of learning
Memorability

The users should reach its aim in short time while using the site rst time
The users should be adapted the site in short time
The users should remember the functions presented by the site

Visual Aspects
C21
C22
C23

Layout
Graphics
Text

The site should present good visual organization

The site should present good tonality
The site should present readable font

Technical adequacy
C31
C32
C33
C34

System availability
Speed
Accessibility
Navigation

The
The
The
The

Security
C41
C42
C43

Reliability
Accuracy
Privacy

The service protect users from hackers attack while downloading a le or surng
The service provide correct information
The site protects users information

Communication
C51
C52
C53

Contact Info
Online Help
Responsiveness

The site should provide contact addresses and phone numbers

The site should provide an assistance service through phone or internet
The site should handle users problems and return to users in a short time

Prestige
C61
C62
C63

Reputation
Sustainability
Currency

The site should be well known

The site should guarantee to serve for a long time
The site should provide continuous improvement

C2

C3

C4

C5

C6

site
site
site
site

must be reached any time

should provide quick loading, accessing, and using
should provide easy access to materials
should provide easy navigation to reach services

Fig. 1. Structure of the proposed methodology.

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135

2011; Shieh et al. 2010; Tsai and Hsu 2010; Tzeng et al. 2011a,
2011b; Wu et al. 2011; Wu and Tsai 2011; Yang and Tzeng 2011;
Zhou et al. 2011).
The DEMATEL methods procedure is as follows (Tseng and Lin
2009; Wu and Tsai 2011):
 Step 1. An initial direct relation matrix is obtained by evaluating the direct inuence between any two criteria.
 Step 2. A normalization procedure is applied to the initial
direct relation matrix.
 Step 3. The total relation (T) matrix is calculated by following formula:

T Z  I  Z1

where Z and I are the normalized matrix and identity matrix.

 Step 4. The cause and effect diagram and the importance
degrees of criteria are obtained.
In the rst step of the DEMATEL procedure, experts assessments are used. In the literature, Fuzzy approaches are widely used
to deal with the ambiguity of human assessments among researchers. Therefore, the preferences of decision makers are extended to
fuzzy numbers by adopting fuzzy linguistic scale. I use a fuzzy version of DEMATEL to analyze interrelations among website design
characteristics in uncertain environment (Wu et al. 2011; Zhou
et al. 2011). Fuzzy triangular numbers has been adopted into DEMATEL procedure for linguistic assessments instead of numerical
values (Mohamadnejad et al. 2011).
3.2. The generalized Choquet integral

127

they may have different impacts on the nal decision. One

of the most used approaches to determine the weights
involves a moderator that assigns expertise points in the
range [0100] for each expert. Evaluations are normalized
to obtain weights in the range of [01]. A pairwise comparison also can be done to determine the experts weights. If
there is not a moderator, each expert also can grade the
other experts in the range [0100] based on knowledge of
their expertise (Cebi and Kahraman 2010).
Step 2. Collect the experts preferences. Interactions among
design characteristics, website design quality and expected
design quality are evaluated by experts.
Step 2.1. Create the direct relation matrix. The pairwise come k describes the interactions between
parison matrix M
design characteristics, and is constructed by each expert
in linguistic form, with the following assessments: no inuence, very weak inuence, weak inuence, strong inuence, and extremely strong inuence. The pairwise
comparison matrix is given by:

 
ek m
~ ij nn
M

~ ij is the preference of the kth expert for the ith criterion in

where m
comparison to the jth criterion, and n represents the number of de~ ij is a triangular fuzzy
sign characteristics (Tzeng et al. 2010). m


u
~ ij mlij ; mm
number represented with three points: m
ij ; mij .
e k of
Step 2.2. Create the quality matrix. The design quality Q
the alternatives under consideration are evaluated by the
team of experts

 
ek q
~ij lxn
Q

The Choquet and Sugeno integrals are used as aggregation operators (Choquet 1954; Murofushi and Sugeno 1989; Chiang 2000;
Tsai and Lu 2006; Beliakov et al. 2007; Grabisch et al. 2009; Tan
and Chen 2011). There are similiarities and slightly differences between the Choquet and Sugeno integrals since bothinvolve the
same formal model and take into account interactions between design parameters (de Campos and Jorge 1992; Torra and Narukawa
2007; Grabisch and Labreuche 2008). The Choquet integral depends on an appropriate representation with fuzzy measures,
which capture the importance of individual criteria or combinations of them. The Choquet integral represents the importance of
a criterion and interactions between criteria (Demirel et al. 2010;
Tsai and Lu 2006; Laurence et al. 2000). In this research, the generalized Choquet integral proposed by Auephanwiriyakul et al.
(2002) and developed by Tsai and Lu (2006), which extends the
standard version is used. In the generalized Choquet integral, measurement is made in terms of intervals instead of real numbers
(Auephanwiriyakul et al. 2002; Tsai and Lu 2006). The main reason
why it is used for the evaluation phase is that it considers existing
interactions between design characteristics, and involves linguistic
expressions, as well as information fusion between different criteria to overcome the vagueness and the imprecision of linguistic
terms (Tsai and Lu 2006). It offers good performance when the
decisions are made under uncertainty (Schmeidler 1989; Grabisch
and Labreuche 2008).

e k represents the design quality of the jth alternative for the

where Q
ith design characteristic according to the kth member, and l repre~ij is a triangular fuzzy number
sents the number of alternatives. q


u
~ij qlij ; qm
represented with three points as follows: q
ij ; qij .

3.3. Integrated quality evaluation model

~ij and ~eij symbolize the aggregated elements of the di~ ij , q

where m
e and
rect relation matrix, quality matrix, and expected quality. Let A
e be two triangular fuzzy numbers represented with three points as
B
(al, am, au) and (bl, bm, bu). Then, the arithmetic operations  and 
are dened as:

I next will discuss an integrated quality evaluation model and

its various phases.
 Phase I. Initial Phase: In the initial phase, the basic denitions and boundaries of the problem are identied.
Step 1. Construct a team of experts. After constructing a team
of experts, it is necessary to determine their weights, since

Step 2.3. Determine the expected quality of the shopping website. The expected quality e
E k of a shopping website is determined for each design criteria i by each member of expert
team via:

e
E k ~ei 1n

~ei also is a triangular fuzzy number represented with three points



u
as follows: ~ei eli ; em
i ; ei .
Step 3. Aggregate expert preferences. The experts preferences for the direct-relation, expected quality, and quality
matrices are aggregated. The aggregation of fuzzy numbers
is performed by applying a fuzzy weighted averaging operator, dened by:

~ ij m
~ 1ij  we1  m
~ 2ij  we2      m
~ kij  wek
m

~ij q
~1ij  we1  q
~2ij  we2      qkij  wek
q

~ei e1i  we1  e2i  we2      eki  wek



 
eB
e al ; am ; au  bl ; bm ; bu
A


l
m
u
al b ; am b ; au b

128

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135



eB
e al ; am ; au  bl ; bm ; bu
A


l
m
u
al  b ; am  b ; au  b :
 Phase II. Determining interactions between design characteristics: fuzzy DEMATEL
Step 4. Normalize the direct relation matrix. The normalized
e ij is obtained in which all principal
direct relation matrix Z
diagonal elements are equal to zero (Mohamadnejad et al.
2011; Lin and Tzeng 2009; Wu and Lee 2007), as follows:

"
#
1
1
e
e
P
P
M
Z min
;
i;j
max16i6n nj1 muij max16j6n ni1 muij

e represents the direct efSince the sum of each row j of matrix M

fects that design characteristic and i gives to the other design char

1P
1P
acteristic, minij
;
represents the direct
n
n
u
u
max16i6j

j1

mij

max16j6j

i1

mij

effects of the design characteristics (Liou et al. 2008).

Step 5. Obtain the total relation matrix. The total relation
e is calculated by following formula:
matrix T


1
e IZ
e
Te Z

where I is identity matrix (Mohamadnejad et al. 2011).

e matrices. R
e are n  1 and
e and C
e and C
Step 6. Obtain the R
1  n vectors representing the sum of the rows and the
e . Let ~r i
sum of the columns of the total relation matrix T
be the sum of the ith row, and ~cj the sum of jth column
e . While ~ri presents both the direct and indirect
in matrix T
effects given by the ith design characteristic to the other
design characteristics, ~cj shows both the direct and indirect
effects by the jth design characteristic from the other
e shows
eC
design characteristics. If j = i, the sum e
SR
the total effects given and received by the ith design characteristic. The sum indicates the importance of the ith
design characteristic (Wu et al. 2010; Wu and Tsai 2011).


e depicts the net efe R
eC
In contrast, the difference D
fect that ith design characteristic contributes to the system


e dl ; dm ; du . Based on the difference values, design
where D
i

characteristics are classied: (1) if the difference is positive



~ P 0 , the ith design characteristic is a net cause
rli P cuj ; d
i
 m

(NC), (2) if the difference is weakly positive di P 0 , the
ith design characteristic is a weak cause (WC), (3) if the dif

~i < 0 , the ith design characference is negative cl P r u ; d
i

teristic is a net receiver (NR), (4) if the difference is weakly

m
negative di < 0, the ith design characteristic is a weak receiver (WR), and otherwise, (5) the design characteristic is
inconsistent.
e values. Since the e
e val Step 7. Standardize the e
S and D
S and D
ues are non-standardized fuzzy numbers, they need to be
converted to standardized fuzzy numbers. The standardization operation is as follows
(Olcer and Odabasi 2005).

u
Assume that X xli ; xm
is a non-standardized fuzzy numi ; xi
ber. The standardized fuzzy numbers may be calculated by



u
X s xli =m; xm
i =m; xi =m

10

u
0 6 xls 6 xm
s 6 xs 6 1

11

and
where m is the maximum value of xui under each criterion level.
After the application of the standardization procedure, crisp
weights and overall weights are calculated by using a suitable

defuzzication procedure to present the priorities of the design

characteristics in a graphical way.
 Phase III. Determining the design quality of the designs: generalized Choquet integral.
Step 8. Normalize design quality. Normalize the design quality of each design characteristic based on (Tsai and Lu
2006):

~f
i

k f ai

a20;1

k
a20;1

fi;a ; fi;a

12

where fi 2 FS is a fuzzy-valued function. e

F S is the set of all fuzh
i pa ea 1;1
ai and eai are a-lezy-valued functions f ; fia fi;a ; fi;a i i2
,p
~i and ~ei for all a 0; 1.
vel cuts of p
Step 9. Calculate design quality. The design quality of the
main design characteristic j is calculated based on: (Tsai
and Lu 2006).

~f dg~

k
a0;1



Z
Z


C fa dg a ; C fa dg a ;

13



a
where gi : PS ! IR , gi g 
i ; g i , g i g i;a ; g i;a , f i : S ! IR ,
and fi fi ; fi  for i = 1, 2, 3, . . . , nj. To calculate design quality, a k
value and the fuzzy measures g(A(i)), I = 1, 2, . . . , n, are needed.
These are obtained from the following equations (Demirel et al.
2010; Tsai and Lu 2006).

gS g

n
[
Ai
i1

)
8 ( n
Y
>
>
1
>
1

kgA


1
if k 0
i
>k
<
i1

n
>
X
>
>
>
gAi
:

if k 0

i1

14
where A \ B u for all i, j = 1, 2, . . . , n and i j.
In the last equation, k 0 indicates that the k-fuzzy measure g is
non-additive; otherwise, the k-fuzzy measure g indicates that there
is no interaction between Ai and Aj for i j. Thus, k > 0 implies that
g A [ B > gA gB and the set {A, B} has a multiplicative effect;
and k < 0 indicates the substitutive effect of the set {A, B} (Chen
and Tzeng 2001).
Let g be a fuzzy measure on S. The Choquet integral of a function f : S ! 1; 0, with respect to fuzzy measures g is dened
R
P
by
C f dg ni1 f si  f si1 gAi ,
where
0 6 f s1 6 f s2 6    6 f sn 6 1, f s0 0 and Ai
fsi ; . . . ; sn g (Tsai and Lu 2006). With k-fuzzy measure of
Sugeno can be solved recursively by following equations
(Tsai and Lu 2006):

gAn gfsn g g n

15

gAi g i gAi1 kg i gAi1 ; where 1 6 i < n

16

where Ai \ Aj u for all i, j = 1, 2, 3, . . . , n and i j, and k 2 1; 1.

Here, f si represents the ith smallest normalized discrepancy between the perceived design quality and the expected
design quality for all design parameters corresponding to the
fuzzy measure gi for all i.
Step 10. Calculate overall quality. Overall design quality is
obtained by using a hierarchical process that applies a
two-stage aggregation process of the generalized Choquet
integral. The overall performance levels yield a fuzzy nume (Tsai and Lu 2006):
ber, V

R
main criterion1 C f dg
..
.

iV C

main criterion dg

main criterionm C f dg
17

129

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135

Table 3
Linguistic scale for interactions among website design characteristics (Olcer and
Odabasi 2005).
Linguistic term

Acronym

Corresponding triangular fuzzy

numbers

No inuence
Very weak inuence
Weak inuence
Strong inuence
Extreme strong
inuence

N
V
W
S
E

(0.00,
(0.00,
(0.30,
(0.50,
(0.70,

0.00,
0.25,
0.50,
0.75,
1.00,

0.30)
0.50)
0.70)
1.00)
1.00)

Table 4
Linguistic scale for quality evaluation (Wu et al. 2011).
Linguistic term

Acronym

Corresponding triangular fuzzy numbers

Very low
Low
Moderate
High
Very high

VL
L
M
H
VH

(0.00,
(0.00,
(0.25,
(0.50,
(0.75,

0.00,
0.25,
0.50,
0.75,
1.00,

0.25)
0.50)
0.75)
1.00)
1.00)

Step 11. Defuzzify the fuzzy numbers. Assume that the meme is l x. The fuzzy number V
e can be defuzzbership of V
e
V
ied into a crisp value v using a suitable method to make
comparisons among the design quality drivers (Tsai and
Lu 2006).
 Phase IV. Output.
Step 12. Set up a threshold value to obtain the interaction diagrams. It is necessary to defuzzify the total relation matrix,
and set up a threshold value to lter out some negligible
effects by using a suitable defuzzication procedure. Thus,
only effects bigger than the threshold value are chosen and
shown in the diagram (Wu et al. 2010; Wu and Tsai 2011).
Step 13. Interpret the design qualities of the alternatives. The
weakness and advantages of the characteristics among
design alternatives are compared (Tsai and Lu 2006).
4. Application
To demonstrate the application of the proposed approach in this
research, I applied the integrated model three of the frequentlyused online shopping websites in Turkey. These websites offer a
wide range of products, including cosmetics, clothing, electronics,
and white goods.
 Phase I. Initial Phase.
Step 1. Construct a team of experts. In this step, two phases of
evaluation are conducted. The rst is to evaluate the interactions between the design characteristics in Table 2. This was
implemented by an expert team consisting of ve members.
The second is to evaluate the quality of the online shopping
websites. This evaluation was accomplished through a survey
of eighteen participating graduate students whose ages were
between 18 and 25. All participants were required to spend
time to evaluate each online shopping website.
Step 2. Collect the experts preferences. In this step, the preferences of experts and the survey group are transformed into triangular fuzzy numbers. Recall that the evaluation responses are
given are in linguistic form, since the evaluators preferences
deal with interactions among the design characteristics and
website quality.
Step 2.1. Create a direct relation matrix. The linguistic terms for
pairwise comparisons to obtain interactions between criteria
are transformed into triangular fuzzy numbers by using the linguistic scale given in Table 3.

Step 2.2. Create a quality matrix. The design qualities of the alternatives next are evaluated by the survey group. To transform
the linguistic response terms used in the survey, the scale given
in Table 4 is utilized.
Step 2.3. Determine the expected quality. The expected quality of
a shopping website is determined based on a consensus of the
team members opinions. See Table 9.
Step 3. Aggregate the preferences. Then, the preferences for the
direct relation matrix and quality matrix are aggregated. In this
step, the weights of the expert team members and the survey
participants are viewed as being equal. The aggregated matrices
for the direct relation and design quality of websites are given
in Tables 5 and 9.
Phase II. Determining interactions between the design characteristics: fuzzy DEMATEL.
Step 4. Normalize the direct relation matrix. The normalized direct
e ij in Table 6 is obtained with Eq. (8).
relation matrix Z
e
Step 5. Obtain the total relation matrix. The total relation matrix T

given in Table 7 is calculated based on Eq. (9).

e matrices. The R
e , matrices, the
e and C
e and C
Step 6. Obtain the R
e , and the type of
eC
weights of the design characteristics R
e are presented in Table 8.
eC
design characteristics based on R
e The fuzzy weights given
Step 7. Standardize the values of e
S and D.
in Table 8 are defuzzied. Then, the relative and overall weights
are obtained. Based on the interactions between the main
design characteristics, the ranking of the design characteristics
from the most important to least important is: technical adequacy, prestige, security, usability, visual aspects, and communication. See Fig. 2.
 Phase III. Determining the quality of the designs: generalized Choquet integral.
Step 8. Normalize design quality. Table 9 presents the standardized fuzzy weights of the design characteristics, the expected
design quality, and the aggregated performance evaluations of
the online shopping websites. In Table 9, standardized weights
and performance evaluations are given in trapezoidal form,
since the expected design quality is described by trapezoidal
fuzzy numbers. The performance quality of websites under
the each criterion is normalized by using Eq. (12) (Tsai and Lu
2006), as follows:

ai  eai 1; 1
p
2
0:49; 0:92  0:25; 1 1; 1
0:24; 0:83

f10 f1;0
; f1;0


f20 0:24; 0:84;

f30 0:26; 0:85

Their corresponding degrees of importance are:

g01 0:61; 0:77;

g02 0:64; 0:74;

g03 0:64; 0:74


Step 9. Calculate design quality. To obtain design quality, fi;0
is



sorted as follows: f1;0 0:237 6 f2;0 0; 243 6 f3;0 0:243. The
corresponding importance measures for the design characteris02 0:64, and g03 0:64 also. After Eq. (14) is
tics are: g01 0:61, g
solved, the k value obtained is 0.928. Then, the related fuzzy
measures can be calculated based on Eqs. (15) and (16) (Tsai
and Lu 2006), as follows:

gA3 a0 g3;0 0:64

gA2 a0 g 2;0 gA3 a0 kg2;0 gA3 a0 0:90
gA1 a0 g 1;0 gA2 a0 kg1;0 gA2 a0 1
R
P


According to Eq. (13), C f  dg 3i1 fi  fi1
gAi 0:252 and
R
R  
C f dg 0:845. So C f dg a0 0:252; 0:845.

130

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135

Table 5
Pairwise comparisons representing dependencies among the design parameter (aggregated direct relation matrices).

C1
C2
C3
C4
C5
C6

C1

C2

C3

C4

C5

C6

(0, 0, 0.3)
(0.58, 0.85, 0.85)
(0.26, 0.45, 0.7)
(0.2, 0.3, 0.58)
(0.3, 0.5, 0.63)
(0.06, 0.1, 0.38)

(0.16, 0.3, 0.56)

(0, 0, 0.3)
(0.24, 0.35, 0.48)
(0.16, 0.25, 0.52)
(0.06, 0.15, 0.42)
(0.1, 0.15, 0.44)

(0.26, 0.4, 0.66)

(0.38, 0.55, 0.53)
(0, 0, 0.3)
(0.34, 0.6, 0.75)
(0.2, 0.3, 0.58)
(0.26, 0.4, 0.66)

(0.16, 0.25, 0.52)

(0, 0.1, 0.38)
(0.34, 0.6, 0.75)
(0, 0, 0.3)
(0.2, 0.3, 0.4)
(0.24, 0.45, 0.58)

(0.24, 0.35, 0.48)

(0.06, 0.15, 0.42)
(0.2, 0.35, 0.62)
(0.2, 0.3, 0.4)
(0, 0, 0.3)
(0.06, 0.1, 0.38)

(0.3, 0.45, 0.58)

(0.3, 0.5, 0.76)
(0.58, 0.85, 1)
(0.58, 0.85, 0.85)
(0.24, 0.4, 0.62)
(0, 0, 0.3)

C11

C12

C13

C11
C12
C13

(0, 0, 0.3)
(0.46, 0.7, 0.94)
(0.46, 0.7, 0.85)

(0.54, 0.8, 1)
(0, 0, 0.3)
(0.58, 0.85, 0.85)

(0.54, 0.8, 0.9)

(0.58, 0.85, 0.85)
(0, 0, 0.3)

C31

C32

C33

C34

C31
C32
C33
C34

(0, 0, 0.3)
(0.2, 0.3, 0.4)
(0.2, 0.35, 0.62)
(0.1, 0.2, 0.48)

(0.24, 0.35, 0.48)

(0, 0, 0.3)
(0.38, 0.6, 0.6)
(0.16, 0.35, 0.6)

(0.4, 0.65, 0.9)

(0.62, 0.9, 1)
(0, 0, 0.3)
(0.12, 0.25, 0.5)

(0.3, 0.5, 0.63)

(0.58, 0.85, 0.85)
(0.28, 0.45, 0.68)
(0, 0, 0.3)

C51

C52

C53

(0, 0, 0.3)
(0.2, 0.35, 0.45)
(0.12, 0.2, 0.46)

(0.2, 0.3, 0.4)

(0, 0, 0.3)
(0.28, 0.5, 0.72)

(0.28, 0.45, 0.68)

(0.46, 0.7, 0.94)
(0, 0, 0.3)

C51
C52
C53

C21
C22
C23

C41
C42
C43

C61
C62
C63

Step 10. Calculate overall quality. The last two steps are repeated
to obtain the overall design quality of the online shopping websites given. See Table 10.
Step 11. Defuzzify the fuzzy numbers. The overall website design
quality values are defuzzied using the centroid method. It provides a crisp value based on the center of gravity, and is the
most commonly used method in the literature (Opricovic and
Tzeng 2004). The results suggest that Website 2 has the best
and website 3 has the worst design quality.
 Phase IV. Output.
Step 12. Set up a threshold value to obtain the interaction diagrams. The diagrams show causal relations among the design
characteristics. See Fig. 3.
Step 13. Interpret the design qualities of alternatives. The strengths
and weaknesses of the websites are presented in Table 10. The
ranking of the alternatives based on their design characteristics
are presented in Fig. 4.

C21

C22

C23

(0, 0, 0.3)
(0.38, 0.6, 0.82)
(0.28, 0.5, 0.72)

(0.22, 0.45, 0.68)

(0, 0, 0.3)
(0.06, 0.25, 0.5)

(0.26, 0.45, 0.7)

(0.18, 0.35, 0.58)
(0, 0, 0.3)

C41

C42

C43

(0, 0, 0.3)
(0.54, 0.8, 0.9)
(0.52, 0.8, 0.75)

(0.44, 0.65, 0.83)

(0, 0, 0.3)
(0.16, 0.3, 0.56)

(0.26, 0.5, 0.6)

(0.06, 0.2, 0.46)
(0, 0, 0.3)

C61

C62

C63

(0, 0, 0.3)
(0.48, 0.75, 0.83)
(0.54, 0.8, 0.9)

(0.22, 0.45, 0.68)

(0, 0, 0.3)
(0.4, 0.65, 0.8)

(0.16, 0.35, 0.6)

(0.14, 0.35, 0.45)
(0, 0, 0.3)

5. Results
We rst analyze the importance of each design characteristics
based on their interactions. From Table 8, we see that technical adequacy is the most important design characteristic, and communication criterion is the least important. The characteristics prestige and
security come second and third. Except for usability, a net receiver
factor, the rest of the main characteristics are net cause factors.
Moreover, from Fig. 3, all of the design characteristics directly affect prestige, while prestige only affects technical adequacy. In addition to prestige, communication has an impact on usability without
being inuenced by any of the other design characteristics. Visual
aspects affect usability and technical adequacy, however prestige is
a design characteristic that no other characteristics affect. Furthermore, technical adequacy seems to impact usability, security, and
prestige, while security affects usability, technical adequacy, and
prestige. In addition, usability inuences technical adequacy,

Table 6
Normalized relation matrix.

C1
C2
C3
C4
C5
C6

C11
C12
C13

C31
C32
C33
C34

C51
C52
C53

C1

C2

C3

(0, 0, 0.06)
(0.25, 0.24,
(0.11, 0.13,
(0.09, 0.08,
(0.13, 0.14,
(0.03, 0.03,

(0.07, 0.08, 0.12)

(0, 0, 0.06)
(0.1, 0.1, 0.1)
(0.07, 0.07, 0.11)
(0.03, 0.04, 0.09)
(0.04, 0.04, 0.09)

(0.11, 0.11,
(0.17, 0.15,
(0, 0, 0.06)
(0.15, 0.17,
(0.09, 0.08,
(0.11, 0.11,

C11

C12

C13

(0, 0, 0.14)
(0.41, 0.42, 0.43)
(0.41, 0.42, 0.39)

(0.48, 0.48, 0.45)

(0, 0, 0.14)
(0.52, 0.52, 0.39)

(0.48, 0.48, 0.41)

(0.52, 0.52, 0.39)
(0, 0, 0.14)

C31

C32

C33

C34

(0, 0, 0.11)
(0.14, 0.15, 0.15)
(0.14, 0.17, 0.23)
(0.07, 0.1, 0.18)

(0.17, 0.17, 0.18)

(0, 0, 0.11)
(0.27, 0.29, 0.22)
(0.11, 0.17, 0.22)

(0.29, 0.32, 0.33)

(0.44, 0.44, 0.37)
(0, 0, 0.11)
(0.09, 0.12, 0.19)

(0.21, 0.24, 0.23)

(0.41, 0.41, 0.31)
(0.2, 0.22, 0.25)
(0, 0, 0.11)

C51

C52

C53

(0, 0, 0.16)
(0.27, 0.3, 0.23)
(0.16, 0.17, 0.24)

(0.27, 0.26, 0.21)

(0, 0, 0.16)
(0.38, 0.43, 0.38)

(0.38, 0.39, 0.35)

(0.62, 0.61, 0.49)
(0, 0, 0.16)

0.18)
0.15)
0.12)
0.13)
0.08)

0.14)
0.11)
0.16)
0.12)
0.14)

C4

C5

C6

(0.07, 0.07, 0.11)

(0, 0.03, 0.08)
(0.15, 0.17, 0.16)
(0, 0, 0.06)
(0.09, 0.08, 0.08)
(0.1, 0.13, 0.12)

(0.1, 0.1, 0.1)

(0.03, 0.04, 0.09)
(0.09, 0.1, 0.13)
(0.09, 0.08, 0.08)
(0, 0, 0.06)
(0.03, 0.03, 0.08)

(0.13, 0.13, 0.12)

(0.13, 0.14, 0.16)
(0.25, 0.24, 0.21)
(0.25, 0.24, 0.18)
(0.1, 0.11, 0.13)
(0, 0, 0.06)

C21
C22
C23

C41
C42
C43

C61
C62
C63

C21

C22

C23

(0, 0, 0.16)
(0.58, 0.55, 0.45)
(0.42, 0.45, 0.39)

(0.33, 0.41, 0.37)

(0, 0, 0.16)
(0.09, 0.23, 0.27)

(0.39, 0.41, 0.38)

(0.27, 0.32, 0.32)
(0, 0, 0.16)

C41

C42

C43

(0, 0, 0.15)
(0.51, 0.5, 0.46)
(0.49, 0.5, 0.38)

(0.42, 0.41, 0.42)

(0, 0, 0.15)
(0.15, 0.19, 0.29)

(0.25, 0.31, 0.31)

(0.06, 0.13, 0.24)
(0, 0, 0.15)

C61

C62

C63

(0, 0, 0.15)
(0.47, 0.48, 0.41)
(0.53, 0.52, 0.44)

(0.22, 0.29, 0.33)

(0, 0, 0.15)
(0.39, 0.42, 0.39)

(0.16, 0.23, 0.3)

(0.14, 0.23, 0.22)
(0, 0, 0.15)

131

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135

Table 7
Total relation matrix.

C1
C2
C3
C4
C5
C6

C1

C2

C3

(0.09, 0.1, 0.3)

(0.33, 0.33, 0.41)
(0.23, 0.26, 0.43)
(0.2, 0.2, 0.37)
(0.2, 0.22, 0.35)
(0.1, 0.11, 0.29)

(0.12,
(0.07,
(0.17,
(0.14,
(0.08,
(0.08,

0.14, 0.31)
0.08, 0.26)
0.18, 0.33)
0.15, 0.32)
0.1, 0.27)
0.09, 0.26)

(0.21,
(0.28,
(0.15,
(0.27,
(0.18,
(0.18,

0.21,
0.27,
0.17,
0.29,
0.18,
0.19,

0.38)
0.36)
0.36)
0.42)
0.35)
0.35)

C11

C12

C13

C11
C12
C13

(4.56, 5.25, 8.16)

(4.75, 5.47, 8.14)
(4.75, 5.47, 7.85)

(5.56, 6.25, 8.53)

(5.12, 5.84, 8.04)
(5.46, 6.18, 7.97)

(5.56, 6.25, 8.24)

(5.46, 6.18, 7.98)
(5.12, 5.84, 7.52)

C41

C42

C43

C31
C32
C33
C34

(0.17, 0.28, 0.92)

(0.35, 0.48, 1.04)
(0.29, 0.41, 0.99)
(0.15,0.26, 0.86)

(0.41,
(0.35,
(0.47,
(0.22,

C61

C62

C63

(0.35, 0.43, 0.92)

(0.65, 0.78, 1.15)
(0.46, 0.58, 1.07)

(0.73, 0.83, 1.15)

(0.66, 0.81, 1.3)
(0.75, 0.92, 1.37)

(0.97, 1.06, 1.47)

(1.28, 1.41, 1.82)
(0.64, 0.79, 1.44)

C51
C52
C53

0.55,
0.51,
0.62,
0.39,

1.05)
1.07)
1.05)
0.95)

(0.57,
(0.77,
(0.34,
(0.24,

C4

C5

(0.15, 0.17, 0.32)

(0.1, 0.15, 0.3)
(0.25, 0.29, 0.41)
(0.11, 0.13, 0.29)
(0.16, 0.16, 0.28)
(0.15, 0.2, 0.3)

(0.15,
(0.11,
(0.17,
(0.16,
(0.06,
(0.07,

C6
0.16,
0.12,
0.19,
0.16,
0.06,
0.08,

0.28)
0.28)
0.35)
0.28)
0.23)
0.24)

C21
C22
C23
C44

0.73,
0.93,
0.48,
0.41,

1.42)
1.55)
1.19)
1.14)

(0.53,
(0.78,
(0.52,
(0.17,

0.69,
0.94,
0.68,
0.31,

1.26)
1.42)
1.24)
0.99)

security, and prestige. Thus, we can conclude that these characteristics have strong interactions.
The ranking for the design sub-characteristics for usability is
ease of learning rst, memorability second, and ease of use third.
Ease of learning and memorability are net receivers, while ease of
e value. In Fig. 3, we see that ease
use is a net cause based on the D
of use affects both ease of learning and memorability, but is not
inuenced itself by any other design sub-characteristics. Moreover, ease of learning and memorability appear to affect each other
though.

C41
C42
C43

C61
C62
C63

(0.26, 0.27, 0.4)

(0.29, 0.31, 0.44)
(0.42, 0.43, 0.54)
(0.4, 0.41, 0.48)
(0.23, 0.24, 0.39)
(0.11, 0.13, 0.32)
C21

C22

C23

(0.73, 1.28, 2.79)

(1.23, 1.71, 3.08)
(0.84, 1.42, 2.75)

(0.65, 1.24, 2.53)

(0.49, 1.01, 2.41)
(0.41, 1.02, 2.27)

(0.85, 1.33, 2.68)

(0.88, 1.35, 2.69)
(0.44, 0.91, 2.3)

C51

C52

C53

(0.59, 0.74, 2.09)

(0.87, 1.01, 2.29)
(0.91, 1.06, 2.16)

(0.73, 0.84, 2.12)

(0.41, 0.51, 1.87)
(0.57, 0.71, 1.93)

(0.44, 0.65, 1.73)

(0.3, 0.51, 1.64)
(0.26, 0.42, 1.51)

C71

C72

C73

(0.33, 0.6, 1.72)

(0.76, 1.06, 1.9)
(1, 1.28, 2.28)

(0.4, 0.69, 1.7)

(0.29, 0.56, 1.52)
(0.71, 1.01, 2.04)

(0.27, 0.53, 1.4)

(0.3, 0.6, 1.32)
(0.26, 0.53, 1.51)

The design sub-characteristics of layout, text, and graphics under

visual aspects occupy the rst, second and third positions in terms
of importance. Also, graphics is a net causes, while layout and text
are net receivers. The graphics characteristic inuences both the
layout and the text design characteristics without being inuenced
itself by any other design characteristics. Additionally, layout and
text also seem to affect each other.
The ranking of design sub-characteristics for technical adequacy
is speed and accessibility occupy the rst position, ahead of navigation, which is followed by system availability. System availability and

Table 8
The importance degrees of the design characteristics.
R
C1
C2
C3
C4
C5
C6

(0.98,
(1.18,
(1.37,
(1.28,
(0.91,
(0.69,

C
1.05,
1.27,
1.52,
1.34,
0.96,
0.79,

1.99)
2.05)
2.41)
2.16)
1.86)
1.76)

(1.14,
(0.67,
(1.26,
(0.92,
(0.72,
(1.71,

1.22, 2.14)
0.74, 1.74)
1.31, 2.22)
1.1, 1.89)
0.77, 1.66)
1.79, 2.57)

Fuzzy weights
(R + C)

Standardized fuzzy
weights

RC

Type of
criterion

Weights Overall
weights

(2.13, 2.27, 4.13)

(1.85, 2.01, 3.79)
(2.63, 2.83, 4.63)
(2.2, 2.44, 4.05)
(1.63, 1.74, 3.51)
(2.41, 2.58, 4.34)

(0.46, 0.49, 0.89)

(0.4, 0.43, 0.82)
(0.57, 0.61, 1)
(0.47, 0.53, 0.87)
(0.35, 0.38, 0.76)
(0.52, 0.56, 0.94)

(0.16, 0.17, 0.16)

(0.51, 0.53, 0.31)
(0.11, 0.21, 0.2)
(0.37, 0.23, 0.26)
(0.19, 0.19, 0.2)
(1.02, 0.99, 0.81)

NR
NC
NC
NC
NC
NR

0.61
0.55
0.73
0.62
0.50
0.67

0.17
0.15
0.20
0.17
0.13
0.18

C11 (15.68, 17.75,

24.93)
C12 (15.33, 17.49,
24.16)
C13 (15.33, 17.49,
23.34)

(14.06, 16.19,
24.15)
(16.14, 18.27,
24.54)
(16.14, 18.27,
23.74)

(29.74, 33.94,
(0.61, 0.69, 1)
49.08)
(31.47, 35.76, 48.7) (0.64, 0.73, 0.99)

(1.62, 1.56, 0.78)

NC

0.77

0.33

(0.81, 0.78, 0.38)

NR

0.79

0.34

(31.47, 35.76,
47.08)

(0.81, 0.78, 0.4)

NR

0.78

0.33

C21 (2.23, 3.85, 8)

C22 (2.6, 4.07, 8.18)
C23 (1.69, 3.35, 7.32)

(2.8, 4.41, 8.62)

(1.55, 3.27, 7.21)
(2.17, 3.59, 7.67)

(5.03, 8.26, 16.62) (0.3, 0.5, 1)

(4.15, 7.34, 15.39) (0.25, 0.44, 0.93)
(3.86, 6.94, 14.99) (0.23, 0.42, 0.9)

(0.57, 0.56, 0.62)

(1.05, 0.8, 0.97)
(0.48, 0.24, 0.35)

NR
NC
NR

0.60
0.54
0.52

0.36
0.33
0.32

4.65)
5.08)
4.47)
3.94)

(0.96, 1.43, 3.81)

(1.23, 1.68, 3.17)
(1.68, 2.14, 4.16)
(2, 2.62, 4.91)

(2.64, 3.68, 8.46)

(3.48, 4.54, 8.25)
(3.3, 4.33, 8.63)
(2.78, 3.99, 8.85)

(0.3, 0.42, 0.96)

(0.39, 0.51, 0.93)
(0.37, 0.49, 0.98)
(0.31, 0.45, 1)

(0.72, 0.82, 0.84)

(1.02, 1.18, 1.91)
(0.06, 0.05, 0.31)
(1.22, 1.25, 0.97)

NC
NC
WC
NR

0.56
0.61
0.61
0.59

0.23
0.25
0.26
0.25

C41 (1.76, 2.23, 5.94)

C42 (1.58, 2.03, 5.8)
C43 (1.74, 2.19, 5.6)

(2.37, 2.81, 6.54)

(1.71, 2.06, 5.92)
(1, 1.58, 4.88)

(4.13, 5.04, 12.48) (0.33, 0.4, 1)

(3.29, 4.09, 11.72) (0.26, 0.33, 0.94)
(2.74, 3.77, 10.48) (0.22, 0.3, 0.84)

(0.61, 0.58, 0.6)

(0.13, 0.03, 0.12)
(0.74, 0.61, 0.72)

NR
NR
NC

0.58
0.51
0.45

0.37
0.33
0.29

C51 (2.05, 2.32, 3.54)

C52 (2.59, 3, 4.27)
C53 (1.85, 2.29, 3.88)

(1.46, 1.79, 3.14)

(2.14, 2.56, 3.82)
(2.89, 3.26, 4.73)

(3.51, 4.11, 6.68)

(4.73, 5.56, 8.09)
(4.74, 5.55, 8.61)

(0.41, 0.48, 0.78)

(0.55, 0.65, 0.94)
(0.55, 0.64, 1)

(0.59, 0.53, 0.4)

(0.45, 0.44, 0.45)
(1.04, 0.97, 0.85)

NC
NC
NR

0.56
0.71
0.73

0.28
0.36
0.37

C61 (1, 1.82, 4.82)

C62 (1.35, 2.22, 4.74)
C63 (1.97, 2.82, 5.83)

(2.09, 2.94, 5.9)

(1.4, 2.26, 5.26)
(0.83, 1.66, 4.23)

(3.09, 4.76, 10.72) (0.29, 0.44, 1)

(2.75, 4.48, 10)
(0.26, 0.42, 0.93)
(2.8, 4.48, 10.06)
(0.26, 0.42, 0.94)

(1.09, 1.12, 1.08)

(0.05, 0.04, 0.52)
(1.14, 1.16, 1.6)

NR
NR
NC

0.58
0.54
0.54

0.35
0.33
0.33

C31
C32
C33
C34

(1.68,
(2.25,
(1.62,
(0.78,

2.25,
2.86,
2.19,
1.37,

(0.64, 0.73, 0.96)

132

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135

Fig. 2. The importance degrees of sub-design characteristics based on interactions.

speed are net causes, while accessibility and navigation are weak
causes and net receivers. According to Fig. 3, navigation is affected
by other design characteristics, though it does not affect any design
characteristics itself. Also, system availability and speed have direct
impacts on both accessibility and navigation, while system availability and speed are not inuenced by other characteristics.
Reliability is the most important among the design sub-characteristics under security, whereas privacy is the least important. In
addition, accuracy and reliability are net receivers, while privacy is
a net cause. From Fig. 3, we can see that privacy affects the others
without being inuenced, and accuracy and reliability seem to affect each other.
The importance of the sub-criteria of communication design
characteristics is ordered as follows: responsiveness is rst, online
help is next, and contact info is third. However, online help and contact information are net causes, whereas responsiveness is a net receiver. In addition, both online help and contact information only
affect responsiveness, but responsiveness inuences online help.

For the design sub-characteristics prestige, the ranking is reputation is more important than sustainability, which, in turn, is more
important than currency. Also, reputation and sustainability are
net causes, while currency is a net receiver. From Fig. 3, we can further see that currency affects the others without being inuenced,
and that reputation is inuenced by other criteria without affecting
them.
When we consider the overall weights, privacy, reliability, and
reputation are the most important design sub-characteristics. The
ranking for all the design sub-characteristics is presented in
Fig. 2 and their weights included in Table 8.
According to Table 10, the perceived design quality of Website 1
is close to Website 2. Although Website 1 has advantages with respect to certain design characteristics, its design quality must be
improved in terms of usability, security, communication (except for
online help), and prestige (except for currency). Website 2 has the
best design quality, since it has more advantages than the others
with respect to the various criteria. However, the design quality

Table 9
The standardized fuzzy weights of design characteristics, expected design quality, and aggregated performance evaluations.
Standardized importance degrees

Expected quality

Performance evaluation of online shopping websites

Website 1

Website 2

Website 3

C1
C11
C12
C13

(0.46,
(0.61,
(0.64,
(0.64,

0.89)
1)
0.99)
0.96)

(0.25, 0.75, 1, 1)
(0.25, 0.75, 1, 1)
(0.25, 0.75, 1, 1)

(0.49, 0.74, 0.74, 0.92)

(0.49, 0.74, 0.74, 0.93)
(0.51, 0.76, 0.76, 0.94)

(0.5, 0.75, 0.75, 0.96)

(0.49, 0.74, 0.74, 0.94)
(0.57, 0.82, 0.82, 0.97)

(0.39, 0.63, 0.63, 0.85)

(0.44, 0.69, 0.69, 0.89)
(0.4, 0.64, 0.64, 0.85)

C2
C21
C22
C23

(0.4, 0.43, 0.43, 0.82)

(0.3, 0.5, 0.5, 1)
(0.25, 0.44, 0.44, 0.93)
(0.23, 0.42, 0.42, 0.9)

(0.25, 0.75, 1, 1)
(0.25, 0.75, 1, 1)
(0.25, 0.75, 1, 1)

(0.51, 0.76, 0.76, 0.96)

(0.49, 0.74, 0.74, 0.93)
(0.5, 0.75, 0.75, 0.96)

(0.51, 0.76, 0.76, 0.96)

(0.46, 0.71, 0.71, 0.9)
(0.44, 0.69, 0.69, 0.9)

(0.4, 0.65, 0.65, 0.88)

(0.4, 0.64, 0.64, 0.85)
(0.47, 0.72, 0.72, 0.92)

C3
C31
C32
C33
C34
C4
C41
C42
C43

(0.57, 0.61, 0.61, 1)

(0.3, 0.42, 0.42, 0.96)
(0.39, 0.51, 0.51, 0.93)
(0.37, 0.49, 0.49, 0.98)
(0.31, 0.45, 0.45, 1)
(0.47, 0.53, 0.53, 0.87)
(0.33, 0.4, 0.4, 1)
(0.26, 0.33, 0.33, 0.94)
(0.22, 0.3, 0.3, 0.84)

(0.5,
(0.5,
(0.5,
(0.5,

(0.57,
(0.61,
(0.57,
(0.47,

(0.53,
(0.54,
(0.54,
(0.44,

(0.53,
(0.49,
(0.51,
(0.42,

(0.5, 0.75, 1, 1)
(0.5, 0.75, 1, 1)
(0.5, 0.75, 1, 1)

(0.58, 0.82, 0.82, 0.92)

(0.53, 0.76, 0.76, 0.93)
(0.53, 0.76, 0.76, 0.92)

(0.6, 0.85, 0.85, 0.96)

(0.54, 0.79, 0.79, 0.93)
(0.57, 0.82, 0.82, 0.94)

(0.38, 0.61, 0.61, 0.82)

(0.4, 0.64, 0.64, 0.85)
(0.43, 0.68, 0.68, 0.86)

C5
C51
C52
C53

(0.35,
(0.41,
(0.55,
(0.55,

0.38,
0.48,
0.65,
0.64,

0.38,
0.48,
0.65,
0.64,

0.76)
0.78)
0.94)
1)

(0.5, 0.75, 1, 1)
(0.5, 0.75, 1, 1)
(0.25, 0.75, 1, 1)

(0.43, 0.67, 0.67, 0.85)

(0.32, 0.56, 0.56, 0.78)
(0.38, 0.58, 0.58, 0.81)

(0.46, 0.71, 0.71, 0.9)

(0.25, 0.47, 0.47, 0.72)
(0.4, 0.65, 0.65, 0.88)

(0.4, 0.65, 0.65, 0.86)

(0.25, 0.47, , 0.47, 0.72)
(0.29, 0.51, 0.51, 0.74)

C6
C61
C62
C63

(0.52,
(0.29,
(0.26,
(0.26,

0.56,
0.44,
0.42,
0.42,

0.56,
0.44,
0.42,
0.42,

0.94)
1)
0.93)
0.94)

(0.25, 0.75, 1, 1)
(0.25, 0.75, 1, 1)
(0.25, 0.75, 1, 1)

(0.57, 0.82, 0.82, 0.97)

(0.56, 0.81, 0.81, 0.96)
(0.61, 0.86, 0.86, 1)

(0.63, 0.88, 0.88, 0.99)

(0.6, 0.85, 0.85, 0.99)
(0.54, 0.78, 0.78, 0.92)

(0.46, 0.71, 0.71, 0.89)

(0.49, 0.74, 0.74, 0.93)
(0.47, 0.72, 0.72, 0.9)

0.49,
0.69,
0.73,
0.73,

0.49,
0.69,
0.73,
0.73,

0.75,
0.75,
0.75,
0.75,

1,
1,
1,
1,

1)
1)
1)
1)

0.82,
0.86,
0.82,
0.72,

0.82,
0.86,
0.82,
0.72,

0.97)
0.97)
0.99)
0.94)

0.78,
0.79,
0.79,
0.69,

0.78,
0.79,
0.79,
0.69,

0.92)
0.94)
0.96)
0.93)

0.78,
0.74,
0.75,
0.67,

0.78,
0.74,
0.75,
0.67,

0.93)
0.9)
0.93)
0.89)

133

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135

Table 10
The design qualities of online shopping websites.
Website design quality

Defuzzied website quality

Website 1

Website 2

Website 3

Website 1

Website 2

Website 3

C1
C11
C12
C13

(0.275,
(0.252,
(0.243,
(0.243,
(0.257,

(0.282, 0.412, 0.545, 0.865)

(0.271, 0.4, 0.523, 0.859)
(0.25, 0.375, 0.5, 0.854)
(0.243, 0.368, 0.493, 0.847)
(0.285, 0.41, 0.535, 0.861)

(0.258,
(0.214,
(0.194,
(0.222,
(0.201,

0.389,
0.339,
0.313,
0.347,
0.319,

0.519,
0.463,
0.438,
0.472,
0.444,

0.835)
0.814)
0.799)
0.819)
0.799)

0.512
0.494
0.484
0.486
0.498

0.526
0.513
0.495
0.488
0.523

0.500
0.458
0.436
0.465
0.441

C2
C21
C22
C23

(0.249, 0.377, 0.504, 0.853)

(0.257, 0.382, 0.507, 0.854)
(0.243, 0.368, 0.493, 0.84)
(0.25, 0.375, 0.5, 0.854)

(0.235,
(0.257,
(0.229,
(0.222,

0.367,
0.382,
0.354,
0.347,

0.498,
0.507,
0.479,
0.472,

0.848)
0.854)
0.826)
0.826)

(0.209,
(0.201,
(0.201,
(0.236,

0.339,
0.326,
0.319,
0.361,

0.474,
0.451,
0.444,
0.486,

0.827)
0.813)
0.799)
0.833)

0.496
0.5
0.486
0.495

0.487
0.5
0.472
0.467

0.463
0.448
0.441
0.479

C3
C31
C32
C33
C34

(0.285,
(0.285,
(0.306,
(0.285,
(0.236,

0.416, 0.541, 0.743)

0.41, 0.535, 0.736)
0.431, 0.556, 0.736)
0.41, 0.535, 0.743)
0.361, 0.486, 0.722)

(0.262,
(0.264,
(0.271,
(0.271,
(0.222,

0.391,
0.389,
0.396,
0.396,
0.347,

0.516,
0.514,
0.521,
0.521,
0.472,

0.729)
0.708)
0.722)
0.729)
0.715)

(0.248,
(0.264,
(0.243,
(0.257,
(0.208,

0.376,
0.389,
0.368,
0.375,
0.333,

0.501, 0.715)
0.514, 0.715)
0.493, 0.701)
0.5, 0.715)
0.458, 0.694)

0.496
0.491
0.507
0.493
0.451

0.474
0.469
0.477
0.479
0.439

0.46
0.47
0.451
0.462
0.424

C4
C41
C42
C43

(0.273,
(0.292,
(0.264,
(0.264,

0.393, 0.527, 0.71)

0.41, 0.535, 0.708)
0.382, 0.507, 0.715)
0.382, 0.507, 0.708)

(0.284,
(0.299,
(0.271,
(0.285,

0.411, 0.544, 0.727)

0.424, 0.549, 0.729)
0.396, 0.521, 0.715)
0.41, 0.535, 0.722)

(0.281,
(0.188,
(0.201,
(0.215,

0.409, 0.543, 0.727)

0.306, 0.431, 0.66)
0.319, 0.444, 0.674)
0.34, 0.465, 0.681)

0.476
0.486
0.467
0.465

0.491
0.5
0.476
0.488

0.49
0.396
0.41
0.425

C5
C51
C52
C53

(0.193, 0.309, 0.445, 0.721)

(0.215, 0.333, 0.458, 0.674)
(0.16, 0.278, 0.403, 0.639)
(0.188, 0.292, 0.417, 0.778)

(0.196,
(0.229,
(0.125,
(0.201,

0.326,
0.354,
0.236,
0.326,

0.464,
0.479,
0.361,
0.451,

0.782)
0.701)
0.611)
0.813)

(0.164,
(0.201,
(0.125,
(0.146,

0.287,
0.326,
0.236,
0.257,

0.43, 0.723)
0.451, 0.681)
0.361, 0.611)
0.382, 0.743)

0.417
0.42
0.37
0.418

0.442
0.441
0.333
0.448

0.401
0.415
0.333
0.382

C6
C61
C62
C63

(0.287,
(0.285,
(0.278,
(0.306,

(0.292,
(0.313,
(0.299,
(0.271,

0.421,
0.438,
0.424,
0.389,

0.556,
0.563,
0.549,
0.514,

0.866)
0.868)
0.868)
0.833)

(0.235,
(0.229,
(0.243,
(0.236,

0.362,
0.354,
0.368,
0.361,

0.49, 0.836)
0.479, 0.819)
0.493, 0.84)
0.486, 0.826)

0.531
0.523
0.516
0.542

0.534
0.545
0.535
0.502

0.481
0.47
0.486
0.477

0.416, 0.527, 0.828)

0.378, 0.503, 0.845)
0.368,0.493, 0.833)
0.368, 0.493, 0.84)
0.382, 0.507, 0.847)

0.417, 0.548, 0.871)

0.41, 0.535, 0.861)
0.403, 0.528, 0.854)
0.431, 0.556, 0.875)

The bold values depict the best design quality in the criterion compared with other websites.

Fig. 3. The casual relations among the design characteristics.

Fig. 4. The rank of the alternatives with respect to the design characteristics.

of Website 1 is better than Website 2 with respect to technical adequacy and visual aspects. These criteria must be improved to increase the perceived design quality of Website 2. Website 3 has
the poorest design quality since its performance is weak under
all of the criteria.

6. Conclusion
Firms try to improve design quality of their websites in order to
strengthen their competitiveness. In this research, the most recent
evaluation methods and criteria were used in the evaluation of

134

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135

design quality of websites. Nineteen sub-criteria and six main criteria were used to evaluate the perceived design quality of shopping websites. No research to date has addressed the interactions
among the design characteristics of websites. Thus, we prposed
an integrated methodology, the quality evaluation model (QuEM),
based on fuzzy sets, DEMATEL and the generalized Choquet integral, to evaluate the design quality of shopping websites. The DEMATEL technique is used to determine the relative importance of
design parameters based on interactions among them. Then, the
generalized Choquet integral is used to evaluate different website
design alternatives. The main reasons why the generalized Choquet integral is used for the evaluation phase include: it can take
into account existing interactions between design parameters; it
is represented with intervals; and it involves linguistic expressions
as well as information fusion between criteria to overcome the
vagueness and imprecision of linguistic terms in evaluation (Tsai
and Lu 2006). The Choquet integral offers good performance, particularly when decisions are made under uncertainty (Schmeidler
1989). Further, it is the most used technique in multiple criteria
decision-making. The key contribution of this research is to take
into account interactions among the design characteristics of websites by utilizing fuzzy sets, to assess the perceived design quality
of websites. I also dened a new classication for design characteristics of websites based on the interactions among them.
A primary conclusion of this research is that technical adequacy
is the most important characteristic among the main design characteristics for online shopping websites. Prestige and security are
second and third. Based on the overall weights, privacy, reliability,
and reputation are the most important design sub-characteristics.
Based on my analysis, I have shown that interactions among the
website design characteristics affect the perceived design quality
of website designs overall.

References
Akbulut, . E., and Akbulut, K. Web site designers opinions about the visual
elements. Procedia Social and Behavioral Sciences, 2, 2, 2010, 15491553.
Auephanwiriyakul, S., Keller, J. M., and Gader, P. D. Generalized Choquet fuzzy
integral fusion. Information Fusion, 3, 1, 2002, 6985.
Bai, B., Law, R., and Wen, I. The impact of website quality on customer satisfaction
and purchase intentions: Evidence from Chinese online visitors. International
Journal of Hospitality Management, 27, 3, 2008, 391402.
Beliakov, G., Pradera, A., and Calvo, T. Aggregation Functions: A Guide for Practitioners.
Springer-Verlag, Berlin, Germany, 2007.
Bonnardel, N., Piolat, A., and Le Bigot, L. The impact of colour on Website appeal and
users cognitive processes. Displays, 32, 2, 2011, 6980.
Cebi, S. Determining importance degrees of website design parameters based on
interactions and types of websites. Decision Support Systems, 54, 2013, 1030
1043.
Cebi, S., and Kahraman, C. Developing a group decision support system based on
fuzzy information axiom. Knowledge-Based Systems, 23, 1, 2010, 316.
Chang, K. H., and Cheng, C. H. Evaluating the risk of failure using the fuzzy OWA
and DEMATEL method. Journal of Intelligent Manufacturing, 22, 2, 2011, 113
129.
Chen, Y. W., and Tzeng, G. H. Using fuzzy integral for evaluating subjectively
perceived travel costs in a trafc assignment model. European Journal of
Operational Research, 130, 3, 2001, 653664.
Chiang, J. H. Aggregating membership values by a Choquet-fuzzy-integral based
operator. Fuzzy Sets and Systems, 114, 3, 2000, 367375.
Chiou, W. C., Lin, C. C., and Perng, C. A strategic website evaluation of online travel
agencies. Tourism Management, 32, 6, 2011, 14631473.
Chiu, H. C., Hsieh, Y. C., and Kao, C. Y. Website quality and customers behavioural
intention: An exploratory study of the role of information asymmetry. Total
Quality Management in Business, 16, 2, 2005, 185197.
Choquet, G. Theory of capacities. Annales de lInstitut Fourier, 5 (Grenoble), 1954,
131295.
Cocquebert, E., Trentesaux, D., and Tahon, C. WISDOM: a website design method
based on reusing design and software solutions. Information and Software
Technology, 52, 12, 2010, 12721285.
Cormany, D., and Baloglu, S. Medical travel facilitator websites: An exploratory
study of web page contents and services offered to the prospective medical
tourist. Tourism Management, 32, 4, 2011, 709716.
de Campos, L. M., and Jorge, M. Characterization and comparison of Sugeno and
Choquet integrals. Fuzzy Sets and Systems, 52, 1, 1992, 6167.

Demirel, T., Demirel, N. C., and Kahraman, C. Multi-criteria warehouse location

selection using Choquet integral. Expert Systems with Applications, 37, 5, 2010,
39433952.
thier, J., Hadaya, P., Talbot, J., and Cadieux, J. Interface design and emotions
experienced on B2C Web sites: Empirical testing of a research model. Computers
in Human Behavior, 24, 6, 2008, 27712791.
Fan, W. S., and Tsai, M. C. Factors driving website success: The key role of Internet
customisation and the inuence of website design quality and Internet
marketing strategy. Total Quality Management in Business, 21, 11, 2010, 1141
1159.
Fang, X., and Holsapple, C. W. An empirical study of web site navigation structures
impacts on web site usability. Decision Support Systems, 43, 2, 2007, 476491.
Fontela, E., and Gabus, A. DEMATEL: Progress achieved. Futures, 6, 4, 1974,
361363.
Grabisch, M., and Labreuche, C. A decade of application of the Choquet and Sugeno
integrals in multi-criteria decision aid. 4OR, 6, 1, 2008, 144.
Grabisch, M., Marichal, J. L., Mesiar, R., and Pap, E. Aggregation Functions. Cambridge
University Press, Cambridge, UK, 2009.
Gregg, D. G., and Walczak, S. The relationship between website quality, trust and
price premiums at online auctions. Electronic Commerce Research and
Applications, 10, 1, 2010, 125.
Hasan, L., and Abuelrub, E. Assessing the quality of web sites. Applied Computing and
Informatics, 9, 1, 2011, 1129.
Hu, Y.-C., and Liao, Y.-C. Finding critical criteria of evaluating electronic service
quality of Internet banking using fuzzy multiple-criteria decision making.
Applied Soft Computing, 11, 4, 2011, 37643770.
Huang, T., and Huang, C. H. An integrated decision model for evaluating educational
web sites from the fuzzy subjective and objective perspectives. Computers and
Education, 55, 2, 2010, 616629.
Huang, A. Y., Lee, W. S., Chang, Y. Y., and Cheng, C. M. Analysis of decision-making
factors for equity investment by DEMATEL and analytic network process. Expert
Systems with Applications, 38, 7, 2011, 83758383.
Jones, C., and Kim, S. Inuences of retail brand trust, off-line patronage, clothing
involvement and website quality on online apparel shopping intention.
International Journal of Consumer Studies, 34, 6, 2010, 627637.
Kaya, T. Multi-attribute evaluation of website quality in e-business using an
integrated fuzzy AHP-TOPSIS methodology. International Journal of
Computational Intelligence Systems, 3, 3, 2010, 301314.
Kaya, T., and Kahraman, C. A fuzzy approach to e-banking website quality
assessment based on an integrated AHP-ELECTRE method. Technological and
Economic Development of Economy, 17, 2, 2011, 313334.
Kim, H., and Niehm, L. S. The impact of website quality on information quality,
value, and loyalty intentions in apparel retailing. Journal of Interactive
Marketing, 23, 3, 2009, 221233.
Kim, S., and Stoel, L. Dimensional hierarchy of retail website quality. Informational
and Management, 41, 5, 2004, 619633.
Laurence, P., Basset, M., Coutant, P., and Gissinger, G. Lateral vehicle behaviour:
Comparison of subjective/objective assessment using the Choquet integral.
Vehicle System Dynamics, 34, 5, 2000, 357379.
Lee, S., and Koubek, R. J. The effects of usability and web design attributes on user
preference for e-commerce web sites. Computers in Industry, 61, 4, 2010, 329
341.
Lee, Y., and Kozar, K. A. Investigating the effect of website quality on e-business
success: An analytic hierarchy process approach. Decision Support Systems, 42, 3,
2006, 13831401.
Lin, H. F. The impact of website quality dimensions on customer satisfaction in the
B2C e-commerce context. Total Quality Management in Business, 18, 34, 2007,
363378.
Lin, H. F. An application of fuzzy AHP for evaluating course website quality.
Computers in Education, 54, 4, 2010, 877888.
Lin, C. L., and Tzeng, G. H. A value-created system of science (technology) park by
using DEMATEL. Expert Systems with Applications, 36, 6, 2009, 96839697.
Lin, Y. T., Yang, Y. H., Kang, J. S., and Yu, H. C. Using DEMATEL method to explore
the core competences and causal effect of the IC design service company:
An empirical case study. Expert Systems with Applications, 38, 5, 2011, 6262
6268.
Liou, J. J. H., Yen, L., and Tzeng, G. H. Building an effective safety management
system for airlines. Journal of Air Transport Management, 14, 1, 2008, 2026.
Liu, G. Z., Liu, Z. H., and Hwang, G. J. Developing multi-dimensional evaluation
criteria for English learning websites with university students and professors.
Computers and Education, 56, 1, 2011, 6579.
McCoy, S., Everard, A., and Loiacono, E. T. Online ads in familiar and unfamiliar sites:
Effects on perceived website quality and intention to reuse. Information Systems
Journal, 19, 4, 2009, 437458.
Mohamadnejad, F., Jassbi, J., and Nasrollahzadeh, H. A fuzzy DEMATEL framework
for modeling cause and effect relationships of strategy map. Expert Systems with
Applications, 38, 5, 2011, 59675973.
Moshagen, M., and Thielsch, M. T. Facets of visual aesthetics. International Journal of
HumanComputer Studies, 68, 10, 2010, 689709.
Murofushi, T., and Sugeno, M. An interpretation of fuzzy measures and the Choquet
integral as an integral with respect to a fuzzy measure. Fuzzy Sets and Systems,
29, 2, 1989, 201227.
Olcer, A. I., and Odabasi, A. Y. A new fuzzy multiple attributive group decision
making methodology and its application to propulsion/manoeuvring system
selection problem. European Journal of Operational Research, 166, 1, 2005, 93
114.

S. Cebi / Electronic Commerce Research and Applications 12 (2013) 124135

Opricovic, S., and Tzeng, G. H. Compromise solution by MCDM methods: A
comparative analysis of VIKOR and TOPSIS. European Journal of Operational
Research, 156, 2, 2004, 445455.
Schmeidler, D. Subjective probability and expected utility without additivity.
Econometrica, 57, 3, 1989, 571587.
Sengel, E., and nc, S. Conducting preliminary steps to usability testing:
Investigating the website of Uludag University. Procedia Social and
Behavioral Sciences, 2, 2, 2010, 890894.
Shen, Y. C., Lin, G. T. R., and Tzeng, G. H. A novel MCDM combining DEMATEL
technique for technology evaluation. In Proceedings of the PICMET 2010:
Technology Management for Global Economic Growth, July 1822, 2010.
Institute for the Management of Technology, National Chiao Tung University,
Hsinchu, Taiwan, 2010.
Shen, Y. C., Lin, G. T. R., and Tzeng, G. H. Combined DEMATEL techniques with novel
MCDM for the organic light emitting diode technology selection. Expert Systems
with Applications, 38, 3, 2011, 14681481.
Shieh, J. I., Wu, H. H., and Huang, K. K. A DEMATEL method in identifying key success
factors of hospital service quality. Knowledge-Based Systems, 23, 3, 2010, 277
282.
Tan, C. Q., and Chen, X. H. Induced intuitionistic fuzzy Choquet integral operator for
multicriteria decision making. International Journal of Intelligent Systems, 26, 7,
2011, 659686.
Torra, V., and Narukawa, Y. Modelling Decisions: Information Fusion and Aggregation
Operators. Springer-Verlag, Berlin, Germany, 2007.
Tsai, W. H., and Hsu, W. A novel hybrid model based on DEMATEL and ANP for
selecting cost of quality model development. Total Quality Management in
Business, 21, 4, 2010, 439456.
Tsai, H. H., and Lu, I. Y. The evaluation of service quality using generalized Choquet
integral. Information Sciences, 176, 6, 2006, 640663.
Tseng, M. L., and Lin, Y. H. Application of fuzzy DEMATEL to develop a cause and
effect model of municipal solid waste management in metro Manila.
Environmental Monitoring and Assessment, 158, 14, 2009, 519533.
Tuch, A. N., Bargas-Avila, J. A., and Opwis, K. Symmetry and aesthetics in website
design: Its a mans business. Computers in Human Behavior, 26, 6, 2010, 1831
1837.
Tzeng, G. H., Chen, W. H., Yu, R., and Shih, M. L. Fuzzy decision maps: A
generalization of the DEMATEL methods. Soft Computing, 14, 11, 2010, 1141
1150.
Tzeng, G. H., Chen, F. H., and Hsu, T. S. A balanced scorecard approach to establish a
performance evaluation and relationship model for hot spring hotels based on a

135

hybrid MCDM model combining DEMATEL and ANP. International Journal of

Hospitality Management, 30, 4, 2011, 908932.
Tzeng, G. H., Ho, W. R. J., Tsai, C. L., and Fang, S. K. Combined DEMATEL technique
with a novel MCDM model for exploring portfolio selection based on CAPM.
Expert Systems with Applications, 38, 1, 2011, 1625.
Wells, J. D., Parboteeah, V., and Valacich, J. S. Online impulse buying: Understanding
the interplay between consumer impulsiveness and website quality. Journal of
the Association for Information Systems, 12, 1, 2011, 3256.
Wu, W. W., and Lee, Y. T. Developing global managers competencies using the fuzzy
DEMATEL method. Expert Systems with Applications, 32, 2007, 499507.
Wu, H. H., and Tsai, Y. N. A DEMATEL method to evaluate the causal relations among
the criteria in auto spare parts industry. Applied Mathematical Computing, 218, 5,
2011, 23342342.
Wu, H. H., Chen, H. K., and Shieh, J. I. Evaluating performance criteria of
employment service outreach program personnel by DEMATEL method.
Expert Systems with Applications, 37, 7, 2010, 52195223.
Wu, C. H., Chang, B., and Chang, C. W. Fuzzy DEMATEL method for developing
supplier selection criteria. Expert Systems with Applications, 38, 3, 2011, 1850
1858.
Yang, J. L., and Tzeng, G. H. An integrated MCDM technique combined with
DEMATEL for a novel cluster-weighted with ANP method. Expert Systems with
Applications, 38, 3, 2011, 14171424.
Yiu, C. S., Grant, K., and Edgar, D. Factors affecting the adoption of Internet banking
in Hong Kong: Implications for the banking sector. International Journal of
Information Management, 27, 5, 2007, 336351.
Yu, X., Guo, S., Guo, J., and Huang, X. Rank B2C e-commerce websites in e-alliance
based on AHP and fuzzy TOPSIS. Expert Systems with Applications, 38, 4, 2011,
35503557.
Zeng, L., Salvendy, G., and Zhang, M. Factor structure of web site creativity.
Computers in Human Behavior, 25, 2, 2009, 568577.
Zhou, T., Lu, Y. B., and Wang, B. The relative importance of website design quality
and service quality in determining consumers online repurchase behavior.
Information Systems Management, 26, 4, 2009, 327337.
Zhou, Q., Huang, W. L., and Zhang, Y. Identifying critical success factors in
emergency management using a fuzzy DEMATEL method. Safety Science, 49, 2,
2011, 243252.
Zviran, M., Glezer, C., and Avni, I. User satisfaction from commercial web sites: The
effect of design and use. Information and Management, 43, 2, 2006, 157178.