On The Pareto Optimum Sensitivity Analysis in Multicriteria Optimization

INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERING
Int. J. Numer. Meth. Engng 2003; 58:955977 (DOI: 10.1002/nme.890)

On the Pareto optimum sensitivity analysis in
multicriteria optimization
W. H. Zhang
,
SinoFrench Laboratory of Concurrent Engineering, Department of Aircraft Manufacturing Engineering,
Northwestern Polytechnical University, P.O. Box 552, 710072 Xian, Shaanxi, China
SUMMARY
To analyse the trade-o relations among the set of criteria in multicriteria optimization, Pareto optimum
sensitivity analysis is systematically studied in this paper. Original contributions cover two parts: the-
oretical demonstrations are rstly made to validate the gradient projection method in Pareto optimum
sensitivity analysis. It is shown that the projected gradient direction evaluated at a given Pareto optimum
in the design variable space rigorously corresponds to the tangent direction of the Pareto curve}surface
at that point in the objective space. This statement holds even for the change of the set of active
constraints in the perturbed problem. Secondly, a new active constraint updating strategy is proposed,
which permits the identication of the active constraint set change, to determine the inuence of this
change upon the dierentiability of the Pareto curve and nally to compute directional derivatives in
non-dierentiable cases. This work will highlight some basic issues in multicriteria optimization. Some
numerical problems are solved to illustrate these novelties. Copyright ? 2003 John Wiley & Sons, Ltd.
KEY WORDS: multicriteria optimization; Pareto optimum sensitivity analysis; gradient projection
method; directional derivative
1. INTRODUCTION
Considering the complexity and multidisciplinary nature of large-scale engineering design
problems in the real-life, multicriteria optimization is a rational approach for the design syn-
thesis. With this procedure, structures and mechanical systems can be appropriately designed
to satisfy a variety of task requirements or multidisciplinary performance demands. In multi-
criteria optimization, conicting among relevant criteria is a basic issue which makes it im-
possible to nd an optimum solution minimizing simultaneously all individual criteria. For
Correspondence to: W. H. Zhang, SinoFrench Laboratory of Concurrent Engineering, Department of Aircraft

Manufacturing Engineering, Northwestern Polytechnical University, P.O. Box 552, 710072 Xian, Shaanxi, China.
E-mail: zhangwh@nwpu.edu.cn
Contract}grant sponsor: National Natural Science Foundation; contract}grant number: 10172072
Contract}grant sponsor: Aeronautical Foundation; contract}grant number: 00B53005
Received 17 October 2001
Revised 23 May 2003
Copyright
?
2003 John Wiley & Sons, Ltd. Accepted 23 May 2003
956 W. H. ZHANG
this reason, a vector optimization problem has to be formulated and solved to produce a set
of compromise solutions.
Today, a variety of optimization methods are available and have been put in use (see
References [1, 2]). Among others, a conventional approach is to transform the original problem
into a scalarized one. The weighting method is a basic approach. However, as indicated by
Koski [3], Messac et al. [4], the disadvantage and defectiveness of such a method are that
non-convex Pareto optimum parts cannot be captured. To bypass this obstacle, Das and Dennis
[5] established the normal boundary method. Alternatively, Athan and Papalambros [6], Chen
et al. [7] suggested the compromise programming method in which a nonlinearly scalarized
objective function with curvature is adopted. To enhance the reliability of the latter, Messac
et al. [4] studied the selection of inherent parameters in order to attribute an appropriate
curvature to the objective function.
Pareto optimum sensitivity analysis studied here is another important concern after the
solving of the optimization problem. In order to explore the Pareto optimum set for decision-
makings, it is necessary to have a further insight into the neighbourhood of each solution.
Pareto optimum sensitivity analysis reects relative variations among dierent criteria. This
information can be used to access quickly relative changes of design variables, constraints and
criteria when a Pareto optimum is shifted from one solution to another. Besides, this trade-o
information is essential to understand precisely how about the non-linearity, dierentiability
and discontinuity of the Pareto curve since the latter may be highly non-linear, non-smooth
and even discontinuous in practice. For example, with known derivatives, the Pareto optimum
curve which is unknown in advance can be well approximated by tting a few available Pareto
optimum points. Within this scope, a comprehensive discussion was given by Tappeta et al.
[8, 9]. In their work, a Pareto optimum sensitivity analysis procedure was implemented jointly
with the physical programming method. The concept of trade-o matrix was introduced to
characterize the inuence of one criterion upon remaining ones. Likewise, the trade-o matrix
was also used in the approximation of the Pareto curve. Meanwhile, Diaz [10], Tappeta et al.
[8, 9] presented an interactive compromise programming procedure which employs sensitivity
analysis capabilities to update the aspiration levels. The SQP and gradient projection method
(GPM) were adopted for sensitivity analysis, respectively. Later, Hernandez [11] evaluated
the Pareto optimum sensitivity by solving a linear system associated with the set of active
constraints. Recently, Fadel et al. [12] attempt to use Pareto optimum sensitivity for the
evaluation of design robustness. A sensitivity measure using the so-called c-optimality concept
is applied for the preferred selection of robust design in bicriteria optimization.
Basically, almost all formulations used by Tappeta et al. [8, 9], Diaz [10], Hernandez [11]
and even those by Sobieszczanski-Sobieski et al. [13] and Vanderplaats and Yoshida [14]
for optimum sensitivity to problems parameters in single objective problems were performed
under the common assumption. It is assumed that the active constraint set remains unchanged
for the perturbed problem and that the strict complementary slackness holds at the current
optimum. Tseng and Lu [15] had to appeal to the nite dierence computing to bypass
this assumption indirectly. In their work, the Pareto optimum sensitivity analysis is used to
study the variation of optimum design with respect to the upper bounds of criterion-dened
constraints after the solving of multicriteria problems by the trade-o method.
As is well-known, the main diculty associated with Pareto optimum sensitivity analy-
sis is the non-dierentiability of the Pareto curve in the objective space. This is due to the
change of the active constraint set when a perturbation is slightly made. Relevant problems
Copyright ? 2003 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Engng 2003; 58:955977
PARETO OPTIMUM SENSITIVITY ANALYSIS 957
exist practically and readers can refer to the truss design with discontinuous Pareto curve dis-
cussed in Reference [3]. Moreover, numerical tests given in References [16, 17] show that the
non-dierentiability is inevitable when the same function is used both for the criterion def-
inition and for the active constraint denition like in physical programming. In the recent
work of Zhang and Yang [18], it is revealed that the normal direction at a non-dierentiable
optimum point on the Pareto curve will not be unique and that an interval of subgradients
exists, and can be obtained by linear programming.
In this work, a new approach is proposed in consideration of the change of the active
constraint set. To evaluate directional derivatives and the normal of the Pareto optimum curve
in non-dierentiable cases, a modied GPM method is proposed originally. In addition to this,
a new active constraint strategy is established for the updating of the gradient matrix of active
constraints. This new strategy is quite dierent from the classical one that deletes the active
constraint having the most negative Lagrangian multiplier value. To ensure the validity of the
GPM method, theoretical demonstrations are given for the rst time. Finally, numerical tests
are used to validate the proposed method and the nite dierence scheme is used to justify
sensitivity results obtained.
2. PARETO OPTIMUM SENSITIVITY ANALYSIS OF MULTICRITERIA
OPTIMIZATION PROBLEMS
2.1. Basis of the multicriteria optimization
Before addressing the Pareto optimum sensitivity analysis, it is necessary to have a brief outline
of the multicriteria optimization problem. Consider the following mathematical programming
statement:
Min F(X)
q
)
(X)60, ) =1, . . . , m
(1)
where F(X) = [[
1
(X), [
2
(X), . . . , [
r
(X)]
T
denotes a set of conicting criteria to be mini-
mized, q
)
(X) denotes the )-th constraint. Unlike scalar optimization problems, the solution
of the above problem is not unique and a compromise solution set exists. According to the
Pareto optimality, a feasible solution X
is dened as the Pareto optimum if there is no other

improved feasible point X such that [
k
(X)6[
k
(X
) with strict inequality for at least one con-

dition. Theoretically, the optimality conditions satised by the Pareto optimum are expressed
as
r
k=1
w
k
[
k
(X
) +
m
)=1
z
)
q
)
(X
) =0 (2a)
z
)
q
)
(X
) =0, z
)
0, ) =1, . . . , m (2b)
in which {w
k
} is a set of positive weightings satisfying the normalization equality
r
k=1
w
k
=1.
z
)
denotes the )-th Lagrangian multiplier and equals zero when the related constraint is in-
active. In fact, it is easily observed that the above expressions are also the KuhnTucker
958 W. H. ZHANG
Figure 1. Illustration of the Pareto curve in the objective space.
conditions of the following scalarized weighting problem:
Min
r
k=1
w
k
[
k
(X)
q
)
(X)60 ) =1, . . . , m
(3)
As illustrated in Figure 1, the compromise solution set in a bicriteria case, i.e. Pareto optimum
set is geometrically represented by curve segments AB and CD in the objective space. Tangent
lines correspond to sensitivities between [
1
and [
2
at points P and P
, respectively. For the

non-dierentiable point P
, only directional derivatives exist. At point P, weighting vector

{w
k
} represents the normal. In fact, according to the notion of the gradient projection method,
Equation (2a) is physically equivalent to a zero-descent direction associated with the projected
gradient of the weighting sum. It can be then written in matrix form as
S=
r
k=1
w
k
[
k
N[ =P
r
_
k=1
w
k
[
k
_
=0 (4)
that is
r
k=1
w
k
[
k
+ N[ = 0 (5)
where N={q
)
}, ( ) J, rank(N) =J) is the gradient matrix whose columns consist of the
gradients of J active constraints (J6M) at X
; P
r
=I N(N
T
N)
1
N
T
denotes the projection
operator.
2.2. Pareto optimum sensitivity analysis
At a given Pareto optimum X
, Pareto optimum sensitivity analysis consists in calculating the

derivatives of criteria with respect to a certain one when the latter is perturbed. As illustrated in
Figure 1, the derivative of one criterion [
2
with respect to another [
1
geometrically corresponds
to the tangent direction of the Pareto curve in the objective space. In this example, P is a
dierentiable Pareto optimum whereas P
is a non-dierentiable one with only directional

derivatives. B is non-dierentiable but also discontinuous since a zero left-hand derivative
exists.
Now, suppose that under the perturbation of the q-th criterion, the design variable vector
varies along the direction S. This variation can then be written as
X=S: (6)
with : to be the step length. Therefore, the variation of the k-th criterion can be evaluated
by
[
k
=[
T
k
S: (7)
Now, the derivative quantifying the trade-o relation between the k-th criterion and the q-th
one is expressed as
d[
k
d[
q
= lim
:0
[
k
[
q
=
[
T
k
S
[
T
q
S
, k =1, r, k =q (8)
Obviously, the derivative is independent of :. In the same way, derivatives of the constraints
and design variables with respect to the criterion [
q
are easily derived
dq
)
d[
q
=
q
T
)
S
[
T
q
S
, ) =1, . . . , m (9a)
dx
i
d[
q
=
S
i
[
T
q
S
, i =1, . . . , n (9b)
Meanwhile, if the perturbation [
q
is prescribed a priori, the step length : can be evaluated as
: =
[
q
[
T
q
S
(10)
As to the constraint variation, it can be estimated correspondingly as follows:
q
)
(X) =q
)
(X
) +q
T
)
X=q
)
(X
) +q
T
)
S
[
q
[
T
q
S
, ) =1, . . . , m (11)
2.3. Determination of tangent and normal directions of the Pareto curve
The above study points out that the basic work in sensitivity analysis is the determination
of the search direction S. Clearly, this direction has to be feasible in the design variable
space and its mapping in the objective space has to be the tangent of the Pareto curve. For a
Pareto surface, tangent directions are innite at a given optimum X
, whereas S is a principal
direction associated with the descent direction of a certain criterion and sensitivities refer to
the directional derivatives along S. In the following work, it will be demonstrated that the
desired S satisfying relevant conditions can be derived by means of the GPM method.
960 W. H. ZHANG
Without loss of generality, suppose that a perturbation is made in order to reduce a pref-
erence function as follows:
[=
r
k=1
a
k
[
k
(12)
Naturally, the projected descent direction associated with this function is
S=
r
k=1
a
k
[
k
N\ (13a)
in which the vector of Lagrangian multipliers is dened by
\ =(N
T
N)
1
N
T
r
k=1
a
k
[
k
(13b)
For example, consider a particular case where only the th criterion involves in the preference
function with
a
k
=
_
1, k =
0, k =
(14)
such that
S=[
N\ (15)
Two cases may exist in the evaluation of S by Equation (13a):
(i) If S is not a zero-vector, the multiplication of Equation (5) by S produces
r
k=1
w
k
[
T
k
S + [
T
N
T
S=0 (16)
Due to the orthogonality between the projected direction S and the gradient matrix N of active
constraints, the above expression can be further simplied as
r
k=1
w
k
[
k
=0 (17)
in which [
k
=[
T
k
S: refers to the k-th criterion variation along S with step length :. It
is worthwhile noting that : is just a scaling parameter having no inuence on the direction
of vector {[
k
}. Equation (17) means that {[
k
} is orthogonal to the normal vector {w
k
}.
Hence, it follows that the mapping of the projected direction S dened in the design variable
space corresponds to the tangent direction {[
k
} of the Pareto curve in the objective space.
(ii) Secondly, if the search direction S obtained is a zero-vector. Namely
r
k=1
a
k
[
k
+ N\ =0 (18)
Equation (13b) has to be further investigated. Firstly, if all Lagrangian multipliers in \ related
to active inequality constraints take non-negative values, it means that no descent direction
exists, that will reduce [ and that the current Pareto optimum is the minimum of [. Hence, the
Pareto optimum sensitivity analysis is trivial. In practice, this often happens at the extremity
end points of the Pareto curve. Note that when equality constraints involve in the denition
of N in Equation (13b), no control is demanded to check the signs of associated Lagrangian
multipliers. Secondly, if the q-th Lagrangian multiplier in \ is negative (j
q
0), it means that
the q-th active constraint will become inactive when a further reduction of [ is done. Hence,
a new modied projected descent direction needs to be revaluated by dropping the gradient,
denoted by n
q
, of the q-th active constraint from the matrix N. Suppose that

N is the reduced
matrix, a modied projected direction

S will be then evaluated by
S=
r
k=1
a
k
[
k

N \ (19)
Now, let us prove that the mapping of

S also denes the tangent of the Pareto curve in
the objective space. In virtue of the matrix decomposition of N, one can write the following
expressions:
[
T
N
T

S =

[
T

N
T

S + z
q
n
T
q

S (20a)
\
T
N
T

S = \
T

N
T

S + j
q
n
T
q

S (20b)
In view of the orthogonality

N
T
S=0 imbedded in Equation (19), it follows that

[
T
N
T

S =z
q
n
T
q

S (21a)
\
T
N
T

S =j
q
n
T
q

S (21b)
Based on Equation (21b), the multiplication of Equation (18) by

S gives rise to
r
k=1
a
k
[
T
k

S + \
T
N
T

S=
r
k=1
a
k
[
T
k

S + j
q
n
T
q

S=0 (22)
namely
j
q
n
T
q

S=
r
k=1
a
k
[
T
k

S (23)
The multiplication of Equation (5) by

S with the retention of Equations (21) and (23)
results in
r
k=1
w
k
[
T
k

S + [
T
N
T

S=
r
k=1
w
k
[
T
k

S + z
q
n
T
q

S=
r
k=1
w
k
[
T
k

S
_
z
q
j
q
_
r
k=1
a
k
[
T
k

S=0 (24)
namely
r
k=1
_
w
k

z
q
j
q
a
k
_
[
k
=0 (25)
962 W. H. ZHANG
with
[
k
=[
T
k

S: (26)
Here, : is added just to keep the conformity of the expression with Equation (17). By
normalizing the weightings to the unit sum value, Equation (25) can be written in the same
compact form as Equation (17)
r
k=1
w
k
[
k
=0 (27)
with normalized weightings to be
w
k
=
w
k
(z
q
}j
q
)a
k
r
t=1
(w
t
(z
q
}j
q
) a
t
)
=
w
k
(z
q
}j
q
)a
k
1 (z
q
}j
q
)
r
t=1
a
t
(28)
If the preference function dened by Equation (14) is used, the normalized weightings will
be reduced to
w
k
=
_
_
_
w
z
q
j
q
___
1
z
q
j
q
_
, k=
w
k
__
1
z
q
j
q
_
, k =
(29)
Equation (27) points out that when an active constraint leaves the set of active constraints
and becomes inactive due to the perturbation, the normal direction vector of the Pareto curve
at the same optimum will be shifted from {w
k
} to {w
k
}. The shifting can be evaluated by
using the general expression (28). In other words, the Pareto optimum is identied to be a
non-dierentiable point in the objective space provided that a shifting of weightings occurs.
Because the non-dierentiability depends uniquely upon the shifting of the weighting vector,
the change of the active constraint set does not truly reect a non-dierentiable optimum
point of the Pareto curve. For example, if the q-th constraint is a degenerate inequality active
constraint at the optimum, i.e. the Lagrangian multiplier z
q
=0 in Equation (28), then no
shifting occurs between {w
k
} to {w
k
}. This problem will be further illustrated by numerical
examples in the following section.
Now, the remaining work is to prove that weightings obtained in Equation (25) consti-
tute indeed another set satisfying the KuhnTucker optimality conditions of the multicriteria
problem (1) at the same X
. From Equation (25), we can write

r
k=1
_
w
k

z
q
j
q
a
k
_
[
k
=
r
k=1
w
k
[
k

z
q
j
q
r
k=1
a
k
[
k
(30)
In view of Equations (5) and (18), the above relation can be developed as
r
k=1
w
k
[
k

z
q
j
q
r
k=1
a
k
[
k
=N[ +
z
q
j
q
N\ =N
_
[
z
q
j
q
\
_
(31)
which can be further transformed into
r
k=1
_
w
k

z
q
j
q
a
k
_
[
k
+ N
_
[
z
q
j
q
\
_
=0 (32)
Similarly, weightings in the above expression can be normalized as those in Equation (28).
As a result, Equation (32) can be written in a compact form
r
k=1
w
k
[
k
+ N[
=0 (33)
with
z
)
=
_
_
_
z
)
z
q
j
q
j
)
___
1
z
q
j
q
r
t=1
a
t
_
, ) =q
0, )=q
(34)
Therefore, Equation (33) validates the statement. To ensure the non-negative values of z
)
in
Equation (34), the following relation must be satised:
z
)
z
q
j
q
j
)
0, ) =q (35)
Evidently, if each j
)
( ) =q) takes a non-negative value, the above relation will be veried
automatically. On the other hand, if several j
)
s are negative, we need then
z
)
j
)
6
z
q
j
q
, ) =q (36)
This inequality indicates that when more than one active constraint have negative Lagrangian
multipliers, the q-th active constraint which becomes inactive and will be deleted from N has
to be identied as such that has the maximum ratio
Max
)
_
z
)
j
)
, j
)
0
_
(37)
This active constraint strategy is quite dierent from that used in the traditional GPM method.
The latter identies the active constraint to be eliminated by taking the most negative
Lagrangian multiplier with
Min
)
{j
)
, j
)
0} (38)
Obviously, Equations (37) and (38) will be the same when only one active constraint with
negative j
)
exists. Therefore, the traditional GPM method is conditionally valid when applied
in Pareto optimum sensitivity analysis.
On the other hand, It is important to remark that Equation (34) is an analytical expression
describing the inherent relationship between Lagrangian multipliers varying from {z
)
} to {z
)
}.
Since j
q
0, all weightings {w
k
} obtained in Equation (28) together with the Lagrangian
multipliers in Equation (34) are ensured to take only non-negative values. Therefore, it follows
that Equation (33) is indeed an alternative form of the KuhnTucker optimality conditions
964 W. H. ZHANG
Figure 2. Flowchart of Pareto optimum sensitivity analysis.
for the same optimum satisfying Equation (5). To have a summary of the above presentation,
a owchart of the sensitivity analysis computing procedure is given in Figure 2.
In this procedure, basically, the use of the weighting method in combination with the dual
optimization algorithm is an ideal approach which will provide directly weighting values and
Lagrangian multipliers in Step 2. Nevertheless, this is not obligatory and even not convenient
for non-convex problems. In fact, once the Pareto optimum X
is obtained by any available

optimization method, a couple of non-negative sets {w
k
} and {z
)
} do exist and can be derived
from KuhnTucker conditions (2) by linear programming [18] or other methods.
Until now, the problem of having a constraint becoming inactive is thoroughly discussed.
For the case of having an inactive constraint becoming active, because sensitivity analysis
refers to function variations with respect to an innitesimal perturbation, an inactive constraint
becoming active has always a distance to traverse and such a distance is always greater than
the innitesimal perturbation. Therefore, such a case does not need to be considered.
3. NUMERICAL EXAMPLES
3.1. Linear bicriteria optimization
Consider an illustrative linear bicriteria problem that is stated as
Min {[
1
(X) = x
1
, [
2
(X) =x
2
}
q
1
(X) =
x
1
4

x
2
4
+ 160
q
2
(X) =
x
1
3

x
2
6
+ 160
q
3
(X) = x
1
60
q
4
(X) = x
2
60
Because each criterion is just dened by a proper design variable, the design variable space
coincides with the objective space. Therefore, the Pareto curve can be easily plotted as the
segments AB and BC shown in Figure 3.
Now, let us focus on the Pareto optimum sensitivity analysis at point B(2,2). Clearly, this
is a non-dierentiable Pareto optimum point which can be obtained by the weighting method
with weighting coecients w
1
=
5
9
and w
2
=
4
9
. At point B, the rst two constraints are active.
By denition, the gradient matrix is
N=
_
1
4

1
3
1
4

1
6
_
From the KuhnTucker optimality conditions, the Lagrangian multipliers related to both
active constraints can easily be deduced as z
1
=
4
3
and z
2
=
2
3
. Firstly, sensitivity is
evaluated for the reduction of [
1
, i.e. [
1
0. The projection of ([
1
(X
)) generates
S
1
=P
r
([
1
(X
)) =0 with \ =(N
T
N)
1
N
T
[
1
=[4, 6]
T
. The rst negative component
of \ indicates that the rst constraint will become inactive as shown in Figure 3. Therefore,
its gradient should be taken away from N so that the reduced gradient matrix is

N=[
1
3
,
1
6
]
T
.
From Equation (19), the modied descent direction equals

S
1
=[
1
5
,
2
5
]
T
. Thus, sensitivity re-
sults with respect to [
1
are then d[
2
}d[
1
=[
T
2

S
1
}[
T
1

S
1
=2, dq
1
}d[
1
=0.25, dq
2
}d[
1
=0,
dq
3
}d[
1
=1, dq
4
}d[
1
=2. From Equation (28), the critical weightings at B are found to be
(w
1
, w
2
) =(
2
3
,
1
3
). It can be observed that this is the normal vector of the Pareto curve dened
by the second active constraint in Figure 3. From Equation (34), the associated Lagrangian
multipliers are (z
1
, z
2
) =(0, 2).
In the case of [
1
0, the search direction will be evaluated using the descent direction of
[
2
. It is found that \ =[4, 6]
T
so that the second active constraint becomes inactive. Simi-
larly, the projected direction is evaluated with

S
2
=[
1
2
,
1
2
]
T
so that sensitivities correspond to
d[
2
}d[
1
=[
T
2

S
2
}[
T
1

S
2
=1, dq
1
}d[
1
=0, dq
2
}d[
1
=
1
6
, dq
3
}d[
1
=1 and dq
4
}d[
1
=1. On
966 W. H. ZHANG
Figure 3. Pareto curve. Figure 4. Variation of the weighting
component w
2
.
the other hand, by means of Equations (28) and (34), one can obtain the shifting of weight-
ings and Lagrangian multipliers, respectively. They are (w
1
, w
2
) =(
1
2
,
1
2
) and (z
1
, z
2
) =(2, 0).
Obviously, the weighting vector is the normal direction of the Pareto curve dened by the
rst active constraint in Figure 4. This example shows that sensitivity analysis, the shifting
of weightings and Lagrangian multipliers can be exactly evaluated by the proposed method at
the non-dierentiable optimum. To have a direct understanding of the variation of the normal
direction (w
1
, w
2
) along the Pareto curve AB and BC, the variation of the weighting com-
ponent w
2
satisfying the condition w
1
+ w
2
=1 can be easily determined from the slope of
AB and BC and is plotted in Figure 4. At two end points A, C and point B, the weighting
jumps mean that they are non-dierentiable points and all weighting values within each jump
interval will produce the same optimum when used in the weighting method.
3.2. Non-linear bicriteria optimization
This test was originally used by Li et al. [19] to study the approximation of the Pareto
optimum curve. It is reconsidered here for the purpose of sensitivity analysis. The problem
is stated as
Min {[
1
(X) = (x
1
2)
2
+ (x
2
1)
2
, [
2
(X) =x
2
1
+ (x
2
6)
2
}
q
1
(X) = x
2
1
x
2
60
q
2
(X) = 5x
2
1
+ x
2
610
q
3
(X) = x
2
65
q
4
(X) = x
1
60
By using the goal attainment method of the MATLAB optimization tool, the set of Pareto
optimum solutions is evaluated and 65 discrete solution points are obtained. The optimum
trajectory is plotted as dot lines in the design variable space (see Figure 5) and objective
space (see Figure 6), respectively. The variation of related weighting w
2
from one point to
Figure 5. Description of the Pareto optimum solution set in the design variable space.
Figure 6. Pareto set in the objective space.
968 W. H. ZHANG
Figure 7. Variation of the weighting component w
2
along the Pareto optimum trajectory.
another is depicted along the optimum trajectory in Figure 7. Here, attention is focused on
four singular points A(
_
15}3,
5
3
), B(1.24817,2.87956), C(0.4,5) and D(0,5).
At point A, only the rst two constraints are active. Therefore, the gradient matrix N is
dened as
N=
_
2
15
3
10
15
3
1 1
_
By using the KuhnTucker optimality conditions, two pairs of extreme weighting vectors
exist with
(1) (w
1
, w
2
) =(0.86, 0.14) with (z
1
, z
2
) =(0, 0.0665)
(2) (w
1
, w
2
) =(0.9072, 0.0928) with (z
1
, z
2
) =(0.4054, 0)
When [
1
0, we have j
2
=0.13070 so that

N=[2
15}3, 1]
T
. Consequently, the pro-
jected direction is

S
1
=[0.13204, 0.3409]
T
. It can be veried that this is the tangent direc-
tion of the rst constraint as shown in Figure 5. Based on (28), it can be veried that the
rst set (w
1
, w
2
) =(0.86, 0.14) will be shifted to the second set. Conversely, when [
2
0,
we have j
1
=7.38890 which leads to

S
2
=[0.05556, 0.71728]
T
as shown in Figure 5.
In this case, the second weighing set will be automatically shifted to the rst set. This jump
identies the non-dierentiability of point A in the objective space. Directional derivatives
can be evaluated as
d[
2
d[
1
=
[
T
2

S
[
T
1

S
=
_
9.7526, [
1
0
6.1428, [
1
0
Note that the negative derivative results make sure the conict between both criteria. From
Equation (9a), constraint sensitivities are obtained
dq
1
d[
1
=
_
0, [
1
0,
0.8315, [
1
0,
dq
2
d[
1
=
_
7.6525, [
1
0
0, [
1
0
and
dq
3
d[
1
=
_
1.2753, [
1
0,
0.6929, [
1
0,
dq
4
d[
1
=
_
0.4939, [
1
0
0.0536, [
1
0
As to point B in Figure 5, only q
2
is active with N=[12.4817, 1]
T
. One can see that it is
not only the Pareto optimum of the constrained problem but also the optimum of the same
problem excluding q
2
with z
2
=0. For dierent perturbations associated with [
1
0 and
[
1
0, two search directions S
3
and

S
4
will be evaluated by means of Equations (13a) and
(19), respectively. Solutions of search directions are S
3
=P
r
([
1
) =[0.30884, 3.8548]
T
and
S
4
=[
2
=[2.49634, 6.24088]
T
as illustrated in Figure 5. Note that q
2
will leave the active
constraint set when

S
4
is calculated. Sensitivities of criteria are then
d[
2
d[
1
=
[
T
2
S
[
T
1
S
=
_
1.66019, [
1
0
1.66019, [
1
0
Additionally, sensitivities of constraints are
dq
1
d[
1
=
_
0.30931, [
1
0,
0.45832, [
1
0,
dq
2
d[
1
=
_
0, [
1
0
0.91562, [
1
0
and
dq
3
d[
1
=
_
0.25775, [
1
0,
0.22932, [
1
0,
dq
4
d[
1
=
_
0.02065, [
1
0
0.09173, [
1
0
Although S
3
and

S
4
are not aligned in the design variable space, derivatives of criteria remain
the same. This important result ensures that B is a dierentiable point. In fact, this conclusion
can also be validated by checking the weighting variation. When [
1
0, it can be found
from Equation (18) that q
2
will become inactive with j
2
0. By putting z
2
=0 in Equation
(29), we get
w
1
=
_
w
1
z
2
j
2
___
1
z
2
j
2
_
=w
1
w
2
=w
2
__
1
z
2
j
2
_
=w
2
Because no shifting occurs for weightings, points B is a dierentiable Pareto optimum. In the
same way, point C is studied. Because z
3
=0, j
3
0, point C is also a dierentiable point
970 W. H. ZHANG
Figure 8. The 25-bar problem.
with the following search directions and derivatives:
S
5
=[3.2, 8]
T
and S
6
=[0.8, 0]
T
d[
2
d[
1
=
[
T
2
S
[
T
1
S
=
_
0.25, [
1
0
0.25, [
1
0
Point D is an end point of the Pareto curve, where side constraints q
3
, q
4
are active. The
corresponding weightings are then (w
1
, w
2
) =(0, 1). The gradient matrix N is
N=
_
0 1
1 0
_
From KuhnTucker conditions, we have z
3
=2, z
4
=0. For [
2
0, we get S=0 from Equa-
tion (13a). This solution is evident since [
2
attains its minimum value at D. For [
1
0,
we obtain j
3
=8, j
4
=4 by means of Equation (18). As a result, q
4
has to be eliminated
according to Equation (37). Hence, the search direction obtained is horizontal with

S
7
=[4, 0]
T
as shown in Figure 5. However, if we use the traditional GPM approach with Equation (38)
by taking away the constraint having the most negative Lagrangian multiplier j
3
=8, an
erroneous vertical search direction will produce.
3.3. The 25-bar problem
Figure 8 illustrates the 25-bar problem that is frequently used to validate the optimization
procedure (see Reference [20]). The structure is subject to two load cases as listed in Table I.
Table I. Load denition.
Load components (lbf)
Load case Node X Y Z
1 1 1000 10 000 5000
2 0 10 000 5000
3 500 0 0
6 500 0 0
2 1 0 20 000 5000
2 0 20 000 5000
Table II. Optimum solution and Pareto optimum sensitivity analysis.
Descent direction (s)
( q0)
Optimum
Design variables section area GPM FDM
(bar no.) (in
2
) method method
1 (1) 0.01 0 0
2 (25) 4.6739 0.003886 0.758
3 (69) 6.5583 0.003180 0.62
4 (10,11) 0.01 0 0
5 (12,13) 0.01 0 0
6 (1417) 1.2418 0.020256 0.015336
7 (1821) 3.8784 0.001351 0.000235
8 (2225) 4.8626 0.000637 0.042198
Optimum weight W
=1155.13 (lbm)
Here, the problem is reformulated as a bicriteria optimization problem. Both the total weight
of the truss denoted by W and the maximum of three displacement components (u, t, w) at
nodes 1 and 2, denoted by q, will be simultaneously minimized. To ensure the structural
symmetry, eight design variables of section areas are dened by grouping bar elements as
listed in Table II. A lower bound is imposed for each design variable. Besides, an allowable
stress o =3000 psi is imposed for stresses in all bars. Initial data are as follows:
Youngs modulus: E =10
7
psi, Material density: j=0.1 lbm}in
3
Lower bound of section areas: a =0.01 in
2
, Allowable stress: o =3000 psi
Mathematically, the problem is stated as
Min
_
W(a), q = Max
k=1, 2, !=1, 2
{u
k, !
(a), t
k, !
(a), w
k, !
(a)}
_
o
), !
(a)6 o, ) =1, 25
a6a
i
, i =1, 8
972 W. H. ZHANG
Symbols k and ! denote the node number and load case, respectively. To study the trade-
o relation between the weight and the truss exibility measured by the maximum nodal
displacement, we will calculate the sensitivity between W and q at the desired optimum
solution. Here, the diculty is that q is a non-dierentiable function when compared with
above examples. By means of the MBF scheme proposed in Reference [21], it can be proved
that the actual problem can be transformed into an equivalently form as follows:
Min {W(a), q}
u
k,!
(a)6 q, k =1, 2, ! =1, 2
t
k,!
(a)6 q, k =1, 2, ! =1, 2
w
k,!
(a)6 q, k =1, 2, ! =1, 2
o
), !
(a)6 o, ) =1, 25, ! =1, 2
a6a
i
, i =1, 8
in which q is regarded as an additional articial design variable. Based on the c-constraint
method, suppose now that q =0.175 in is prescribed for the upper bound of nodal displace-
ments. By using the dual approach with CONLIN approximation [22], a weight minimiza-
tion problem is solved. The optimum solution is obtained as given in Table II. At this op-
timum, displacement components t
1,1
and t
2, 1
attain their upper bound simultaneously in
the rst load case, and both components have also the same gradient due to the struc-
tural symmetry. However, as listed in Table III, because the associated Lagrangian mul-
tipliers take a zero-value, both constraints become degenerate and the optimum is an ir-
regular point. Following KuhnTucker conditions (2), weightings associated with W and
q are derived as w
1
=1, w
2
=0. In the meantime, stresses in bars 16, 25 attain the allow-
able stress in the rst load case, whereas stresses in bars 2, 5, 7, 8, 19, 20 attain the
allowable stress in the second load case. Nevertheless, due to the structural symmetry, it
is observed that the same stress gradients hold for bar pairs (2,5), (7,8) and (19,20). Ac-
cordingly, only ve among eight active stress constraints are retained to form the gradi-
ent matrix N. Besides, three design variables numbered 1, 4, 5 attain the lower bound.
As a result, there exist totally eight active constraints whose gradients are linearly inde-
pendent and will be used to constitute the gradient matrix N. By using the dual approach, the
Lagrangian multipliers associated with these eight active constraints are provided automatically as
a by-product after optimization. They are z =[3.0106E2, 9.1991E2, 8.6904E2, 8.7608E2,
8.8358E2, 8.8712, 4.5885, 8.9330]. Based on the GPM method, the search direction is now
evaluated. For a reduction of the weight, it concludes that the projection of the negative gra-
dient of W gives rise to a zero-direction with all non-negative Lagrangian multipliers (j0).
This implies therefore that the weight is no more reducible even with a bound relaxation
of the displacement constraints. Conversely, if a reduction of the maximum displacements
is made by projecting the negative gradient of Y-component t
1,1
or t
2, 1
, a zero-direction is
also obtained but with negative Lagrangian multipliers being \ =[8.9989E6, 2.7264E5,
5.3219E6, 1.0168E5, 6.5811E6, 7.8388E4, 3.1628E3, 1.1564E3]. By means
of Equation (37), it turns out that the biggest value among all negative ratios (z
)
}j
)
) refers
to the rst stress constraint attached to bar 16. Therefore, the gradient of this constraint
has to be cancelled from N. Consequently, a modied search direction is reevaluated and
Table III. List of active response constraints with associated Lagrangian multipliers.
Optimum Weight W Displacement q List of active Lagrangian
point (i) (lb) (in) response constraints multipliers (z)
1 1155.13 0.175 Displacements: t
1,1
, t
2, 1
0.0000000E + 00
Stresses: o
16.1
, o
25.1
, o
2.2
, 0.0000000E + 00
o
5.2
, o
7.2
, o
8.2
, o
19.2
, o
20, 2
3.01068425E 02
9.19912364E 02
4.34518871E 02
4.34518871E 02
4.38042233E 02
4.38042233E 02
4.41791559E 02
4.41791559E 02
1,1
, t
2, 1
1.84754826E + 03
Stresses: o
2, 2
, o
5, 2
, o
7, 2
, o
8, 2
, 1.84754826E + 03
o
19, 2
, o
20, 2
3.35478362E 02
3.35478362E 02
2.50888437E 02
2.50888437E 02
3.20807300E 02
3.20807300E 02
1,1
, t
2, 1
2.25729654E + 03
Stresses: o
2, 2
, o
5, 2
, o
7, 2
, o
8, 2
, 2.25729654E + 03
o
19, 2
, o
20, 2
3.10648197E 02
3.10648197E 02
2.11683644E 02
2.11683644E 02
2.95968917E 02
2.95968917E 02
4 1271.511 0.15 Displacements: 2.67736163E + 03
t
1, 2
, t
2, 2
, t
1, 2
, t
2, 2
2.67736163E + 03
Stresses: o
19,2
, o
20,2
1.50114296E + 03
1.50114296E + 03
2.94039349E 03
2.94039349E 03
5 1362.082 0.14 Displacements: 3.06680393E + 03
t
1,1
, t
2, 1
, t
1, 2
, t
2, 2
3.06680393E + 03
1.79656594E + 03
1.79656594E + 03
6 1466.832 0.13 Displacements: 3.55696194E + 03
t
1,1
, t
2, 1
, t
1, 2
, t
2, 2
3.55696194E + 03
2.08339617E + 03
2.08339617E + 03
7 1589.040 0.12 Displacements: 4.17471949E + 03
t
1,1
, t
2, 1
, t
1, 2
, t
2, 2
4.17471949E + 03
2.44486762E + 03
2.44486762E + 03
974 W. H. ZHANG
Table III. Continued.
Optimum Weight W Displacement q List of active Lagrangian
point (i) (lb) (in) response constraints multipliers (z)
8 1733.467 0.11 Displacements: 4.96853448E + 03
t
1,1
, t
2, 1
, t
1, 2
, t
2, 2
4.96853448E + 03
2.90932144E + 03
2.90932144E + 03
9 1906.780 0.10 Displacements: 6.01225773E + 03
t
1,1
, t
2, 1
, t
1, 2
, t
2, 2
6.01225773E + 03
3.51994813E + 03
3.51994813E + 03
Figure 9. Pareto optimum curve.
the solution is given in Table II. By using Equation (8), the criteria sensitivity equals then
dW}d q =3345.6122. This solution can be validated by using nite dierence computing.
To do this, the upper bound q of displacement constraints will be relaxed and tightened,
respectively. In the rst case when q =0.18 in is imposed, it turns out that the optimum
solution is insensitive to the relaxation and remains unchanged. This is consistent with the
solution obtained by GPM method. In the second case when the upper bound is tightened
with q =0.16 in, the perturbation causes the optimum solution to change with an increase
of W =58.4492 lb and it is easily observed that the stress constraint associated with bar
16 does leave the active constraint set and becomes inactive. The search direction dening
the variation of design variable vector is given in Table II. Although the latter is not com-
patible with the previous one, the derivative evaluated by nite dierence approximation is
W} q =3368 with only a relative error of 0.7% with respect to the exact value.
To validate further the proposed method, we study now the trade-o relations between
W and q at other prescribed values of q. Optimum points are obtained in the same way
and plotted in Figure 9. At such points (29), active constraints are listed in Table III with
related values of Lagrangian multipliers. Because the same multipliers correspond to those
constraints having the same gradient and upper bound values, the corresponding gradients
have to be countered and used one time in the formation of the gradient matrix N. At each
optimum, search directions are evaluated by the GPM method for reduction perturbations of
W and q, respectively. Each pair of search directions obtained is listed in Table IV. Both are
Table IV. Search directions obtained by the GPM method at dierent optimum points.
(W0)
Optimum Search directions
point (i) (q0)
1 0 0 0 0 0 0 0 0
0 0.003886 0.003180 0 0 0.020256 0.001351 0.00064
2 0 10.13595 8.30248 0 0 70.5749 3.4941 53.811
0 0.00274 0.002246 0 0 0.019098 0.00094 0.01456
3 0 9.26089 7.5850 0 0 71.0197 3.1957 53.711
0 0.00205 0.001680 0 0 0.015730 0.00070 0.01189
4 0 69.5067 52.5741 0 0 38.1401 3.55047 21.281
0 0.008317 0.006291 0 0 0.004563 0.0004 0.00254
5 0 59.8823 44.6597 0 0 60.9811 79.569 43.912
0 0.00615 0.00459 0 0 0.00626 0.00818 0.00451
6 0 59.88647 44.66261 0 0 60.98227 79.5735 43.9129
0 0.005308 0.003959 0 0 0.005405 0.00705 0.003892
7 0 59.8899 44.6650 0 0 60.9827 79.5768 43.9131
0 0.004523 0.0033737 0 0 0.0046062 0.006010 0.003316
8 0 59.8929 44.66705 0 0 60.9827 79.5794 43.9130
0 0.00380 0.00283 0 0 0.00387 0.00505 0.00278
9 0 59.8955 44.6687 0 0 60.9823 79.5815 43.9125
0 0.00314 0.00234 0 0 0.003198 0.004174 0.002303
Figure 10. Sensitivity curve for trade-o analysis.
aligned and oriented in opposite directions that produce the same sensitivity results as plotted
in Figure 10. In this calculation, it is found that there are no active constraints to be eliminated
from the gradient matrix. Therefore, points (29) are all regular optima. In fact, according
to the theory of the optimum sensitivity to the bound of active constraint [23], an alternative
sensitivity analysis scheme is available for regular optimum points with non-degenerate active
976 W. H. ZHANG
constraints. For points (29), the trade-o between W and q can be analytically evaluated by
means of relevant Lagrangian multipliers
dW
d q
=
J
t=1
z
t
where J denotes the number of active displacement constraints. The sum is used because the
same upper bound q is imposed for all active displacement constraints. In this way, it can be
easily veried that the addition of related Lagrangian multipliers in Table III gives exactly
the same sensitivity results as those by the GPM method (see Figure 10).
4. CONCLUSIONS
This paper presents a new gradient-projection based Pareto optimum sensitivity analysis
method to study the trade-o relations in multicriteria problems. Theoretical demonstrations
are made originally. It is shown that the gradient projection scheme constitutes the basis for
the determination of tangent and normal directions of the Pareto optimum curve. Depend-
ing on the projection direction of the selected function, analytical relations are obtained to
evaluate the weighting shifting that depicts directional derivatives in non-dierentiable cases.
Theoretically, it is revealed that the variation of non-degenerate active inequality constraint set
is the cause of the non-dierentiability. Thanks to the new active constraint strategy, an auto-
matic identication of the active constraint to be inactive is made possible and validated by
numerical solutions. In addition, the proposed method can also be generalized to study the de-
pendence between the objective function and active constraints in single objective problems.
Numerical applications indicate that the optimum sensitivity with respect to the constraint
bound can be rigorously evaluated if the latter is considered as a design parameter.
ACKNOWLEDGEMENTS
This work is supported by the National Natural Science Foundation under the Grant No. 10172072 and
the Aeronautical Foundation under the Grant No. 00B53005. Numerical tests are carried out thanks to
the help of graduate students Tong GAO and Jihong ZHU.
REFERENCES
1. Eschnauer H, Koski J, Osyczka A. Multicriteria Design Optimization. Springer: Berlin, Heidelberg, New York,
1990.
2. Stadler W. Multicriteria Optimization in Engineering and in Sciences. Plenum Press: New York, 1988.
3. Koski J. Defectiveness of weighting methods in multicriterion optimization of structures. Communications in
Applied Numerical Methods 1985; 1:333337.
4. Messac A, Sundararaj GJ, Tappeta RV, Renaud JE. Ability of objective functions to generate points on non-
convex Pareto frontiers. AIAA Journal 2000; 38(6):10841091.
5. Das I, Dennis JE. Normal-boundary intersection: a new method for generating the Pareto surface in nonlinear
multicriteria optimization problems. SIAM Journal on Optimization 1998; 8(3):631657.
6. Athan TW, Papalambros Y. A note on weighted criteria methods for compromise solutions in multi-objective
optimization. Engineering Optimization 1996; 27:155176.
7. Chen W, Wiecek MM, Zhang J. Quality utilitya compromise programming approach to robust design. Journal
of Mechanical Design (ASME) 1999; 121:179187.
8. Tappeta RV, Renaud JE. Interactive multiobjective optimization procedure. AIAA Journal 1999; 37(7):
881889.
9. Tappeta RV, Renaud JE, Messac A, Sundararaj GJ. Interactive physical programming: tradeo analysis and
decision making in multicriteria optimization. AIAA Journal 2000; 38(5):917926.
10. Diaz A. Interactive solution to multiobjective optimization problems. International Journal for Numerical
Methods in Engineering 1987; 24:18651877.
11. Hernandez S. Post-optimal procedures for structural optimization. Advances in Engineering Software 2000;
31:401409.
12. Fadel G, Konda S, Blouin V, Wiecek M. c-optimality in bi-criteria optimization. AIAA Paper 2002; 1233:
256265.
13. Sobieszczanski-Sobieski J, Barthelemy JF, Riley KM. Sensitivity of optimum solutions of problem parameters.
AIAA Journal 1982; 20(9):12911299.
14. Vanderplaats GN, Yoshida N. Ecient calculation of optimum design sensitivity. AIAA Journal 1985;
23(11):17981803.
15. Tseng CH, Lu TW. Minmax multiobjective optimization in structural design. International Journal for
Numerical Methods in Engineering 1990; 30:12131228.
16. Zhang WH, Yang HC. A study of the method for a certain type of multicriteria structural optimization problems.
Computers and Structures 2001; 79(31):27412749.
17. Das I, Dennis JE. A closer look at drawbacks of minimizing weighted sums of objectives for Pareto set
generation in multicriteria optimization problems. Structural Optimization 1997; 14:6369.
18. Zhang WH, Yang HC. Ecient gradient calculation of the Pareto optimal curve in multicriteria optimization.
Structural and Multidisciplinary Optimization 2002; 23(4):311319.
19. Li YS, Fadel GM, Wiecek MM. Approximating Pareto curves using the hyper-ellipse. ISSMO}NASA}AIAA
First Internet Conference on Approximation and Fast Reanalysis Techniques in Engineering Optimization,
1999, http://www.aero.u.edu/issmo/webconf.htm.
20. Fleury C, Schmit LA. Dual methods and approximation concepts in structural synthesis. NASA CR-3226,
December 1980.
21. Zhang WH, Domaszewski M, Fleury C. An improved weighting method with multibounds formulation and
convex programming for multicriteria structural optimization. International Journal for Numerical Methods in
Engineering 2001; 52(9):889902.
22. Fleury C. First and second order convex approximation strategies in structural optimization. Structural
Optimization 1989; 1:310.
23. Luenberger DG. Introduction to Linear and Nonlinear Programming. Addison-Wesley: Reading, MA, 1973.

On The Pareto Optimum Sensitivity Analysis in Multicriteria Optimization

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

On The Pareto Optimum Sensitivity Analysis in Multicriteria Optimization

Uploaded by

Copyright:

Available Formats

INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERING

Int. J. Numer. Meth. Engng 2003; 58:955977 (DOI: 10.1002/nme.890)

Correspondence to: W. H. Zhang, SinoFrench Laboratory of Concurrent Engineering, Department of Aircraft

is dened as the Pareto optimum if there is no other

) with strict inequality for at least one con-

, respectively. For the

, only directional derivatives exist. At point P, weighting vector

, Pareto optimum sensitivity analysis consists in calculating the

is a non-dierentiable one with only directional

S=0 imbedded in Equation (19), it follows that

. From Equation (25), we can write

is obtained by any available

You might also like