Struct Multidisc Optim (2006) 32: 191202
DOI 10.1007/s00158-006-0021-2
R E S E A R C H PA P E R
S. Kitayama M. Arakawa K. Yamazaki
Penalty function approach for the mixed discrete nonlinear
problems by particle swarm optimization
Received: 5 September 2005 / Revised manuscript received: 12 January 2006 / Published online: 8 July 2006
Springer-Verlag 2006
Abstract In this paper, the basic characteristics of particle
swarm optimization (PSO) for the global search are discussed
at first, and then the PSO for the mixed discrete nonlinear
problems (MDNLP) is suggested. The penalty function ap-
proach to handle the discrete design variables is employed, in
which the discrete design variables are handled as the contin-
uous ones by penalizing at the intervals. As a result, a useful
method to determine the penalty parameter of penalty term
for the discrete design variables is proposed. Through typical
mathematical and structural optimization problems, the va-
lidity of the proposed approach for the MDNLP is examined.
Keywords Global optimization Particle swarm
optimization Mixed discrete nonlinear problems
Penalty function approach
1 Introduction
The particle swarm optimization (PSO), which mimics social
behavior, is an optimization technique developed by Kennedy
and Eberhart (2001). It has been reported that the PSO is
suitable for the minimization of nonconvex and continuous
function through many numerical examples, and it is often
said that the PSO can find a global minimum of nonconvex
function with high certainty. Many methods, called meta-
heuristics, have some common characteristics as follows:
(1) Metaheuristic methods are applicable to the nondifferen-
tiable optimization problems because these methods do
not require the gradient information of functions.
(2) Compared with the one-point search method, it is possi-
ble to search the design space widely. This is the reason
S. Kitayama (B) K. Yamazaki
Kanazawa University, Kakuma-machi, Kanazawa, 920-1192, Japan
Tel.: +81-76-2344758
Fax: +81-76-2344668
e-mail: kitagon@t.kanazawa-u.ac.jp
e-mail: yamazaki@t.kanazawa-u.ac.jp
M. Arakawa
Kagawa Univeristy, Hayashi-cho, Takamatsu, Kagawa, 761-0396,
Japan
e-mail: arakawa@eng.kagawa-u.ac.jp
why many metaheuristic methods utilize multiple-point
search.
Few researches on the PSO for the discrete design vari-
ables problems have been reported (Venter and Sobieski
2003; Venter and Sobieski 2004). However, these reports
handled the discrete design variables as the continuous ones,
directly. That is, firstly, all design variables may be handled
as the continuous design variables, and optimized. Secondly,
the round-off or cut-off procedure can be applied to get a
discrete optimum. These approaches to handling the discrete
design variables may be valid in a sense, but some problems
are included. Let us consider the following two cases shown
in Fig. 1a,b.
Figure 1a shows a case where an optimum xL is obtained
when the discrete design variables are handled as the con-
tinuous ones. Points A and B represent the discrete design
variables close to xL . In this case, point B is selected as
the neighborhood of xL by the round-off procedure. How-
ever, the objective function at point B makes a change of
the function value worse in comparison with the objective
function at point A. This case shows that the better discrete
design point is not always selected by the round-off procedure
(Rao 1996). Another case shown in Fig. 1b is well known.
That is, the discrete optimum obtained by the round-
off or cut-off procedure does not satisfy all feasibilities
(Fu et al. 1991).
The PSO is suitable for the optimization of nonconvex and
continuous function. Therefore, all design variables should
be handled as the continuous ones whenever the PSO is
applied to the mixed or discrete design variables problems.
That is, the discrete design variables should be transformed
into the continuous ones by introducing some approaches.
The optimization methods for the discrete design variables
are summarized by Arora et al. (1994).
On the other hand, the penalty function approach for the
discrete design variables has been proposed by Shin et al.
(1990). The algorithm of their approach is described briefly
as follows:
(STEP1) It is supposed that all design variables are the con-
tinuous ones. In this stage the penalty function for
the discrete design variables is not introduced.
192
Fig. 1 Nature of the discrete optimization
(STEP2) Any gradient method (e.g., sequential linear pro-
gramming or sequential quadratic programming)
can be employed to find an optimum in the design
space of the continuous design variables.
(STEP3) The penalty function for the discrete design vari-
ables is introduced at this step.
(STEP4) By updating the penalty parameter of penalty term
for the discrete design variables, a discrete opti-
mum is found. If the discrete optimum does not
satisfy all feasibilities, another initial point is taken.
Return to STEP1.
This penalty function approach may be valid; however,
some problems still remain. In STEP1, the penalty function
for the discrete design variables is not introduced because the
augmented objective function, which consists of the objec-
tive, behavior constraints, and penalty term for the discrete
design variables, becomes a nonconvex function when the
penalty function for the discrete design variables is intro-
duced at the first stage. As a result, a local minimum may be
obtained using any gradient method. In STEP 4, another ini-
tial point has to be taken when the discrete optimum does not
satisfy all feasibilities. This means that the multistart algo-
rithm is employed implicitly. Additionally, however, a con-
stant positive number is usually utilized to update the penalty
parameter; this parameter should be adjusted. This means
that some trial errors to select an appropriate constant pos-
itive number are required. It is very difficult to select such
an appropriate constant positive number with a few trials.
Finally, the application to the mixed discrete nonlinear prob-
lems (MDNLP) was not reported by Shin et al. (1990). That
is, the penalty function approach may be valid only for the
optimization problems of discrete design variables. The main
subjects of this penalty function approach for the discrete de-
sign variables are summarized as follows:
(P1) When the augmented objective function is constructed
at the initial search stage, it is necessary to apply some
global optimization method for the continuous design
variables.
(P2) It is necessary to propose a method to escape from a
local minimum, instead of the multistart algorithm.
S. Kitayama et al.
(P3) It is necessary to propose a new update scheme of the
penalty parameter of penalty function for the discrete
design variables.
(P4) It is also necessary to propose a new method applicable
to the MDNLP.
To overcome P1, the PSO is adopted. To overcome P2 and
P3, a new method to determine the penalty parameter and its
update scheme is also proposed. Finally, the PSO is applied
to the optimum design of pressure vessel, which is one of the
most famous benchmark problems of MDNLP.
In Section 2, the basic characteristics of PSO are dis-
cussed. It is shown that the PSO has similar structure to the
gradient methods, where it is found that the PSO has the
search direction vector and stochastic step-size even though
the PSO does not utilize the gradient information of func-
tion. Additionally, the neighborhood of PSO is also clarified.
In Section 3, the problem definition and the penalty func-
tion approach for the MDNLP are described. In this sec-
tion, a new method to determine the penalty parameter and
its update scheme are described. In Section 4, the algorithm
and flowchart of proposed approach are shown. Through nu-
merical examples, the validity of the proposed approach is
examined.
2 Particle swarm optimization
The PSO is one of the global optimization methods for the
continuous design variables. Several models of PSO have
been proposed. Amongst them, the most popular model may
be the so-called g-best model (Fourie and Groenwold 2002).
2.1 Position and velocity
When the position and velocity of particle d are represented
by x kd and v kd , respectively, and k represents the iteration
counter, the position and velocity of particle d at k+1 iter-
ations are calculated by the following equations.
x k+1
d = x kd + v k+1
d 1t (1)


c1r1 ( pkd x kd ) c2 r2 pkg x kd
v k+1 = wv kd + + (2)
d
1t 1t
where 1t represents the time-step and is set as unit. The co-
efficient w in (2) is called the inertia term. The parameters r1
and r2 denote random numbers between [0,1]. The weighting
coefficients c1 and c2 are recommended to keep the following
relationship (Parsopoulos and Vrahatis 2002).
c1 + c2 4 (3)
In general, c1 =c2 =2 are often used. The position vector pkd ,
called the p-best, represents the best position of particle d
till the k-th iteration, and pkg , called the g-best, represents the
best position in the swarm at the k-th iteration.
Penalty function approach for the mixed discrete nonlinear problems by particle swarm optimization 193

2.3 The inertia term

In the PSO, the inertia term in (4) gradually decreases during

the search iteration by (7).
wmax wmin
w = wmax k (7)
kmax
where wmax and wmin represent the maximum and minimum
values to control the inertia, respectively. In general, the fol-
lowing values are recommended (Yoshida et al. 2000):
wmax = 0.9, wmin = 0.4 (8)
Fig. 2 Neighborhood of PSO
The approach represented by (7) is sometimes called the in-
ertia weight approach.
2.2 The neighborhood and search direction vector of PSO

The following equation can be obtained from (1) and (2) after 2.4 Best position-keeping model
some replacements.
The g-best pkg in (2) represents the best particle in the swarm
at the k-th iteration, and then it may be considered that pkg
x k+1 = x kd + wv kd + q x kd


d (4) is updated at each iteration. As a result, there is no certainty
that the following relationship between pkg and pk+1 g may be
where and q are given by (5) and (6), respectively. established.
f pk+1 f pkg



g (9)
= c1r1 + c2 r2 (5) It is preferable to have the relationship represented by
(9) because of the descent property of swarm. Consequently,
the model keeping the best particle is also considered. This
c1r1 pkd + c2 r2 pkg model, called as the best position-keeping model, is expressed
q= (6)
c1 r 1 + c2 r 2 as follows:
v k+1 = wv kd + c1r1 pkd x kd + c2 r2 pg x kd



d (10)
The neighborhood of PSO is generated from (4), as shown in
Fig. 2, where

the magnitude of neighborhood is represented where pg represents the best position among pkd until now.
by  q x kd .
Additionally, it is possible to interpret that q x kd repre-
sents the search direction when we imagine the similarity to
the gradient methods. in (5) may also be regarded as the
stochastic step-size, in which its lower and upper bounds are
0 and c1 +c2 , and the mean value is (c1 + c2 )/2. From these
relationships, it is possible to consider that the PSO has the
search direction vector and stochastic step-size, even though
the PSO does not utilize the gradient information of function.
Fig. 3 Relationship between pkd and pg
Because pg is selected among pkd , pg is not updated until the
better pkd appears. As a result, it is expected that the (local)
search ability will be enhanced, and fast convergence will be
expected.

Figure 3a shows the relationship of f pkd , f ( pg ) and
f (q) when pg and pkd locate in the same convex space. From
Fig. 3a, the following relationship can be obtained:
f (q) a f pkd + (1 a) f pg



(11)
194 S. Kitayama et al.

where a represents a real number in an interval [0,1]. Indeed, 3.2 Penalty function
(11) shows the condition for the convex space. Additionally,
the following relationship is established between pkd and pg . In this paper, the following penalty function suggested by
Shin et al. (1990) is adopted.
f pg f pkd



(12) n
"  c
#
X 1 2 xm+i 0.25 di, j+1 + 3di, j
(x) = sin +1
From (12), the following equation can be obtained: 2 di, j+1 di, j
i=1

f (q) f pg


(13) (19)
where di,j and di,j+1 represent the discrete design variables.
The above equation is always satisfied when pkd and pg locate c
xm+i is the continuous design variables between di,j and di,j+1 .
in the same convex space. Additionally, pkd represents the Therefore, the augmented objective function F(x) is con-
best position of particle d till the k-th iteration. As a result, structed by using the above penalty function as follows:
the following equation can be obtained.
ncon
X
f (q) f (x) (14) F(x) = f (x) + s(x) + r max [0, gk (x)] (20)
k=1
Hence, the search direction vector always turns to the direc- where s and r denote the penalty parameters of penalty terms
tion which improves the objective function when pkd and pg for the discrete design variables and the behavior constraints,
locate in the same convex space. respectively. Finally, the MDNLP is transformed into the fol-
On the other hand, there is no certainty that (11) is satisfied lowing problem of continuous design variables.
in the case of the situation shown in Fig. 3b. In this case,
the search direction vector turns to the direction which does F(x) min (21)
not improve the objective function. This means that some
particles can escape from a local minimum. xiL xi 6 xiU i = 1, 2, , m (22)
c
di,1 xm+i di,q i = 1, 2, , n (23)
For simplicity, the design variables are supposed as the
3 Penalty function approach for the discrete design discrete ones in the following discussion. The case of mixed
variables design variables will be discussed in Section 3.8 separately.

3.1 Problem definition

In general, the MDNLP is described as follows (Arora
et al. 1994):
f (x) min (15)
xiL xi xiU i = 1, 2, , m (16)
xm+i Di Di = di,1 , di,2 , , di,q i = 1, 2, , n (17)

gk (x) 0 k = 1, 2, , ncon (18)
where x denotes the design variables, which consist of the
continuous and discrete design variables. f (x) is the objec-
tive function to be minimized, and gk (x) is the behavior con-
straint. ncon represents the number of behavior constraints. xi
represents the design variables, and m and n are the total num-
ber of continuous and discrete design variables, respectively.
xiL and xiU denote the lower and upper bounds of continuous
design variables, respectively. Di is the set of discrete values
for the i-th discrete design variable. di,j is the j-th discrete
value for the i-th discrete design variables. q represents the
number of discrete values. The lower and upper bounds of
discrete design variables are given by di,1 and di,q , respec-
tively. In general, the number of available discrete values q
for each discrete design variable may be different.
3.3 Characteristics of penalty function for the discrete
design variables
The value of (19) becomes small around the neighborhood
of the discrete value. On the other hand, the value of (19)
becomes large, turning from the discrete value. When pkg
satisfies the following equation, the discrete value resides
around the neighborhood of pkg .
pkg


(24)
where represents a small positive value. As a result, the
penalty parameter s in (20) must be updated so that (24)
is satisfied. Let us consider the following simple problem to
examine the effect of penalty parameter s (example)
8
f (x) = x 4 x 3 2x 2 + 8x min (25)
3
x {1, 0, 1, 2} (26)
In this simple example, the objective f (x) and the augmented
objective function F(x) are shown in Fig. 4. The penalty pa-
rameter s in Fig. 4 is set as s=10.
It is apparent that F(x) becomes a nonconvex and contin-
uous function, and many local minima are generated around
the neighborhood of discrete values. As a result, the problem
to find the discrete optimum is transformed into finding the
Penalty function approach for the mixed discrete nonlinear problems by particle swarm optimization
Fig. 4 Objective and augmented objective functions
global minimum of F(x). Therefore, it is natural to consider
that the PSO is applied to F(x). Additionally, the discrete val-
ues are given at the points where the relative error between
f (x) and F(x) becomes small. The following equation is uti-
lized as the terminal criteria, instead of using (24) directly.
195
3.4 Initial penalty parameter for the discrete design
variables
The initial search point x d of particle d is selected at random.
Then the value of (19) is calculated for each particle. The
penalty parameter s in (20) is determined as follows:
sd = 1 + (x d ) d = 1, 2, , agent (28)
where sd represents the penalty parameter of particle d. The
number agent in (28) is the total number of particles, and the
initial penalty parameter sinitial is determined as follows:
sinitial = min s1 , s2 , , sagent

(29)
At the initial stage to search the optimum, F(x) is actively
transformed into a nonconvex and continuous function, and
many local minima are generated around the neighborhood
of discrete values.
3.5 Update scheme of penalty parameter



 F pk f pk 

 The following equation is used to update the penalty para-
g


g
(27) meter s.
 F pk 
g
s = s exp 1 + pkg



(30)
In case of  F pkg  , the numerator in (27) is used as


The behavior of F(x) by updating the penalty parameter
the terminal criteria (that is,  F pkg f pkg  ).



As mentioned above, the PSO does not use the gradient
information of function; therefore, it is difficult to satisfy (24)
strictly. Therefore, (27) is used instead. Behaviors of F(x) for
the various penalty parameter s are shown in Fig. 5. It is found
from this figure that to determine the penalty parameter s in
advance is very difficult.
Fig. 5 Augmented objective function for the various penalty parameters
s is shown schematically in Fig. 6, in which the solid line
shows F(x) at the k-th iteration, and the dotted line shows
F(x) at the k+1-th iteration.
As shown in Fig. 6, F(x) at the k+1-th iteration becomes
a highly nonconvex and continuous function in comparison
with F(x) at the k-th iteration. For example, a point A in Fig. 6
corresponds to the point pkg at the k-th iteration. By updating
196
Fig. 6 Update of the penalty parameter
the penalty parameter s, pkg corresponds to the point A0 on the
dotted line, which represents F(x) at the k+1-th iteration. It
is apparent that F(x) at the k+1-th iteration becomes a highly
nonconvex function in comparison with F(x) at the k-th itera-
tion. As discussed in Section 2.3, the PSO has a similar struc-
ture to the gradient methods. For this reason, it is expected
that pkg moves to the direction in Fig. 6. Finally, it is also
expected that pkg will satisfy (27). The proposed approach
need not select an appropriate constant positive number to
update the penalty parameter s in advance. By using (30), the
penalty parameter s is determined automatically.
3.6 Initialization of penalty parameter
When (27) is satisfied, the discrete value resides around the
neighborhood of pkg . Then, the initial penalty parameter ob-
tained by (29) is utilized to find another discrete value because
F(x) becomes a highly nonconvex and continuous function
by updating the penalty parameter s. It may be assumed that
pkg fails to escape from a local minimum. In such occasions,
F(x) is relaxed by using the initial penalty parameter when
(27) is satisfied. As a result, it may be expected that pkg can
escape from a local minimum. The concept of relaxation of
F(x) to escape from a local minimum is shown in Fig. 7.
3.7 Difference between the traditional and proposed
methods
The penalty function for the discrete design variables adopted
in this paper is basically the same as that of Shin et al. (1990).
However, the approaches are very different from each other.
Shin et al. searched the optimum by handling all design vari-
ables as the continuous ones at the initial stage, where the
penalty parameter s in (20) has been set as zero. After the op-
timum obtained by handling all design variables as the con-
S. Kitayama et al.
Fig. 7 Escape from a local minimum by relaxation of augmented
objective function
tinuous ones was found, the penalty function for the discrete
design variables was introduced to avoid the search proce-
dure of a global minimum amongst many local minima of
the augmented objective function. Additionally, Shins ap-
proach cannot overcome the case shown in Fig. 1b.
On the other hand, the penalty parameter s is actively
introduced at the initial stage in the proposed approach.
Obviously, the augmented objective function becomes a
nonconvex and continuous function. However, this is not
a serious problem because the PSO is applied to this non-
convex and continuous function. The proposed approach ob-
viously overcomes the case shown in Fig. 1b because of its
flexible and strict search ability. The new update scheme of
penalty parameter s is proposed by (30). In past works (Fu
et al. 1991; Shin et al. 1990), a constant positive number
was used to update the penalty parameter. This means that
the update scheme of the penalty parameter depends on the
problem. On the other hand, the penalty parameter s may al-
ways change in the proposed approach because the value of
penalty function pkg is utilized. It may be expected that the


penalty parameter s is determined automatically, and flexible
applications are possible. Finally, the initialization of penalty
parameter s is also proposed to relax the augmented objective
function. As a result, it is expected that pkg can escape from
a local minimum.
With the binary PSO, it is also easy to handle the dis-
crete design variables (He et al. 2004; Kennedy and Eberhart
1997); however, the search process of binary PSO is stochas-
tic. Additionally, it is not sure that the objective function or
design space is not always continuous. On the other hand, the
characteristics of PSO and penalty function for the discrete
design variables have been clarified through Sections 2 and 3.
The approach adopted here utilizes the characteristics of PSO
Penalty function approach for the mixed discrete nonlinear problems by particle swarm optimization 197

and penalty function for the discrete design variables to find

the optimum. That is, the proposed approach may be deter-
ministic in comparison with the binary PSO.

3.8 In the case of the mixed design variables

In the case of the mixed design variables, the component of

pkg can be expressed as follows:
 cont 
x
pkg = (31)
x discr t
where x cont = (x1 , x2 , , xm )T and x discr t = xm+1
c
, xm+2
c
,
, xm+n ) represent the components of continuous and dis-
c T

crete design variables, respectively. Then, the components of

continuous design variables xcont in pkg are neglected when
the terminal criteria by (27) are applied. That is, only the com-
ponents of discrete design variables xdiscont in pkg are adopted
when the terminal criteria by (27) is utilized. Fig. 8 The algorithm for MDNLP by PSO

4 Algorithm
The basic algorithm of PSO for the MDNLP is described in
the following:
(STEP1) Number of particles and maximum search itera-
tions k max are determined. Set iteration counter
k=1, initial position, and velocity of each particle
at random.
(STEP2) Calculate the value of (19) for each particle.
(STEP3) Calculate the penalty parameter s for each particle
by (28).
(STEP4) From STEP3, determine the initial penalty para-
meter sinitial by (29).
(STEP5) Calculate the augmented objective function.
(STEP6) Calculate of the g-best and the p-best.
(STEP7) Check the terminal criteria by (27). If (27) is satis-
fied, the penalty parameter s is initialized. Other-
wise, the penalty parameter s is updated by (30).
(STEP8) Update the inertia term by (7).
(STEP9) Update the position and velocity by (1) and (2).
(STEP10) Increase iteration counter as k=k+1.
(STEP11) Check the iteration counter. If kk max , go to
STEP5. Otherwise, the algorithm will terminate
by taking pkg as optimum.
The algorithm mentioned above is shown in Fig. 8. When
the best position-keeping model is used, (11) is used to update
the velocity.
5 Numerical examples
In this section, the validity of the proposed approach is exam-
ined through numerical examples. At first, benchmark prob-
lems of the continuous design variables are considered to
examine the search ability between the g-best and the best
position-keeping model. Then, typical benchmark problems
for the MDNLP are treated.
5.1 Continuous design variables problem
Let us consider the following problem:
m n
X p o
f (x) = 418.9829 m + xi sin |xi | min (32)
i=1
500 x 500 (33)
The global minimum of this benchmark problem is well
known as follows:
xi,G = 420.9687 i = 1, 2, , m (34)
The behavior of this function in two dimensions and the
global optimum are shown in Fig. 9.
The number of particles is set as 10, and the maximum
number of search iterations k max is set as 200. Examples of
typical histories of the g-best are shown in Fig. 10 for the
g-best model, and Fig. 11 for the best position-keeping model.
From Fig. 10, the g-best of the g-best model moves at
each iteration because the g-best is selected among the best
particles at the k-th iteration. As considered that each particle
goes to the g-best, it is apparent that the g-best model may
show slow convergence. On the other hand, the g-best of best
position-keeping model shows fast convergence because the
g-best is selected among the p-best. Additionally, the g-best of
the best position-keeping model can find some local minima
during the search process, as shown in Fig. 11, where points
a, b, c, and d represent local minima. The search process of
g-best in the best position-keeping model is also shown in
Fig. 9.
198 S. Kitayama et al.

Fig. 9 Behavior and contour of objective

Fig. 10 History of the g-best of the g-best model

Fig. 11 History of the g-best of the best position-keeping model

Penalty function approach for the mixed discrete nonlinear problems by particle swarm optimization 199

5.2 Topology optimization problem of truss structure

This problem is taken from Kitayama and Yamazaki (2005),

in whose work an optimum topology at global minimum was
reported. The objective is to minimize the total volume of
structure subject to the behavior constraints of the displace-
ments. The problem is described as follows:

m
X
f (A) = Ai L i min (35)
i=1
Fig. 12 Ground structure of topology optimization
g j (A) = u k (A) u a (A) 1 0 j = 1, 2, , ncon

(36)

is a=100[mm]. The vertical displacements at nodes 4 and 7

Ai,min Ai Ai,max i = 1, 2, , m (37)
where A = (A1 , A2 , , Am )T denotes the cross-sectional
areas of each element, which are the continuous design vari-
ables. f (A) is the total volume of structure, and gj (A) is the
behavior constraint. uk and ua represent the k-th nodal dis-
placement and the allowable limit of displacement, respec-
tively. Ai,max and Ai,min are the upper and lower bounds of
cross-sectional areas of the i-th member. The analysis model,
which consists of nine nodes and 28 elements, is shown in
Fig. 12.
Nodes 1 and 3 are fixed and the external forces P = 1.00
103 [N ] are applied at nodes 4 and 7. The distance of each node
are constrained less than the allowable displacement u a =
1.50 102 [mm]. All upper and lower bounds of cross-
sectional areas of each element are set as Ai,max = 1.00
103 [mm 2 ] and Ai,min = 1.00[mm 2 ]. The Youngs modulus
is set as E=210[GPa]. At the global minimum, two con-
straints of vertical displacements are active. To compare the
results, the optimum topologies of the same problem are also
searched by the simple genetic algorithm (GA), the distrib-
uted GA, and the simulated annealing (SA). In the PSO, the
number of particles is set as 100, and the maximum number
of search iterations k max is set as 2,000. The best position-
keeping model is utilized. Some parameters in the simple
GA, the distributed GA, and the SA are taken from Kitayama

Table 1 Comparison of optimum

topologies
200 S. Kitayama et al.

and Yamazaki (2005) [in the simple GA, the population size
is set as 100, the maximum number of search iterations is set
as 3,000, the crossover ratio is set as 0.6, and the mutation
ratio is set as 0.01. In the distributed GA, the population size
is set as 100, the maximum number of search iterations is
set as 10,000, the crossover and mutation ratios are the same
as those of simple GA, The number of islands in distributed
GA is four. In the SA, initial temperature is set as 1.0. TcT
(c=0.85) is utilized as the cooling schedule. Minimum tem-
perature is Tmin = 1.00 105 . The number of search itera-
tions for each temperature step is 1,000].
The final results are shown in Table 1. In the topology il-
lustration some members that have the cross-sectional areas
on the lower bound are omitted. Additionally, two behavior
constraints of vertical displacements at nodes 4 and 7 are
active at the global minimum. The result of PSO is consid-
ered that the active constraints can be practically obtained.
From Table 1, the PSO is superior to the GA in the case of
continuous design variables.

5.3 Problem of the discrete design variables problem

As shown in the above two examples, the excellent search

abilities of the best position-keeping model have been shown.
Therefore, the best position-keeping model is applied to the
following problems. in (27) is set as 1.010-2 (Rosen-
brocks test function) (Kannan and Kramer 1994).
f (x) = 100(x2 x12 )2 + (1 x1 )2 min (38)
0.55 x 4.95, interval 0.55 (39)
D
In this problem, the discrete optimum is x opt = (1.65,
T
2.75) . Let us specify that variables x1 and x2 must be the
discrete integer multiples of 0.55. In Fig. 13, the symbol
shows the discrete design points. The continuous and dis-
crete optima are also shown in Fig. 13. When the discrete
Fig. 14 Contour plots of augmented objective functions
design variables are handled as the continuous ones directly,
the discrete optimum cannot be obtained by the rounding-off
or cutting-off procedure.
The number of particles is set as 10 and the maximum
number of search iterations k max is set as 100. The behavior
of the augmented objective function is shown in this session.
The changes of contour of augmented objective function are
shown in Fig. 14 for a couple of penalty parameters s.
The contour of the augmented objective function is shown
in Fig. 14a when the initial penalty parameter is s=1.344. In
this case, the difference between the contour of continuous
design variables and the one of discrete design variables is
not clear. However, the behavior of the augmented objec-
tive function gradually changes by increasing s, as shown
in Fig. 14b,c. From Fig. 14, the augmented objective func-
tion becomes highly nonconvex when the g-best attains at the
global minimum. In this case, the g-best can always attain at
the global minimum xopt D
= (1.65, 2.75)T .

Fig. 13 The discrete and continuous optima Fig. 15 Optimum design problem of pressure vessel
Penalty function approach for the mixed discrete nonlinear problems by particle swarm optimization 201

Table 2 Comparison of results

Sandgren (1990) Qian et al. (1993) Kannan and Kramer (1994) Hsu et al. (1995) He et al. (2004) This research
R (inch) 47.00 58.31 58.29 N/A 42.10 42.37
L (inch) 117.70 44.52 43.69 N/A 176.64 173.42
Ts (inch) 1.125 1.125 1.125 N/A 0.8125 0.8125
Th (inch) 0.625 0.625 0.625 N/A 0.4375 0.4375
g1 (x) 0.19 0.00 0.00 N/A 0.00 0.00
g2 (x) 0.28 0.11 1.11 N/A 0.08 0.08
g3 (x) 0.51 0.81 0.82 N/A 0.26 0.28
g4 (x) 0.05 0.02 1.11 N/A 0.00 0.00
Objective ($) 8, 129.80 7, 238.83 7, 198.20 7,021.67 6, 059.71 6, 029.87

5.4 Optimum design of pressure vessel 1296000 43 x13

g4 (x) = 10 (46)
x12 x2
Let us consider an optimum design problem of pressure ves-
sel, as shown in Fig. 15. This problem is one of the most in which x3 and x4 are the discrete design variables and are
famous benchmarks for the MDNLP, and many researchers the integer multiples of 0.0625.
have tried to solve it (Kannan and Kramer 1994; Sandgren The behavior constraints from (43) to (46) are included
1990; Qian et al. 1993; Hsu et al. 1995; He et al. 2004). Sev- in the augmented objective as the penalty function by using
eral results are shown in Table 2. From Table 2, it is clear that (20). From the past researches, it was considered that twice
it is very difficult to find a global optimum, even though this the value of the objective function is suitable for the penalty
problem consists of only four design variables. The design parameter of the behavior constraints. Therefore, r=1.0108
variables are radius R=x1 , length L=x2 , thickness Ts=x3 , and is used.
thickness Th=x4 . The objective is to minimize the cost given The objective considering all design variables as the
as continuous ones is f (xopt )=5,885.32, and the optimum is
x opt = (40.2, 200, 0.778, 0.384)T . Noting that x3 and x4 are
f (x) = 0.6224x1 x2 x3 + 1.7781x12 x4 the discrete design variables, the round-off or cut-off pro-
cedure can be applied to these design variables. However,
+3.1661x2 x32 + 19.84x1 x32 min (40)
g1 (x) and g2 (x) are not satisfied by the round-off or cut-off
procedure.
subject to the side and behavior constraints given as
The number of particles is set as 50, and the maximum
number of search iterations k max is set as 500. Ten trials are
10 x1 , x2 200 (41) performed with different random seeds. The best result during
the 10 trials is shown in the last column in Table 2. From
Table 2, the best result could be obtained by the proposed
0.0625 x3 , x4 6.1875 (42) method. The results of 10 trials are also shown in Table 3.
The search history of g-best in the worst case is shown in
0.0193x1 Fig. 16. This case corresponds to the trial 10 in Table 3. The
g1 (x) = 10 (43) average of function calls in the 10 trials is 22,500. On the
x3
other hand, the binary PSO requires 30,000 function calls to
0.00954x1
g2 (x) = 10 (44) obtain the result. The proposed approach has achieved the
x4 reduction of 7,500 function calls. It is clear that some local
x2 minima have been found through this search process. From
g3 (x) = 10 (45) Fig. 16, it is found that the g-best can escape from some
240

Table 3 Results of 10 trials

Trial no. R (inch) L (inch) Ts (inch) Th (inch) g1 (x) g2 (x) g3 (x) g4 (x) Objective
1 42.37 173.42 0.8125 0.4375 0.00 0.08 0.28 0.0 6, 029.87
2 45.79 135.72 0.875 0.4375 0.00 0.00 0.43 0.0 6, 039.84
3 45.78 135.81 0.875 0.4375 0.00 0.00 0.43 0.0 6, 040.78
4 45.73 136.27 0.875 0.4375 0.00 0.00 0.43 0.0 6, 046.33
5 45.70 136.27 0.875 0.4375 0.00 0.00 0.43 0.0 6, 049.72
6 45.70 136.59 0.875 0.4375 0.00 0.00 0.43 0.0 6, 049.97
7 45.41 136.62 0.875 0.4375 0.00 0.00 0.42 0.0 6, 081.93
8 45.31 139.48 0.875 0.4375 0.00 0.01 0.41 0.0 6, 094.76
9 45.30 140.59 0.875 0.4375 0.00 0.01 0.41 1.0 6, 094.30
10 45.10 142.68 0.875 0.4375 0.00 0.02 0.41 0.0 6, 117.81
202
Fig. 16 History of the g-best for the worst case in 10 trials
local minima, and the effect of the initialization of penalty
parameter s for the discrete design variables may be valid.
6 Conclusions
In this paper, the basic characteristics of PSO have been ex-
amined. As a result, the following characteristics of PSO have
been clarified:
(1) The PSO has the neighborhood.
(2) It is possible to consider that the PSO also has the search
direction vector.
(3) It is possible to consider that the PSO has the stochastic
step-size.
Then, the PSO was applied to the MDNLP. The penalty
function for the discrete design variables was introduced
to handle as the continuous design variables. To utilize the
penalty function for the discrete design variables, it is pos-
sible to handle all design variables as the continuous ones.
Additionally, the augmented objective function becomes non-
convex and continuous by introducing the penalty function.
As it is considered that the PSO is naturally suitable for the
global search of the nonconvex and continuous function, the
proposed approach may be valid. The method to determine
the penalty parameter s and the new update scheme is also
proposed. By introducing the initialization of the penalty pa-
rameter s, it is expected that the g-best can escape from a local
minimum. As results, it is considered that problems P1, P2,
P3, and P4 described in the introduction may be overcome.
S. Kitayama et al.
Through typical numerical examples, the basic validity of
PSO for the MDNLP has been examined.
Acknowledgements The authors would like to say many thanks to
Prof. Yamakawa, H. (Waseda University) and Prof. Nakayama, H.
(Konan University), who gave us useful and contributive comments.
References
Arora JS et al (1994) Methods for optimization of nonlinear problems
with discrete variables: a review. Struct Optim 8:6985
Fourie PC, Groenwold AA (2002) The particle swarm optimization
algorithm in size and shape optimization. Struct Multidisc Optim
23(4):259267
Fu JF et al (1991) A mixed integer-discrete continuous programming
and its application to engineering design optimization. Eng Optim
17:263280
He S et al (2004) An improved particle swarm optimizer for mechanical
design optimization problems. Eng Optim 355:585605
Hsu YH et al (1995) A two stage sequential approximation method for
non-linear discrete variable optimization. ASME Design Engineer-
ing Technical Conference, Boston, pp 197202
Kannan BK, Kramer SN (1994) An augmented lagrange multiplier
based method for mixed integer discrete continuous optimization
and its applications to mechanical design. Trans ASME J Mech Des
116:405411
Kennedy J, Eberhart RC (1997) A discrete binary version of the particle
swarm optimization algorithm. Proceedings of the 1997 Conference
on Systems. Man, and Cybernetics, pp 41044109
Kennedy J, Eberhart RC (2001) Swarm intelligence. Morgan Kauf-
mann, San Mateo
Kitayama S, Yamazaki K (2005) Generalized random tunneling algo-
rithm for continuous design variables. Trans ASME J Mech Des
127:408414
Parsopoulos KE, Vrahatis MN (2002) Recent approaches to global
optimization problems through particle swarm optimization. Nat
Comput 1:235306
Qian Z et al (1993) A genetic algorithm for solving mixed discrete
optimization problems, DE-Vol.65-1. Adv Des Autom 1:499503
Rao SS (1996) Engineering optimization: theory and practice. Wiley
InterScience, New York
Sandgren E (1990) Nonlinear and discrete programming in mechanical
design optimization. Trans ASME J Mech Des 112:223229
Shin DK et al (1990) A penalty approach for nonlinear optimization
with discrete design variables. Eng Optim 16:2942
Venter G, Sobieski JS (2003) Particle swarm optimization. AIAA J
41(8):15831589
Venter G, Sobieski JS (2004) Multidisciplinary optimization of a
transport aircraft wing using particle swarm optimization. Struct
Multidisc Optim 26(12):121131
Yoshida H et al (2000) A particle swarm optimization for reactive power
and voltage control considering voltage security assessment. IEEE
Trans Power Syst 15(4):12321239