Professional Documents
Culture Documents
Dimitri P. Solomatine
Professor of Hydroinformatics
1
Optimization is in Greek, maximization
Single-objective optimiz.
X squared
250
Optimization problem
2
Decision making in planning
Classical optimization
3
Unconstrained optimization (1)
X squared
250
200
150
X squared
100
50
0
0 5 10 15 20 25 30 35
4
Constrained optimization (1)
250
200
150
X squared
100
50
0
0 5 10 15 20 25 30 35
250
200
Minimum
150
X squared
100
50
0
0 5 10 15 20 25 30 35
5
Constrained optimization (3)
6
Constrained optimization (5): penalty
functions
f(x) min
gk (x) < 0 k=1,,
k=1 m
7
Steepest descent
8
Possible problems with the convergence
9
Optimization
O i i i alongl the
h line,
li
without calculation of derivatives
a b c
10
Line optimization principle of bracketing
Minimum is originally F(x)
bracketed by points 1,3,2.
Choose some point 4 between
2 (worst) and 3 (best)
Evaluate function at 4
If 4 is better than 2, replace 2
by 4 F(x1)
Select some point 5 between 1
(worst) and 3 (best)
If 5 is better than 1, replace 1
by 5 5 6
Then 6, which replaces 4.
The rule at each stage is to
keep a center point that is 3
lower than the two outside
start
points
After the steps shown, the X
minimum is bracketed by
points 3,6,5
x1 x3 x2
How to choose the points?
D.P. Solomatine. Introduction to Optimization 21
Solution:
imagine b is known so that
points a, b, c are given.
minimum is between b and c
how to find x ? x ? c
a b
(b a ) (c b )
W 1-W
W 1W
(c a ) (c a )
( x b) Z
Z
(c a )
Z
Minimize worst case possibility choose x W Z 1 W Scale similarity: W
to make the next interval equally probable 1 W
to be to the left and to the right of x Combine:
3 5
W 2 3W 1 0 Solution: W 0.38197
2
D.P. Solomatine. Introduction to Optimization 22
11
Line optimization Brent search (1)
original points:
1,2,3
F(x)
parabola is drawn
-------
its minimum (4)
replaces 3
new parabola is F(x1)
drawn through
1,4,2
its minimum is at
5, which is close F(x5)
to the function
minimum x
x1 x2 x3
The best estimate of the minimum
D.P. Solomatine. Introduction to Optimization 24
12
Stopping rules of iterative algorithms
Relative tolerance f ( xn ) f ( xn 1 )
f ( xn )
Additional
dd l stopping rules:
l xn xn 1
x
Ensuring changes in x are not too large xn
Linear programming
13
Linear optimization (linear programming)
14
Dynamic programming
15
Dynamic programming
volume s2=Q-x1 s3=s2-x2 s3-x3 Inflow qi
Volume Q, s2, s3,
Q
0 State
Decisions: x1 x2 x3 Decision
NB3(x)
4 possible states s3
Dynamic programming
s2=Q-x1 s3=s2-x2
Q
NBj(x) = net benefit of making decision x at stage j
fj (s) = total net benefit of having state s at stage j
((includes benefits at all subsequent
q stages)
g ) 0
Calculate all immediate benefits NBi(x)
of all possible decisions x at each stage j Decisions: x1 x2 x3
at stages 1,2,3
Determine max possible benefit f3(s3) for each state s3 : f3 (s3 ) maxNB3 (x3 )
0 x3s3
Determine max possible benefit f2(s2) associated
with decisions x2 and x3 given initial state s2 (for each state s2)
16
Multi--criterial (multi
Multi (multi--objective) optimization
33
17
Pareto optimal set of solutions
dominates all other solutions
OF 2 Best for OF2
ideal point
(does not exists)
Inflow qi
Volume Q, s2, s3,
Example:
Multi--purpose reservoir optimization:
Multi
use of a Neural Network
as a replicator of hydraulic/hydrologic model
18
Multi--objective optimization: Apure river
Multi
37
19
Decision space and objective (criteria) space
Evaluation by
running a
model
OF 1
Decision space
D Difference between target and
D=Difference RN
achieved power production
ideal point Alternative
(does not exists) solutions
R1
R2 Rt releases (1..N)
D.P. Solomatine. Introduction to Optimization 40
20
Converting multi
multi--objective problem to a
single--objective problem
single
consider 2-objective problem:
Z1 min and Z2 min
Z1
weighting method:
Z = w1 Z1 + w2 Z2
disadvantage: how to choose w1, 2?
one is compensated by another
method of ideal point
Z = sqrt (Z12 + Z22) Z2
disadvantage: data must be
normalized
one is compensated by another
Z1
constraint method
consider only Z2 < FixedValue
then Z1 min
disadvantage: less sensitive
to the value of Z2 Z2
optimal solution
D.P. Solomatine. Introduction to Optimization 41
21
Reservoir optimisation in Apure river basin
using a surrogate ANN model
Mike11/NAM Neural Network emulator
Surrogate modelling: Optimisation
Mike11/NAM was run Start
offline for a variety of
scenraios and inputs generate release schedules
The generated data was run Mike11/NAM model river and
used to train a Neural run Neural network replicating catchment
to produce water
Mike11/NAM model levels
Network surrogate model modelling
(model of a model) check navigability
This ANN model was used constraints
instead of Mike11/NAM in
the optimization loop optimal solution
reached?
Stop
43
D.P. Solomatine. Role of ICT:
22
Multi--extremum functions
Multi
250
200
Minimum
150
X squared
100
50
0
0 5 10 15 20 25 30 35
Multi--extremum problems
Multi
0
-1.5 -1 -0.5 0 0.5 1 1.5 2
23
Multi--extremum problems
Multi
24
Process of calibration as an optimization
problem (1)
Input X
Physical Model
system y = f (X, P)
Error small
enough? No, update model parameters P
Yes
Stop
D.P. Solomatine. Introduction to Optimization 49
25
Global (multi-
(multi-extremum) optimization
26
Example: Hosaki function of 2 var., two
minima, global at (4, 2), local at (1, 2)
7 3 1 4 2 - x2
f ( x1 , x2 ) = ( 1 - 8 x1 + 7 x12 - x1 + x1 ) x 2 e
3 4
27
Another example
28
Random search methods
29
Random search methods: some algorithms
30
Genetic and evolutionary algorithms
31
GA: genes, chromosomes, binary coding
In 2 dimensions
recombination can be
seen as the exchange
of coordinates:
S=(xS1 xS2)
T=(xT1 xT2)
Offspring:
S=(xS1 xT2)
T (xT1 xS2)
T=(x
32
GA: recombination in multidimensional case
33
GA: recombination with binary coding
Two offsprings:
S' = { S1,...,S,T+1,...,Tnl }
T' = { T1,...,T
, , ,
,S+1,
,...,S
, nl }
34
GA: mutation operator
In a chromosome:
S = { S1,...,S S,...,SSnl }
randomly change bit S
35
GA algorithm iterate until the desired
accuracy is achieved
0.Set population number t = 0.
1 (initialize population). Construct a set P(t) of N points by drawing
them
h from
f X according
d to a certain rule.
l
2 (evaluate initial population). Compute function values in all points
of the population.
3 (recombine). Recombine points from P(t) according to the
recombination operator, creating new points (individuals) called
offsprings; put them to the population thus creating set P1(t).
4 (mutate). Modify (mutate) coordinates of some points from P1(t)
according to the mutation operator, creating set P2(t).
5 (evaluate population). Compute function values in all points of the
population P2(t).
6 (select). Apply selection operator and select some best points from
P2(t), thus creating the next population P(t+1).
7. If termination criterion is not fulfilled, then increase t and go to
step 3, otherwise stop.
D.P. Solomatine. Introduction to Optimization 71
36
Simplex descent method
evaluate new
point
reflect the
worst point worst point
repeat the
process
D.P. Solomatine. Introduction to Optimization 74
37
Simplex descent method: possible operators
38
Main ideas used in ACCO
39
Adaptive cluster covering (ACCO): step 1
randomly generate
points
i t
reduce (leave only good)
cluster
iterate for each cluster
(box):
generate more points
shift the center
shrink the box
40
Adaptive cluster covering (ACCO) : step 3
41
Adaptive cluster covering (ACCO)
Main principles:
reduction
clustering
adaptation
periodic
randomization
42
Optimization tool GLOBE (by Solomatine) in
calibration of a rainfall-
rainfall-runoff model
43
Global optimization tool GLOBE:
the following algorithms are implemented
CRS2 (controlled random search, by Price 1983)
CRS4 (modification by Ali & Storey 1994)
GA with
ith a one-point
i t crossover, and
d with
ith a choice
h i between
b t the
th
real-valued or binary 15-bit coding, various random bit mutation,
between the tournament and fitness rank selection, and between
elitist and non-elitist versions
Multis - multistart algorithm (based on Powell-Brent direct
minimization)
M-Simplex - multistart algorithm (based on downhill simplex
method of Nelder and Mead)
adaptive cluster covering (ACCO)
adaptive cluster covering with local search (ACCOL)
adaptive cluster descent (ACD);
adaptive cluster descent with local search (ACDL)
Case study:
using global optimization in calibration of
models
44
Process of calibration as an optimization
problem (1)
Input X
Physical Model
system y = f (X, P)
Error small
enough? No, update model parameters P
Yes
Stop
D.P. Solomatine. Introduction to Optimization 89
45
Process of calibration as an optimization
problem (2)
Popular way to measure error is root mean squared error:
1 T
wt OBSt MODt ( P) min
2
E ( P) i
T t 1
OBSt = observed output variables values (numbers),
MODt = values generated by a model
time moments t = 1T
P is the parameters vector {P1, , Pn} = independent
variables in the optimization
p problem
p
Problems in solving this problem:
there is no analytical expression for E(P), so we
cannot find the derivatives (gradient).
typically it is multi-extremum
often calibration is a multi-objective problem
D.P. Solomatine. Introduction to Optimization 91
46
Experience in using GO for model calibration
47
Tank model of Sugawara (1974)
eight parameters
that cannot be
measured:
d1, d2
k1, k2, k3, k4
s1, s2
they have to be
identified
(calibrated)
48
Rainfall--runoff model: calibrated
Rainfall
49
Performance of various GO algorithms
CRS2
GA
8
ACCOL/3LC
Multis
Function value
7 M-Simplex
CRS4
ACDL/3LC
6
4
0 1 2 3 4
Thousands
Function evaluations
D.P. Solomatine. Introduction to Optimization 99
50
Water distribution networks
Using EPANET models
and single
single- and multi
multi-objective
objective optimization
Abebe, A.J., and Solomatine D. (1998). Application of global optimization to the design
of pipe networks. Proc. Int. Conf. on Hydroinformatics, Balkema, Rotterdam.
L. Alfonso, A. Jonoski, D.P. Solomatine (2009). Multi-objective optimization of
operational responses for contaminant flushing in water distribution networks. ASCE J.
Water Res. Planning & Management.
101
30 24 18 7
18 7
24 19
31
6
19
30 29 23 20 3 4 5
29 28 23 20 3 4 5 6
21 2 RESERVOIR
34 Pipes 21 2 100
31 Nodes 22 1
1 Reservoir 22
1
51
Example 2: optimization of water distribution
network (WDN) rehabilitation
52
Network optimization based on EPANET
model
Hydraulic model
Optimization tool
Optimal solutions
Example 2: Multi
Multi--objective optimisation of operational
responses for contaminant flushing in WDN
Trace J9
0.00 V5
0.00 Contam
0.00
50.00
percent
V4 H1
Trace J9
0.00
0.00
H2 V3
0.00
50.00
percent
V2
53
Organization of computation
GLOBE
(GA)
Solutions
Contam V5
Trace J9
0.00
0.00
0.00 V4 H1
50.00
percent
Trace J9
0.00
0.00 H2 V3
0.00
50.00
percent
V2
54
Combining criteria: method of "ideal point"
50
40
30
Number o
20
10
0
0 5 10 15 20 25 30 35
Number of changes in the network
D.P. Solomatine. Introduction to Optimization 109
45
40
35
unctions
30
25
Affected ju
20
15
10
5
0
0 5 10 15 20 25 30 35 40 45 50
Number ofto
D.P. Solomatine. Introduction movements
Optimization 110
55
Urban drainage networks optimization
Rehabilitation of drainage
networks using hydrodynamic
models and multi-objective
optimization
W. Barreto Cordero, R.K. Price, D.P. Solomatine, Z. Vojinovic.
Approaches to multi-objective multi-tier optimization in urban
drainage planning. Proc. 7th Intern. Conf. on Hydroinformatics,
Nice, Research Publishing, 2006.
W. Barreto Cordero, Z. Vojinovic, R. Price, D.P. Solomatine. A
Multi-objective Evolutionary Approach for Rehabilitation of Urban
Drainage Systems. J Water Res. Planning & Mang., 2009 (in
review).
111
56
Flood damage functions
Above ground
Under gground
L(m) D(m) Aging Cost
Cost Pipes f ( L * C )
Objective functions
Multi-objective approach:
Flooding damages:
ncells nlu
f1 fc
i 1 k 1
k
i Cmax
k
depi
Rehabilitation costs (pipe renewal)
n
f 2 Min (Li * Ci )
i 1
57
Multi-objective optimization:
Multi-
example of urban drainage system rehabilitation
Z1 = flood damage due to overflows dominated (inferior)
Criteria space solutions
non-inferior
i f i (the
(th
best) solutions,
uncomparable,
Pareto layer
Decision space
Z2 = costs of implementing
p g DN
a particular rehabilitation
option
ideal point
(does not exists)
D1
A decision (rehabilitation option)=
set of pipe diameters Di (i=1..N)
D2 Di pipe diameters (1..N)
D.P. Solomatine. Introduction to Optimization 115
58
Example: Drainage network
59
NSGAX optimization software:
search for optimal pipe replacement option
1
Flood damage du
Costs of implementing
rehabilitation option
60
Parallel computing in computationally
intensive optimization problems
121
Parallel Computing
Clusters of computers:
1. Parallel Virtual Machine (PVM):
Heterogeneous Cluster Networks
Context management (virtual machine)
Fault tolerant
2. Message-Passing Interface (MPI):
Standard for Super Computers (MPP)
Commercial and free implementations
Future of Parallel computations ?
61
Parallel Computing: organisation of the
computational process
Master
Slave 1
Slave 2 Master/Slave
Slave 3
Slave 4
Serial Time
Parallel Time
Communication Time
D.P. Solomatine. Introduction to Optimization 123
Init
Nsgas.exe
Random
Population
Ifacemouse.exe
Nsgam.exe
Mouse.exe
Population Parallel
Evaluation evaluation
62
Parallel Computing: computers used in the
cluster
Tested cluster configuration:
63
Parallel Computing: XVPM interface
64
Water networks optimization: conclusions and
recommendations
65
Optimization: conclusions
132
66