Optimization theory and advanced

computing methods

Outlines of lectures

Introduction
OPTIMIZATION THEORY AND
Optimization problem definition and classification

Linear programming problem – the solution methods for continuous
ADVANCED COMPUTING METHODS variables

Linear programming methods for integer variables
Faculty of Electronics
Embedded Robotics
Nonlinear programming methods :
M.Sc. level
Optimization methods without constraints

Optimization methods with constraints

Ph.D Ewa Szlachcic
Local optimization methods – , Nelder-Meade algorithm, non-gradient

Department of Automation, Mechatronics and Control Systems
Wrocław University of Science and Technology
room 219 C-3
email: ewa.szlachcic@pwr.edu.pl
algorithms, gradient algorithms

Global optimization methods– (Genetic algorithms – GA, Evolution
algorithms – EA, Ant Colony algorithms , Particle Swarm optimization
algorithms, Harmony Search algorithms – HSA).

Multi-criterial optimization problems
Optimization theory and adv. comp. methods Faculty of Electronics
Ph.D. eng. Ewa Szlachcic Embedded Robotics

Literature Optimization of a multi-dimenstional problem

x = [x1, x 2,..., xn ]
 Nocedal J. and Wright S., Numerical optimization, 2006. T
 Ruszczyński ., Nonlinear optimization, Princeton University Press, 2006. Desicion variables vector x:
 Fletcher R., Practical methods of optimization, J. Wiley Ed. 2000.
 Minoux M., Mathematical programming – Theory and algorithms, J. Wiley &Sons where: n – number of decision variables.
Ed. 2008.
Rardin R.L., Optimization in operations research. Prentice Hall Ed. 1998.
f (x ): R n 

Objective function f(x) : → R1
 Stephan Boyd, Convex optimization, Cambridge University Press, 2004
(bv_cvxbook.pdf)
 Thomas Weise, Global optimization algorithms – Theory and Applications
(book.pdf) and m constraint functions gi(x):
 Eckart Zitzler: Evolutionary Algorithms for Multiobjective Optimization: Methods
g i (x ) : R n 
and Applications, Zürich 1999.
 Jeffrey Horn, Nicholas Nafpliotis, David E. Goldberg: A Niche Pareto Genetic → R1 dla i = 1,..., m
Algorithm for Multiobjective Optimalization, IEEE 1994 , Volume 1, pp. 82-87.
 http://delta.cs.cinvestav.mx/~ccoello/EMOO/

Optimization theory and adv. comp. methods Faculty of Electronics Optimization theory and adv. comp. methods Faculty of Electronics
Ph.D. eng. Ewa Szlachcic Embedded Robotics Ph.D. eng. Ewa Szlachcic Embedded Robotics

∧
Let us find the decision variables vector x, which belongs to a set X of
admissible solutions in the form: Applications

{
X = x g i (x) ≤0, i =1,...,m }  Optimization of technological processes : model formulation,

 Identification of technological processes

such, that for ∀x ∈ X
 Optimal management of an enterprise – cost minimization
∧
f  x  ≤ f (x )
   The maximization of consumption, profit maximization or the
minimization of waste in a production process
∧
 
It’s equivalent to: min f (x) = f  x 
x∈ X  Control of technological processes

Optimization theory and adv. comp. methods Faculty of Electronics Optimization theory and adv. comp. methods Faculty of Electronics
Ph.D. eng. Ewa Szlachcic Embedded Robotics Ph.D. eng. Ewa Szlachcic Embedded Robotics
Optimization theory and advanced
computing methods

Classification of optimization problems

Applications cont.:
 Model of the process:
 Static problem
 Dynamic problem
 Optimal flows in networks ( water distribution network, gas
distribution network, computer network).  The solution space of decision variables:
 Real variables
 Discrete variables (integer variables, binary variables)
 Products distribution, optimal scheduling in technological
 Mixed variables
processes.
 Objective function:
 Cutting stocks problems – algorithms for a furniture industry  Scalar objective function
 Vector objective function ( multi-criteria problem)
 Vehicle Routing Problem VRP - VRPTW, CVRP, CVRPTW
 Data necessary for optimization algorithms:
 Objective function values
 Gradient of an objective function
 Hessian of an objective function

Optimization theory and adv. comp. methods Faculty of Electronics Optimization theory and adv. comp. methods Faculty of Electronics
Ph.D. eng. Ewa Szlachcic Embedded Robotics Ph.D. eng. Ewa Szlachcic Embedded Robotics

Linear programming problem LP Quadratic programming problem

max f (x ) = cT x
max f (x ) = 0.5xT Ax + bT x
x∈X
On the set of constraints: A1 x ≤ b1 where: :
A2 x ≥ b2 {
X = x : DT x ≤ e, x ≥ 0 }
x ≥0
Nonlinear multi-dimentional optimization problem with constraints
dim x=n, dim c=n
min f (x) = ( x1 − 2 ) + ( x2 − 1)
2 2
Matrices A1, A2 for m1 and m2 constraints
dim A1 =[m 1 x n], dim A2 =[m 2 x n] x∈X
With constraints:
 − x 2 + x 12 ≤ 0 
Vectors b1, b2 : dim b1=m1, dim b2=m2 X =  
 x1 + x 2 ≤ 2 

Optimization theory and adv. comp. methods Faculty of Electronics Optimization theory and adv. comp. methods Faculty of Electronics
Ph.D. eng. Ewa Szlachcic Embedded Robotics Ph.D. eng. Ewa Szlachcic Embedded Robotics