Liberti or Modelling SW Slides

Mathematical Programming:
Modelling and Software

Leo Liberti
LIX, Ecole Polytechnique, France
INF572/ISC610A p. 1
The course
Title: Mathematical Programming: Modelling and Software
Code: INF572 (DIX) / ISC610A (ISIC)

Teacher: Leo Liberti (liberti@lix.polytechnique.fr)
Assistants: Sonia Cafieri (cafieri@lix.polytechnique.fr),
Antonio Mucherino (mucherino@lix.polytechnique.fr),
Giacomo Nannicini (giacomon@lix.polytechnique.fr),
Fabien Tarissan (tarissan@lix.polytechnique.fr)
Timetable INF572: tue 23,30/9, 7,14/10, 4,18,25/11, 2,9/12
0830-1000 (SI34/72), 1015-1200 (SI34); exam 16/12
Timetable ISC610A: thu 6,10,13,27/11, 4,11,18/12; exam 8/1
URL INF572: http://www.lix.polytechnique.fr/~liberti/
teaching/inf572
URL ISC610A: http://www.lix.polytechnique.fr/~liberti/
teaching/isc610a
INF572/ISC610A p. 2
Contents
1. Introduction
2. AMPL basics
3. Formal definition of MP language
4. AMPL grammar
5. Solvers
6. Sudoku
7. Kissing Number Problem
8. Some useful MP theory
9. Reformulations
10. Symmetry
11. The final attack on the KNP
INF572/ISC610A p. 3
Introduction
INF572/ISC610A p. 4
Example: Set covering
There are 12 possible geographical positions A1 , . . . , A12 where some
discharge water filtering plants can be built. These plants are supposed to
service 5 cities C1 , . . . , C5 ; building a plant at site j (j {1, . . . , 12}) has
cost cj and filtering capacity (in kg/year) fj ; the total amount of discharge
water produced by all cities is 1.2 1011 kg/year. A plant built on site j can
serve city i if the corresponding (i, j)-th entry is marked by a * in the
table below.
A1 A2 A3 A4 A5 A6 A7 A8 A9 A10 A11 A12
C1 * * * * * *
C2 * * * * * *
C3 * * * * *
C4 * * * * * *
C5 * * * * * * *
cj 7 9 12 3 4 4 5 11 8 6 7 16
fj 15 39 26 31 34 24 51 19 18 36 41 34
What is the best placement for the plants?
INF572/ISC610A p. 5
Example: Sudoku
Given the Sudoku grid below, find a solution or prove that
no solution exists
2 1
4 1 9 2 8 6
5 8 2 7
5 1 3
9
7 8 6
3 2 6 4 9
1 9 4 5 2 8
8 6
INF572/ISC610A p. 6
Example: Kissing Number
How many unit balls with disjoint interior can be placed
adjacent to a central unit ball in Rd ?
In R2
2
-1
-2
2 1 0 -1
0 1 2-2
-2 -1
In R3
(D = 3: problem proposed by Newton in 1694, settled by

[Schtte and van der Waerden 1953] and [Leech 1956])
INF572/ISC610A p. 7
Mathematical programming
The above three problems seemingly have nothing in
common!
Yet, there is a formal language that can be used to
describe all three: mathematical programming (MP)
Moreover, the MP language comes with a rich supply of
solution algorithms so that problems can be solved right
away
Problem Reformulation
formulation and choice of so- Solution process
in MP lution algorithm
Human intelligence
AMPL Solver
(for now)
INF572/ISC610A p. 8
Modelling questions
Asking yourself the following questions should help you get started with
your MP model
The given problem is usually a particular instance of a
problem class; you should model the whole class, not just
the instance (replace given numbers by parameter
symbols)
What are the decisions to be taken? Are they logical,
integer or continuous?
What is the objective function? Is it to be minimized or
maximized?
What constraints are there in the problem? Beware
some constraints may be hidden in the problem text
If expressing objective and constraints is overly difficult, go
back and change your variable definitions
INF572/ISC610A p. 9
Set covering 1
Let us now consider the Set Covering problem
What is the problem class?
We replace the number 12 by the parameter symbol n,

the number 5 by m and the number 1.2 1011 by d
We already have symbols for costs (cj ) and capacities
(fj ), where j n and i m
We represent the asterisks by a 0-1 matrix A = (aij )
where aij = 1 if there is an asterisk at row i, column j of
the table, and 0 otherwise
INF572/ISC610A p. 10
Set covering 2
What are the decisions to be taken?
The crucial text in the problem is what is the best placement
for the plants?; i.e. we need to place each plant at some
location
1. geographical placement on a plane? (continuous
variables)
2. yes/no placement? (should the j -th plant be placed
here? logical 0-1 variables)
Because the text also says there are n possible geographical
positions. . . , it means that for each position we have to
decide whether or not to build a plant there
For each of geographical position, introduce a binary
variable (taking 0-1 values):
j n xj {0, 1}
Set covering 3
What is the objective function?
In this case we only have the indication best placement in

the text
Given our data, two possibilities exist: cost
(minimization) and filtering capacity (maximization)
However, because of the presence of the parameter d, it
wouldnt make sense to have more aggregated filtering
capacity than d kg/year
Hence, the objective function is the cost, which should
be minimized: X
min cj xj
jn
Set covering 4
What are the constraints?
The total filtering capacity must be at least d:

X
fj xj d
jn
Each city must be served by at least one plant:

X
i m aij xj 1
jn
Because there are no more constraints in the text, this

concludes the first modelling phase
Analysis
What category does this mathematical program belong
to?
Linear Programming (LP)
Mixed-Integer Linear Programming (MILP)
Nonlinear Programming (NLP)
Mixed-Integer Nonlinear Programming (MINLP)
Does it have any notable mathematical property?
If an NLP, are the functions/constraints convex?
If a MILP, is the constraint matrix Totally Unimodular
(TUM)?
Does it have any apparent symmetry?
Can it be reformulated to a form for which a fast solver is
available?
Set covering 5
The objective function and all constraints are linear forms
All the decision variables are binary
Hence the problem is a MILP
Good solutions can be obtained via heuristics (e.g. local branching,
feasibility pump, VNS, Tabu Search)
Exact solution via Branch-and-Bound (solver: CPLEX)
No need for reformulation: CPLEX is a fast enough solver
CPLEX 11.0.1 solution: x4 = x7 = x11 = 1, all the rest 0 (i.e. build
plants at positions 4,7,11)
Notice the paradigm model & solver solution
Since there are many solvers already available,

solving the problem reduces to modelling the problem
AMPL Basics
AMPL
AMPL means A Mathematical Programming
Language
AMPL is an implementation of the Mathematical
Programming language
Many solvers can work with AMPL
AMPL works as follows:
1. translates a user-defined model to a low-level
formulation (called flat form) that can be understood
by a solver
2. passes the flat form to the solver
3. reads a solution back from the solver and interprets
it within the higher-level model (called structured form)
Model, data, run
AMPL usually requires three files:
the model file (extension .mod) holding the MP formulation
the data file (extension .dat), which lists the values to be
assigned to each parameter symbol
the run file (extension .run), which contains the (imperative)
commands necessary to solve the problem
The model file is written in the MP language
The data file simply contains numerical data together with the
corresponding parameter symbols
The run file is written in an imperative C-like language (many notable
differences from C, however)
Sometimes, MP language and imperative language commands can
be mixed in the same file (usually the run file)
To run AMPL, type ampl < problem.run from the command line
An elementary run file
Consider the set covering problem, suppose we have
coded the model file (setcovering.mod) and the data
file (setcovering.dat), and that the CPLEX solver is
installed on the system
Then the following is a basic setcovering.run file
# basic run file for setcovering problem
model setcovering.mod; # specify model file
data setcovering.dat; # specify data file
option solver cplex; # specify solver
solve; # solve the problem
display cost; # display opt. cost
display x; # display opt. soln.
Set covering model file
# setcovering.mod
param m integer, >= 0;
param n integer, >= 0;
set M := 1..m;
set N := 1..n;
param c{N} >= 0;

param a{M,N} binary;
param f{N} >= 0;
param d >= 0;
var x{j in N} binary;
minimize cost: sum{j in N} c[j]*x[j];

subject to capacity: sum{j in N} f[j]*x[j] >= d;
subject to covering{i in M}: sum{j in N} a[i,j]*x[j] >= 1;
Set covering data file
param m := 5;
param n := 12;
param : c f :=
1 7 15
2 9 39
3 12 26
4 3 31
5 4 34
6 4 24
7 5 51
8 11 19
9 8 18
10 6 36
11 7 41
12 16 34 ;
param a: 1 2 3 4 5 6 7 8 9 10 11 12 :=
1 1 0 1 0 1 0 1 1 0 0 0 0
2 0 1 1 0 0 1 0 0 1 0 1 1
3 1 1 0 0 0 1 1 0 0 1 0 0
4 0 1 0 1 0 0 1 1 0 1 0 1
5 0 0 0 1 1 1 0 0 1 1 1 1 ;
param d := 120; INF572/ISC610A p. 21
AMPL+CPLEX solution
liberti@nox$ cat setcovering.run | ampl
ILOG CPLEX 11.010, options: e m b q use=2
CPLEX 11.0.1: optimal integer solution; objective 15
3 MIP simplex iterations
0 branch-and-bound nodes
cost = 15
x [*] :=
1 0
2 0
3 0
4 1
5 0
6 0
7 1
8 0
9 0
10 0
11 1
12 0
;
Formal definition of MP language
Formal languages
An alphabet is any well-defined set A
A word is any finite sequence of letters of A; the empty
word is denoted by
The set of words is denoted by A
A formal language is a subset L of A
The main problem linked to formal languages is: given a
word , does it belong to L? (i.e. determine whether a
given sentence is correctly formulated w.r.t. the
language)
Main tool: context-free grammars (CFGs)
Grammars
Informally A grammar is a set of rules used to break
down a sentence in its constituent parts
For example, the English language:
sentence : subj_obj predicate
subj_obj : article noun
noun
predicate : verb complements
: verb
complements : complement complements
: complement
complement : subj_obj
: time_complement
: space_complement
: ... INF572/ISC610A p. 25
Grammars
article : the
: a Potentially, each English

noun : aardvark sentence can be decom-
: Aalto posed in its logical com-
: ... ponents
: zulu
The sentence a zulu watches the aardvark would be anal-

ysed as:
sentence subj obj predicate
article noun verb complements
a zulu watches complement
a zulu watches subj obj
a zulu watches article noun
a zulu watches the aardvark
Grammars
Aim Given a sentence, decide whether it belongs to a
language via a grammar
For example, the sentence aardvark zulu a the watches is
made up of English words but is not English
The grammar allows us to decide this:
zuluwould be classified as a verb because it appears in
the predicate after subj obj but then no rule verb zulu
exists
the procedure stops before all words have been
analysed
the sentence does not belong to the language
If letters in strings represent stored states, a grammar is like a set of im-
perative statements whose application does not follow a fixed succession
Context free grammars
A CFG is a quadruplet (N, , R, S)
L is a finite set of terminal symbols
N is a finite set of nonterminal symbols s.t. N = and S N
R is a relation of N (N ) such that w (N ) ((S, w) R)
(, w) is written : w for all N , and w is a parsing expression (PEs);
these are defined recursively as follows:
1. terminal or nonterminal symbols and empty string (atomic PEs):
2. given any PE e, e, the following are PEs:
ee (sequence), e|e (or)
e (zero-or-more) equivalent to e | (for ),
e+ (one-or-more) equivalent to e e
e? (optional) equivalent to e|
&e (and), !e (not)
square brackets are used to indicate precedence when
necessary
Non-atomic parsing expressions
Assume = {a, . . . , z, _}
(Sequence) : cat matches = cattle but not tomcat
(Or) : cat|dog matches {catalysis, dogbert} but not
my cat is a dog
(Zero-or-more) : ca[t] matches
{cane, catalogue, cattle}, but not copper
(One-or-more) : ca[t]+ matches {catalogue, cattle}
but not cane or copper
(Optional) : ca[t]?e matches {case, category} but not
catters or cars
(And) : cat&ego matches = category but not cattle
(Not) : cat!ego matches = cattle but not category
Mathematical expressions
Assume = {[0-9], [a-zA-Z], ., (, ), *, /, +, -},
N = {float, variable, leaf, value, product, sum, expression}.
The following CFG (call it A) recognizes arithmetical expressions with
the operators +,-,*,/ acting on floating point numbers and variables of
the form name number (such as e.g. x1)
integer : [0-9]+
float : [0-9] .[0-9]+
number : integer|float
variable : [a-z]+ [0-9]
leaf : number|variable
value : leaf| ( expression )
product : value[[ * | / ]value]
sum : product[[ + | - ]product]
expression : sum
It is easy to extend A to also recognize the power operator and

transcendental functions such as exp, log, sin, cos
Quantified expressions
P Q
Assume = { , , , , , , =, , :} A,
N = {index, set, param, ivar, ileaf, prod1, sum1, iprod, isum, iexpr,
qatom, qlist, catom, clist, quantifier}
The following CFG (call it B) recognizes mathematical expressions
quantified over given sets
indexatom : [a-zA-Z] [ integer]? ! Y

iprod : quantifier ival|prod1
index : [index ,]? indexatom X
isum : quantifier ival|sum1
set : [a-zA-Z]+ [ index]?
iexpr : isum
param : [a-zA-Z]+ [ index]?
catom : iexpr [ | = | ] iexpr
ivar : variable[ index]?
clist : [clist [|]]? catom
ileaf : float|param|ivar
qatom : index set
ival : ileaf|[(]? iexpr [)]?
qlist : [qlist ,]? qatom
prod1 : ival[[ * | / ]ival]
quantifier : qlist [: clist]?
sum1 : prod1[[ + | - ]prod1]
An iexpr f q is written xq
Extension to power and transcendental functions equally easy
MP Language
= {Z, min, max} B, N =
{objective, constraint, integrality, objlist, conlist, intlist}
The following CFG (call it M) recognizes mathematical
programming formulations
objective : [quantifier]? [min | max] iexpr

objlist : objlist objective
constraint : [quantifier]? catom
conlist : conlist constraint
integrality : [quantifier]? ivar Z
intlist : intlist integrality
Example: set covering
M = {1, . . . , m}, N = {1, . . . , n}
P
min cj xj
j N : quantifier
jN

cj xj : iexpr
P
fj xj d

jN P
jN cj xj : iexpr
P
i M aij xj 1
jN
xj 0 : catom
j N xj 0

j N xj 1 : constraint

j N xj 1

j N xj Z j N xj Z : integrality
objlist only has one objective which has an empty

quantifier and a fairly simple iexpr consisting of a
quantified sum whose argument is a product
conlist has two constraints, the second of which is
quantified
intlist only has one (quantified) integrality element
MP language implementations
Software packages implementing (sub/supersets of the) MP language:
AMPL (our software of choice, mixture of MP and near-C language)
commercial, but student version limited to 300 vars/constrs is
available from www.ampl.com
quite a lot of solvers are hooked to AMPL
GNU MathProg (subset of AMPL)
free, but only the GLPK solver (for LPs and MILPs) can be used
it is a significant subset of AMPL but not complete
GAMS (can do everything AMPL can, but looks like COBOL ugh!)
commercial, limited demo available from www.gams.com
quite a lot of solvers are hooked to GAMS
Zimpl (free, C++ interface, linear modelling only)
LINDO, MPL, . . . (other commercial modelling/solution packages)
AMPL Grammar
AMPL MP Language
There are 5 main entities: sets, parameters, variables, objectives and
constraints
In AMPL, each entity has a name and can be quantified
set name [{quantifier}] attributes ;
param name [{quantifier}] attributes ;
var name [{quantifier}] attributes ;
minimize | maximize name [{quantifier}]: iexpr ;
subject to name [{quantifier}]: iexpr <= | = | >= iexpr ;
Attributes on sets and parameters is used to validate values read
from data files
Attributes on vars specify integrality (binary, integer) and limit
constraints (>= lower, <= upper)
Entities indices: square brackets (e.g. y[1], x[i,k])
The above is the basic syntax there are some advanced options
AMPL data specification
In general, syntax is in map-like form; a
param p{i in S} integer;
is a map S Z, and each pair (domain, codomain) must be
specified:
param p :=
1 4
2 -3
3 0;
The grammar is simple but tedious, best way is

learning by example or trial and error
AMPL imperative language
model model filename.mod ;
data data filename.dat ;
option option name literal string, ... ;
solve ;
display [{quantifier}]: iexpr ; / printf (syntax similar to C)
let [{quantifier}] ivar :=number;
if (clist) then { commands } [else {commands}]
for {quantifier} {commands} / break; / continue;
shell command line; / exit number; / quit;
cd dir name; / remove file name;
In all output commands, screen output can be redirected to a file by
appending > output filename.txt before the semicolon
These are basic commands, there are some advanced ones
Reformulation commands
fix [{quantifier}] ivar [:=number];
unfix [{quantifier}] ivar;
delete entity name;
purge entity name;
redeclare entity declaration;
drop/restore [{quantifier}] constr or obj name;
problem name[{quantifier}] [:entity name list] ;
This list is not exhaustive
Solvers
Solvers
In order of solver reliability / effectiveness:
1. LPs: use an LP solver (O(106 ) vars/constrs, fast, e.g. CPLEX, CLP,
GLPK)
2. MILPs: use a MILP solver (O(104 ) vars/constrs, can be slow,
e.g. CPLEX, Symphony, GLPK)
3. NLPs: use a local NLP solver to get a local optimum (O(104 )
vars/constrs, quite fast, e.g. SNOPT, MINOS, IPOPT)
4. NLPs/MINLPs: use a heuristic solver to get a good local optimum
(O(103 ), quite fast, e.g. B ON M IN, MINLP_BB)
5. NLPs: use a global NLP solver to get an (approximated) global
optimum (O(103 ) vars/constrs, can be slow, e.g. C OUENNE, BARON)
6. MINLPs: use a global MINLP solver to get an (approximated) global
optimum (O(103 ) vars/constrs, can be slow, e.g. C OUENNE, BARON)
Not all these solvers are available via AMPL
Solution algorithms (linear)
LPs: (convex)
1. simplex algorithm (non-polynomial complexity but
very fast in practice, reliable)
2. interior point algorithms (polynomial complexity,
quite fast, fairly reliable)
MILPs: (nonconvex because of integrality)
1. Local (heuristics): Local Branching, Feasibility Pump
[Fischetti&Lodi 05], VNS [Hansen et al. 06] (quite
fast, reliable)
2. Global: Branch-and-Bound (exact algorithm,
non-polynomial complexity but often quite fast,
heuristic if early termination, reliable)
Solution algorithms (nonlinear)
NLPs: (may be convex or nonconvex)
1. Local: Sequential Linear Programming (SLP), Sequential
Quadratic Programming (SQP), interior point methods
(linear/polynomial convergence, often quite fast, unreliable)
2. Global: spatial Branch-and-Bound [Smith&Pantelides 99]
(-approximate, nonpolynomial complexity, often quite slow,
heuristic if early termination, unreliable)
MINLPs: (nonconvex because of integrality and terms)
1. Local (heuristics): Branching explorations [Fletcher&Leyffer 99],
Outer approximation [Grossmann 86], Feasibility pump [Bonami
et al. 06] (nonpolynomial complexity, often quite fast, unreliable)
2. Global: spatial Branch-and-Bound [Sahinidis&Tawarmalani 05]
(-approximate, nonpolynomial complexity, often quite slow,
heuristic if early termination, unreliable)
LP example: .mod
# lp.mod
param n integer, default 3;
param m integer, default 4;
set N := 1..n;
set M := 1..m;
param a{M,N};
param b{M};
param c{N};
var x{N} >= 0;

minimize objective: sum{j in N} c[j]*x[j];
subject to constraints{i in M} :
sum{j in N} a[i,j]*x[j] <= b[i];
LP example: .dat
# lp.dat
param n := 3; param m := 4;
param c :=
1 1
2 -3
3 -2.2 ;
param b :=
1 -1
2 1.1
3 2.4
4 0.8 ;
param a : 1 2 3 :=
1 0.1 0 -3.1
2 2.7 -5.2 1.3
3 1 0 -1
4 1 1 0 ;
LP example: .run
# lp.run
model lp.mod;
data lp.dat;
option solver cplex;
solve;
display x;
LP example: output
CPLEX 11.0.1: optimal solution; objective -11.30153
0 dual simplex iterations (0 in phase I)
x [*] :=
1 0
2 0.8
3 4.04615
;
MILP example: .mod
# milp.mod
set N := 1..n;
set M := 1..m;
param a{M,N};
param b{M};
param c{N};
var x{N} >= 0, <= 3, integer;

var y >= 0;
minimize objective: sum{j in N} c[j]*x[j];
subject to constraints{i in M} :
sum{j in N} a[i,j]*x[j] - y <= b[i];
MILP example: .run
# milp.run
model milp.mod;
data lp.dat;
solve;
display x;
display y;
MILP example: output
CPLEX 11.0.1: optimal integer solution; objective -
0 MIP simplex iterations
0 branch-and-bound nodes
x [*] :=
1 0
2 3
3 3
;
y = 2.2
NLP example: .mod
# nlp.mod
set N := 1..n;
set M := 1..m;
param a{M,N};
param b{M};
param c{N};
var x{N} >= 0.1, <= 4;
minimize objective:
c[1]*x[1]*x[2] + c[2]*x[3]2 + c[3]*x[1]*x[2]/x[3];
subject to constraints{i in M diff {4}} :
sum{j in N} a[i,j]*x[j] <= b[i]/x[i];
subject to constraint4 : prod{j in N} x[j] <= b[4];
NLP example: .run
# nlp.run
model nlp.mod;
data lp.dat;
## only enable one of the following methods
## 1: local solution
option solver minos;
# starting point
let x[1] := 0.1;
let x[2] := 0.2; # try 0.1, 0.4
let x[3] := 0.2;
## 2: global solution (heuristic)
#option solver bonmin;
## 3: global solution (guaranteed)
#option solver boncouenne;
solve;
display x;
NLP example: output
MINOS 5.51: optimal solution found.
47 iterations, objective -38.03000929
Nonlin evals: obj = 131, grad = 130, constrs = 131,
x [*] :=
1 2.84106
2 1.37232
3 0.205189
;
MINLP example: .mod
# minlp.mod
set N := 1..n;
set M := 1..m;
param a{M,N};
param b{M};
param c{N};
param epsilon := 0.1;
var x{N} >= 0, <= 4, integer;
minimize objective:
c[1]*x[1]*x[2] + c[2]*x[3]2 + c[3]*x[1]*x[2]/x[3] +
x[1]*x[3]3;
subject to constraints{i in M diff {4}} :
sum{j in N} a[i,j]*x[j] <= b[i]/(x[i] + epsilon);
subject to constraint4 : prod{j in N} x[j] <= b[4];
MINLP example: .run
# minlp.run
model minlp.mod;
data lp.dat;
## only enable one of the following methods:

## 1: global solution (heuristic)
#option solver bonmin;
## 2: global solution (guaranteed)
option solver boncouenne;
solve;
display x;
MINLP example: output
bonmin: Optimal
x [*] :=
1 0
2 4
3 4
;
Sudoku
Sudoku: problem class
The class of all sudoku grids

Replace {1, . . . , 9} with a set K
Will need a set H = {1, 2, 3} to define 3 3 sub-grids
An instance is a partial assignment of integers to
cases in the sudoku grid
We model an empty sudoku grid, and then fix certain
variables at the appropriate values
Modelling the Sudoku
Q: What are the decisions to be taken?
A: Whether to place an integer in K = {1, . . . , 9} in the
case at coordinates (i, j) on the square grid (i, j K )
We might try integer variables yij K
Q: What is the objective function?
A: There is no natural objective; we might wish to
employ one if needed
Q: What are the constraints?
A: For example, the first row should contain all numbers
in K ; hence, we should express a constraint such as:
if y11 = 1 then y1 6= 1 for all 1;
if y11 = 2 then y1 6= 2 for all 2;
. . . (for all values, column and row indices)
Sudoku constraints 1
In other words,
i, j, k K, 6= j (yij = k yi 6= k)
Put it another way: a constraint that says all values should
be different
In constraint programming (a discipline related to MP) there
is a constraint
i K AllDiff(yij | j K)
that asserts that all variables in its argument take

different values: we can attempt to implement it in MP
A set of distinct values has the pairwise distinctness property:
i, p, q K yip 6= yiq , which can also be written as:
i, p < q K |yip yiq | 1

Sudoku constraints 2
We also need the same constraints in each column:
j, p < q K |ypj yqj | 1
. . . and in some appropriate 3 3 sub-grids:

1. let H = {1, . . . , 3} and = |K|/|H|; for all h H
define Rh = {i K | i > (h 1) i h} and
Ch = {j K | j > (h 1) j h}
2. show that for all (h, l) H H , the set Rh Cl
contains the case coordinates of the (h, l)-th 3 3
sudoku sub-grid
Thus, the following constraints must hold:
h, l H, i < p Rh , j < q Cl |yij ypq | 1
The Sudoku MINLP
The whole model is as follows:

min 0

i, p < q K |yip yiq | 1

j, p < q K |ypj yqj | 1

h, l H, i < p Rh , j < q Cl |yij ypq | 1

i K, j K yij 1

i K, j K 9

yij

i K, j K yij Z

This is a nondifferentiable MINLP

MINLP solvers (B ON M IN, MINLP_BB, C OUENNE) cant
solve it
Absolute value reformulation
This MINLP, however, can be linearized:
|a b| >= 1 a b >= 1 b a >= 1
For each i, j, p, q K we introduce a binary variable
pq
wij = 1 if yij ypq 1 and 0 if ypq yij 1
For all i, j, p, q K we add constraints
pq
1. yij ypq 1 M (1 wij )
pq
2. ypq yij 1 M wij
where M = |K| + 1
pq
This means: if wij = 1 then constraint 1 is active and 2
is always inactive (as ypq yij is always greater than
pq
|K|); if wij = 0 then 2 is active and 1 inactive
Transforms problematic absolute value terms into
added binary variables and linear constraints
The reformulated model
The reformulated model is a MILP:

min 0

iq
i, p < q K yip yiq 1 M (1 wip )

iq

i, p < q K yiq yip 1 M wip

qj
j, p < q K ypj yqj 1 M (1 wpj )

qj
j, p < q K yqj ypj 1 M wpj

pq
h, l H, i < p Rh , j < q Cl yij ypq 1 M (1 wij )

pq
h, l H, i < p Rh , j < q Cl ypq yij 1 M wij

i K, j K yij 1

i K, j K yij 9

i K, j K yij Z
It can be solved by CPLEX; however, it has O(|K|4 ) binary variables

on a |K|2 cases grid with |K| values per case (O(|K|3 ) in total)
often an effect of bad modelling
A better model
In such cases, we have to go back to variable definitions
One other possibility is to define binary variables
i, j, k K (xijk = 1) if the (i, j)-th case has value k , and 0
otherwise
Each case must contain exactly one value
X
i, j K xijk = 1
kK
For each row and value, there is precisely one column
that contains that value, and likewise for cols
X X
i, k K xijk = 1 j, k K xijk = 1
jK iK
Similarly for each Rh Ch sub-square
X
h, l H, k K xijk = 1
iRh ,jCl
The Sudoku MILP
The whole model is as follows:
min 0

P
i K, j K xijk = 1

kK

P
i K, k K xijk = 1

jK
P
j K, k K xijk = 1

PiK

h H, l H, k K xijk = 1

iRh ,jCl

i K, j K, k K {0, 1}

xijk
This is a MILP with O(|K|3 ) variables
P
Notice that there is a relation i, j K yij = kxijk between the
kK
MINLP and the MILP
The MILP variables have been derived by the MINLP ones by disaggregation
Sudoku model file
param Kcard integer, >= 1, default 9;
param Hcard integer, >= 1, default 3;
set K := 1..Kcard;
set H := 1..Hcard;
set R{H};
set C{H};
param alpha := card(K) / card(H);
param Instance {K,K} integer, >= 0, default 0;
let {h in H} R[h] := {i in K : i > (h-1) * alpha and i <= h * alpha};
let {h in H} C[h] := {j in K : j > (h-1) * alpha and j <= h * alpha};
var x{K,K,K} binary;
minimize nothing: 0;
subject to assignment {i in K, j in K} : sum{k in K} x[i,j,k] = 1;

subject to rows {i in K, k in K} : sum{j in K} x[i,j,k] = 1;
subject to columns {j in K, k in K} : sum{i in K} x[i,j,k] = 1;
subject to squares {h in H, l in H, k in K} :
sum{i in R[h], j in C[l]} x[i,j,k] = 1;
Sudoku data file
param Instance :=
1 1 2 1 9 1 2 2 4 2 3 1 2 4 9
2 6 2 2 7 8 2 8 6 3 1 5 3 2 8
3 8 2 3 9 7 4 4 5 4 5 1 4 6 3
5 5 9 6 4 7 6 5 8 6 6 6 7 1 3
7 2 2 7 3 6 7 8 4 7 9 9 8 2 1
8 3 9 8 4 4 8 6 5 8 7 2 8 8 8
9 1 8 9 9 6 ;
Sudoku run file
# sudoku
# replace "/dev/null" with "nul" if using Windows
option randseed 0;
option presolve 0;
option solver_msg 0;
model sudoku.mod;
data sudoku-feas.dat;
let {i in K, j in K : Instance[i,j] > 0} x[i,j,Instance[i,j]] := 1;
fix {i in K, j in K : Instance[i,j] > 0} x[i,j,Instance[i,j]];
display Instance;
solve > /dev/null;
param Solution {K, K};
if (solve_result = "infeasible") then {
printf "instance is infeasible\n";
} else {
let {i in K, j in K} Solution[i,j] := sum{k in K} k * x[i,j,k];
display Solution;
}
Sudoku AMPL output
liberti@nox$ cat sudoku.run | ampl
Instance [*,*]
: 1 2 3 4 5 6 7 8 9 :=
1 2 0 0 0 0 0 0 0 1
2 0 4 1 9 0 2 8 6 0
3 5 8 0 0 0 0 0 2 7
4 0 0 0 5 1 3 0 0 0
5 0 0 0 0 9 0 0 0 0
6 0 0 0 7 8 6 0 0 0
7 3 2 6 0 0 0 0 4 9
8 0 1 9 4 0 5 2 8 0
9 8 0 0 0 0 0 0 0 6
;
instance is infeasible
Sudoku data file 2
But with a different data file. . .
param Instance :=
1 1 2 1 9 1 2 2 4 2 3 1 2 4 9
2 6 2 2 7 8 2 8 6 3 1 5 3 2 8
3 8 2 3 9 7 4 4 5 4 5 1 4 6 3
5 5 9 6 4 7 6 5 8 6 6 6 7 1 3
7 2 2 7 8 4 7 9 9 8 2 1
8 3 9 8 4 4 8 6 5 8 7 2 8 8 8
9 1 8 9 9 6 ;
Sudoku data file 2 grid
. . . corresponding to the grid below. . .
2 1
4 1 9 2 8 6
5 8 2 7
5 1 3
9
7 8 6
3 2 4 9
1 9 4 5 2 8
8 6
Sudoku AMPL output 2
. . . we find a solution!
liberti@nox$ cat sudoku.run | ampl
Solution [*,*]
: 1 2 3 4 5 6 7 8 9 :=
1 2 9 6 8 5 7 4 3 1
2 7 4 1 9 3 2 8 6 5
3 5 8 3 6 4 1 9 2 7
4 4 7 8 5 1 3 6 9 2
5 1 6 5 2 9 4 3 7 8
6 9 3 2 7 8 6 1 5 4
7 3 2 7 1 6 8 5 4 9
8 6 1 9 4 7 5 2 8 3
9 8 5 4 3 2 9 7 1 6
;
Kissing Number Problem
KNP: problem class
There is no number in the problem definition:

How many unit balls with disjoint interior can be placed
adjacent to a central unit ball in Rd ?
Hence the KNP is already defined as a problem class
Instances are given by assigning a positive integer to
the only parameter D
Modelling the KNP
Q: What are the decisions to be taken?
A: How many spheres to place, and where to place them
For each sphere, two types of variables
1. a logical one: yi = 1 if sphere i is present, and 0 otherwise
2. a D-vector of continuous ones: xi = (xi1 , . . . , xiD ), position of
i-th sphere center
Q: What is the objective function?
A: Maximize the number of spheres
Q: What are the constraints?
A: Two types of constraints
1. the i-th center must be at distance 2 from the central sphere if the
i-th sphere is placed (center constraints)
2. for all distinct (and placed) spheres i, j, for their interior to be
disjoint their centers must be at distance 2 (distance constraints)
Assumptions
1. Logical variables y
Since the objective function counts the number of placed
P
spheres, it must be something like i yi
What set N does the index i range over?
Denote k (d) the optimal solution to the KNP in Rd
Since k (d) is unknown a priori, we cannot know N a priori;
however, without N , we cannot express the objective function
Assumewe know an upper bound k to k (d); then we can define
N = {1, . . . , k} (and D = {1, . . . , d})
2. Continuous variables x
Since any sphere placement is invariant by translation, we assume
that the central sphere is placed at the origin
Thus, each continuous variable xik (i N, k D) cannot attain
values outside [2, 2] (why?)
Limit continuous variables: 2 xik 2
Problem restatement
The above assumptions lead to a problem restatement
Given a positive integer k , what is the maximum

number (smaller than k ) of unit spheres with dis-
joint interior that can be placed adjacent to a unit
sphere centered at the origin of Rd ?
Each time assumptions are made for the sake of modelling, one
must always keep track of the corresponding changes to the
problem definition
The Objective function can now be written as:
X
max yi
iN
Constraints
Center constraints:
i N ||xi || = 2yi
(if sphere i is placed then yi = 1 and the constraint

requires ||xi || = 2, otherwise ||xi || = 0, which implies
xi = (0, . . . , 0))
Distance constraints:
i N, j N : i 6= j ||xi xj || 2yi yj
(if spheres i, j are both are placed then yi yj = 1 and the

constraint requires ||xi xj || 2, otherwise
||xi xj || 0 which is always by the definition of norm)
KNP model
P
max yi

iN
rP

i N x2ik

= 2yi

kD

rP

i N, j N : i 6= j (xik xjk )2 2yi yj

kD

i N yi 0

i N yi 1

i N, k D xik 2

i N, k D xik 2

i N yi Z

For brevity, we shall write yi {0, 1} and xik [2, 2]
Reformulation 1
Solution times for NLP/MINLP solvers often also
depends on the number of nonlinear terms
We square both sides of the nonlinear constraints, and
notice that since yi are binary variables, yi2 = yi for all
i N ; we get:
X
i N x2ik = 4yi
kD
X
i 6= j N (xik xjk )2 4yi yj
kD
which has fewer nonlinear terms than the original

problem
Reformulation 2
Distance constraints are called reverse convex (because if
we replace with the constraints become convex);
these constraints often cause solution times to lengthen
considerably
Notice that distance constraints are repeated when i, j
are swapped
Change the quantifier to i N, j N : i < j reduces the
number of reverse convex constraints in the problem;
get:
X
i N x2ik = 4yi
kD
X
i < j N (xik xjk )2 4yi yj
kD
KNP model revisited
P
max yi

iN

x2ik = 4yi
P
i N

kD
(xik xjk )2 4yi yj
P
i N, j N : i < j
kD

i N, k D xik [2, 2]

i N yi {0, 1}
This formulation is a (nonconvex) MINLP
KNP model file
# knp.mod
param d default 2;
param kbar default 7;
set D := 1..d;
set N := 1..kbar;
var y{i in N} binary;

var x{i in N, k in D} >= -2, <= 2;
maximize kstar : sum{i in N} y[i];
subject to center{i in N} : sum{k in D} x[i,k]2 = 4*y[i];

subject to distance{i in N, j in N : i < j} :
sum{k in D} (x[i,k] - x[j,k])2 >= 4*y[i]*y[j];
KNP data file
Since the only data are the parameters d and k (two

scalars), for simplicity we do not use a data file at all, and
assign values in the model file instead
KNP run file
# knp.run
model knp.mod;
option solver boncouenne;
solve;
display x,y;
display kstar;
KNP solution (?)
We tackle the easiest possible KNP instance (d = 2),
and give it an upper bound k = 7
It is easy to see that k (2) = 6 (place 6 circles adjacent
to another circle in an exagonal lattice)
Yet, after several minutes of CPU time C OUENNE has not
made any progress from the trivial feasible solution
y = 0, x = 0
Likewise, heuristic solvers such as B ON M IN and
MINLP BB only find the trivial zero solution and exit
What do we do next?
In order to solve the KNP and deal with other difficult

MINLPs, we need more advanced techniques
Some useful MP theory
Canonical MP formulation

minx f (x)

s.t. l g(x) u
[P ] (1)
xL x xU

i Z {1, . . . , n} xi Z

where x, xL , xU Rn ; l, u Rm ; f : Rn R; g : Rn Rm
F(P ) = feasible region of P , L(P ) = set of local optima,
G(P ) = set of global optima
Nonconvexity G(P ) ( L(P )
1
min 4 x + sin(x)
3
0 6 x
x[3,6]
Open sets
In general, MP cannot directly model problems involving
sets which are not closed in the usual topology (such as
e.g. open intervals)
The reason is that the minimum/maximum of a
non-closed set might not exist
E.g. what is min x? Since (0, 1) has no minimum (for
x(0,1)
each (0, 1), 2 < and is in (0, 1)), the question has
no answer
This is why the MP language does not allow writing constraints that
involve the <, > and 6= relations
Sometimes, problems involving open sets can be
reformulated exactly to problems involving closed sets
Best fit hyperplane 1
Consider the following problem:
Given m points p1 , . . . , pm Rn , find the hyperplane
w1 x1 + + wn xn = w0 minimizing the piecewise linear form
P P
f (p, w) = | wj pij w0 |
iP jN
Mathematical programming formulation:

1. Sets: P = {1, . . . , m}, N = {1, . . . , n}
2. Parameters: i P pi Rn
3. Decision variables: j N wj R, w0 R
4. Objective: minw f (p, w)
5. Constraints: none
Trouble: w = 0 is the obvious, trivial solution of no interest
We need to enforce a constraint (w1 , . . . , wn , w0 ) 6= (0, . . . , 0)
Bad news: Rn+1 r {(0, . . . , 0)} is not a closed set
We can implicitly impose such a constraint by transforming the
(p,w)
objective function to minw f||w|| (for some norm || ||)
This implies that w is nonzero but the feasible region is Rn+1 , which
is both open and closed
Obtain fractional objective difficult to solve
Suppose w = (w , w0 ) Rn+1 is an optimal solution to the above

problem
Then for all d > 0, f (dw , p) = df (w , p)
Hence, it suffices to determine the optimal direction of w , because the
actual vector length simply scales the objective function value
Can impose constraint ||w|| = 1 and recover original objective
Solve reformulated problem:
min{f (w, p) | ||w|| = 1}

The constraint ||w|| = 1 is a constraint schema: we must
specify the norm
Some norms can be reformulated to linear constraints,
some cannot
max-norm (l ) 2-sphere (square), Euclidean norm (l2 )
2-sphere (circle), abs-norm (l1 ) 2-sphere (rhombus)
max- and abs-norms are piecewise linear, they can be

linearized exactly by using binary variables (see later)
Convexity
A function f (x) is convex if the following holds:
x0 , x1 dom(f ) [0, 1]
f (x0 + (1 )x1 ) f (x0 ) + (1 )f (x1 )
f
f (x1 )
f (x0 ) + (1 )f (x1 )
f (x0 )
f (x0 + (1 )x1 )
x0 x1 x
x0 + (1 )x1
A set C Rn is convex if x0 , x1 C, [0, 1] (x0 + (1 )x1 C)
If g : Rm Rn are convex, then {x | g(x) 0} is a convex set
If f, g are convex, then every local optimum of min f (x) is global
g(x)0
A local NLP solver suffices to solve the NLP to optimality

Convexity in practice
Recognizing whether an arbitrary function is convex is
an undecidable problem
For some functions, however, this is possible
Certain functions are known to be convex (such as all
affine functions, cx2n for n N and c 0, exp(x),
log(x))
Norms are convex functions
The sum of two convex functions is convex
Application of the above rules repeatedly sometimes
works (for more information, see Disciplined Convex
Programming [Grant et al. 2006])
Warning: problems involving integer variables are in general not
convex; however, if the objective function and constraints are convex
forms, we talk of convex MINLPs
Recognizing convexity 1
Consider the following mathematical program
min 8x2 17xy + 10y 2
x,y[0,10]
xy 1
1
xy
x
Objective function and constraints contain nonconvex
term xy
Constraint 2 is x1 ; the function x1 is convex (only in
x 0) but constraint sense makes it reverse convex
Is this problem convex or not?
can be written as (x, y)T Q(x, y)
The objective function !
8 8
where Q =
9 10

The eigenvalues of Q are 9 73 (both positive), hence
the Hessian of the objective is positive definite, hence
the objective function is convex
The affine constraint x y 1 is convex by definition
1
xy x can be reformulated as follows:
1. Take logarithms of both sides: log xy log x1
2. Implies log x + log y log x 2 log x log y 0
3. log is a convex function, sum of convex functions is
convex, convex affine is a convex constraint
Thus, the constraint is convex
model;
var x <= 10, >= -10;
var y <= 10, >= -10;
minimize f: 8*x2 -17*x*y + 10*y2;
subject to c1: x-y >= 1;
subject to c2: x2*y >= 1;
printf "solving with sBB (couenne)\n";
option solver boncouenne; solve > /dev/null;
display x,y;
printf "solving with local NLP solver (ipopt)\n";
option solver ipopt; let x := 0; let y := 0;
solve > /dev/null; display x,y;
Get same solution (1.5, 0.5) from C OUENNE and IPOPT
Total Unimodularity
A matrix A is Totally Unimodular (TUM) if all invertible
square submatrices of A have determinant 1
Thm.
If A is TUM, then all vertices of the polyhedron
{x 0 | Ax b}
have integral components

Consequence: if the constraint matrix of a given MILP is
TUM, then it suffices to solve the relaxed LP to get a
solution for the original MILP
An LP solver suffices to solve the MILP to optimality
INF572/ISC610A p. 100
TUM in practice 1
If A is TUM, AT and (A|I) are TUM
An m n matrix A is TUM if:
TUM Sufficient conditions.
1. for all i m, j n we have aij {0, 1, 1};
2. each column of A contains at most 2 nonzero
coefficients;
3. there is a partition R
P1 , R2 of theP
set of rows such that
for each column j , iR1 aij iR2 aij = 0.
Example: take R1 = {1, 3, 4}, R2 = {2}

1 0 1 1 0 0
0 1 0 1 1 1

1 1 0 0 0 1

0 0 1 0 1 0
INF572/ISC610A p. 101
TUM in practice 2
Consider digraph G = (V, A) with nonnegative variables
xij R+ defined on each arc
P P
Flow constraints i V xij xji = bi yield a
(i,j)A (j,i)A
TUM matrix (partition: R1 = all rows, R2 = prove it)

Maximum flow problems can be solved to integrality by
simply solving the continuous relaxation with an LP
solver
The constraints of the set covering problem do not form a TUM. To
prove this, you just need to find a counterexample
INF572/ISC610A p. 102
Maximum flow problem
Given a network on a directed graph G = (V, A) with a
source node s, a destination node t, and integer capacities
uij on each arc (i, j). We have to determine the maximum
integral amount of material flow that can circulate on the
network from s to t. The variables xij Z, defined for each
arc (i, j) in the graph, denote the number of flow units.
X xij
maxx xsi

4

(s,i)A

i 6= s X X

i V, xij = xji i
i 6= t
(i,j)A (j,i)A

1
(i, j) A 0 xij uij 3

xji

(i, j) A xij Z

INF572/ISC610A p. 103
Max Flow Example 1
1
5
1 2 3
2
1 2
4 1
6 3
7
4 5 6 7
7 5
arc capacities as shown in italics: find the maximum flow

between node s = 1 and t = 7
INF572/ISC610A p. 104
Max Flow: MILP formulation
Sets: V = {1, . . . , n}, A V V
Parameters: s, t V , u : A R+
Variables: x : A Z+
P
Objective: max xsi
(s,i)A
P P
Constraints: i V r {s, t} xij = xji
(i,j)A (j,i)A
INF572/ISC610A p. 105
Max Flow: .mod file
# maxflow.mod
param n integer, > 0, default 7;
param s integer, > 0, default 1;
param t integer, > 0, default n;
set V := 1..n;
set A within {V,V};
param u{A} >= 0;
var x{(i,j) in A} >= 0, <= u[i,j], integer;
maximize flow : sum{(s,i) in A} x[s,i];
subject to flowcons{i in V diff {s,t}} :

sum{(i,j) in A} x[i,j] = sum{(j,i) in A} x[j,i];
INF572/ISC610A p. 106
Max Flow: .dat file
# maxflow.dat
param : A : u :=
1 2 5
1 4 4
1 5 1
2 4 2
3 2 1
3 4 1
3 7 2
4 5 7
5 3 6
5 6 5
6 2 3
6 7 7 ;
INF572/ISC610A p. 107
Max Flow: .run file
# maxflow.run
model maxflow.mod;
data maxflow.dat;
solve;
for {(i,j) in A : x[i,j] > 0} {
printf "x[%d,%d] = %g\n", i,j,x[i,j];
}
display flow;
INF572/ISC610A p. 108
Max Flow: MILP solution
1
5
1 2 3
2
1 2
4 1
6 3
7
4 5 6 7
7 5
1unit of flow 5 units of flow

2 units of flow 6 units of flow
4 units of flow maximum flow = 7
x[1,2] = 2 x[4,5] = 6
x[1,4] = 4 x[5,3] = 2
x[1,5] = 1 x[5,6] = 5
x[2,4] = 2 x[6,7] = 5
x[3,7] = 2 flow = 7
INF572/ISC610A p. 109
Max Flow: LP solution
Relax integrality constraints (take away integer keyword)
1
5
1 2 3
2
1 2
4 1
6 3
7
4 5 6 7
7 5
1unit of flow 5 units of flow

2 units of flow 6 units of flow
4 units of flow maximum flow = 7
Get the same solution
INF572/ISC610A p. 110
Reformulations
INF572/ISC610A p. 111
Reformulations
If problems P, Q are related by a computable function f
through the relation f (P, Q) = 0, Q is an auxiliary problem
with respect to P .
Opt-reformulations: preserve all optimality properties
Narrowings: preserve some optimality properties
Relaxations:provide bounds to the optimal objective
function value
formulation Q depending on a
Approximations:
parameter k such that lim Q(k) is an
k
opt-reformulation, narrowing or relaxation
INF572/ISC610A p. 112
Opt-reformulations
P
Q
G G
|G L
F L
|L
F

Main idea:if we find an optimum of Q, we can map it back to the same
type of optimum of P , and for all optima of P , there is a correspond-
ing optimum in Q. Also known as exact reformulation
INF572/ISC610A p. 113
Narrowings
P
Q
G
G
|G
F
F

Main idea: if we find a global optimum of Q, we can map
it back to a global optimum of P . There may be optima
of P without a corresponding optimum in Q.
INF572/ISC610A p. 114
Relaxations
A problem Q is a relaxation of P if the globally optimal

value of the objective function min fQ of Q is a lower
bound to that of P .
INF572/ISC610A p. 115
Approximations
Q is an approximation of P if there exist: (a) an auxiliary problem
Q of P ; (b) a sequence {Qk } of problems; (c) an integer > 0;
such that:
1. Q = Q
2. objective f in Q there is a sequence of objectives fk of Qk
converging uniformly to f ;
3. constraint li gi (x) ui of Q there is a sequence of constraints
lik gik (x) uki of Qk such that gik converges uniformly to gi , lik
converges to li and uki to ui
There can be approximations to opt-reformulations, narrowings,
relaxations.
Q1 , Q2 , Q3 , . . . Q ,. . . Q hauxiliary problem ofi P

approximation of P
INF572/ISC610A p. 116
Fundamental results
Opt-reformulation, narrowing, relaxation, approximation
are all transitive relations
An approximation of any type of reformulation is an approximation
A reformulation consisting of opt-reformulations,
narrowings, relaxations is a relaxation
A reformulation consisting of opt-reformulations and narrowings is a
narrowing
A reformulation consisting of opt-reformulations is an
opt-reformulation
opt-reformulations narrowings relaxations approximations
INF572/ISC610A p. 117
Reformulations in practice
Reformulations are used to transform problems into
equivalent (or related) formulations which are somehow
better
Basic reformulation operations :
1. adding / deleting variables / constraints
2. replacing a term with another term (e.g. a product xy
with a new variable w)
INF572/ISC610A p. 118
Product of binary variables
Consider binary variables x, y and a cost c to be added
to the objective function only of xy = 1
Add term cxy to objective
Problem becomes mixed-integer (some variables are
binary) and nonlinear
Reformulate xy to MILP form (P ROD B IN reform.):
replace xy by z
add z y , z x
z 0, z x + y 1
x, y {0, 1}
z = xy
INF572/ISC610A p. 119
Application to the KNP
In the RHS of the KNPs distance constraints we have 4yi yj , where
yi , yj are binary variables
We apply P ROD B IN (call the added variable wij ):
P 9
min yi >
>
>
iN >
>
>
x2ik
P
i N = 4yi
>
>
>
>
kD
>
>
>
>
(xik xjk )2
P
i N, j N : i < j 4wij
>
>
>
>
kD >
>
>
>
i N, j N : i < j wij yi =
i N, j N : i < j wij yj >

>
>
>
>
i N, j N : i < j wij yi + yj 1 >
>
>
>
>
>
i N, j N : i < j wij [0, 1] >
>
>
>
>
i N, k D xik [2, 2]
>
>
>
>
>
>
i N yi {0, 1} ;
Still a MINLP, but fewer nonlinear terms

Still numerically difficult (2h CPU time to find k (2) 5)
INF572/ISC610A p. 120
Product of bin. and cont. vars.
P ROD B IN C ONT reformulation
Consider a binary variable x and a continuous variable
y [y L , y U ], and assume product xy is in the problem
Replace xy by an added variable w
Add constraints:
w yU x
w yLx
w y + y L (1 x)
w y y U (1 x)
Exercise 1 : show that P ROD B IN C ONT is indeed a reformulation
Exercise 2 : show that if y {0, 1} then P ROD B IN C ONT is equivalent to

P ROD B IN
INF572/ISC610A p. 121
Prod. cont. vars.: approximation
B ILIN A PPROX approximation
Consider x [xL , xU ], y [y L , y U ] and product xy
Suppose xU xL y U y L , consider an integer d > 0
Replace [xL , xU ] by a finite set
xU xL
D = {xL + (i 1) | 1 i d}, where = d1
INF572/ISC610A p. 122
B ILIN A PPROX
Replace the product xy by a variable w
Add binary variables zi for i d
Add assignment constraint for zi s
X
zi = 1
id
Add definition constraint for x:
X
x= (xL + (i 1))zi
id
(x takes exactly one value in D)
Add definition constraint for w
X
w= (xL + (i 1))zi y (2)
id
Reformulate the products zi y via P ROD B IN C ONT
INF572/ISC610A p. 123
B ILIN A PPROX 2
B ILIN A PPROX 2 : problem P has a term xy where x
[xL , xU ], y [y L , y U ] are continuous; assume xU
xL y U y L
1. choose integer k > 0; add q = {qi | 0 i k}
to P so that q0 = xL , qk = xU , qi < qi+1 for all i
2. add continuous variable w [wL , wU ] (com-
puted from ranges of x, y by interval arithmetic)
and replace term xy by w
3. add binary variables zi for 1 i k and con- k : get identity
P
straint ik zi = 1 opt-reformulation
4. for all 1 i k add constraints:

k
P k
P
qj1 zj xi q j zj

j=1 j=1
qi +qi1 U L qi +qi1 U L

2 y (w w )(1 zi ) w 2 y + (w w )(1 zi ),
INF572/ISC610A p. 124
Relaxing bilinear terms
RRLTR ELAX : quadratic problem P with terms xi xj (i < j) and constrs
Ax = b (x can be bin, int, cont); perform opt-reformulation RRLT first:
1. add continuous variables wij (let wi = (wi1 , . . . , w1n ))
2. replace product xi xj with wij (for all i, j)
3. add the reduced RLT (RRLT) system k Awk bxk = 0
4. find a partition (B, N ) of basic/nonbasic variables of k Awk = 0
such that B corresponds to variables with smallest range
5. for all (i, j) N add constraints wij = xi xj ()
then replace nonlinear constraints () with McCormicks envelopes
wij max{xL L L L U U U U
i xj + xj xi xi xj , x i xj + xj xi xi xj } 1
0.5
C
B
D
min{xU xL xU L L
xU xL U 0.5
wij i xj + j xi i xj , x i xj + j xi i xj }
1
1
A
0
0.5
1
0.5
0 0.5
0.5 1
1
The effect of RRLT is that of using information in Ax = b to eliminate

some of the problematic product terms (those with indices in B)
INF572/ISC610A p. 125
Linearizing the l norm
I NF N ORM [Coniglio et al., MSc Thesis, 2007]. P has
vars x [1, 1]d and constr. ||x|| = 1,
s.t. x F(P ) x F(P ) and f (x ) = f (x ).
1. k d add binary var uk
2. delete constraint ||x|| = 1
3. add constraints:
k d xk 2uk 1
X
uk = 1.
kd
Narrowing I NF N ORM(P ) cuts away all optima having

maxk |xk | = 1 with xk < 1 for all k d
INF572/ISC610A p. 126
Approximating squares
I NNER A PPROX S Q: P has a continuous variable
x [xL , xU ] and a term x2 appearing as a
convex term in an objective or constraint
xU xL
1. add parameters n N, = n1 , 1
0
111111
000000
111111
000000 s
000000
111111
1
0
000000
111111
000000
111111
000000
111111
111111
000000 000000
111111
111111
000000
0
1 000000
111111
0
1
xi = xL + (i 1) for i n 111111
000000
0
1
111111
000000
111111
000000
111111
000000
111111
000000
000000
111111
0
1
000000
111111
000000
111111
000000
111111
000000
111111
111111
000000 000000
111111
111111
000000 000000
111111
111111
000000 000000
111111
2. add a continuous variable w [wL , wU ], 111111
000000
111111
000000
111111
000000
111111
000000
111111
000000
000000
111111
000000
111111
000000
111111
000000
111111
00000000
11111111
0000000000
1111111111 000000
111111
00000000
11111111
0000000000
1111111111
111111
000000 000000
111111
00000000
11111111
0000000000
1111111111
111111
000000 000000
111111
11111111
00000000
0000000000
1111111111
where wL = 0 if xL xU 0 or 111111
000000
0
1
111111
000000
0
1
111111
000000
111111
000000
111111
000000
0
1
0
1
11111111
0
1
1
0
1
000000
111111
00000000
0000000000
1111111111 0
1
000000
0 111111
00000000
11111111
0000000000
1111111111 000000
111111
00000000
11111111
0000000000
1111111111
0
1 000000
111111
00000000
11111111
0000000000
1111111111 000000
111111
11111111
00000000
0000000000
1111111111
111111
000000 0
1 000000
111111
00000000
11111111
0000000000
1111111111
111111
000000 0
1 000000 x
111111
00000000
11111111
0000000000
1111111111
min((xL )2 , (xU )2 ) otherwise and 111111
000000
0000000000
1111111111
wU = max((xL )2 , (xU )2 )
n : get
3. replace all occurrences of term x2 with w
identity opt-
4. add constraints
reformulation
i n w (xi + xi1 )x xi xi1 .
Replace convex term by piecewise linear ap-

proximation
INF572/ISC610A p. 127
Conditional constraints
Suppose a binary variable y and a constraint g(x) 0
in the problem
We want g(x) 0 to be active iff y = 1
Compute maximum value that g(x) can take over all x,
call this M
Write the constraint as:
g(x) M (1 y)
This sometimes called the big M modelling technique
Example:
Can replace constraint (2) in B ILIN A PPROX as follows:
i d M (1 zi ) w (xL + (i 1))y M (1 zi )
where M s.t. w (xL + (i 1))y [M, M ] for all w, x, y
INF572/ISC610A p. 128
Symmetry
INF572/ISC610A p. 129
Example
Consider the problem

min x1 + x2

3x1 + 2x2 1
2x1 + 3x2 1

x1 , x2 {0, 1}

AMPL code: The solution (given by

set J := 1..2; CPLEX) is x1 = 1, x2 = 0
var x{J} binary;
minimize f: sum{j in J} x[j]; If you swap x1 with x2 , you
subject to c1: 3*x[1] + 2*x[2] >= 1; obtain the same problem, with
subject to c2: 2*x[1] + 3*x[2] >= 1;
swapped constraints
solve; Hence, x1 = 0, x2 = 1 is
display x; also an optimal solution!
INF572/ISC610A p. 130
Permutations
We can represent permutations by 0
maps N
1
N
1 2
The permutation of our example is @ A
2 1
Permutations are usually written as cycles: e.g. for a
0 1
1 2 3
permutation @ A, which sends 1 3, 3 2 and
3 1 2
2 1, we write (1, 3, 2) to mean 1 3 2( 1)
The permutation of our example is (1, 2) a cycle of
length 2 (also called a transposition, or swap)
INF572/ISC610A p. 131
Cycles
Cycles can be multiplied together, but the multiplication
is not commutative: (1, 2, 3)(1, 2) = (1, 3) and
(1, 2)(1, 2, 3) = (2, 3)
The identity permutation e fixes all N
Notice (1, 2)(1, 2) = e and (1, 2, 3)(1, 3, 2) = e, so
(1, 2) = (1, 2)1 and (1, 3, 2) = (1, 2, 3)1
Cycles are disjoint when they have no common element
Thm. Disjoint cycles commute
Thm. Every permutation can be written uniquely (up to

order) as a product of disjoint cycles
For each permutation , let () be the set of its disjoint
cycles
INF572/ISC610A p. 132
Groups
A group is a set G together with a multiplication
operation, an inverse operation, and an identity element
e G, such that:
1. g, h G (gh G) (multiplication closure)
2. g G (g 1 G) (inverse closure)
3. f, g, h G ((f g)h = f (gh)) (associativity)
4. g G (eg = g) (identity )
5. g G (g 1 g = e) (inverse)
The set {e} is a group (denoted by 1) called the trivial
group
The set of all permutations over {1, . . . , n} is a group,
called the symmetric group of order n, and denoted by Sn
For all B {1, . . . , n} define SB as the symmetric group
over the symbols of B
INF572/ISC610A p. 133
Generators
Given any subset T Sn , the smallest group containing
the permutations in T is the group generated by T , denoted
by hT i
For example, if T = {(1, 2), (1, 2, 3)}, then hT i is
{(1), (1, 2), (1, 3), (2, 3), (1, 2, 3), (1, 3, 2)} = S3
For any n N, h(1, . . . , n)i is the cyclic group of order n,
denoted by Cn
Cn is commutative, whereas Sn is not
Commutative groups are also called abelian
Thm. h(1, 2), (1, . . . , n)i = h(i, i + 1) | 1 i < ni = Sn
INF572/ISC610A p. 134
Subgroups and homomorphisms
A subgroup of a group G is a subset H of G which is also a group
(denoted by H G); e.g. C3 = {e, (1, 2, 3), (1, 3, 2)} is a subgroup of
S3
Given two groups G, H, a map : G H such that
f, g G ( (f g) = (f )(g) ) is a homomorphism
Ker = {g G | (g) = e} is the kernel of (Ker G)
Im = {h H | g G (h = (g))} is the image of (Im H)
If is injective and surjective (i.e. if Ker = 1 and Im = H), then is
an isomorphism, denoted by G =H
Thm.[Lagrange] For all groups G and H G, |H| divides |G|
Thm.[Cayley] Every finite group is isomorphic to a subgroup of Sn for

some n N
INF572/ISC610A p. 135
Normal subgroups
Let H G; for all g G, gH = {gh | h H} and
Hg = {hg | h H} are in general subsets (not
necessarily subgroups) of G, and in general gH 6= Hg
If g G (gH = Hg) then H is a normal subgroup of G,
denoted by H G (e.g. C3 S3 )
If H G, then {gH | g G} is denoted by G/H and has
a group structure with multiplication (f H)(gH) = (f g)H ,
inverse (gH)1 = (g 1 )H and identity eH = H
For every group homomorphism , Ker G and
G/Ker = Im
INF572/ISC610A p. 136
Group actions
Given a group G and a set X, the action of G on X is a set of
mappings g : X X for all g G, such that g (x) = (gx) X for
all x X
Essentially, the action of G on X is the definition of what happens to
x X when g is applied to it
For example, if X = Rn and G = Sn , a possible action of G on X is
given by gx being the vector x with components permuted according

to g (e.g. if x = (0.1, 2, 2) and g = (1, 2), then gx = (2, 0.1, 2))
Convention: left multiplication if x is a column vector (g (x) = gx), right
if x is a row vector (g (x) = xg): treat g as a matrix
For Y X and H G, HY = {hy | h H, y Y } and
Y H = {yh | h H, y Y } are the left and right orbits of Y in H (also
denoted orb(Y, H)); notice orb(Y, H) X
stab(Y, G) = {g G | gY Y } is the stabilizer of Y in G; notice
stab(Y, G) G
INF572/ISC610A p. 137
Groups and graphs
Given a digraph G = (V, A) with V = {v1 , . . . , vn }, the
action of Sn on G is the natural action of on V
is a graph automorphism if (i, j) A ((i), (j)) A
For example:
1 2 3 2 4 1
4 3 4 1 3 2
G1 G2 G3
G2 = (1, 3)G1 is a graph automorphism of G1
G3 = (1, 2, 3, 4)G1 is not an automorphism of G1 : (4, 2) A but
((4), (2)) = (1, 3) 6 A
The automorphism group of G1 is he, (1, 3)i
= C2 (denoted
by Aut(G1 ))
INF572/ISC610A p. 138
Back to MP: Symmetries and BB
Symmetries are bad for Branch-and-Bound techniques:
many branches will contain (symmetric) optimal
solutions and therefore will not be pruned by bounding
deep and large BB trees
-Inf
BB tree for symmetric
-Inf
problem
1.5 2
BB tree for problem
2 2
modulo symmetries
2 2
How do we write a mathematical programming formu-

lation modulo symmetries?
INF572/ISC610A p. 139
Solution symmetries
The set of solutions of the following problem:
min x11 +x12 +x13 +x21 +x22 +x23
x11 +x12 +x13 1
x21 +x22 +x23 1
x11 +x21 1
x12 +x22 1
x13 +x23 1
{(0, 1, 1, 1, 0, 0), (1, 0, 0, 0, 1, 1), (0, 0, 1, 1, 1, 0),
is G(P ) =
(1, 1, 0, 0, 0, 1), (1, 0, 1, 0, 1, 0), (0, 1, 0, 1, 0, 1)}
G = stab(G(P ), Sn ) is the solution group (variable permutations keeping

G(P ) fixed)
For the above problem, G is
h(2, 3)(5, 6), (1, 2)(4, 5), (1, 4)(2, 5)(3, 6)i
= D12
For all x G(P ), G x = G(P ) only 1 orbit only one
solution in G(P ) (modulo symmetries)
How do we find G before solving P ?
INF572/ISC610A p. 140
Formulation symmetries
The cost vector cT = (1, 1, 1, 1, 1, 1) is fixed by all (column)
permutations in S6
The vector b = (1, 1, 1, 1, 1) is fixed by all (row) permutations in S5
Consider P s constraint matrix:

1 1 1 0 0 0

0 0 0 1 1 1

1 0 0 1 0 0

0 1 0 0 1 0

0 0 1 0 0 1
Let S6 be a column permutation such that a row permutation

S5 with (A) = A
Then permuting the variables/columns in P according to does not
change the problem formulation (the constraint order is not important)
INF572/ISC610A p. 141
The formulation group
For a MILP with binary variables only,
GP = { Sn | c = c Sm (A = A b = b)}
is called the formulation group of P

= D12
In the example above, we get GP = G
Thm.
GP G .
Result can be extended to all MILPs [Margot 02, 03, 07]
INF572/ISC610A p. 142
Symmetries in MINLPs
Consider a MINLP P

min f (x)

g(x) 0 (3)

x X.
where the set X may contain integrality constraints on x

For a row permutation Sm and a column
permutation Sn , we define P as follows:

min f (x)

g(x) 0 (4)

x X.
Define GP = { Sn | Sm (P = P )}
INF572/ISC610A p. 143
Representing g(x)
In the linear case, writing Ax is easy how do we deal with g(x)?
How do we decide whether gi (x) = gh (x) for i, h m?
Answer: consider the expression DAG (DAG=Directed Acyclic Graph)
representation of g

P3
xi yi log(x4 /y4 )
i=1 + log
List of expressions expres-
/
sion DAG sharing variable leaf
nodes x1 y1x2 y2x3 y3 x4 y4
Every function g : Rn Rm is represented by a DAG whose leaf nodes are

variables and constants and whose intermediate nodes are mathematical operators
Look for relationships between the DAGs representing g(x) and

g(x)
INF572/ISC610A p. 144
Example
+0 +1
c0 : x6 x7 + x8 x9 = 1
2 3 4 5
c1 : x6 x8 + x7 x9 = 1
x6 6 x7 7 x8 8 x9 9
GDAG = group of automorphisms of expression DAG fixing: (a)
root node set having same constr. direction and
coeff. (constraint permutations), (b) operators with same label
and rank and (c) leaf node set (variable permutations)
GDAG = h(45)(67)(89), (23)(68)(79), (01)(24)(35)(78)i
GP is the projection of GDAG to variable indices
h(6, 7)(8, 9), (6, 8)(7, 9), (7, 8)i
= D8
INF572/ISC610A p. 145
Node colors
Let DP = (V, A) be the union of all objective and constraint DAGs in
the MINLP (a.k.a the DAG of P )
Colors on the DAG nodes V are used to identify those subsets of nodes which can
be permuted (e.g. variable and operator nodes cant be permuted)
1. Root nodes (i.e. constraints) can be permuted if they have the same RHS
2. Operator nodes (including root nodes) can be permuted if they have the
same DAG rank and label; if an operator node is non-commutative, then
the order of the children node must be maintained
3. Constant nodes can be permuted if they have the same DAG rank level
and value
4. Variable nodes can be permuted if they have the same bounds and inte-
grality constraints
The relation (u v u, v have the same color) is an equivalence
relation on V (reflexive, symmetric, transitive)
partitions V into a disjoint union V/ of equivalence classes V1 , . . . , Vp
INF572/ISC610A p. 146
MINLP formulation groups
Let P be a MINLP and D = (V, A) be the DAG of P
Let GDAG be the group of automorphisms of D that fix
each color class in V/
Define : GDAG Sn by () =projection of on
variable indices; then
Thm.
is a group homomorphism and Im = GP
Hence can find GP by computing Im
Although the complexity status (P/NP-complete) of the
G RAPH I SOMORPHISM problem is currently unknown,
nauty is a practically efficient software for computing
GDAG
So now we have GP , how do we write P modulo GP ?
INF572/ISC610A p. 147
Symmetry-breaking reformulation
Consider our first example P :

min x1 + x2

3x1 + 2x2 1

2x1 + 3x2 1

x1 , x2 {0, 1}
P has G(P ) = {(0, 1), (1, 0)}, G = h(1, 2)i

= C2 and
GP = G
The orbit GP {(0, 1)} is the whole of G(P )
We look for a reformulation of P where only one
representative of each orbit is feasible
Let Q be the reformulation of P consisting of P with the
added constraint x1 x2
We have G(Q) = {(0, 1)} and G = GQ = 1
INF572/ISC610A p. 148
Breaking orbital symmetries 1
Every group G Sn acting on the variable indices N = {1, . . . , n}
partitions N into disjoint orbits (all subsets of N )
This follows from the equiv. rel. i j g G (g(i) = j)
Let be the set of nontrivial orbits ( || > 1)
Thm. G acts transitively on each of its orbits
This means that i 6= j g G (g(i) = j)
Applied to MP, if i, j are distinct variable indices belonging to the same orbit
of GP acting on N , then there is GP sending xi to xj
Pick x G(P ); if P is bounded, for all i s.t. xi is a
component having minimum value over all components of x
By theorem above, GP sending xi to xmin
Hence x is s.t. xmin is minimum over all other components of x,
and since GP G , x G(P )
INF572/ISC610A p. 149
Breaking orbital symmetries 2
Thus, for all there is at least one optimal solution
of P which is feasible w.r.t. the constraints
j (xmin xj )
Such constraints are called (orbit-based) symmetry
breaking constraints (SBCs)
Adding these SBCs to P yields a reformulation Q of P
of the narrowing type (prove it!)
Thm. If g (x) 0 are SBCs for each orbit with ap-
propriate properties, then (g (x) 0) are also
SBCs
Thus we can combine orbit-based SBCs for
appropriate properties
Yields narrowings with fewer symmetric optima
INF572/ISC610A p. 150
Breaking the symmetric group
The above SBCs work with any group GP , but their
extent is limited (they may not break all that many
symmetries)
If we find such that the action of GP on
is S , then there are much tighter SBCs
For all let = r {max } and for all j let
j + be the successor of j in
The following are valid SBCs:
j xj xj +
which are likely to break many more symmetries
INF572/ISC610A p. 151
The final attack on the KNP
INF572/ISC610A p. 152
Decision KNP
Recall the binary KNP variables are used to count the
number of spheres
Suggests simply considering whether a fixed number of
spheres can be placed around a central sphere in a
kissing configuration, or not
This is the decision version of the KNP (dKNP):
Given positive integers n, d, can n unit spheres with disjoint
interior be placed adjacent to a unit sphere centered at the
origin of Rd ?
Should eliminate binary variables, yielding a
(nonconvex) NLP, simpler than the original MINLP
In order to find the maximum value for n, we proceed by
bisection on n and solve the dKNP repeatedly
INF572/ISC610A p. 153
The dKNP formulation
Let N = {1, . . . , n}; the following formulation P correctly
models the dKNP:

max 0

P 2
i N xik = 4

kD
(xik xjk )2 4
P
i N, j N : i < j

kD

i N, k D xik [2, 2]

If F(P ) 6= then the answer to the dKNP is YES,

otherwise it is NO
However, solving nonconvex feasibility NLPs is
numerically extremely difficult
INF572/ISC610A p. 154
Feasibility tolerance
We therefore add a feasibility tolerance variable :

max

x2ik = 4
P
i N

kD

(xik xjk )2 4
P
i N, j N : i < j
kD

i N, k D xik [2, 2]

0
The above formulation Q is always feasible (why?)

Much easier to solve than P , numerically
Q also solves the dKNP: if the optimal is 1 then the
answer is YES, otherwise it is NO
INF572/ISC610A p. 155
The KNP group
The dKNP turns out to have group Sd (i.e. each spatial dimension
can be swapped with any other)
Rewriting the distance constraints as follows:
X
2
||xi xj || = (xik xjk )2
kD
X
= (x2ik + x2jk + 2xik xjk )
kD
X
= 2(d + xik xjk )
kD
(for i < j n) yields an opt-reformulation Q of Q (prove it)

The formulation group GQ turns out to be Sd Sn (pairs of distinct
spatial dimensions can be swapped, and same for spheres), much
larger than Sd
Yields more effective SBC narrowings
INF572/ISC610A p. 156
Results
Instance Solver Without SBC With SBC
D N Time Nodes OI Gap Time Nodes OI Gap
516000
2 6 Couenne 4920.16 110150
1 0.04% 100.19 14672 1 0%
45259
2 6 BARON 1200 6015
1 10% 59.63 2785 131 0%
465500 469780
2 7 Couenne 7200 127220
1 41.8% 7200 38693
1 17.9%
259800
2 7 BARON 10800 74419
442 83.2% 16632 693162 208 0%
OI: Iteration where optimum was found

: default Couenne CPU time limit
: default BARON CPU time limit
total nodes
nodes: still on tree
Thus, we finally established by MP that k (2) = 6

Actually, solutions for k (3) and k (4) can be found by using MINLP
heuristics (VNS)
INF572/ISC610A p. 157
The end
INF572/ISC610A p. 158

Liberti or Modelling SW Slides

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Liberti or Modelling SW Slides

Uploaded by

Copyright:

Available Formats

Mathematical Programming:

Modelling and Software

LIX, Ecole Polytechnique, France

Code: INF572 (DIX) / ISC610A (ISIC)

What is the best placement for the plants?

(D = 3: problem proposed by Newton in 1694, settled by

We replace the number 12 by the parameter symbol n,

In this case we only have the indication best placement in

The total filtering capacity must be at least d:

Each city must be served by at least one plant:

Because there are no more constraints in the text, this

Since there are many solvers already available,

param c{N} >= 0;

var x{j in N} binary;

minimize cost: sum{j in N} c[j]*x[j];

: a Potentially, each English

The sentence a zulu watches the aardvark would be anal-

It is easy to extend A to also recognize the power operator and

indexatom : [a-zA-Z] [ integer]? ! Y

objective : [quantifier]? [min | max] iexpr

objlist only has one objective which has an empty

The grammar is simple but tedious, best way is

var x{N} >= 0;

var x{N} >= 0, <= 3, integer;

## only enable one of the following methods:

The class of all sudoku grids

that asserts that all variables in its argument take

i, p < q K |yip yiq | 1

. . . and in some appropriate 3 3 sub-grids:

h, l H, i < p Rh , j < q Cl |yij ypq | 1

This is a nondifferentiable MINLP

It can be solved by CPLEX; however, it has O(|K|4 ) binary variables

subject to assignment {i in K, j in K} : sum{k in K} x[i,j,k] = 1;

There is no number in the problem definition:

Given a positive integer k , what is the maximum

(if sphere i is placed then yi = 1 and the constraint

(if spheres i, j are both are placed then yi yj = 1 and the

For brevity, we shall write yi {0, 1} and xik [2, 2]

which has fewer nonlinear terms than the original

This formulation is a (nonconvex) MINLP

var y{i in N} binary;

maximize kstar : sum{i in N} y[i];

subject to center{i in N} : sum{k in D} x[i,k]2 = 4*y[i];

Since the only data are the parameters d and k (two

In order to solve the KNP and deal with other difficult

Mathematical programming formulation:

Suppose w = (w , w0 ) Rn+1 is an optimal solution to the above

min{f (w, p) | ||w|| = 1}

max- and abs-norms are piecewise linear, they can be

A local NLP solver suffices to solve the NLP to optimality

have integral components

TUM matrix (partition: R1 = all rows, R2 = prove it)

arc capacities as shown in italics: find the maximum flow

var x{(i,j) in A} >= 0, <= u[i,j], integer;

maximize flow : sum{(s,i) in A} x[s,i];

subject to flowcons{i in V diff {s,t}} :

1unit of flow 5 units of flow

1unit of flow 5 units of flow

Get the same solution

A problem Q is a relaxation of P if the globally optimal

Q1 , Q2 , Q3 , . . . Q ,. . . Q hauxiliary problem ofi P

opt-reformulations narrowings relaxations approximations

i N, j N : i < j wij yj >