Linear Programming

IE 5551, Spring 2011

Xi Chen

Outline
Background
Definition (canonical form)
Standard form
Excel solver
Branch-and-Bound algorithm

Background
Optimization (Wikipedia)
Choosing the best elements from some set of available alternatives
Subfields:
Convex programming (includes linear programming)
Integer programming
Nonlinear programming
Stochastic programming
Combinatorial optimization

Components:
Variables
Objectives
constraints

Linear Programming
A Linear Program (LP) is a mathematical optimization problem in which

the objective function and all constraints are linear.

Canonical form:
n

min

ci xi

min
s.t.

i 1
n

s.t.

aij xi

bi ,

i 1

Uses:

Operations research
Electrical engineering
Portfolio management
Heuristics

j 1, 2,..., m

cT x
Ax b

Linear Programming
Examples (LP?):

max

x1 x2

s.t.

sin( x1 ) x2

max

x1

x2

s.t.

x1 x2

x2 {0,1}

max

x1 x1 x2

s.t.

x1 x2 1
x1

max

x1

x2

s.t.

x1 x2
x1 , x2

2
0

Linear Programming
Standard form:
An LP is said to be in its standard form if it is of the form:

min

cT x

s.t.

Ax b
x 0

Any LP can be written in the standard form using a few tricks

(Ronald L. Rardin, Optimization in Operations Research,

1998).
Standard form is used to apply algorithm and use software

Linear Programming
Classes of LP problems:
Feasible and bounded

min

x1

x2

s.t.

x1 x2
x1 , x2

Infeasible

min

x1 x2

s.t.

x1 x2
x1 , x2

Unbounded

min

x1

x2

s.t.

x1

x2

2
0

Linear Programming
Important theorems:
Global optimal, local optimal
Unimodal Function:

Convex region:
For any two points within the region, all the points on the line segment between them are all
within the region

Unimodal Objective function & Convex feasible set implies

Global optimal exists
Local optimal is global optimal
LP has UOF & CFS
Global optimal exists
Local optimal is global optimal
Global optimal are among the vertices

Linear Programming
Solving LP:
Graphical method (2-dimentional LP)

min
s.t.

x1 x2
x1 2 x2

2 x1

x1 , x2
Simplex method
Interior point method
Ellipsoid algorithm

Computer programming tools:

Excel Solver
CPLEX

x2
0

Excel Solver
Load Solver Add-In:
Menu -> Excel Options -> Add-Ins -> Manage Excel Add-Ins

-> select Solver Add-in

Example:
min

s.t.

x1 2 x2 3x3
x1

x3
x2

x1

x4
1
x4

x2

x1 , x2 , x3 , x4 , x5

1
x5 1.5

Branch-and-Bound
Background Integer Programming
An integer program (IP) is just a linear program with the additional

constraint imposed that all variables must be integers.

min
s.t.

cT x
Ax b
x 0
x

zn

Although a linear program is easily solved, the presence of the

integral make the problem much harder (NP-hard)

One natural idea is to take the solution to the linear-programming
relaxation and round it to the nearest integer point
This may not be feasible, but even if it is, there is no guarantee that
we're at an optimal point.
The most commonly applied technique is branch-and-bound

Branch-and-Bound
Basic idea: solve the linear programming relaxation, and hope

that the solution is integral

Example:

max

17 x1 12 x2

s.t.

10 x1 7 x2
x1

x2

x1 , x2

40

5
Z

Solving the linear relaxation gives us: ( x1 , x2 )

We split the search space into two nodes:

( 5 ,10 )
3
3

P1 {( x1 , x2 ) :10 x1 7 x2

40, x1

x2

5, x1 1}

P2

40, x1

x2

5, x1

{( x1 , x2 ) :10 x1 7 x2

2}

Branch-and-Bound