Lecture 11 and 12 Material

Instructor: K. Gita Ayu December 19, 2011

Integer Programming

Integer programming is also called as discrete optimization models or combinatorial optimization problems. A discrete variable is limited to a xed or countable set of values while a continuous variable can take on any value in a specied interval. Continuous variable provides better tractability than discrete variable. Thus, shall fraction have no practical importance, modeling with continuous variables is preferable than discrete variables. Three types of IP: Pure IP: all variables are integers Mixed IP: some variables are integers 0-1 IP: all variables must be binary (0, 1) An integer linear programming model has one objective function and all main constraints are linear while an integer nonlinear programming model has one objective function and any of its main constraints is nonlinear.

Example 1 Determine whether each of the following mathematical programs is best described as an LP, NLP, ILP, or INLP. Max z = x1 + 5x2 + x3 s.t. x1 x2 x1 + x2 + x3 = 10 xj = 0 or 1, j = 1, 2, 3 This is a pure Integer Linear Programming (ILP) because all variables are binary which is an integer. Min z = 3x1 + 4x2 2x3 s.t. x1 x3 x2 x1 + x2 + x3 = 10 xj 0 or 1, j = 1, 2, 3 x1 integer This is a mixed Integer Non Linear Programming (INLP) because at least one of the variables is an integer and the rst constraint is nonlinear.

 Min z = 3x1 + 5 log x3 x2 s.t. x1 x2 x1 + x2 + x3 = 10 x1 0 x2 , x3 1 This is a Non Linear Programming (NLP) because none of the variables is integer and the objective function is nonlinear. Max z = 10x1 s.t. x1 x2 x1 + x2 + x3 = 10 x1 0 This is a Linear Programming (LP) because both the objectives and constraints are linear. LP relaxation is an LP without all integer or 0 1 constraints on variables. Thus, we can say, IP = LP relaxation + additional constraint (1)

which means the feasible region of the IP must be contained in the feasible region for the corresponding LP relaxation. For Maximization problem: Optimal z-value of LP Relaxation Optimal z-value of IP (2)

Modeling

As mentioned previously, the most challenging part from mathematical programming is to translate the problem into mathematical forms. A stream of examples are given in the following subsection to develop the essential IP modeling skills as well as to illustrate the wide range of model forms available.

2.1

Simple Modeling Example

Let j = project number = {1, 2, 3, . . . , n} xj = 1 0 if project j is selected otherwise

Express these constraints in mathematical model: At least one of the rst ve projects must be selected: At most two of the rst ve projects can be selected: At least two of the last six projects must be selected: At most three of the last ten projects can be selected: Either project 2 or 5 must be selected, but not both: If project 3 is selected, project 6 must be selected: x1 + x2 + x3 + x4 + x5 1 x1 + x2 + x3 + x4 + x5 2 xn5 + xn4 + + xn 2 xn9 + xn8 + + xn 3 x2 + x5 = 1 x10 x1 x3 x6

Project 10 can be selected only if project 1 is also selected:

If project 4 is selected then project 8 and 9 must be selected: 2x4 x8 + x9

2.2

Capital Budgeting, Winston p. 478

Stockco is considering four investments. Investment 1 will yield a net present value (NPV) of $16,000; investment 2, an NPV of $22,000; investment 3, an NPV of $12,000; and investment 4, an NPV of $8000. Each investment requires a certain cash outow at the present time: investment 1, $5,000; investment 2, $7,000; investment 3, $4,000; and investment 4, $3,000. Currently, $14,000 is available for investment. Formulate an IP whose solution will tell Stockco how to maximize the NPV obtained from investments 1-4. Solution yj (j = 1, 2, 3, 4) = Max s.t. 1 0 if investment j is made otherwise z = 16000y1 + 22000y2 + 12000y3 + 8000y4 5000y1 + 7000y2 + 4000y3 + 3000y4 14000 yj = BIN, j (NPV) (available budget) (0-1 variable)

Note: knapsack problem is an integer programming with only one constraint. The Stockco problem above is a knapsack problem. The following problem is another example of knapsack problem.

2.3

Overnight Hiking

Suppose you are planning to go on an overnight hike. There are ve items you are considering taking along on the trip. The weight of each item along with its benet are listed in the table below. Your backpack can only hold up to 8 lbs. Formulate the LP which will maximize your total benet. Item Water Survival kits Camping kits Snack bars Medical pack Solution yj (j = 1, 2, 3, 4, 5) = 1 if you decided to take item j 0 otherwise Max s.t. z = 16y1 + 20y2 + 15y3 + 3y4 + 8y5 2y1 + 3y2 + 3y3 + y4 + y5 8 yj = BIN, j (benet) (maximum weight) (0-1 variable) Weight(lbs) 2 3 3 1 1 Benet 16 20 15 3 8

2.4

Capital Budgeting Revisited

y1 + y2 + y3 + y4 2 y1 + y2 + y3 + y4 1 y2 y1 y2 + y4 1 2y3 y1 + y2 y3 + y4 2y1

Write the constraint for each additional requirement below: Stockco can invest in at most two investment: Stockco must invest at least one investment:

If Stockco invests in investment 2, they must also invest in investment 1: If Stockco invests in investment 2, they cannot invest in investment 4:

If Stockco invests in investment 3, they must also invest in investment 2 and 1: If Stockco does not invest in investment 1, they must not invest in investment 3 and 4:

2.5

Fixed-Charge Problem, Winston p. 480

Gandhi Cloth Company is capable of manufacturing three types of clothing: shirts, shorts, and pants. The manufacture of each type of clothing requires Gandhi to have the appropriate type of machinery. The machinery needed to manufacture each type of clothing must be rented at the following rates: shirt machinery, $200 per week; shorts machinery, $150 per week; pants machinery, $100 per week. The manufacture of each type of clothing also requires the amounts of cloth and labor show in the table below. Each week, 150 hours of labor and 160 sq yd of cloth are available. The variable unit cost and selling price for each type of clothing are also shown in the table. Formulate an IP whose solution will maximize Gandhis weekly prots. Clothing type Shirt Shorts Pants labor (hrs) 3 2 4 cloth (sq.yd) 4 3 4 sales price ($) 12 8 15 variable cost ($) 6 4 8

Solution Let i = {shirt, shorts, pants} xi = # clothing type i produced 1 if machine to produce clothing type i is rented yi = 0 otherwise Max s.t. z = 200y1 150y2 100y3 + (12 6)x1 + (8 4)x2 + (15 8)x3 3x1 + 2x2 + 6x3 150 4x1 + 3x2 + 4x3 160 x1 M y 1 x2 M y 2 x3 M y 3 x1 , x2 , x3 0 yi = BIN, i (prot = sales - xed - variable) (available labor hours) (available clothes) (shirt production) (shorts production) (pants production) nonnegative constraint (0-1 variable)

Note: A xed-charge problem has a cost associated with performing an activity at a nonzero level that does not depend on the level of the activity. Another example of xed-charge problem is facility location model (e.g. plants, warehouses, business oces, retail store).

2.6

The Lockbox Problem, Winston p. 483

J.C. Nickles receives credit card payments from four regions of the country (West, Midwest, East, and South). The average daily value of payments mailed by customers from each region is as follows: the West, $70,000; the Midwest, $50,000; the East $60,000; the South $40,000. Nickles must decide where customers should mail their payments. Because Nickles can earn 20% annual interest by investing these revenues, it would like to receive payments as quickly as possible. Nickles is considering setting up operations to process payments (often referred to as lockboxes) in four dierent cities: Los Angeles, Chicago, New York, and Atlanta. The average number of days (from time payment is sent) until a check clears and Nickles can deposit the money depends on the city to which the payment is mailed, as show in table below. For example, if a check is mailed from the West to Atlanta, it would like an average of 8 days before Nickles could earn interest on the check. The annual cost of running a lockbox in any city is $50,000. Formulate an IP that Nickles can use to minimize the sum of costs due to lost interest and lockbox operations. Assume that each region must send all its money to a single city and that there is no limit on the amount of money that each lockbox can handle.

To From Region Region Region Region 1: 2: 3: 4: West Midwest East South

city 1 LA 2 6 8 8

city 2 Chicago 6 2 5 5

city 3 New York 8 5 2 5

city 4 Atlanta 8 5 5 2

Solution i = region = {west, midwest, east, south} j = city = {LA, CHI, NY, ATL} xij = yj = 1 if region i send all of the payments to lockbox in city j 0 otherwise 1 0 if lockbox is setup in city j otherwise

The objective of this problem is to minimize the total annual cost which consists of annual cost to operate lockboxes and annual lost interest cost. To determine the annual lost interest cost, we must determine the revenue would be lost if payments from region i were sent to region j. For example, on any given day, average daily value of payment mailed from West is $70,000. It takes 2 days to transfer the payment from West to LA which means 2 days worth of West payment will be in the mail and will not be earning interest, or 2(70, 000) = $140, 000. Because Nickles can earn 20% annually, each year, West funds will result in (.20)($140, 000) = $28, 000 in lost interest. Similar calculations for the remaining possible assignment of region i to city j. To From Region Region Region Region 1: 2: 3: 4: West Midwest East South city 1 LA 2(70K)(.20)=28K 6(50K)(.20)=60K 8(60K)(.20)=96K 8(40K)(.20)=64K city 2 Chicago 6(70K)(.20)=84K 2(50K)(.20)=20K 5(60K)(.20)=60K 5(40K)(.20)=40K city 3 New York 8(70K)(.20)=112K 5(50K)(.20)=50K 2(60K)(.20)=24K 5(40K)(.20)=40K city 4 Atlanta 8(70K)(.20)=112K 5(50K)(.20)=50K 5(60K)(.20)=60K 2(40K)(.20)=16K

Min

z = 50, 000(y1 + y2 + y3 ) + 28, 000x11 + 84, 000x12 + 112, 000x13 + 112, 000x14 +60, 000x21 + 20, 000x22 + 50, 000x23 + 50, 000x24 +96, 000x31 + 60, 000x32 + 24, 000x33 + 60, 000x34 +64, 000x41 + 40, 000x42 + 40, 000x43 + 16, 000x44

s.t.

x11 + x12 + x13 + x14 = 1 x21 + x22 + x23 + x24 = 1 x31 + x32 + x33 + x34 = 1 x41 + x42 + x43 + x44 = 1 x11 + x21 + x31 + x41 4y1 x12 + x22 + x32 + x42 4y2 x13 + x23 + x33 + x43 4y3 x14 + x24 + x34 + x44 4y4 yi = BIN, i xij = BIN, i, j (0-1 variable) (0-1 variable)

Second set x11 y1 , x12 y2 , x13 y3 , x14 y4 ,

of constraints can be written as follow: x21 y1 , x31 y1 , x41 y1 x22 y2 , x32 y2 , x42 y2 x23 y3 , x33 y3 , x43 y3 x24 y4 , x34 y4 , x44 y4

2.7

Set-Covering Problem, Winston p. 486

There are six cities in Kilroy County. The county must determine where to build re stations. The county wants to build the minimum number of re stations needed to ensure that at least one re station is within 15 minutes (driving time) of each city. The times (in minutes) required to drive between the cities in Kilroy County are shown in the table below. Formulate an IP that will tell Kilroy how many re stations should be built and where they should be located. To From City 1 City 2 City 3 City 4 City 5 City 6 Solution i = city = {1,2,3,4,5,6} xi = 1 0 if a re station is built in city i otherwise Min z = x1 + x2 + x3 + x4 + x5 + x6 s.t. x1 + x2 x1 + x2 x3 + x4 x3 + x4 + x5 x4 + x5 + x6 x2 + x5 + x6 x1 , x2 , x3 , x4 , x5 , x6 xi + x6 1 1 1 1 1 1 0 = BIN, i (number of re stations) (city 1 constraint) (city 2 constraint) (city 3 constraint) (city 4 constraint) (city 5 constraint) (city 6 constraint) nonnegative constraint (0-1 variable) city 1 0 10 20 30 30 20 city 2 10 0 25 35 20 10 city 3 20 25 00 15 30 20 city 4 30 35 15 0 15 25 city 5 30 20 30 15 0 14 city 6 20 10 20 25 14 0

Note: Assume there exists two sets, lets call it as set 1 and set 2. The objective of set-covering problem is to minimize the number of elements that are required in set 2 to cover all elements in set 1. Examples of set-covering problem are airline crew scheduling, airline scheduling, truck routing, nurse scheduling, market research, and political districting.

2.8

Either-Or Constraints

One of the most commonly constraints occurs in mathematical programming problems. Given two constraints: f (x1 , x2 , . . . , xn ) 0 and g(x1 , x2 , . . . , xn ) 0 and we want to ensure that at least

one of the two constraints is satised, often called either-or constraint. Adding the following two constraints will ensure that at least one of the two constraints is satised: f (x1 , x2 , . . . , xn ) g(x1 , x2 , . . . , xn ) where y is 0 1 variable and M is any big number. Understanding the either f (x) or g(x) constraint Given (3) and (4), if f (x) > 0, it will force y = 1 which will force the second constraint, g(x) 0. Choosing y = 1 will force f (x) 0 and g(x) 0. In summary, we can say eitherf (x) is true or g(x) is true. Example (Winston p. 488 ) Dorian Auto is considering manufacturing three types of autos: compact, midsize, and large. The resources required for, and the prots yielded by, each type of car are shown in table below. Currently, 6,000 tons of steel and 60,000 hours of labor are available. For production of a type of car to be economically feasible, at least 1,000 cars of that type must be produced. Formulate an IP to maximize Dorians prot. Resource Steel required (tons) Labor required (hrs) Prot yielded ($) Solution i = car type = {compact, midsize, large} xi = # cars type i produced yi = 1 if car type i is produced 0 otherwise Max z = 2000x1 + 3000x2 + 4000x3 s.t. 1.5x1 + 3x2 + 5x3 30x1 + 25x2 + 40x3 x1 1000 x1 x2 1000 x2 x3 1000 x3 x1 , x2 , x3 yi (prot) 6000 60000 M1 y1 M2 y2 M3 y3 0 = BIN, i (available steel) (available labor hours) (either produce 1000 compact cars or 0) (either produce 1000 midsize cars or 0) (either produce 1000 large cars or 0) (nonnegative constraint) (0-1 variable) Compact 1.5 30 2,000 Car Type Midsize 3 25 3,000 Large 5 40 4,000 My M (1 y) (3) (4)

M1 (1 y1 ) M2 (1 y2 ) M3 (1 y3 )

The maximum value of M1 can be found by assuming all available steel are used to produce compact cars, the value M1 can not be any larger than 6000/1.5 = 4000. Applying the same assumption to available labor hours, M1 can not take numbers any larger than 60000/30 = 2000. Given the two possibilities of the max value of M1 can take, we can set M1 = 2000. By the same token, the maximum value of M2 can be either 6000/3 = 2000 or 60000/25 = 2400 by assuming all available steel or labor hours are used to produce midsize cars. Thus, M2 can be at most 2000.

Lastly, the maximum value of M3 can be either 6000/5 = 1200 or 60000/40 = 1500 with the assumption of all available steel or labor hours are used to produce large cars. Thus, M3 can be set at most 1200. Writing the IP can be quite challenging for this problem especially for the producing over 1000 or not at all constraint (x1 1000 or x1 0). The rst step is to determine the f (x) 0 and g(x) 0 function: 1000 x1 0 and x1 0

Let f (x) = x1 and g(x) = 1000 x1 , by applying equation (3), the either-or constraint can be written as: x1 1000 x1 where y1 is binary. The same rule can be applied for the midsize and large cars production constraints: x2 1000 x2 x3 1000 x3 M2 y2 M2 (1 y2 ) M3 y3 M3 (1 y3 ) M1 y1 M1 (1 y1 )

2.9

If-Then Constraint

One of the most commonly constraints occurs in mathematical programming problems. Given two constraints: f (x1 , x2 , . . . , xn ) > 0 and g(x1 , x2 , . . . , xn ) 0 and we want to ensure that if f (x1 , x2 , . . . , xn ) > 0 is satised then the g(x1 , x2 , . . . , xn ) 0 constraint must be satised, often called if-then constraint. Adding the following two constraints will ensure that at least one of the two constraints is satised: f (x1 , x2 , . . . , xn ) g(x1 , x2 , . . . , xn ) where y is 0 1 variable and M is any big number. Understanding the if f (x) then g(x) constraint Given (5) and (6), if f (x) > 0, it will force y = 0 which will force the second constraint, g(x) 0. If f (x) > 0 is not satised, then (5) allows y = 0 or y = 1. Thus, if f > 0 is not satised, then the values of x1 , x2 , . . . , xn are unrestricted and g < 0 or g 0 are both possible. In summary, we can say if f (x) is true then g(x) must be true. Example (Winston p. 490 ) If customers in region 1 send payment to city 1, then no other customers may send their payments to city 1. Solution The above statement can be written in a mathematical model as if x11 = 1, then x21 = x31 = x41 = 0 M (1 y) My (5) (6)

Rewrite the above constraint to f (x) and g(x) form: if x11 > 0, then x21 + x31 + x41 0 or x21 x31 x41 0

Let f (x) = x11 and g(x) = x21 x31 x41 and write the if-then constraint as: x11 x21 + x31 + x41 M (1 y) My

It can shown that allowing f (x) > 0 forces y to zero, which forces g(x) to be zero. Note that the constraints can also be written as follow: x11 x21 + x31 + x41 My M (1 y)

Similarly, if f (x) > 0, y is forced to be 1, which forces g(x) to be zero. Because f (x) and g(x) cannot exceed 3, the value of M can be set to be 3.

Piecewise Linear Function*

A piecewise linear function consists of several straight-line segments connected to each other. Break points are the points where the slope of each of the piecewise linear function changes or the range ends. This linear function commonly occurs in purchasing division.

Branch-and-Bound Method

B&B (Branch and Bound) method is commonly used in practice due its simplicity and implicit enumeration on all points in the IPs feasible region. For IP with large number of integer points in the feasible region, B&B procedure can be very ecient for eliminating non-optimal points from consideration. When all variables in the optimal solution of the LP relaxation of a pure IP are integers, then the optimal solution to the LP relaxation is also the optimal solution to the IP. The optimal solution of LP relaxation is an upper bound for the optimal solution of the IP.

4.1

Pure Integer Programming

(Telfa Corporation, Winston p. 513 ) The Telfa Corporation manufactures tables and chairs. A table requires 1 hour of labor and 9 square board feet of wood, and a chair requires 1 hour of labor and 5 square board feet of wood. Currently, 6 hours of labor and 45 square board feet of wood are available. Each table contributes $8 to prot and each chair contributes $5 to prot. Formulate and solve an IP to maximize Telfas prot. Solution x1 = # of tables manufactured x2 = # of chairs manufactured Max z = 8x1 + 5x2 s.t. x1 + x2 9x1 + 5x2 x1 , x2 x1 , x2 6 45 0 INT 9 (prot) (available labor hours) (available sq.ft.of wood) nonnegative constraint integers

We begin the branch-and-bound method by solving the LP relaxation of the IP through dropping all integer constraints: Max z = 8x1 + 5x2 s.t. x1 + x2 9x1 + 5x2 x1 , x2
15 4

6 45 0

We nd the optimal solution of the LP relaxation x1 = and x2 = 9 with zvalue of 164 . The optimal 4 4 z-value of the LP relaxation is an upper bound for this particular problem (see Figure 1).

Figure 1: Feasible Region of the IP and Its LP Relaxation Note that both variables are non-integer in the optimal solution which means the optimal solution of the LP relaxation is not the optimal solution for the IP itself. Next we want to partition the feasible region for the LP relaxation in an attempt to nd out the location of the IPs optimal solution. Arbitrarily, perform branching by adding constraint to variable that is fractional in the optimal solution to the LP relaxation, for instance, variable x1 . The nearest integers for x1 = 15 are 3 and 4. With this in mind, add constraint 4 to drop the feasible solution of 3 < x1 < 4 for the IP by creating the following two additional subproblems (see Figure 2): Subproblem 2: subproblem 1 + x1 3 Subproblem 3: subproblem 1 + x1 4 A display of all subproblems that have been created is called a tree where each subproblem is referred as a node and each line connecting two nodes is called arc. Choose any subproblem. Arbitrarily choose to solve subproblem 2. The optimal solution is x1 = x2 = 3 with z-value of 39. Note that all variables integers which satises all constraints. Thus, this solution is called candidate solution and fathomed (further branching on the subproblem cannot yield any useful information). Because the candidate solution may be optimal, keep the candidate solution until a better feasible solution to the IP (if any exists) is found by setting its z-value as the lower bound (LB).

10

Figure 2: B&B Method: First Branching

Figure 3: Feasible Region for Subproblem 2 and 3 Now move to solve subproblem 3. The current LB is 39. Solving the subproblem 3, we obtained the optimal solution x1 = 4, x2 = 9 with z-value of 41. Because x2 is the only fractional variable in the optimal 5 9 solution to subproblem 3, we branch on x2 by partitioning the feasible region. Note that 5 is a value between 2 and 3. With this in mind, add constraint to drop the feasible solution of 1 < x2 < 2 for the IP by creating the following two additional subproblems (see Figure 4): Subproblem 4: subproblem 3 + x2 1 Subproblem 5: subproblem 3 + x2 2 Again, choose any subproblem that has not yet been solved as an LP. Arbitrarily choose to solve subproblem 4. The optimal solution is x1 = 40/9, x2 = 1 with z-value of 365 . 9

11

Figure 4: B&B Method: Second Branching

Figure 5: B&B Method: Feasible Region for Subproblem 4 Two approaches which are commonly used to determine which subproblems should be solved next. The most widely used is the LIFO rule (solve the most recently created subproblem which leads us down one side of the branch-and-bound tree and quickly nds a candidate). This method is called backtracking or depth. The second method is jumptracking or breadth in which when branching on a node, it solves all the problems created by the rst branching prior branching from the branch. 12

In this example, using backtracking means branching subproblem 4 rst, while jumptracking means solving subproblem 5 rst. Lets use backtracking method for the time being considering it does not have tendencies to create more subproblems and require more computer storage. Branching subproblem 4 by creating two additional subproblems (see Figure 6): Subproblem 6: subproblem 4 + x1 4 Subproblem 7: subproblem 4 + x1 5

Figure 6: B&B Method: Third Branching The optimal solution for subproblem 6 is x1 = 4, x2 = 1, with z-value of 37. Note that the previous candidate solution has set the lower bound of the future candidate solution to be at least 39 and the optimal solution of this subproblem is below the lower bound, further branching on this subproblem will not yield any useful information, thus we fathomed this node. Next, nd the optimal solution of subproblem 7 which is x1 = 5, x2 = 0 with z-value of 40 which is another candidate solution. Since the z-value of this new candidate solution is greater than the lower bound, set 40 as the new lower bound and cross out the candidate solution obtained in subproblem 2. Fathomed this subproblem because no further branching is needed (see Figure 6).

13

Figure 7: B&B Method: Feasible Region for Subproblem 6 and 7 Return to subproblem 5 because we cannot branch any further down the tree. The optimal solution for the subproblem 5 is infeasible which automatically fathomed this node. Thus, the optimal solution is found with z-value of 40 and variables x1 = 5 and x2 = 0.

Figure 8: B&B Method: Feasible Region for Subproblem 5

14

The complete Branch and Bound tree for Telfa problem can be seen in the gure below:

Figure 9: B&B Tree for Telfa Problem Note: the key aspects of the branch-and-bound method for solving pure IP can be summarized as follows Fathomed the subproblem shall branching further is unnecessary due to the subproblem: is infeasible yields an optimal solution in which all variables have integer values has optimal z-value which does not exceed (max problem) the current LB Eliminate subproblem from consideration given the following situations: the subproblem is infeasible LB is at least as large as the z-value for the subproblem

15

4.2

Mixed Integer Programming

Branch-and-bound can also be used to solve mixed IP by branching only on variables that are required to be integers and subproblem can be a candidate solution if variables are assigned integer values whenever required. Branch-and-bound are often used for implicit enumeration of all possible solutions to a combinatorial optimization problem. Example (Winston p. 523 ) Max z = 2x1 + x2 s.t. 5x1 + 2x2 x1 + x2 x1 , x2 x1 8 3 0 INT

Using similar method, begin the iteration through solving the LP relaxation of the IP (z = 11 , x1 = 2 , 3 3 7 and x2 = 2 ). Because only x1 is required to be integer, start branching on x1 which yields subproblems 2 and 3 as you can see in Figure 10.

Figure 10: B&B Method: Mixed Integer Programming

3 Optimal solution for this problem is x1 = 1, x2 = 2 , with z-value of 7 . 2

4.3

Knapsack Problem

A knapsack problem is an IP with single constraint. In this section, a knapsack problem in which each variable must be equal to 0 or 1 will be discussed and solved using branch-and-bound method. A knapsack problem can be written as follow: Max z = c1 x1 + c2 x2 + + cn xn s.t. a1 x1 + a2 x2 + + an xn xi = 0 or 1 where: ci = benet obtained if item i is chosen ai = amount of available resource used by item i b (i = 1, 2, . . . , n)

16

Solving knapsack problem using branch-and-bound method is fairly simple. The rst step is to nd the level of importancy of each item and put them in the knapsack in a descending order. Level of importancy c can be calculated through benet-cost (B/C) ratio ( ai ). Note that the best items have the largest while the i worst items have the smallest values of B/C ratio. Rank the items based on the best to the worst and ll the knapsack with as much of these items as possible until the best remaining item will overll the knapsack. Because all variables are binary, the branching will be in equality sign instead of inequality sign (e.g. x1 = 1 and x1 = 0 instead of x1 0 and x1 1). This is the initial solution of the LP relaxation of this knapsack problem. Example (Winston p. 525 ) Stockco problem revisited. Given the model for Stockco problem discussed previously, nd the optimal solution using B&B method.

Max s.t.

z = 16y1 + 22y2 + 12y3 + 8y4 5y1 + 7y2 + 4y3 + 3y4 14 yi = BIN, i

(NPV) (available budget) (0-1 variable)

Find the B/C ratio and rank them in descending order: Item 1:
16 5

= 3.2

Item 2:

22 7

= 3.14

Item 3:

12 4

=3

Item 4:

8 3

= 2.67

Figure 11: B&B Method: Knapsack Model Upon ranking select the best item and assign the maximum value to the decision variable for that item. In this case, select item 1 (y1 = 1) with remaining resources of 14 5 = 9. Next, select the second best item, item 2 (y2 = 1) and the remaining resources becomes 9 7 = 2. The next best item is item 3, however, we cannot assign the maximum value to item 3 because we only have 2 units left in the resources while for item 1 3 we need 4 units. Thus, we set y3 = 2 = 2 . The optimal solution for subproblem 1 is y1 = y2 = 1, y3 = 1 , 4 2 and y4 = 0 with z-value of 44. 17

Next, branch subproblem 1 by creating two additional subproblems: Subproblem 2: subproblem 1 + (y3 = 0) Subproblem 3: subproblem 1 + (y3 = 1) Choose subproblem 2 and nd the optimal solution by setting y3 = 0 and apply the technique used to solve the LP relaxation. Setting y3 = 0 allows us to use the subproblem 1 optimal solution with small modication to variable y4 . The remaining resource is 2 and the maximum value to be assigned for item 4 2 is 3 . Thus, the optimal solution is y1 = y2 = 1, y3 = 0, and y4 = 2 with z-value of 43.33. 3 Next, branch subproblem 2 by creating two additional subproblems (see Figure 11): Subproblem 4: subproblem 2 + (y4 = 0) Subproblem 5: subproblem 2 + (y4 = 1)

Figure 12: B&B Method: Knapsack Problem Second Branching

18

Now solve subproblem 4 and nd the optimal solution by setting y3 = 0 and y4 = 0 and apply the technique used to solve the LP relaxation. The optimal solution is y1 = y2 = 1 and y)3 = y4 = 0 with z-value of 38. Since all variables are integer, subproblem 4 becomes a candidate solution and set the lower bound for the LP to be 38. Fathom subproblem 4 because further branching will not yield better solution.

Figure 13: B&B Tree for Stockco Knapsack Problem Move to the neighbor branch, subproblem 5, set y3 = 0 and y4 = 1 and apply the same technique as above. Set the value for the best time to its maximum level and so forth leads us to the optimal solution of 6 y1 = y4 = 1, y2 = 7 , and y3 = 0 with z-value of 42.857. Next, branch subproblem 5 by creating two additional subproblems: Subproblem 6: subproblem 5 + (y2 = 0) Subproblem 7: subproblem 5 + (y2 = 1) Solve subproblem 6 by setting y3 = y2 = 0, y4 = 1. The optimal solution is y1 = y4 = 1, y2 = y3 = 0 with z-value of 24. Considering the value is below the LB, thus this subproblem is fathomed. Move to the neighbor branch and solve subproblem 7 by setting y3 = 0, y2 = y4 = 1 and yield the optimal solution as 4 y1 = 5 with z-value of 42.8.

19

Branch subproblem 7 by creating two additional subproblems (see Figure 12): Subproblem 8: subproblem 7 + (y2 = 0) Subproblem 9: subproblem 7 + (y2 = 1) Solve subproblem 8 and yield optimal solution of 30 and lower than the LB. Thus, subproblem 8 is fathomed. Subproblem 9 is infeasible, thus no further branching is needed. Move back to the rst tier and solve subproblem 3 and nd the optimal solution by setting y3 = 1 and apply the technique used to solve the LP relaxation. Setting y3 = 1 reduces the resource to 14 4 = 10. Choose the best one (item 1) and set y1 = 1 and recalculate the remaining resource 10 5 = 5. The next best one is item 2, however, we cannot assign the maximum value to item 2, thus set y1 = 5 . The optimal 7 solution for subproblem 2 is y1 = y3 = 1, y2 = 5 , and y4 = 0 with z-value of 43.71. From the tree, we nd 7 the optimal solution is z = 42, y1 = 0, y2 = y3 = y4 = 1 (see Figure 13).

Combinatorial Optimization Problems

In general, any optimization that has a nite number of feasible solutions is a combinatorial optimization problem in which often branch-and-bound approach is the most ecient way to solve them. Combinatorial optimization is a broad eld and is known for its computational complexity, algorithm, and heuristics. The scope is broad which includes but not limited to network ow, integer programming, simulated annealing, and genetic algorithms. Examples of combinatorial optimization problems are single machine scheduling and traveling salesman problem with n! possible solutions must be considered.

5.1

Machine Scheduling Problem, (Winston, p. 528)

Four jobs must be processed on a single machine. The time required to process and due date of each job are shown in the table below. The delay of a job is the number of days after the due date that a job is completed (if a job is completed on time or early, then the jobs delay is zero). In what order should the jobs be processed to minimize the total delay of the four jobs? Job 1 2 3 4 Solution xij = 1 0 if job i is the jth job to be processed otherwise Days required to complete the job 6 4 5 8 Due date end of day end of day end of day end of day

8 4 12 16

Start branch-and-bound method by partitioning all solutions according to the job that is last processed. Draw four branches in which each job is set as the last job to be processed by setting x14 , x24 , x34 , or x44 equals to 1. This yields the lower bound of the total delay (D). For example, if x14 = 1 then job 4 will be nished the last which equals to the total processing time.

20

D (total processing time) (job 4s due date) D (6 + 4 + 5 + 8) 16 D 7 days late Perform the same technique for the remaining three nodes (jumptracking method). Branch on the node that has the smallest lower bound of D (in this case we choose to branch node 4 ) by setting the job which be nished second to the last. For example, if x13 = 1 then job 1 will be nished second to the last node (third job processed) which equals to the total processing time without job 4. Note that from the previous analysis, job 4 will be delayed by 7 days and has to be added to the total delay (see Figure 14). D (total processing time) (job 1s due date) + (job 4s delay) D (6 + 4 + 5) 8 + 7 D 14 days late

Figure 14: B&B Tree for Machine Scheduling Problem: First Branching Perform the same analysis for the remaining two nodes and branch on the node with the smallest lower bound of D (choose node 7 ). Any job sequence associated with node 7 must have x12 or x22 equals to 1. Arbitrarily, choose x22 = 1, which leads us to node 8 and it corresponds to the order 2134 with total delay of D = 7 (for job 4) + 3 (for job 3) + 2 (for job 1) + 0 (for job 2) D = 12 days This sets the lower bound of the total delay to be 12 days. Knowing that any node that cannot have a total delay of less than 12 can be eliminated. Node 9 corresponds to processing the jobs in the order 1 2 3 4. Calculating node 9 does not generate better total delay (D = 16) than node 8, thus node 9 may be eliminated from consideration. Node 5 (D 14) and 6 (D 18) which have total delay greater than 12 by default, thus can be eliminated directly. This is also applicable to node 1 (D 15) and 2 (D 19) which leaves node 3 (D 11) as the only node left to be analyzed (see Figure 15). By utilizing the same method, three additional nodes (node 10 12) are added from branching node 3. For node 10, D (delay from processing job 3 last) + (delay from processing job 1 third) D 11 + (6 + 4 + 8) 8 D 21 days late

21

Because the delay will be at least 21 days (lower bound is 12 days per node 8), it will not be necessary to branch further in this node. Thus, this node is fathomed. Performing the same analysis to the remaining nodes, the optimal total delay is 12 days with job processing sequence is in the order 2 1 3 4.

Figure 15: B&B Tree for Machine Scheduling Problem: Second Branching

5.2

Traveling Salesman Problem (TSP)

Suppose TSP consists of cities 1, 2, 3, . . . N . For i = j, let: cij = distance from city i to city j and cii = M (very large number) to ensure no loop within city xij = 1 if the solution to TSP goes from city i to city j 0 otherwise

The solution to a TSP can be found by solving the following IP: Min z =
i j N

cij xij

(7)

s.t.
i=1 N

xij

1 (for j = 1, 2, . . . , N )

(8)

xij
j=1

1 (for i = 1, 2, . . . , N )

(9) (10)

ui uj + N xij All xij

N 1 (for = j, i = 2, 3, . . . , N ; j = 2, 3, . . . , N ) = 0 or 1, All uj 0

The objective function (7) calculated the total length of the arcs included in a tour. The constraints in (8) ensure that we arrive once at each city, while constraints in (9) ensure that we leave each city once. The constraints in (10) plays an important role because it ensure the following:

22

 Any set of xij s constaining a subtour will be infeasible Any set of xij s that forms a tour will be feasible Branch-and-bound methods can be used to solve TSP. However, shall the number of cities increases signicantly, the amount of computer time required will also increase signicantly. Thus, heuristics methods, which quickly lead to a good (not necessarily optimal) solution, are often used. Only use heuristics when an algorithmic approach is impractical. Two types of heuristics for TSP are NNH (nearest-neighbor heuristic) and CIH (cheapest-insertion heuristic). NNH begins with any city then visit the nearest city. Continue in this fashion until a tour (an itinerary that begins and ends at the same city and visits each city once) is obtained. Continue applying this method to all cities and take the best tour obtained. This heuristic can be exhausting. CIH begins with any city and nd its closest neighbor. Then create a subtour joining those two cities and replace an arc in the subtour (i.e. arc (i, j)) by the combination of two arcs (arc (i, k) and (k, j)) that will increase the length of the subtour by the smallest (or cheapest) amount. Let cij be the length of arc (i, j). Replacing arc (i, j) with arc (i, k) and (k, j) increase the length of the subtour by cik + ckj cij . Continue with this procedure until a tour is obtained. Example (Winston p. 530 ) Joe State lives in Gary, Indiana. He owns insurance agencies in Gary, Fort Wayne, Evansville, Terre Haute, and South Bend. Each December, he visits each of his insurance agencies. The distance between each agency (in miles) is shown in table below. Find the order of visiting agencies that will minimize the total distance traveled. City City City City City Solution xij = 1 0 if Joe leaves city i and travels to city j otherwise 1: 2: 3: 4: 5: Gary Fort Wayne Evansville Terre Haute South Bend Gary 0 132 217 164 58 Fort Wayne 132 0 290 201 79 Evansville 217 290 0 113 303 Terre Haute 164 201 113 0 196 South Bend 58 79 303 196 0

ci j = distance between city i and j NNH Arbitrarily choose to begin at city 1. The closest city from city 1 is city 5 (compare the distance: 132 to city 2, 217 to city 3, 164 to city 4, and 58 to city 5, which generated the arc 1 5. Of the remaining cities, city 2 is the closest from city 5 and generated the arcs 1 5 2. Of the remaining cities, city 4 is the closest to city 2, comparing distance: 290 (city 3) and 201 (city 4) and the arcs become 1 5 2 4. Only city 3 left which must be visited and then returns to city 1 and this yields the tour 1 5 2 4 3 1. In this problem, NNH yields an optimal tour with length 58 + 79 + 201 + 113 + 217 = 668 miles. CIH Arbitrarily begins at city 1. The closest city from city 1 is city 5. Thus the subtour becomes (1, 5) (5, 1). Arc (1, 5) can be replaced with arcs: (1, 2) (2, 5), (1, 3) (3, 5), and (1, 4) (4, 5). Arc (5, 1) can also be replaced with arcs: (5, 2) (2, 1), (5, 3) (3, 1), and (5, 4) (4, 1). See table below for calculation on the additional distance of replacing the arc (1, 5) or (5, 1).

23

Replaced arc (1, 5) (1, 5) (1, 5) (5, 1) (5, 1) (5, 1)

arcs added to subtour (1, 2) (2, 5) (1, 3) (3, 5) (1, 4) (4, 5) (5, 2) (2, 1) (5, 3) (3, 1) (5, 4) (4, 1)

additional length c12 c15 = 132 + 79 58 c13 + c35 c15 = 217 + 303 58 c14 + c45 c15 = 164 + 196 58 c52 + c21 c51 = 79 + 132 58 c53 + c31 c51 = 203 + 217 58 c54 + c41 c51 = 196 + 164 58

= 153 = 462 = 302 = 153 = 462 = 302

Considering replacing (1, 5) with (1, 2) (2, 5) and replacing (5, 1) with (5, 2) (2, 1) provides the same minimum additional length to the subtour, we arbitrarily choose replacing arc (1, 5) with (1, 2) (2, 5) and obtain the new subtour (1, 2) (2, 5) (5, 1). Now, consider replacing each arc in the subtour (see table below for relevant computations). Replaced arc (1, 2) (1, 2) (2, 5) (2, 5) (5, 1) (5, 1) arcs added to subtour (1, 3) (3, 2) (1, 4) (4, 2) (2, 3) (3, 5) (2, 4) (4, 5) (5, 3) (3, 1) (5, 4) (4, 1) additional length c13 + c32 c12 = 217 + 290 132 c14 + c42 c12 = 164 + 201 132 c23 + c35 c25 = 290 + 303 79 c24 + c45 c25 = 201 + 196 79 c53 + c31 c51 = 203 + 217 58 c54 + c41 c51 = 196 + 164 58

= 375 = 233 = 514 = 318 = 462 = 302

From table above, arc (1, 2) will be replaced with (1, 4) (4, 2) which yields the subtour (1, 4) (4, 2) (2, 5) (5, 1). Now, consider to replace each arc in the subtour (see the following table). Replaced arc (1, 4) (4, 2) (2, 5) (5, 1) arcs added to subtour (1, 3) (3, 42) (4, 3) (3, 2) (2, 3) (3, 5) (5, 3) (3, 1) additional length c13 + c34 c14 = 217 + 113 164 c13 + c32 c42 = 113 + 290 201 c23 + c35 c25 = 290 + 303 79 c53 + c31 c51 = 203 + 217 58

= 166 = 202 = 514 = 462

This yields the optimal tour (1, 3) (3, 4) (4, 2) (2, 5) (5, 1) and optimal value of 668. Solving the problem above using B&B will require calculation on 20 branches.

Implicit Enumeration

This method is commonly used to solve 0 1 IP. Two types of variable are introduced in this method: xed variables (variables which values are specied at a node) and free variables (variables whose values are unspecied at a node). The main idea used in implicit enumeration is to check whether a node has a feasible completion by looking at each constraint and assigning each free variable the best value (see table below) for satisfying the constraint. For any node, a specication of the values of all the free variables is called a completion of the node. Three main ideas used in implicit enumeration are: 1. Suppose at any node, the values of the xed variables are given. A good completion can be obtained by setting each free variable equal to the value 0 or 1 that makes the objective function largest (for maximization problem) or smallest (for minimization problem) 2. Even if the best completion of a node is not feasible, it gives a bound on the best objective function value that can often be used to eliminate a node from consideration

24

Type of constraint

Sign of free variables coecient in constraint + +

Value assigned to free variable in feasibility check 0 1 1 0

Table 1: How to Determine Whether a Node has a Completion Satisfying a Given Constraint 3. An easy way to determine if all completions of the node are infeasible is through assigning values to the free variables that make the lhs of the equation as small as possible (for constraint) or as large as possible (for constraint) Note: equality constraint should be replaced by a and a constraint. In general, to determine whether a node has a feasible completion or not can be done through looking at each constraint and assigning each free variable the best value for satisfying the constraint (see Table 1 for the summary). Example (Winston p. 542 ) Solve the following 0 1 IP using implicit enumeration: Max s.t. z = 7x1 3x2 2x3 x4 2x5 4x1 2x2 + x3 2x4 5x5 4x1 2x2 4x3 + x4 + 2x5 xi = BIN 3 7 , i (11) (12)

Start with node 1. The best completion of node 1 is to set all variables equal to zero. Unfortunately, the best completion is not feasible because it violates both constraints. Now, check whether node 1 has no feasible completion by using the information available in Table 1. Setting x1 = x2 = x4 = x5 = 1 (a constraint and the free variables coecients are all ) and x3 = 0 (a constraint with coecient of the free variable is +) satises (11). Lets call this set as A (1, 1, 0, 1, 1). Choosing x1 = x2 = x3 = 1 (a constraint and the coecient of free variables are ) and x4 = x5 = 0 (a constraint and the coecient is +) satises (12). Lets call this set as B (1, 1, 1, 0, 0). The infeasibility check (surrogate constraints) does not allow node 1 to be classied as having no feasible completion. Now lets choose to branch on a free variable, arbitrarily, x1 and yields two new nodes (see Figure 16).

Figure 16: Branching on Node 1 Arbitrarily choose to analyze node 2. The best completion of node 2 is to set x1 = 1 and the remaining variables to zero. Unfortunately, the best completion violates (11). Next, we need to determine whether node 2 has a feasible completion by applying the rule dened in Table 1. We obtained the values of x1 , x2 , x3 , x4 , x5 to be (1, 1, 0, 1, 1) and (1, 1, 1, 0, 0) which satises (11) and (12), respectively. 25

The infeasibility check has yielded no information about whether node 2 having no feasible completion. Now, choose to branch on node 2 by setting free variable x2 arbitrarily and obtains two new nodes (see Figure 17).

Figure 17: Branching on Node 2 Arbitrarily, we choose to analyze node 5 and found that the best completion of node 5 to be x1 = 1, x2 = 0 (due to the additional constraints from the precedence nodes) and the remaining free variables equal to zero. Again, unfortunately, the completion violates (12). Now perform a feasibility check on node 5. Completion (1, 0, 0, 1, 1) satises (11) and (1, 0, 1, 0, 0) satises (12). Again, the infeasibility check has yielded no information. Thus, branch node 5 by choosing any free variable, say x3 (see Figure 17).

Figure 18: Branching on Node 5 Choose to analyze node 6 and the best completion of node 6 is x1 = 1, x2 = 0, x3 = 1 and set the remaining free variables equal to zero. This completion is feasible, thus we found a candidate solution with z = 9 is found and set the lower bound to be -9. Next, analyze node 7 and the best completion is x1 = 1, x2 = 0, x3 = 0 and set the remaining free variables equal to zero with zvalue of 7 (better than the lower bound of 9), but unfortunately it violates (12). (1, 0, 0, 1, 1) satises (11) but does not satisfy (12). Thus, node 7 has no feasible completion and can be eliminated from consideration. Now analyze node 4. The best completion of node 4 is x1 = x2 = 1 and set the remaining variables to be equal to zero with z = 10 (which is worse than the lower bound). Thus, this node is eliminated from consideration. 26

Going up the tree, we now analyze node 3 and setting the best completion of node 3 to be x1 = x2 = x3 = x4 = x5 = 0 yields optimal solution to 0 (better than the lower bound), however, it violates all constraints. Now check whether node 3 has feasible completion. (0, 1, 1, 1, 1) does not satisfy (12) as well as (0, 1, 1, 0, 0). Apparently, changing the value for x2 and x3 to 1 (considering these two are the variables with negative coecient) does not satisfy constraint (12). Thus, node 3 may be eliminated from consideration. Because no nodes left to analyze, the node 6 candidate with z = 9 becomes the optimal solution with x1 = 1, x2 = 0, x3 = 1, x4 = 0, x5 = 0.

The Cutting Plane Algorithm

27