Problem-Solving Phases

In this section, the art of problem-solving will be addressed, specifically emphasizing a methodology of
converting a written problem statement into a mathematical model. The approach to be followed here is
based upon the approach given by the well-known mathematician, George Polya, in his text How To
Solve
I. Understanding The Problem
Identify the unknowns, the data, and the conditions. Check for sufficient, contradictory, and redundant
conditions. Make an illustration when possible. Introduce a suitable notation.
II. Devising A Plan (Program/Algorithm)
Is there a computer program available to solve the problem? Think about similar problems. Can
appropriate formulas or theorems be used? State and solve a simpler problem (remove or add
conditions),
if necessary, to get started. Go back to the definitions. List all the steps needed to solve the problem
without regard to order. After this has been done, place a list of these precise steps in order - this is the
algorithm.
III. Carrying Out The Plan
Execute the code/algorithm using the given data. Are all of the steps correct?
IV. Looking Back
Is the solution correct? Can this be proven? Can the problem be solved another way? Does the
solution provide any insight about the problem?

Phase 1. Understanding The Problem
The starting point here is the written statement of the optimization problem. Of the four major phases of
problem-solving, this one is the most crucial. If this is not done correctly, the remaining work (no matter
how good it is) may be wasted with a misleading outcome. A correct solution to the wrong problem is
usually of little use! The objective of this phase is to produce a validated optimization model.
Step 1. Carefully read and re-read the problem statement throughout the entire problem-solving process.

Step 2. Identify and list the single (dependent) optimization variable. Here its mathematical name will be
z.
Also identify its physical units (e.g., dollars, newtons, ft2, etc.). In practice, it could be given an application
name that reflects its role in the problem (e.g., cost, force, area, etc.). Its application name, mathematical
name, range (including data type - real, rational, integer, etc.), and units should be listed.

Note: This is how a problem is identified as an optimization problem. That is, the problem should use a
superlative criterion such as maximum, best, highest, strongest, largest, fastest, etc. (or the opposite like
minimum, worst, lowest, weakest, smallest, slowest etc.). If there is no such term, the problem may not
even be an optimization problem at all.
Step 3. Identify and list all of the (independent) optimization variables x1, x2, ..., xn, and their
corresponding
physical units. The number of these variables will be called n. Continue the listing started in the
previousstep with the application name, mathematical name, domain (including data type), and units for
each.
Step 4. Identify the stated and hidden assumptions about the character of the problem. This includes
indicating which variables are continuous (e.g., real with fractional parts) and which, if any, are required to
be integers.
Step 5. Determine the objective function z = f(x1, x2, ..., xn) which shows how the independent variables
are
to be combined to produce the dependent variable. Verify that the units of z are consistent with the units
used in the terms of f(x1, x2, ..., xn) using any physical principles and principles of dimensional analysis.
Indicate the form of this function in terms of how the independent variables are combined - added,
multiplied,
exponentiated, and exhibit linear, quadratic, exponential, oscillatory, or general nonlinear behavior.

Step 6. Determine if the objective function is to be maximized, minimized, or both.
Step 7. Determine each inequality constraint gi(x1, x2, ..., xn) 0, i=1,2,...,m and identify which of the n
variables (out of x1, x2, ..., xn) influence this constraint. Verify that the units are consistent using the
principles
of dimensional analysis. Indicate the form of this function in terms of how the independent variables
are combined. The number of these inequality constraints will be called m. Do this for each of the m
constraints. Consider the special cases of non-negativity constraints (xi 0) and bounded variables (ai
xi
bi). If there are no inequality constraints, this step may be skipped.
Step 8. Determine each equality constraint hj(x1, x2, ..., xn) = 0, j=1,2,...,k and identify which of the n
variables
(out of x1, x2, ..., xn) influence this constraint. Verify that the units are consistent using the principles
of dimensional analysis. Indicate the form of this function in terms of how the independent variables are
combined. The number of these equality constraints will be called k. Do this for each of the k constraints.
If there are no equality constraints, this step may be skipped.

Step 9. Validate the model, whenever possible, by having the person(s) that formulated the optimization
problem confirm that the optimization model describes the problem with complete accuracy (without
adding,
removing, or changing any information). Specifically try to determine if there is missing information that
must be supplied, redundant information that can be removed, extra data that has not been incorporated,
or contradictory information that has to be resolved.

Step 10. If the model is validated, this phase is complete. Otherwise, repeat the appropriate steps in
Steps 1-9 until the current model is validated.









Phase 2. Devising A Plan
The starting point of this phase is a validated optimization model equivalent to the form:
Maximize or Minimize:
f(x1, x2, ..., xn)
Possibly subject to inequality and equality constraints:
gi(x1, x2, ..., xn) 0, i=1,2,...,m
hj(x1, x2, ..., xn) = 0, j=1,2,...,k
The objective of this phase is to identify the class of the optimization model and to prepare it for solution
by
the most appropriate software.

Step 1. Determine the model classification as precisely as possible. For example, if the objective function
is quadratic and the constraints are linear, then it is a QP, not just an NLP, so that the first attempt to
solve
it should use a QP code, if available. Simplifying assumptions may be necessary to get it into a form like
that shown above. Don't make unjustified assumptions.
Step 2. Seek out reliable software. Select the available software packages that are designed to solve
optimization models of the indicated classification. Pick the most reliable of these packages to use. If
none are available, try the Internet (e.g., use http://www-fp.mcs.anl.gov/otc/Guide/SoftwareGuide/ ).
Alternately
use a search engine (e.g., Google) to locate code in an appropriate programming language or locate
a well-known algorithm that can be coded. The result of this step should be a program that is expected to
solve the model classified in the previous step.

Step 3. If necessary, reformulate (restate) the model in order to match the input requirements of the code
from the previous step. Special variable names may be required. Different constraint forms may be
required such as using pi(x1, x2, ..., xn) ai or qi(x1, x2, ..., xn) bi, for i=1,2,...,m. Similarly, one may
need
to use rj(x1, x2, ..., xn) = aj, for j=1,2,...,k. Some model information may need to be coded in the language
of the software program. Some information may need to be entered at the keyboard or through one or
more files. Interface code and data file preprocessing code may have to be written. Remember that the
more changes that are made, the more likely that the model will be accidentally altered.
Step 4. If the model matches the program's requirements, this phase is complete. Otherwise, repeat
these
steps until the model matches the program's input requirements. If the problem has to be changed, go
back to Phase1.

Phase 3. Carrying Out The Plan
The objective of this phase is to compute and verify that the optimal values of x1 = x1
* , x2 = x2
* , ..., xn = xn
* ,
z = z* , is the solution to the optimization model.
Step 1. Solve the model with the computer program and its information inputs from the previous phase.
Look carefully for any and all error messages. Make any necessary input changes that are consistent with
the validated model and re-execute the program until no error messages or unexplained warning
messages
are produced by the program. Try various algorithm parameter settings, method changes, starting
points, etc. until a solution is obtained. This step can take time, patience, and an organized approach.

Step 2. If the validated model cannot be solved and equality constraints are in the model, consider solving
for one or more of these in terms of the others (it is usually best to solve for those that appear linearly).
Then substitute these wherever they appear in the objective function and in any other constraints. This
will
reduce both n and k, and should make the optimization problem easier to solve. This step is optional.

Step 3. If the validated model still cannot be solved, then some assumptions may need to be added (e.g.,
like bounds on the independent variables) or altered (e.g., like dropping integer requirements for some or
all variables). It will be especially important to re-validate this new model before accepting any solution.
This step is optional.

Step 7. Repeat these steps until the global solution to the model is computed and verified or until the
model
has to be changed. Be sure that the solution isn't just a local one (unless any local solution is
satisfactory).
Remember that the first and second derivative information, if it exists, is typically local information at a
given point (e.g., convexity of the objective function may be required to guarantee a global minimum).
Several widely dispersed starting points and several methods may have to be tried to have confidence
that
the global optimum has been found. However, confidence is not a guarantee. This step is related to Steps
3 and 4 of Phase 4. If the model has to be changed, go back to Phase 2.
Phase 4. Looking Back
The objectives of this phase are to verify and validate (V&V) that the optimal values of x1 = x1
* , x2 = x2
* , ...,
xn = xn
* , z = z* , are the solution to the original optimization problem, to explain the meaning of the solution
within the context of the given problem, and to optionally analyze the optimization model's parameter
sensitivity.

Step 1. Always interpret any solution in terms of the original optimization problem in order to see if it
"makes sense." When possible, verify the solution by determining if it meets the appropriate
unconstrained
or constrained optimality conditions.

Step 2. Always validate the optimal values of the global solution and any local solution in terms of the
original problem. This should ideally be done in collaboration with the poser of the problem - the person
most knowledgeable about the problem's application area.

Step 3. When necessary, verify the solution by trying new initial values for the independent optimization
variables associated with the software of algorithm being used. Any time that a different solution is
obtained, compare its optimum value z* with the previous optimum value and identify the better of the two
solutions. This requires going back to Phase 1.

Step 4. When necessary, also verify the solution by trying different algorithms and/or different programs.
Any time that a different solution is obtained, compare its optimum value z* with the previous optimum
value and identify the better of the two solutions. This requires going back to Phase 1.



Step 5. Optionally analyze the current model and identify any terms in the objective function or any of the
m
+ k constraints that will cause the model to be ill-conditioned where a small change to the model produces
a large change in the solution. This can be done by trial and error or by investigating the appropriate first
partial derivatives of coefficients and constants (now treated as parameters) within these m + k + 1
expressions.
These expressions may be plotted to gain additional insight. Again, the poser of the problem needs
to be involved. This also requires going back to Phase 1.
Note: If a small change in any of the constants, parameters, or decision variables produces a "significant"
change in the optimal value of the objective function, then it is most important to spend enough time on
Step 5.