2. Content
What is Optimization?
Categorization of Optimization Problems
Some Optimization Problems
3. What is Optimization?
Objective Function
– A function to be minimized or maximized
Unknowns or Variables
– Affect the value of the objective function
Constraints
– Restrict unknowns to take on certain values but
exclude others
4. What is Optimization?
Objective Function
– A function to be minimized or maximized
Unknowns or Variables
– Affect the value of the objective function
Constraints
– Restrict unknowns to take on certain values but
exclude others
5. Example:
0-1 Knapsack Problem
Which boxes should be chosen
to maximize the amount of
money while still keeping the
overall weight under 15 kg ?
6. Example:
0-1 Knapsack Problem
1 {0,1}
x
2 {0,1}
x
3 {0,1}
x
5 {0,1}
x
4 {0,1}
x
Objective Function
Unknowns or Variables
Constraints
's, 1, ,5
i
x i
, ,
0,1 1, 5
i
x i
1 2 3 4 5
4 2 10 2 1
Maximize
x x x x x
1 2 3 4 5
12 1
Subject t
15
4 2
o
1
x x x x x
7. Example:
0-1 Knapsack Problem
1 2 3 4 5
4 2 10 2
Maxim 1
ize x x x x x
1 2 3 4 5
12 1 4 2
Subject to 15,
0,1 , 1, ,5
1
i
x x x x x
x i
9. Minimum and maximum value of a
function
This denotes the minimum value of the
objective function , when choosing x from
the set of real numbers . The minimum value
in this case is 1, occurring at x=0 .
10. Maximum value of a function
Similarly, the notation
asks for the maximum value of the objective
function 2x, where x may be any real
number. In this case, there is no such
maximum as the objective function is
unbounded, so the answer is "infinity" or
"undefined".
11. By convention, the standard form of an optimization
problem is stated in terms of minimization.
Generally, unless both the objective function and the
feasible region are convex in a minimization
problem, there may be several local minima, where
a local minimum x* is defined as a point for which
there exists some δ > 0 so that for all x such that
the expression
holds; that is to say, on some region around x* all of
the function values are greater than or equal to the
value at that point. Local maxima are defined
similarly.
13. Optimization Problems w/o
Objective Function
In some cases, the goal is to find a set of
variables that satisfies some constraints only, e.g.,
– circuit layouts
– n-queen
– Sudoku
This type of problems is usually called a
feasibility problem or constraint satisfaction
problem.
• Objective Function
• Unknowns or Variables
• Constraints
14. Optimization Problems w/
Multiple Objective Functions
Sometimes, we need to optimize a number of different
objectives at once, e.g.,
– In the panel design problem, it would be nice to minimize
weight and maximize strength simultaneously.
Usually, the different objectives are not compatible
– The variables that optimize one objective may be far from
optimal for the others.
In practice, problems with multiple objectives are
reformulated as single-objective problems by either
forming a weighted combination of the different objectives
or else replacing some of the objectives by constraints.
• Objective Function
• Unknowns or Variables
• Constraints
15. Variables
• Objective Function
• Unknowns or Variables
• Constraints
Variables are essential.
– Without variables, we cannot define the objective
function and the problem constraints.
Continuous Optimization
– all the variables are allowed to take values from
subintervals of the real line;
Discrete Optimization
– require some or all of the variables to have integer values.
16. Constraints
• Objective Function
• Unknowns or Variables
• Constraints
Constraints are not essential.
Unconstrained optimization
– A large and important one for which a lot of algorithms
and software are available.
However, almost all problems really do have
constraints, e.g.,
– Any variable denoting the “number of objects” in a system
can only be useful if it is less than the number of
elementary particles in the known universe!
– In practice, though, answers that make good sense in
terms of the underlying physical or economic problem can
often be obtained without putting constraints on the
variables.
20. Some Well-Known Algorithms
for Shortest Path Problems
Dijkstra's algorithm — solves single source problem if all
edge weights are greater than or equal to zero.
Bellman-Ford algorithm — solves single source problem if
edge weights may be negative.
A* search algorithm — solves for single source shortest
paths using heuristics to try to speed up the search.
Floyd-Warshall algorithm — solves all pairs shortest paths.
Johnson's algorithm — solves all pairs shortest paths, may
be faster than Floyd-Warshall on sparse graphs.
Perturbation theory — finds the locally shortest path.
21. Maximum Flow Problem
Given: Directed graph G=(V, E),
Supply (source) node O, demand (sink) node T
Capacity function u: E R .
Goal:
– Given the arc capacities,
send as much flow as possible
from supply node O to demand node T
through the network.
Example:
4
4
5
6
4
4
5
5
O
A
D
B
C
T
22. Towards the Augmenting Path Algorithm
Idea: Find a path from the source to the sink,
and use it to send as much flow as possible.
In our example,
5 units of flow can be sent through the path O B D T
Then use the path O C T to send 4 units of flow.
The total flow is 5 + 4 = 9 at this point.
Can we send more?
O
A
D
B
C
T
4
4
5 6
4
4 5
5
5 5
4
4
5
23. Towards the Augmenting Path Algorithm
If we redirect 1 unit of flow
from path O B D T to path O B C T,
then the freed capacity of arc D T could be used
to send 1 more unit of flow through path O A D T,
making the total flow equal to 9+1=10 .
To realize the idea of redirecting the flow in a systematic
way, we need the concept of
residual capacities.
O
A
D
B
C
T
4
4
5 6
4
4 5
5
5 5
5
4
4
4
4
1 5
1 1
26. Hitchcock Transportation Problems
Find a minimal cost transportation.
A problem’s argument
11 12 1
21 22 2
1
1
1
2
1
4
2
2
1 2
1
2
n
m
n
m mn
j
n
s
s
s
n
c c c
c
s
d d d d
c c
c c c
T
m
Supply
Demand
11 12 1
21 22 2
1
1
1
2
1
4
2
2
1 2
1
2
n
m
n
m mn
j
n
s
s
s
x x x
x
s
x
d d d
x
T
x
x x
d
n
m
Supply
Demand
An answer
1 1
m n
i j
j
j i
i x
c
Minimize
27. Hitchcock Transportation Problems
Find a minimal cost transportation.
A problem’s argument
11 12 1
21 22 2
1
1
1
2
1
4
2
2
1 2
1
2
n
m
n
m mn
j
n
s
s
s
n
c c c
c
s
d d d d
c c
c c c
T
m
Supply
Demand
11 12 1
21 22 2
1
1
1
2
1
4
2
2
1 2
1
2
n
m
n
m mn
j
n
s
s
s
x x x
x
s
x
d d d
x
T
x
x x
d
n
m
Supply
Demand
An answer
1 1
m n
i j
j
j i
i x
c
minimize
28. Hitchcock Transportation Problems
1 1
m n
i j
j
j i
i x
c
Minimize
1
1
1, ,
1, ,
0 1, , and 1, ,
i
j
n
j
m
i
ij
ij
ij
i m
j n
i m j n
s
x
x
x
d
Subject to
A problem’s argument
11 12 1
21 22 2
1
1
1
2
1
4
2
2
1 2
1
2
n
m
n
m mn
j
n
s
s
s
n
c c c
c
s
d d d d
c c
c c c
T
m
Supply
Demand
11 12 1
21 22 2
1
1
1
2
1
4
2
2
1 2
1
2
n
m
n
m mn
j
n
s
s
s
n
c c c
c
s
d d d d
c c
c c c
T
m
Supply
Demand
11 12 1
21 22 2
1
1
1
2
1
4
2
2
1 2
1
2
n
m
n
m mn
j
n
s
s
s
x x x
x
s
x
d d d
x
T
x
x x
d
n
m
Supply
Demand
11 12 1
21 22 2
1
1
1
2
1
4
2
2
1 2
1
2
n
m
n
m mn
j
n
s
s
s
x x x
x
s
x
d d d
x
T
x
x x
d
n
m
Supply
Demand
An answer
29. Task Assignment Problems
Example: Machineco has four jobs to be completed. Each
machine must be assigned to complete one job. The time
required to setup each machine for completing each job is
shown in the table below. Machinco wants to minimize the
total setup time needed to complete the four jobs.
Problem
Instance
Time (Hours)
Job1 Job2 Job3 Job4
Machine 1 14 5 8 7
Machine 2 2 12 6 5
Machine 3 7 8 3 9
Machine 4 2 4 6 10
33. Task Assignment Problems
1 1
m m
i j
j
j i
i x
c
Minimize
1
1
1, ,
1, ,
0,1 1, , and 1,
1
,
1
ij
i
j
i
j
i
n
j
m
i m
j m
x
i m j
x
m
x
Subject to
45. Bin-Packing Problems
Determine how to put the given objects in
the least number of fixed space bins.
There are many variants, such as, 3D, 2D,
linear, pack by volume, pack by weight,
minimize volume, maximize value, fixed
shape objects, etc.
46. Bin-Packing Problems
Example: suppose we need a
number of pipes of different,
specific lengths to plumb a
house and we can buy pipe
stock in 5 meter lengths.
How can we cut the 5 meter
pipes to waste as little as
possible, i.e., to minimize the
cost of pipe?
