Добавил:
ivanov666
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Fundamentals of Operations Research. A textbook
.pdf
Heuristic Algorithms
111
and set
Iteration 2.
S1“ t3u
Calculate
. This is a feasible set of value
j2:
cpS1q “ 31 ă `8
.
cp3, 1q “ pmint3, 6u ` mint4, 1u ` mint6, 15u ` mint4, 9u ` mint2, 7u ` mint1, 4uq ` 11 ` 21
“ 49,
cp3, 2q “ pmint3, 2u ` mint4, 9u ` mint6, 2u ` mint4, 11u ` mint2, 23u ` mint1, 3uq ` 11 ` 16
“ 42,
cp3, 4q “ pmint3, 4u ` mint4, 11u ` mint6, 3u ` mint4, 8u ` mint2, 9u ` mint1, 5uq ` 11 ` 24
“ 52,
thus,
j2“ 2
algorithm stops with
. But
cp3, 2q ´ cp3q ě 0
SG“ t3u
.
, so, new element does not improve the solution and the
When applied to a particular problem, the greedy heuristic have to be adapted to the
problem's structure. For example, ¾go to the nearest unvisited city¿ strategy for the Traveling
Salesman Problem lies within the greedy paradigm, though it does not t the outline above.
9.1.2. Local Search
The idea of the Local Search heuristic is somewhat similar to that of the Greedy Algorithm.
Instead of searching for an optimal solution in the whole feasible set, a local search algorithm at
each its step examines a relatively small subset, a
neighborhood
of a given solution, and moves
from one neighborhoodoptimal solution to another, until a local optimum is achieved, or the
number of steps exceeds a given limiting value.
To describe the local search methodology formally, we rewrite a minimization combinatorial
problem (9.1) as follows:
min
tcpSq : gpSq “ 0u,
SĎN
where
gpSq ě 0
is a set function measuring
infeasibility
For a local search algorithm, we need to dene a
S Ď N
and a
goal function
fpSq
which may either be equal to
otherwise, or be a composite function such as
of a subset
local neighborhood
cpSq ` λgpSq
S Ď N
.
QpSq
cpSqifS
is feasible and innity
forλbig enough.
for any solution
(9.2)
Algorithm 9.2
Step 0.
Local Search Heuristic
Initialization
Choose an initial solutionS(non necessarily feasible).
Step 1.
Iteration
Find
if
no such
else
S1P QpSq
set
S “ S1and repeat iteration.
such that
S1exists
then
fpS1q ă fpSq
STOP:
.
S1is local optimum (which can also be infeasible),
Choice of particular neighborhood depends on the problem's structure. For example, a
neighborhood ofSmay consist of subsets in which a single element is added toSor removed
112
Approximation Algorithms
from it. For each solution, its neighborhood contains
enough to be enumerated. Such neighborhood can be used in BKP and other problems for
which a feasible solution may contain arbitrary number of elements.
In case all the feasible solutions have the same cardinality (which is quite common in graph
optimization problems), the following neighborhood can be used:
S, j R Su,|QpSq| “ Op|N |2q
Sometimes, two elements of a neighborhood must dier by at least two elements. TSP is a
good example: there is no tour that diers from a given one by a single edge, but if we remove
two edges, there exists exactly one tour that contains all the remaining edges. That is, when
solving TSP overnnodes by means of the Local Search heuristics, we can use neighborhood
and goal function
called a
can be arbitrarily far from the global one. To leave the neighborhood of a local optimum and
continue search for better solutions, a number of
are listed below.
9.1.3. Tabu Search
2optimal
The main aw of a local search algorithm is that it converges to a local optimum, which
fpSq
tour.
.
QpSq “ t
equal to the weight of cycleS. The resulting local search solution is
hamiltonian cycles
Op|N |q
S1: |S1X S| “ n ´ 2u
metaheuristics
elements and, usually, is small
QpSq “ tS1“ S Y jztiu : i P
was designed. Some of them
When at local optimum, it is natural to move to a secondbest solution in the neighborhood
even though its value is worse. An obvious diculty with this idea is that cycling may occur,
that is, the algorithm returns to the same solution after several iterations. To avoid cycling,
certain solutions or moves are declared forbidden (
The basic version of the tabu search is as follows.
Algorithm 9.3
Start with empty tabu list.
Find an initial solutionS.
repeat
Choose a subset
nd
S1“ argmintf psq : s P Q1pSqu
replaceSwith
until
Output:
1. the choice of subset
some termination conditions are satised.
To implement this heuristic, we must specify
bors (for a small neighborhood) or, when the neighborhood is large, may include a xed
number of solutions from
Tabu Search Heuristic
Q1pSq Ď QpSq
S1and update the tabu-list.
the best solution found by the end of the algorithm.
Q1pSq
of non-tabu solutions,
,
, which either can be equal to the whole set of non-tabu neigh-
QpSq
, chosen randomly or by some heuristic rule;
tabu
).
2. the tabu list, which must be, on the one hand, small enough for its examination does
not increase overall algorithm time consumption, and, on the other hand, big enough to
prevent cycling eectively;
Heuristic Algorithms
3. the termination rule: usually, an algorithm is stopped after performing a given number
of iterations, or after a certain number of iterations without improvement of the goal
function.
For example, for the swap neighborhood described above,
QpSq “ tS1: S1“ pSztiuq Y tju, i R S, j P Su,
113
a tabu list may containtlast excluded elements
for some relatively smallt. A solution
pSztiuq Y tjpu
for
p “ 1, . . . , t
.
ti1, . . . , itu
S1P QpSq
andtlast added elements
is tabu, if
S1“ pSztipuq Y tju
tj1, . . . , jtu
or
S1“
9.1.4. Simulated Annealing
While a tabu search algorithm moves to a point worse than the current best solution only after
considering a local optimum and placing it into tabu list, in many other heuristics the decision
to move towards a worse solution can be made at each point with some probability depending
on the value of the solution.
At each step of a
Simulated Annealing
algorithm, we choose an arbitrary neighbor of a
current solution and pass to it with probability 1, if the new solution is better than the initial
one. The probability to accept a worse solution depends on the dierence of values of two
solutions and to a parameter called
temperature
, which serves to ensure convergence of the
algorithm. The temperature decreases in time, thus, the probability to move towards a slightly
worse solution is relatively high at each stage of the algorithm, though it diminishes from one
stage to another. More formally, the outline of the heuristic can be expressed as follows.
Algorithm 9.4
Simulated Annealing Heuristic
Find an initial solutionS.
Choose an initial temperatureTand a cooling ratio
while
the temperature is high enough (
T ě T
min
r P p0, 1q
)
do
.
repeat
Choose an arbitrary
calculate
if
else
until
∆ ď 0
a required number of iterations are executed.
∆ “ fpS1q ´ fpSq
then
set
S “ S1with probability
Reduce the temperature:
Output:
the best solution found by the end of the algorithm.
set
S1P QpSq
S “ S
1
T “ r ¨ T
,
,
´∆{T
e
.
.
9.1.5. Genetic Algorithm
Rather that working on improvement of an
with a collection of solution called
population
individual
solution, the
Genetic Algorithm
. The population evolves in a way similar to that
of living species. Each iteration of the algorithm incudes the following stages.
Evaluation.
The tness of the solutions is calculated. Similarly to the goal function in
local search algorithms, the tness usually depends on the value of the solution and its
infeasibility measure.
operates
114
Approximation Algorithms
Parent Selection.
A number of solutions are selected as parents for subsequent crossover.
Usually, these are the ones with the highest tness. Also, solutions can be selected
randomly, with probabilities proportional to tness.
Crossover.
Collection of parent solutions is split into pairs, then, each pair is combined to
produce new solutions called osprings. A great variety of crossover operators is used in
practice. In most of the cases, parent solutions are encoded into strings in some alphabet,
then, they exchange substrings to obtain osprings. Several common crossover methods
are listed below.
1-point crossover.
The osprings are
2-point crossover.
t1, . . . , r ´ 1u,p ă q
y1. . . ypx
p`1
. . . xqy
Uniform crossover.
z1z2. . . zr, where
Mutation.
a solution
Some of the osprings are randomly modied. Often, under a mutation operator,
z1z2. . . zris changed by replacing a randomly selected symbol
symbolm. The result is
two random positionspandq: the result is
Given strings
x1. . . xpy
Given strings
x1x2. . . xrand
. . . yrand
p`1
y1y2. . . yr, and index
y1. . . ypx
x1x2. . . xrand
. . . xr.
p`1
y1y2. . . yr, and two indices
. The osprings are
. . . yr.
q`1
Given strings
x1x2. . . xrand
y1y2. . . yr. The ospring is string
ziis a randomly chosen symbol of pair
z1. . . z
p´1mzp`1
. . . zr. Another approach is to swap symbols in
z1. . . z
p´1zqzp`1
p P t1, . . . , r ´ 1u
x1. . . xpy
p`1
. . . yqx
txi, yiu,i “ 1, . . . , r
. . . z
q´1zpzq`1
q`1
zpwith random
. . . zr.
.
p, q P
. . . xrand
.
Population Selection.
A new generation is formed by replacing some or all of the elements
of the initial population with the osprings.
9.2. Algorithms with Performance Guarantees
It was said above that most of heuristic algorithms, though simple in formulation, are very
dicult in analysis, and their performance qualities are usually stated empirically, in a series of
computational experiments. Another approach to estimate performance of an algorithm solving
some optimization problem is
which holds for any instance of the problem.
As was mentioned in chapter 2, it is common to measure the performance of an algorithm
A
for problemΠby means of its absolute and relative accuracy:
αA“ sup
where
respectively, and
the
ApIq
and
OP T pIq
Π`Ď Π
ratio
of algorithmA. It is greater or equal to 1 for minimization problems and is less or
equal to 1 for maximization problems. The ratio is 1 for exact algorithms. Note that it is also
common to give an estimate in terms of
a priori analysis
, in which a theoretically proved estimate is given,
ApIq
|ApIq ´ OP T pIq|, ρA“ sup
IPΠ
IPΠ
OP T pIq
`
,
are values of the solution constructed byAand the optimal solution,
is the set of instances with positive optima. Value
relative error
εAinstead of relative accuracy:
ρAis often called
εA“ sup
IPΠ
|ApIq ´ OP T pIq|
`
OP T pIq
, ρA“ 1 ˘ εA,
Algorithms with Performance Guarantees
115
the plus sign is used for minimization problems and the minus sign for maximization problems.
A solution, the value of which is within aKfactor of the optimal value, is calledKoptimal
or
suboptimal
performance guarantees
. An algorithm, for which a performance estimate is obtained, is said to have
.
Denition 9.1.
mation scheme
We call a parametric family of algorithms
forΠ, if for each
ε ą 0
tAεu
for problemΠan
εą0
approxi-
|AεpIq ´ OP T pIq|
ď ε, @I P Π.
OP T pIq
Denition 9.2.
ε ą 0
is xed, the time-complexity is polynomial in input length:
polynomial-time(F P TAS
T
“ O`pol`n,
A
ε
That is, a PTAS may have time-complexity
An approximation scheme
˘˘
1
.
ε
), if its time-complexity is polynomial in both input length and
tAεu
is called
εą0
O´n22
polynomial-time(P T AS
T
“ Oppolpnqq
A
ε
¯
1
ε
or even
O
´
`
˘
n
ε
), if when
, and
¯
1
ε
, which becomes
fully
1
ε
enormous whenεis close to zero, but is still polynomial innfor each xedε. An FPTAS demon-
strates a moderate increase in complexity whenεapproaches zero: a typical time-complexity
´
is
O
¯
`
˘
3
n
.
ε
In this section, we conduct a priory analysis of approximation algorithms for the Knapsack
Problem and the Metric Traveling Salesman Problem (that is, TSP with a symmetrical matrix
and distances satisfying the triangle inequality).
9.2.1. WorstCase Analysis of Approximation Algorithms
:
Linear Knapsack Problem
Estimate performance guarantee of the greedy heuristics applied to the Linear Knapsack Prob-
lem (LKP)
where
cją 0
for eachj, and values
max
xPZ`
#
n
ÿ
cjxj:
n
j“1
aj,
j “ 1, . . . , n
n
ÿ
ajxjď A+,
j“1
, andAare integers. Without loss of
(9.3)
generality we assume
• ajď A
c
1
•
a
1
The solution provided by the greedy algorithm
for eachj, and
c
j
ě
for
a
j
j “ 2, . . . , n
.
AGfor instanceIis
xG“ ptA{a1u, 0, . . . , 0q, AGpIq “ pc, xGq “ c1tA{a1u.
(9.4)
Clearly, the algorithm has linear time-complexity.
Theorem 9.1.
For any instanceIof the Linear Knapsack Problem (9.3), the inequality holds
AGpIq
OP T pIq
ě
1
.
2
116
Approximation Algorithms
Proof.
Note that vector
xL“ pA{a1, 0, . . . , 0q
is the optimal solution for the instance of the
problem with relaxed integrality condition. That implies,
c1A
Let
A{a1“ tA{a1u ` f,0 ď f ă 1
A
ď
a
1
OP T pIq ď OP TLpIq “ pc, xLq “
. Since
Z
^
A
` f ă
a
1
a1ď A,tA{a1u ě 1
Z
^
A
Z
A
` 1 ď
a
1
a
1
^`Z
.
a
1
, and, therefore,
^
Z
A
^
A
“ 2
a
1
a
1
.
Thus,
Remark
AGpIq
OP T pIq
9.1.Though the greedy algorithm gives us a solution of LKP, the value of which is at
AGpIq
ě
OP TLpIq
c1A{a
“
c1tA{a1u
1
1
ě
.
2
least 50% of the optimum, a greedy solution of the Boolean Knapsack Problem (BKP) can be
arbitrarily far from the optimal one.
Consider the following instance of BKP:
n “ 2,c “ pk, kA ´ kq,a “ p1, Aq
for some
k P N
Greedy policy prescribes to choose the rst item, since
but the value of solution
x˚“ p0, 1q
.
xH“ p1, 0q
c
1
“ k ą k ´
a
1
is`1 ´
k
c
2
“
A
˘
1
A
times less than that of the optimal solution
k
,
a
2
.
Metric Traveling Salesman Problem
As was shown in chapter 2, for any given
k ą 1
, there are no polynomial-time algorithms
constructing akapproximate solution for the general Traveling Salesman Problem. Below we
see that adding metricity conditions makes the problem
Denition 9.3.
We call an unordered graph
G “ pV, Eq
3{2
approximable.
Eulerian
, if degrees of all its nodes
are even.
Lemma 9.1.
Each connected Eulerian graph possesses a
Eulerian circuit
, that is, a circuit that
passes each edge of the graph exactly once.
Theorem 9.2.
weights of the edges satisfy the triangle inequality. Let
H
. There is a hamiltonian cycle of length at most
Proof.
Let
LetHbe a complete edge-weighted undirected graph with node setV, such that
G “ pV, Eq
ř
be a Eulerian subgraph of
ceinH.
ePE
m “ |E|,n “ |V |
, and
C “ tv0, e1, v1, e2, . . . , em, vmu
, where
v0“ vm, be a Eulerian
circuit inG. Clearly, the circuit passes each node of the graph, and some of the nodes may
occur inCmore than once. Let
C
. Consider hamiltonian cycle
From the triangle inequality it follows that the length of edge
exceed the length of the path connecting
v
, v
i
1
C1“ tv
, . . . , v
i
2
, v
i
1
c
f
i
2
v
i
ě
k
be the list of rst occurrences of each node in
i
n
, . . . , v
and
k
j“ik`1
i
k`1
ÿ
, v
i
i
n
v
i
k`1
c
.
e
j
u
. Estimate its length.
1
fk“ pv
in circuitC:
, v
i
i
k
k`1
q
does not
Algorithms with Performance Guarantees
2
b
1
b
3
b
4
b
5
b
2
b
1
b
3
b
4
b
5
b
2
b
1
b
3
b
4
b
5
b
117
That yields the following estimate on the length of cycle
n
ÿ
k“1
c
ď
f
k
n
ÿ
k“1
i
k`1
ÿ
j“ik`1
c
e
j
“
C1:
ÿ
ce.
ePE
The theorem above allows to formulate an approximate algorithm for Metric TSP.
Algorithm 9.5A2
Given:
a complete undirected graph
approximation Algorithm For Metric TSP
G “ pV, Eq
with edge-weights satisfying the triangle
inequality.
Step 1.
Step 2.
Step 3.
Find a minimum-weight spanning treeTinG.
Duplicate each edge inTto obtain a connected Eulerian graph
Find a Eulerian circuit in
T1and construct a hamiltonian cycle as in the proof of
T1.
Theorem 9.2.
(a) A spanning tree
T
(b) Eulerian graph
cuit
t1, 4, 3, 2, 3, 5, 3, 4, 1u
T1with cir-
(c) A cycle related to the cir-
cuit
Figure 9.1: Implementation of Algorithm 9.5
Note that the most dicult stage of Algorithm 9.5 is nding a minimum spanning tree, so,
the complexity of the algorithm is
Theorem 9.3.
For any instanceIof Metric TSP, the estimate holds:
Opn2q
.
ApIq
ď 2,
OP T pIq
where
ApIq
is the value of solution obtained by Algorithm 9.5.
ř
Proof.
cycle
is a spanning tree. Thus,
second step of the algorithm is
Easy to see, value
OP T pIq
. Indeed, removing any edge from the cycle we obtain a hamiltonian path, which
cT“
ceis a lower bound for the length of the shortest hamiltonian
ePT
OP T pIq ě cT. The weight of Eulerian circuit constructed at the
2cT, and, due to the triangle inequality,
ApIq ď 2cT. That
implies,
ApIq
OP T pIq
ď
2c
T
“ 2.
T
c
118
2
b
1
b
3
b
4
b
5
b
2
b
1
b
3
b
4
b
5
b
2
b
1
b
3
b
4
b
5
b
Approximation Algorithms
The approach described above can be used to obtain other approximation algorithms, it is
sucient to replace the doubled spanning tree with some other spanning Eulerian subgraph. In
1970ies, Nicos Christodes and Anatoly Serdyukov proposed an algorithm with better perfor-
mance guarantee, in which a combination of a minimum-weight spanning tree an a minimum-
weight matching is used.
Algorithm 9.6
Given:
a complete undirected graph
inequality.
Step 1.
Find a minimum-weight spanning treeTinG. Let
degree inT.
Step 2.
Construct
weight perfect matching
Step 3.
Find a Eulerian circuit in
of Theorem 9.2.
(a) A spanning tree
3
A
approximation Algorithm For Metric TSP
2
G1“ pV1, E1q
G “ pV, Eq
such that
E1“ tpi, jq P E : i, j P V1u
with edge-weights satisfying the triangle
MinG1.
M Y T
T
(b) Eulerian graph
circuit
and construct a hamiltonian cycle as in the proof
M Y T
t1, 3, 2, 5, 3, 4, 1u
with
V1Ď V
be a set of nodes of odd
and nd a minimum-
(c) A cycle related to the cir-
cuit
Figure 9.2: Implementation of Algorithm 9.6
Theorem 9.4.
where
Proof.
in
ApIq
Similar to that of Theorem 9.5. Note that
G1. Graph
For any instanceIof Metric TSP, the estimate holds:
ApIq
OP T pIq
ď
3
,
2
is the value of solution obtained by Algorithm 9.6.
|V1|
is even, thus, a perfect matching exists
M Y T
is Eulerian, since each odd-degree node inTis incident to an additional
edge ofM. Thus, the algorithm is correct.
Estimate its performance ratio. For the matter of simplicity, we assume nodes of the graph
to be numbered such that
of nodes of
V1in increasing order. Consider two matchings
t1, 2, 3, . . . , n, 1u
is the optimal cycle. Let
M1“ tpi1, i2q, pi3, i4q, . . . , pi
M2“ tpi2k, i1q, pi2, i3q, . . . , pi
2k´1
2k´2
, i2kqu,
, i
2k´1
ti1, i2, . . . , i2ku
qu
be the list
Algorithms with Performance Guarantees
in
G1. Since
M
is a minimum-weight matching in this graph,
2k
2cMď z
M
1
` c
M
ÿ
2
“
c
il,i
l“1
l`1
ď
2k
ÿ
l“1
i
l`1
ÿ
j“il`1
c
j,j`1
cMď c
n
ÿ
“
j“1
M
c
j,j`1
M
1
2
, c
, and, therefore,
“ z˚.
119
As was noticed before,
cTď OP T pIq
. Also,
ApIq ď cT` cMdue to the triangle inequality.
Thus,
ApIq ď cT` cMď OP T pIq `
OP T pIq
2
3OP T pIq
“
.
2
9.2.2. Approximation Schemes: Existence and Nonexistence
A fully polynomial-time approximation scheme is, in some sense, the best we can hope for when
dealing with an
N P
-hard problem. Here, we give two observations concerning the question of
existence of an FPTAS for a problem. The rst and disappointing one is that almost all strongly
N P
hard problems do not admit FPTAS.
Theorem 9.5.
exists a polynomial
there is no FPTAS forΠ, unless
Proof.
algorithm
Assume an FPTAS
Aεconstructs a solution such that
LetΠbe a strongly
qp¨, ¨q
such that
P “ N P
tAεu
exists. Fix
N P
hard integer minimization problem. Assume there
OP T pIq ă qpLpIq, N pIqq
for each instance
I P Π
.
I P Π
and set
ε “ 1{qpLpI q, NpI qq
. For chosenε,
. Then
OP T pIq
AεpIq ď p1 ` εq ¨ OP T pIq ă OP T pIq `
“ OP T pIq ` 1.
qpLpIq, NpIqq
Due to integrality of the problem, the solution is optimal. Thus, the algorithm nds an exact
solution in time
O`pol`LpIq,
˘˘
1
“ O ppol pLpIq, qpLpIq, NpIqqqq “ OppolpLpIq, N pIqqq
ε
, which
is pseudopolynomal.
Remark
lengths 1 and 2 by reduction from the Hamiltonian Cycle Problem (which is
strong sense). Note that TSP with such weights is metric, so, Metric TSP is strongly
9.2.In chapter 2 (example 2.3), we demonstrated
N P
completeness of TSP with edge-
N P
complete is
N P
hard. Conditions of Theorem 9.5 are, obviously, satised, thus, Metric TSP has no FPTAS if
P ‰ N P
.
What about PTAS? The answer is no again. The argument similar to used in Theorem
2.3 leads to the following conclusion: given
providing a cycle of length`1 `
˘
1
-times optimal for an instance of TSP with edge-lengths 1
n
P ‰ NP
, there is no polynomial-time algorithm
and 2 overnnodes.
That is, if no pseudopolynomial algorithm exists for a problem, it is very likely that neither
does FPTAS. On the other hand, sometimes, if a pseudopolynomial algorithm is known, it can
be used as a basis for FPTAS. Consider the Boolean Knapsack Problem as an example.
Algorithm 9.7
Given:
Choose
for
an instanceIwith weights
ε ą 0
j “ 1ton
Solve the instance
FPTAS for the Boolean Knapsack Problem
and set
do
K “ εa
set
I1with weights
taj: j “ 1, . . . , nu
{n
.
max
X
\
a
“
j
.
K
1
a
using DP-algorithm from chapter 3.
j
1
a
j
.
120
Approximation Algorithms
Theorem 9.6.
sack Problem in
Proof.
Consider an instanceIof the problem. Again, without loss of generality assume
for eachj. That implies
LetSand
Algorithm 9.7 constructs a
´
¯
3
n
O
time.
ε
OP T pIq ě a
Y
]
a
1
a
“
j
K
a
j
“
K
S1be optimal sets of items forIand
ÿ
OP T pIq “
AεpIq “
jPS
ÿ
jPS
aj“ K
ajě K
1
From the inequalities above, it follows
OP T pIq ´ AεpIq ď Kn “ ε ¨ a
The complexity of
DP
-algorithm applied to
p1`εq
, and
max
j
´ fj, 0 ď fjă 1, j “ 1, . . . , n.
ÿ
jPS
ÿ
jPS
approximation solution of the Boolean Knap-
I1. Estimate
1
a
` K
j
1
a
.
j
1
ÿ
fjď K
jPS
max
OP T pIq
ÿ
1
a
j
1
jPS
ď ε ¨ OP T pIq.
´
I1is
Opn2a
1
max
q “ O
n
ε
and
` Kn,
¯
3
.
AεpIq
ajď A
:
To learn more about methods to construct and analyze approximation schemes, see [7].
Соседние файлы в предмете [НЕСОРТИРОВАННОЕ]