Bradley, Manna. The Calculus of Computation, Springer, 2007
.pdf220 8 Quantifier-Free Linear Arithmetic
1.The point x = 0, z = 0 satisfies constraints (1)-(3). It is a vertex.
• It satisfies constraints (2) and (3), for
D10 = 0 ≤ g1 and D20 − 0 = 0 ≤ g2 .
The inequalities hold since g1 ≥ 0 and g2 > 0 according to our construction of g1 and g2.
•It satisfies constraints (1). Indeed, constraints (1) are its defining constraints.
2.The optimum equals vG = 1Tg2 i G is TQ-satisfiable.
•If the optimal value is 1Tg2 at optimal point x , z , then x satisfies G. For
D1x ≤ g1
by (2); and
D2x − z = g2
by constraint (3) of M0 and that the optimal value is 1Tg2, so
D2x = g2 + z ≥ g2
by (1). By construction of D1, D2, g1, and g2, we thus know that Ax ≤ b, so G is satisfiable.
•If x satisfies G, then the optimal value is at least 1Tg2. We have
D1x ≤ g1 , D2x ≥ g2 , and x ≥ 0 .
Choose z = D2x − g2. Then z ≥ 0 and D2x − z = g2, so that all constraints of M0 are satisfied. Additionally, 1T(D2x − z ) = 1Tg2, so that the maximum of M0 is at least 1Tg2. Constraint (3) limits the maximum to 1Tg2.
Because the optimum equals 1Tg2 i G is TQ-satisfiable, we need only solve the optimization problem M0 to determine the satisfiability of G and to find a vertex of G if it is satisfiable. We already have a satisfying vertex x = 0, z = 0 of M0 by characteristic 1.
Example 8.7. Consider the ΣQ-formula
F : x + y ≥ 1 x − y ≥ −1 .
Because F has only weak inequality literals, the corresponding linear program has a constant objective function:
M : max 1 subject to
x + y ≥ 1 x − y ≥ −1
G
8.4 The Simplex Method |
221 |
Solving the corresponding linear program M0 is su cient to determine the TQ-satisfiability of F .
To convert F to the form x ≥ 0 Ax ≤ b, introduce nonnegative x1, x2 for x and y1, y2 for y:
F ′ : (x1 − x2) + (y1 − y2) ≥ 1 (x1 − x2) − (y1 − y2) ≥ −1
x1, x2, y1, y2 ≥ 0
F is TQ-equisatisfiable to F ′. In matrix form, the first two literals of F ′ are
|
1 1 |
1 |
|
1 |
x2 |
|
|
1 |
|
|
|
|
|
|
|
|
|
|
|||
−1 1 −1 |
|
1 |
|
x1 |
|
≤ |
−1 |
|
|
|
|
|
|
|
|
|
|
||||
− |
|
y1 |
|
|
|
|
|
|
|
|
|
|
|||||||||
− |
|
|
|
y |
2 |
|
b |
|
|
|
|
|
|
|
|
|
|
||||
|
A |
|
|
|
|
|
|
|
| |
|
|
|
|
|
|
|
|
|
|
||
| {z } |
|
|
|
|
{z } |
|
|
|
|
|
|
|
|
|
|
||||||
Since b1 < 0 and b2 > 0, separating constraints yields |
|
|
|
|
|||||||||||||||||
|
1 1 1 |
|
1 |
|
x2 |
|
|
[1] |
and |
1 |
1 1 |
|
1 |
|
x2 |
|
[1] . |
||||
− |
|
− |
|
x1 |
|
≤ g1 |
|
|
−D2 |
− |
|
|
|
x1 |
|
≥ |
|
||||
D1 |
y2 |
|
|
y2 |
g2 |
||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| {z } |
y1 |
|
|
|{z} |
| |
{z } |
y1 |
|
|
|{z} |
|||||||||||
|
|
|
|
D2 has only one row, so z = [z]. According to (8.1), pose the following optimization problem:
max 1 |
|
1 1 |
|
1 |
x2 |
|
|
|
[z] |
|
|
|
|
|
|
|
|
||
|
− |
|
− |
|
x1 |
|
− |
|
|
|
|
|
|
|
|
|
|
||
|
y2 |
|
|
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
y1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
subject to |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
x1, x2 |
, y1, y2, z ≥ 0 |
|
|
|
|
|
|
||||||||
|
|
|
1 1 1 |
|
1 |
x2 |
|
|
[1] |
|
|
|
|
|
|
||||
|
|
− |
|
|
|
|
|
|
x1 |
|
≤ |
|
|
|
|
|
|
|
|
|
|
|
|
− y2 |
|
|
|
|
|
|
|
||||||||
|
|
|
|
|
|
x1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
y1 |
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
1 |
− |
1 1 |
− |
1 |
x2 |
|
− |
[z] |
≤ |
[1] |
|
|
|
|
|
|
|||
|
|
|
y2 |
|
|
|
|
|
|
|
|
|
|
||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
y1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
F is TQ-satisfiable i the optimum is vG = |
1Tg2 |
= 1. |
|
≥ |
|||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 2 |
|||
We know that the point |
|
x1 x2 y1 y2 z |
|
T = |
0 0 0 0 0 |
T |
is a vertex. It |
satisfies all constraints and has defining constraints x , x , y1, y2, z 0. |
|
The simplex method requires M0 to be in standard form |
|
222 8 Quantifier-Free Linear Arithmetic
max cTy subject to
Ay ≤ b
Let |
|
z . |
|
|
|
|
|
|
|
||
y = |
|
|
|
|
|
|
|
||||
|
|
|
x |
|
|
|
|
|
|
|
|
Since the objective function of M0 is |
|
||||||||||
1 (D2x − z) = 1 |
D2 |
−I |
z |
, |
|||||||
T |
|
|
|
T |
|
|
|
|
x |
|
|
|
|
|
|
|
|
|
|
|
|
|{z} |
|
|
|
|
|
|
|
|
|
|
|
y |
|
construct vector c as follows: |
|
|
|||||||||
cT = 1T D2 −I . |
|
|
|
|
|
||||||
The constraints have the form |
|
|
|||||||||
−I |
|
I |
x |
|
0 |
, |
|
|
|||
|
|
|
|
|
z ≤ |
|
0 |
|
|
|
|
|
D1 |
− |
I |
|
g1 |
|
|
|
|||
D |
2 |
|
y |
g |
2 |
|
|
||||
|
|
− |
|
|
|
|
|
|
|||
|
|
|
|
|{z} |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
where blank regions of the matrix are filled with 0s. Hence, M0 standard form is written
M0 : max 1 |
|
|
|
D2 −I |
|
z |
|
|
|
|
||||
|
T |
|
|
|
|
|
|
|
x |
|
|
|
|
|
| {z } |{z} |
|
|
|
|
||||||||||
|
|
|
|
|
|
|
|
|
|
y |
|
|
|
|
subject to |
|
I |
|
x |
|
|
0 |
|
||||||
|
−I |
|
|
|
||||||||||
|
|
|
|
|
|
|
|
z |
≤ |
|
0 |
|
||
|
|
D1 |
− |
|
|
g1 |
||||||||
D |
2 |
|
I |
|
y |
|
g |
2 |
||||||
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
− |
|{z} |
|
|
|
|
|||||
|
|
|
|
A |
|
|
|
b |
|
|||||
| {z } |
|
|
|
| {z } |
of (8.1) in
(8.2)
Example 8.8. According to (8.2), rewrite the optimization problem of Example 8.7 as follows:
|
|
|
|
|
|
|
x2 |
|
|
|
|
|
|
|
8.4 The Simplex Method |
223 |
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
|
x1 |
|
|
|
|
|
|
|
|
|
|
max |
1 −1 1 −1 −1 |
y1 |
|
|
|
|
|
|
|
|
||||||||
|
|
|
|
cT |
|
|
|
|
2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
z |
|
|
|
|
|
|
|
|
|
|
| |
|
{z } |
|
y |
|
|
|
|
|
|
|
|
|
||||
subject to |
|
1 |
0 |
0 |
|
|
|
|
x1 |
|
|
0 |
|
|||||
|
|
−0 |
|
0 |
|
|
|
|
||||||||||
|
|
1 |
|
0 |
0 |
0 |
0 |
|
x2 |
|
|
|
0 |
|
|
|||
|
0 −0 1 |
0 |
0 |
|
0 |
|
||||||||||||
|
0 |
|
0 |
−0 |
1 |
0 |
|
y |
1 |
|
|
0 |
|
|||||
|
|
|
|
|
|
−0 |
|
|
|
y |
|
≤ |
|
|
|
|
||
|
|
0 |
|
0 |
0 |
1 |
|
2 |
|
0 |
|
|
||||||
|
|
1 |
|
1 |
1 |
1 −0 |
|
|
|
|
|
|
1 |
|
|
|||
|
|
|
|
|
z |
|
|
|
|
|
||||||||
|
|
−1 |
|
1 |
1 |
−1 |
1 |
|
|
|
|
|
|
|
1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
− |
|
|
− − |
|
|
|
|
|
|
|
|
|
|||
|
| {z } |
|
|
|
|
|
| {z } |
|
||||||||||
|
|
|
|
|
A |
|
|
|
|
|
|
|
|
|
|
b |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
8.4.2 Vertex Traversal |
|
|
|
|
|
|
|
|
|
|
|
|
||||||
Assume that we have a generic optimization problem |
|
|||||||||||||||||
max cTx |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
(8.3) |
||
subject to |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
Ax ≤ b |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
for which we have a satisfying vertex vi. The simplex method traverses vertices of Ax ≤ b to find the vertex v that maximizes cTx. In particular, one iteration of this step seeks a vertex vi+1 adjacent to vi such that cTvi+1 > cTvi.
For the optimization problem M0 of (8.2), the point x = 0, z = 0, which satisfies the constraints of M0, is the initial vertex v1.
Example 8.9. The point v1 = |
x1 x2 y1 y2 z T = |
0 0 0 0 0 |
|
T is a vertex of |
|
the constraints of the |
optimization problem of Example 8.8. |
|
|||
|
|
|
|
Construction of u
To begin the ith iteration, we use the vertex vi to construct a vector u such that uTA = cT. If u ≥ 0 then the Duality Theorem (Theorem 8.6) implies that vi is optimal, as we discuss below, and the process terminates. However, in all but the final iteration, u 6≥0: at least one row of u is negative.
To construct u, choose one of its sets of defining constraints: choose a n×n nonsingular submatrix Ai of A with corresponding rows bi such that
Aivi = bi . |
(8.4) |
Let R be the indices of the rows of A in Ai. Such a subset of constraints exists because vi is a vertex.
224 8 Quantifier-Free Linear Arithmetic
x2 |
|
|
|
cT x |
|
• |
x1 |
|
v1 |
||
|
Fig. 8.2. Illustration of Example 8.10
Second, solve uTA = cT for u: solve
AiT ui = c |
(8.5) |
for ui (which has a solution because Ai is nonsingular), and let u be ui for indices in R with 0s added for rows with indices not in R. Thus,
uTA = cT . |
(8.6) |
Example 8.10. Consider the optimization problem of the form (8.3)
max [−1 1] x |
|
|
|
|
||
| |
{z } |
|
|
|
|
|
|
cT |
|
|
|
|
|
subject to |
1 |
x |
|
0 |
|
|
|
−0 |
|||||
|
1 |
0 |
|
≤ |
0 |
|
2 |
−1 |
2 |
||||
| {z } |
| {z } |
Ab
for which we know that v1 = 0 0 T is a vertex.
The problem and initial vertex is visualized in Figure 8.2. The solid lines represent the constraints of the problem, and the set of satisfying points corresponds to the interior of the triangle. The dashed line indicates cTx; the arrow points in the direction of increasing value.
Given vertex v1 = 0 0 |
T, the first two constraints are the defining con- |
||||||
straints of v1, so |
choose R = [1; 2]: |
|
|||||
|
|
|
|
|
. |
||
A1 = −0 |
−1 |
and |
b1 = |
0 |
|||
1 |
0 |
|
|
|
0 |
|
From (8.5), solving
8.4 The Simplex Method |
225 |
−0 −1 u1 |
= |
−1 |
|
1 |
0 |
|
1 |
| {z } |
|
| {z } |
|
A1 |
T |
|
c |
|
|
|
for u1 (via, for example, Gaussian elimination; or by observing that A1T = −I
and that −Iu1 = c implies that u1 = −c) yields u1 = 1 −1 T. Adding 0s for rows not in R produces
u = 1 −1 0 T , |
T |
|
T |
where the first two elements are from u1. Check that this u satisfies u A = c |
|
||
of (8.6) as desired. |
|
|
Example 8.11. Continuing from Examples 8.8 and 8.9, choose the first five rows of A and b (R = [1; 2; 3; 4; 5]) since
−0 −1 |
0 |
0 |
0 |
0 |
|
0 |
|
|
||||||||
|
1 |
0 |
0 |
0 |
0 |
|
|
0 |
|
|
|
0 |
|
|
|
|
0 |
0 −0 1 |
0 |
0 |
= |
0 |
|
|
|||||||||
|
0 |
0 |
0 |
−0 |
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
|
0 |
|
|
0 |
|
|
||||||||
|
0 |
0 |
1 |
0 |
0 |
|
|
0 |
|
|
|
0 |
|
|
|
|
|
|
|
|
|
− |
|
|
|
|
|
|
|
|
|
|
|
| {z } | {z } |
|
| {z } |
|
|
||||||||||||
|
|
|
A1 |
|
|
|
v1 |
|
|
|
b1 |
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
as desired from (8.4). From (8.5), solving |
|
|
||||||||||||||
−0 −1 |
0 |
0 |
0 |
|
|
|
−1 |
|
|
|||||||
|
1 |
0 |
0 |
0 |
0 |
u1 = |
|
1 |
|
|
|
|||||
0 |
0 −0 1 |
0 |
1 |
|
|
|||||||||||
|
0 |
0 |
0 |
−0 |
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
|
|
|
−1 |
|
|
|||||||||
|
0 |
0 |
1 |
0 |
0 |
|
|
|
|
|
1 |
|
|
|
||
|
|
|
|
|
− |
|
|
|
− |
|
|
|
|
|||
| {z } |
|
|
|
| {z } |
|
|
||||||||||
|
|
|
T |
|
|
|
|
|
|
|
c |
|
|
|
|
|
|
|
|
A1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
yields |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
u1T = −1 1 −1 1 1 . |
|
|
|
|
|
|
|
|
|
|||||||
Then |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
u = −1 1 −1 1 1 0 0 T , |
|
|
|
|
T |
|
T |
|||||||||
where the first five elements are from u1. Check that this u satisfies u A = c |
|
|||||||||||||||
of (8.6) as desired. |
|
|
|
|
|
|
|
|
|
|
|
There are two cases to consider: either u ≥ 0 or u 6≥0.
226 8 Quantifier-Free Linear Arithmetic
Case 1: u ≥ 0
We prove that in this case, vi is actually the optimal point with optimal value cTvi. The crux of the argument is the Duality Theorem (Theorem 8.6).
From equation (8.6), we have
cTvi = uTAvi .
We claim next that
uTAvi = uTb . |
(8.7) |
First, from (8.4)
Aivi = bi implies uiTAivi = uiTbi
so that equation (8.7) holds at rows R. For rows j 6 R, we know that uj = 0 by construction, so that both
(uTAvi)j = 0 and (uTb)j = 0 ,
proving equation (8.7). Reasoning further,
uTb ≥ min{yTb : y ≥ 0 yTA = cT}
since u is a member of the set by (8.6) and the case u ≥ 0. By duality (Theorem 8.6),
min{yTb : y ≥ 0 yTA = cT} = max{cTx : Ax ≤ b} .
In summary, we have by (8.6), (8.7), and Theorem 8.6,
cTvi = uTAvi = uTb ≥
min{yTb : y ≥ 0 yTA = cT} = max{cTx : Ax ≤ b} ,
which proves that vi is actually the optimal point with optimal value cTvi. Figure 8.3 illustrates this case: the vertex vi maximizes cTx. The dashed
line illustrates the objective function cTx. Moving upward relative to it increases its value. In this illustration, cTx cannot be increased without leaving the region defined by the constraints.
Case 2: u 6≥0
In this case, vi is not the optimal point. Thus, we need to move along an edge y to an adjacent vertex to increase the value of the objective function. In moving to an adjacent vertex, we swap one of the defining constraints of vi for another constraint to form the defining constraints of vi+1.
In this second case, there exists some uk < 0. Let k be the lowest index of u such that uk < 0 (it must be one of the indices of R since for all other indices ℓ, uℓ = 0). Let k′ be the index of the row of ui and Ai corresponding to row k of u and Ai.
8.4 The Simplex Method |
227 |
•vi
Ax ≤ b |
cTx |
Fig. 8.3. Case 1
Construction of y
Having fixed the indices k (row k of A and b) and k′ (the corresponding row k′ of Ai and bi) of the o ending constraint, we seek a direction along which to travel away from vertex vi and, in particular, away from the plane that defines the kth constraint.
Define y to be the k′th column of −A−i 1. To find y, solve
Aiy = −ek′ |
(8.8) |
(where, recall, ek′ is the k′th unit vector, which consists of 0s except in position k′) for y. Thus,
aℓy = 0 for every row aℓ of Ai except the k′th row
and
ak′ y = −1 for the k′th row ak′ of Ai .
The vector y provides the direction along which to move to the next vertex. For all rows aℓ but the k′th row of Ai, aℓy = 0 implies that moving in direction y stays on the boundary of the constraints (i.e., both aℓvi = b and, in the next step, aℓvi+1 = b), so aℓ will be a row in Ai+1. However, moving along y moves inward from the boundary of the k′th constraint because ak′ y = −1. This change is desirable, as this constraint is keeping u from being nonnegative.
Example 8.12. Let us examine
u1 = |
1 −1 |
T and u = |
1 −1 0 |
T |
of Example 8.10. Since the second row of u is −1, we are in Case 2 with corresponding to row k′ = 2 of u1. Let y be the second column of −A−1 1
−0 −1 y = |
−1 |
|
1 |
0 |
0 |
| {z } |
| {z } |
k = 2, : solve
A1 |
−e2 |
228 8 Quantifier-Free Linear Arithmetic
x2 |
|
|
• v2 |
|
|
|
cT x |
|
y |
|
|
• |
x1 |
|
v1 |
||
|
Fig. 8.4. Illustration of Example 8.12
for y, yielding y = 0 1 T.
This y is visualized in Figure 8.4 by the dark solid arrow that points up from v1. The vertical and horizontal lines are the defining constraints of v1; in moving in the direction y, we keep the vertical constraint for the next vertex v2 but drop the horizontal constraint. The diagonal constraint will become
the second of v2’s defining constraints. |
|
|
Example 8.13. Let us examine |
T |
|
u1 = −1 1 −1 1 1 |
T and u = −1 1 −1 1 1 0 0 |
of Example 8.11. Since the first row of u is −1, k = 1, corresponding to row k′ = 1 of u1. Thus, solve
−0 −1 |
0 |
0 |
|
0 |
−0 |
|||||
|
1 |
0 |
0 |
0 |
|
0 |
y = |
|
1 |
|
0 |
0 −0 1 |
|
0 |
0 |
||||||
|
0 |
0 |
0 |
−0 |
|
|
|
|
|
|
|
|
1 |
|
0 |
||||||
|
0 |
0 |
1 |
0 |
|
0 |
|
|
0 |
|
|
|
|
|
|
− |
|
|
|
||
| {z } |
|T |
{z } |
||||||||
|
|
|
A1 |
|
|
|
|
−e1 |
|
|
for y, yielding y = |
1 0 0 0 0 |
|
. |
|
Again we have two cases to consider: either the optimum is bounded (Case 2(a)) or it is unbounded (Case 2(b)).
Case 2(a): Optimum is Bounded
In this case, we move along the edge y to a better vertex vi+1, according to the objective function cTx. However, there is a set of rows of A with indices
8.4 The Simplex Method |
229 |
S such that for ℓ S, aℓy > 0. For these constraints, moving in the direction y actually moves toward leaving the satisfying region. These constraints limit how far in direction y we can move. For example, in Figure 8.4, the diagonal constraint limits how far we can move in direction y.
Construction of λi, vi+1 |
|
We want to solve for the greatest λi ≥ 0 such that |
|
A(vi + λi y) ≤ b . |
(8.9) |
Thus, choose λi > 0 such that for some row aℓ, ℓ S, |
|
aℓ(vi + λiy) = bℓ |
(8.10) |
and for all other rows m S \ {ℓ}, |
|
am(vi + λiy) ≤ bm . |
(8.11) |
Set |
|
def |
(8.12) |
vi+1 = vi + λiy . |
Finally, we construct the defining constraints of vi+1 for the next iteration. Construct submatrix Ai+1 of A from Ai: replace row ak′ of Ai with row aℓ of A. Choose the corresponding rows of b for bi+1. Row ℓ and the rows carried over from this iteration comprise the defining constraints of vi+1.
Example 8.14. Continuing from Example 8.12, choose λ1 such that
A(v1 + λ1y) ≤ b , |
|
|
|
|
|||||
specifically |
|
|
|
|
|
|
|
|
|
−0 |
1 |
|
|
0 |
+ λ1 |
1 |
≤ |
|
0 , |
2 |
−1 |
|
|
|
|
|
2 |
||
1 |
0 |
|
|
0 |
|
0 |
|
|
0 |
|
|
|
|
|
|
|
|
|
|
and one constraint is equationally satisfied. Thus, choose λ1 = 2.
Notice that the first and third constraints are equationally satisfied. The first row is already included in R; hence, focus on the third row (ℓ = 3). From (8.12), define vi+1 for the next iteration:
v2 |
= v1 + λ1y = |
0 |
+ 2 |
1 |
= |
2 . |
|
|
0 |
|
0 |
|
0 |
This vertex is visualized in Figure 8.4. Choosing R = [1; 3] and replacing the second row of A1 and b1 with the third row of Ax ≤ b yields