[Boyd]_cvxslides

Lagrange dual of maximum margin separation problem (1)

optimal value is distance between convex hulls

Approximate linear separation of non-separable sets

• at optimum, ui = max{0, 1 − aT xi − b}, vi = max{0, 1 + aT yi + b}

• can be interpreted as a heuristic for minimizing #misclassified points

Support vector classifier

produces point on trade-o curve between inverse of margin 2/kak2 and classification error, measured by total slack 1T u + 1T v

same example as previous page, with γ = 0.1:

Nonlinear discrimination

separate two sets of points by a nonlinear function:

f(xi) > 0, i = 1, . . . , N, f(yi) < 0, i = 1, . . . , M

• choose a linearly parametrized family of functions

f(z) = θT F (z)

F = (F1, . . . , Fk) : Rn → Rk are basis functions

•solve a set of linear inequalities in θ:

θT F (xi) ≥ 1, i = 1, . . . , N, θT F (yi) ≤ −1, i = 1, . . . , M

quadratic discrimination: f(z) = zT P z + qT z + r

xTi P xi + qT xi + r ≥ 1, yiT P yi + qT yi + r ≤ −1

can add additional constraints (E.G., P −I to separate by an ellipsoid) polynomial discrimination: F (z) are all monomials up to a given degree

Placement and facility location

•N points with coordinates xi R2 (or R3)

•some positions xi are given; the other xi’s are variables

•for each pair of points, a cost function fij(xi, xj)

placement problem

variables are positions of free points interpretations

•points represent plants or warehouses; fij is transportation cost between facilities i and j

•points represent cells on an IC; fij represents wirelength

P

example: minimize (i,j) A h(kxi − xjk2), with 6 free points, 27 links

optimal placement for h(z) = z, h(z) = z2, h(z) = z4

Convex Optimization — Boyd & Vandenberghe

9.Numerical linear algebra background

•matrix structure and algorithm complexity

•solving linear equations with factored matrices

•LU, Cholesky, LDLT factorization

•block elimination and the matrix inversion lemma

•solving underdetermined equations

9–1

Matrix structure and algorithm complexity

cost (execution time) of solving Ax = b with A Rn×n

•for general methods, grows as n3

•less if A is structured (banded, sparse, Toeplitz, . . . )

flop counts

•flop (floating-point operation): one addition, subtraction, multiplication, or division of two floating-point numbers

•to estimate complexity of an algorithm: express number of flops as a (polynomial) function of the problem dimensions, and simplify by keeping only the leading terms

•not an accurate predictor of computation time on modern computers

•useful as a rough estimate of complexity

vector-vector operations (x, y Rn)

• inner product xT y: 2n − 1 flops (or 2n if n is large)

• sum x + y, scalar multiplication αx: n flops

matrix-vector product y = Ax with A Rm×n

•m(2n − 1) flops (or 2mn if n large)

•2N if A is sparse with N nonzero elements

•2p(n + m) if A is given as A = UV T , U Rm×p, V Rn×p

matrix-matrix product C = AB with A Rm×n, B Rn×p

•mp(2n − 1) flops (or 2mnp if n large)

•less if A and/or B are sparse

•(1/2)m(m + 1)(2n − 1) ≈ m2n if m = p and C symmetric