2.3.10 Other Problems

In addition to the applications of 0–1 IP discussed above there are a number of other applications of 0–1 IP not discussed here but which can be followed up through the references.

Other problems involve networks and graphs (with nodes and arcs) and are naturally modelled with 0–1 variables representing the presence of particular nodes and the resulting presence of arcs emanating from them. The quadratic assignment problem is an extension of the assignment problem mentioned above (which can be treated as an LP) but is naturally modelled with products of 0–1 variables which can be linearised (with a large increase in model size) in the way shown in Example 2.15 making it a genuine IP. The matching problem that was mentioned above can also be regarded as an extension of the assignment problem. The matching problem, itself, has a number of extensions.

Scheduling problems involve deciding in what order to process jobs on machines. The 0–1 variables can be used to decide, for each pair of competing jobs, which one is processed first.

Capital investment problems (in addition to facilities location, already discussed) can incorporate capital limitation constraints and be modelled over periods of time with resultant cash flows.

2.4 Computational Complexity

2.4.1 Problem Classes and Instances

It is clear that problems in some classes are more difficult to solve than others. For example LPs are relatively ‘easy’ to solve. Models with thousands of constraints and variables are now solved routinely on desktop computers. Models with millions of constraints and variables have also been successfully solved. For IP the picture is very different. While some large models have been successfully solved some models, with only hundreds of variables, can prove very difficult. In logic large satisfiability problems (which are discussed in Chapter 4) can prove very difficult to solve. The subject of computational complexity is concerned with trying to classify problems into the easy and difficult categories. According to the method which we will describe LP is easy but IP, and many of its special cases, and the satisfiability problem are difficult.

The method which we describe looks at classes of problems rather than instances. Within a particular ‘difficult’ class there may be easy instances. The categorisation of classes will be based on the worst cases within that class. In some ways this is unsatisfactory since in practice the worst cases may seldom arise. However if we base the categorisation on worst cases then we can give a ‘guarantee’ of difficulty.

Average case analysis is much more difficult and requires probability distributions to be defined in order to define what the average cases are.

It is important to recognise that the method uses categorises problem classes rather than algorithms. There may be ‘good’ and ‘bad’ algorithms for solving problems in any class. For a problem class to be designated easy it is necessary to show that there is at least one easy algorithm for solving all problems in the class. A problem class will be designated ‘easy’ if there is an algorithm for solving problems in it whose number of steps will never be more than a polynomial function of the size of the problem (storage spaces required in a computer). On the other hand a problem class will be designated ‘difficult’ if there is no known such algorithm and the number of steps in all the algorithms known requires an exponential number of steps. These definitions will be made more precise later. A rather negative aspect of the categorisation, in terms of mirroring reality, is that some successful algorithms for problems in the easy class are exponential (in the worst case). The classic example is the simplex algorithm for LP. Examples can be found (see references) where the simplex algorithm takes an exponential number of steps although in practice (‘average cases’) it is known to solve huge models in reasonable periods of time. However there are known to be polynomial algorithms for LP (one of which sometimes overtakes the Simplex on ‘huge’ problems). For IP and the satisfiability problem no algorithms have ever been found which are not exponential or worse.

2.4.2 Computer Architectures and Data Structures

Another aim of this problem characterisation is to make it, as far as possible, independent of the computer architecture used. In order to do this fairly crude assumptions are made but it turns out that they often do not matter in making the (major) distinction between polynomial and exponential algorithms. One of the assumptions is that ‘elementary’ operations such as addition, multiplication, comparison, etc. all take the same amount of time on ‘small’ numbers. Of course multiplication is inherently more difficult than addition, although by how much depends on the architecture of the computer used.

Also of great significance, in practice, is the data structure used for encoding instances of the problem. We must, however, restrict ourselves to ‘sensible’ data structures. We would not encode a number n by a sequence of n 1’s. We would, rather, encode it using binary notation taking account of the significance of the position of each digit as well as its value. For example 23 would be written as 10111 (since 23 = 1 × 24 + 0 × 23 + 1 × 22 + 1 × 21 + 1 × 20) instead of as a string of 23 1’s. It is useful (and realistic) to use only two symbols 0 and 1 given the binary hardware devices used in computers, although representing numbers to a higher radix would not alter the final analysis. However, what is important is to use some such representation. Clearly the storage required for a number n will be log2 n bits (with possibly another bit to represent the sign of n), as opposed to n bits.

function of M and the distinction between exponential and polynomial growth would be lost.

If, however, we were to encode to a different radix than 2 then this would simply alter the value of c for a polynomial growth rate and the value of a for an exponential growth rate. It would not, therefore, alter the distinction between polynomial and exponential growth rates.

We should also explain that, in this discussion, we are restricting ourselves to data and solutions that are integer or rational. Therefore these can be represented exactly in a computer and we do not have to bring in concepts of solution to some degree of

approximation. Of course it is possible to represent any ‘computable’ number (i.e.

√

any we might wish to talk about such as, for example, 2 or π ) in a computer by an exact ‘formula’ (computer procedure) but such issues of a uniform method of representability are beyond the scope of this book. We restrict ourselves to the usual (binary) encoding of numbers to successive multiples of powers to a given radix. If necessary rational numbers could be represented exactly by storing their numerator and denominator (reduced to be coprime), it is easy to show that the size of the denominator is bounded by the maximum determinant of a basis matrix in LP.

It is necessary to distinguish between ‘small’ and ‘large’ numbers in discussing the encoding. While each ‘number’ in some problems (e.g. the satisfiability problem and some graph problems) can be represented as a small number and elementary operations on such numbers assumed to take unit time, this is unrealistic for other problems. For example LP and IP problems may contain arbitrarily large coefficients and the time to carry out operations on them will be related to (the logarithm of) their size. Therefore, for example, we could take the size of the encoding of an LP or IP to be bounded by (m + 1) × (n + 1) × log maxi j (| Ai j |, |c j |, |bi |), where m, n, A, c, b represent the number of constraints, variables, matrix of coefficients and objective and right-hand-side vectors, respectively. Other ways of measuring the size of the encodings (e.g. log( i j (Ai j + c j + bi )) will make no difference to whether they be classified as polynomial or exponential. With any of these encodings LP becomes polynomially bounded (by the Ellipsoid and Karmarkar algorithms referenced in Sect. 2.5). However no polynomial algorithm has ever been found (or seems likely to be found) which is a function of m and n alone, i.e. which ignores the magnitude of the coefficients. In contrast there are polynomial algorithms for solving simultaneous equations (e.g. by Gaussian elimination) which are functions of m and n alone. Another reason suggesting why it is usually necessary to take account of the magnitude of the coefficients is that it is, in principle, possible to reduce many problems to ‘simpler’ problems with larger coefficients. Any pure IP model, with all equality constraints, can be converted to a knapsack model by aggregating the constraints (but producing very large coefficients in consequence). In one sense the knapsack problem is ‘easier’ than general pure IP (although both are difficult in the classification given here). It would therefore be unrealistic to assume the size of the coefficients in a knapsack problem do not matter. The celebrated work of Godel,¨ mentioned in Chapter 1, was carried out (again in principle) by encoding proofs of