Hedman. A First Course in Logic, 2004 (Oxford)

a function. Given some input, each problem requires a certain output. In each of the examples we have cited, the output is a “yes” or “no” answer. This type of problem is called a decision problem. Other problems may require various output: the problem of finding the largest prime that divides a given number, for example. More generally, let f (¯x) be a k-ary function on the non-negative integers. Consider the following problem:

• The f -computability problem: Given x¯, determine the value of f (¯x).

If this problem is decidable, then the function f is said to be computable. We claim that the f -computability problems are the only problems we need to consider. In fact, we take this as our definition of a problem.

Every problem corresponds to a function on the non-negative integers.

A decision problem corresponds to a function that only takes on the values 0 and 1 (we interpret 0 as “no” and 1 as “yes”). Let A be the set of all k-tuples x¯ for which f (¯x) = 1. The problem represented by f corresponds to the following decision problem: given x¯, determine whether or not x¯ is in A. In this way, every decision problem can be viewed as a subset of (N {0})k for some k.

Every decision problem corresponds to a relation on the non-negative integers.

The Even Problem corresponds to the set of non-negative even numbers. For the 10-Clique Problem, we can code each finite graph as a k-tuple of non-negative integers. Likewise, every formula of first-order logic can be coded as a natural number. We shall discuss the required coding procedures in this chapter and the next.

The goal of this chapter is to give precise meaning to notions such as algorithm, decidable, and polynomial-time. These notions have arisen in earlier chapters. In Chapter 5, we defined decidable theories in terms of algorithms. Given a V-theory T and a V-sentence ϕ, we may ask whether or not ϕ is a consequence of T . This decision problem is decidable if and only if the theory T is decidable. The Satisfiability Problem for First-order Logic (which we shall abbreviate FOSAT), was the central topic of Chapter 3. The corresponding problem for propositional logic (denoted PSAT), was discussed in Chapter 1.

The definitions and results of the present chapter allow us to prove in the final chapters some of the claims that were made in the previous chapters. In Chapter 8, we prove that the theory of arithmetic and other theories are undecidable. In chapter 10, we prove that FOSAT is undecidable. In contrast, PSAT is decidable. However, there is no known algorithm that decides PSAT quickly.

This problem is NP-complete. Again, we do not prove this until the final chapter. In the present chapter, we define the concept of NP-completeness and discuss the relationship between PSAT and the P = NP question.

The topics of computability and complexity do not flow from the stream of ideas embodied in the model theory of the previous chapters. In fact, Sections 7.2 and 7.6.2 are the only sections of this chapter that require logic. However, there are strong connections between the topics of the present chapter and the logic of the previous chapters. The formal nature of computability has historically and philosophically linked this subject with the formal languages of logic. In addition to the connections mentioned in the previous paragraphs, we have the following fact: many of the classes of problems that naturally arise in computability and complexity can be defined in terms of logic. We shall see evidence of this fundamental fact in Sections 7.2, 7.6.2, 8.3, and 10.4.

7.1 Computable functions and Church’s thesis

We consider functions on the non-negative integers. Whenever we refer to a function in this chapter, we always mean a function of the form f : D → N {0} where the domain D of f is a subset of (N 0)k for some k. If the domain D happens to be all of (N {0})k, then the function f is said to be a total function. Otherwise, f is a partial function.

Definition 7.1 Let f be an k-ary function on the non-negative integers. Then f is computable if there exists an algorithm which, given an k-tuple a¯ of non-negative integers,

•outputs f (¯a) if a¯ is in the domain of f , and

•yields no output if a¯ is not in the domain of f .

To make this definition precise, we must state what constitutes an algorithm. Informally, an algorithm is a computer program. We claim that the definition of a computable function is invariant under our choice of programming language. If we have a computer program in C++ that computes the function f (x), then we can translate this to a program in Fortran or any other language.

There are countably many computer programs and uncountably many functions (Propositions 2.47 and 2.48). So most functions are not computable. This fact flies in the face of empirical evidence. Nearly every function that naturally arises in mathematics (every function one encounters in, say, calculus) is computable. We shall demonstrate several functions that are not computable in Section 7.6.1.

The set of computable functions is our primary object of study in this chapter. We take two approaches to circumscribing this set from among the uncountably many functions that are not computable. In Section 7.3, we make definite the notion of an algorithm by specifying a simplified computer language. This facilitates the analysis of the set of computable functions. In the present section, we take a di erent approach. We consider some basic computable functions and show how more complex computable functions can be derived from them. We define a set of computable functions known as the recursive functions. This set is generated from a few functions by applying a few rules. We prove in Sections 7.3 and 7.4 that the two approaches yield the same functions: the recursive functions of the present section are precisely those that are computable by an algorithm in the sense of Section 7.3. Moreover, we claim that these functions are precisely those functions that can be computed by a C++ program, a Maple program, a Pascal program, or any other computer program. Thus, we demonstrate that computability is not a vague notion. It is a robust concept suitable for mathematical analysis.

7.1.1 Primitive recursive functions. We begin our analysis of computable functions with some examples.

Example 7.2 The following functions are unquestionably computable:

the zero function Z(x) = 0,

the successor function s(x) = x + 1, and

the projection functions pki (x1, x2, . . . , xk) = xi.

The projection functions are defined for any k and i in N with i ≤ k.

Definition 7.3 The functions s(x), Z(x), and pki (x) from the previous example are called the basic functions.

From two given unary functions f and g we can define various new functions. One of these is the composition h(x) = f (g(x)). If we can compute both f (x) and g(x), then we can compute h(x). We generalize this idea to k-ary functions.

Definition 7.4 Let S be a set of functions on the non-negative integers. The set is said to be closed under compositions if given any m-ary function h in S and k-ary functions g1, g2, . . . , gm in S (for any k and m in N) the k-ary function f defined by

f (x1, . . . , xk) = h(g1(x1, . . . , xk), . . . , gm(x1, . . . , xk))

is also in S.

If k = 1, then we allow h to be any constant function.

If both h and g are computable, then so is any function defined from h and g be primitive recursion. So the set of computable functions is closed under primitive recursion as well as composition. Also, the computable functions include the basic functions Z(x), s(x), and pki from Example 7.2.

Definition 7.7 The set of primitive recursive functions is the smallest set containing the basic functions and closed under both composition and primitive recursion.

Whereas every primitive recursive function is computable, it is not true that every computable function is primitive recursive. The primitive recursive functions are only part of the set of computable functions. They are, however, a sizable part.

Proposition 7.8 The addition function defined by a(x, y) = x + y is primitive recursive.

Proof Let h(y) = p11(y) = y and g(x, y, z) = s(p33(x, y, z)) = s(z). Since h(x) is a basic function, it is primitive recursive. Since g(x, y, z) is the composition of

basic functions, it is also primitive recursive. We define a(x, y) using primitive recursion as follows:

a(0, y) = h(y) = y, and

a(x + 1, y) = g(x, y, a(x, y)) = s(a(x, y)) = a(x, y) + 1.

It follows that a(x, y) is primitive recursive.

Unlike addition, subtraction does not determine a total binary function on the non-negative integers. We let − denote a modified subtraction operation that is total. This function is defined as follows:

Proposition 7.9 The function sub(x, y) = x − y is primitive recursive.

Proof First we show that the predecessor function pred(x) = x − 1 is primitive

recursive. Let h1(x) = Z(x) and g1(x, y) = p21(x, y). Let pred(0) = h1(0) = 0, and pred(x + 1) = g1(x, pred(x)) = x.

To define sub(x, y) by primitive recursion, let h2(x) = p11(x) and g2(x, y, z) = pred(z). Then sub(x, 0) = h2(x) = x and sub(x, y + 1) = g2(x, y, sub(x, y)) = pred(sub(x, y)) and so sub(x, y) is primitive recursive.

Proposition 7.10 The multiplication function defined by m(x, y) = xy is primitive recursive.

Proof Let h(y) = Z(y) and g(x, y, z) = a(y, z). By Proposition 7.8, a(y, z) is primitive recursive. We define m(x, y) using primitive recursion as follows:

m(0, y) = h(y) = 0, and

m(x + 1, y) = g(x, y, m(x, y)) = a(y, m(x, y)) = y + m(x, y).

It follows that m(x, y) is primitive recursive.

Consider the exponential function exp(x, y) = yx. Since this function is not total, it cannot be primitive recursive. If we define exp(0, 0) to be 1 (or any other number), then the resulting function is primitive recursive.

Proposition 7.11 The function exp(x, y) is primitive recursive.

Proof Let h(y) = s(0) and g(x, y, z) = m(y, z). By the previous proposition, m(y, z) is primitive recursive. We define exp(x, y) using primitive recursion as follows:

exp(0, y) = h(y) = s(0) = 1, and

exp(x + 1, y) = g(x, y, exp(x, y)) = m(y, exp(x, y)) = y · exp(x, y).

It follows that exp(x, y) is primitive recursive.

Proposition 7.12 Let p(x) be any polynomial having natural numbers as coe cients. Then p(x) is primitive recursive.

Proof This follows from the fact that p(x) can be written as a composition of constant functions, a(x, y), m(x, y), and exp(x, y).

By definition, the set of primitive recursive functions is closed under the operations of composition and primitive recursion. By repeatedly using is fact, we can generate more and more primitive recursive functions from the basic functions. We next show that the set of primitive recursive functions is necessarily closed under operations other than composition and primitive recursion.

Definition 7.13 Let S be a set of functions on the natural numbers. The set is closed under bounded sums if, for any k-ary function f (x1, . . . , xk) in S, the

bounded sums.

Thus, we see that the primitive recursive functions form a vast set of computable functions. In fact, it may seem di cult to demonstrate a total computable function that is not primitive recursive. We now give one well known example of such a function.

7.1.2 The Ackermann function. We define a total binary function A(n, x) that is computable but not primitive recursive. We refer to this function as the Ackermann function. It is one of several variations of a function first introduced by Wilhelm Ackermann in 1928. The function A(n, x) is defined as follows. For (n, x) N2,

A(0, x) = x + 1,

A(n, 0) = A(n − 1, 1), and

A(n, x) = A(n − 1, A(n, x − 1)).

This function is computable. Let us compute some specific values of A(n, x). Since A(0, x) = x + 1, we have A(0, 1) = 2, A(0, 2) = 3, and so forth. We also have: A(1, 0) = A(0, 1) = 2, A(2, 0) = A(1, 1) = A(0, A(1, 0)) = A(0, 2) = 3, A(1, 2) = A(0, A(1, 1)) = A(0, 3) = 4, and A(3, 0) = A(2, 1) = A(1, A(2, 0)) =

Although these computations may seem innocuous, it is practically impossible to extend the above table much further to the right. The column for n = 4 begins A(4, 0) = 13, A(4, 1) = 65534, and then the numbers get really big. The column for n = 5 begins

A(5,0) = A(4,1) = 65533.

The computation of A(5, 1) is beyond the capabilities of any computer. For each n N, let an(x) denote the unary function A(n, x). From the above

computations, we see that

a0(x) = A(0, x) = x + 1, a1(x) = A(1, x) = x + 2,

x+3 times

The functions an(x) for n > 4 cannot be expressed using conventional notation. We make two observations regarding these functions. First, note that the function an(x) can be defined inductively from the function an−1(x). This follows from the rules A(n, 0) = A(n − 1, 1) and A(n, x) = A(n − 1, A(n, x − 1)) in the definition of A(n, x). It follows that each an(x) is a primitive recursive function (since a0(x) is). The second observation is that the function an−1(x) is smaller than an(x) for each n N.

Definition 7.19 We say that g(x1, . . . , xk) is smaller than f (x), and write g < f , if g(x1, . . . , xk) < f (m) for all k-tuples x¯ having maximum entry xi = m. If g happens to be unary, then g < f simply means that g(n) < f (n) for all n.

Let a∞(x) = A(x, x). This function is not smaller than any of the functions an(x).

Proposition 7.20 The function a∞(x) = A(x, x) is not primitive recursive.

Proof We show that every primitive recursive function is smaller than am(x) for some m N. The basic functions Z(x) and pki (¯x) are smaller than a1(x) and s(x) is smaller than a2(x).

Suppose now that f is the composition of g and h and g < ai and h < aj for some i and j in N. Further suppose that g and h are unary functions (we claim that what follows generalizes to higher arities). We show that f (x) = g(h(x)) is smaller than am+1 where m is larger than both i and j. To show this, we make two observations: for any x N and m N {0}:

1.x < a(mx−1)(1) (for x > 0 and m > 0), and

2.a(mx+1)(1) = am+1(x),

where a(mx)(1) denotes the composition am(am(· · · (am(1)))). We regard a(mx−1)(1)

x times

as a function of x.

To see that the first observation holds, note that a(0x−1)(1) = x for x > 0. For the second observation, recall from the definition of the Ackermann

function that

am+1(x) = am(am+1(x − 1)) = am(am(am+1(x − 2))) = · · · .

We see that am+1(x) = a(mx)(am+1(0)). Again by the definition of the Ackermann function: am+1(0) = am(1). This establishes the second observation.

We now show that the composition of h and g is smaller than am+1(x). We have

g(h(x)) < ai(aj (x)) < am(am(x)).

By observation 1, am(am(x)) < am(am(a(mx−1)(1))).

By observation 2, am(am(a(mx−1)(1))) = am(x+1)(1) = am+1(x).

Now suppose that f is defined from h and g by primitive recursion. Again, we suppose for simplicity that f is unary. In this case, h(x) = c for some constant c and g is a binary function. The function f is defined for each n N as follows:

f (0) = c and f (n + 1) = g(n, f (n)).

Suppose that g < ai. We show that f < am for some m. Since aj < aj+1 for each j, there exists j such that aj (0) is bigger than the constant c. Let m be greater than both j and i + 1.

We may assume that f (n) > n for each n (since f is smaller than some function with this property). Likewise, we may also assume that g is an increasing function. In particular, we use the inequality g(n, f (n)) < g(f (n), f (n)).

Claim f (n) < am(n) for all n.

Proof We prove this by induction on n.

f (0) = c < am(0) by our choice of m.

f (n + 1) = g(n, f (n)) < g(f (n), f (n)) (by our previous remarks)

<ai(f (n)) < ai(am(n)) (by induction)

<am−1(am(n)) (since m > i + 1)

= am(n + 1) (since A(m, n + 1) = A(m − 1, A(m, n))).

This completes the proof.

We conclude that the inductive method used to define A(n, x) is stronger than primitive recursion. To generate the set of computable functions from the basic functions, we must include closure operations beyond composition and primitive recursion. We claim that, in addition these two operations, we need only one more operation to obtain the entire set of computable functions. As we shall show, it su ces to modify the operation of bounded search.

7.1.3 Recursive functions. We now define the recursive functions. In addition to being closed under composition and primitive recursion, the set of recursive functions is also closed under unbounded search.