8Linear equations and inequalities

Many problems in economics can be modelled as a system of linear equations or a system of linear inequalities. In this chapter, we consider some basic properties of such systems and discuss general solution procedures.

8.1 SYSTEMS OF LINEAR EQUATIONS

8.1.1 Preliminaries

At several points in previous chapters we have been confronted with systems of linear equations. For instance, deciding whether a set of given vectors is linearly dependent or linearly independent can be answered via the solution of such a system. The following example of determining feasible production programmes also leads to a system of linear equations.

Example 8.1 Assume that a firm uses three raw materials R1, R2 and R3 for the production of four goods G1, G2, G3 and G4. There are the following amounts of raw materials available: 120 units of R1, 150 units of R2 and 180 units of R3. Table 8.1 gives the raw material requirements per unit of each good. Denote by xi the quantity of good Gi , i {1, 2, 3, 4}. We are interested in all possible production programmes which fully use the available amounts of the raw materials.

Considering raw material R1 we get the following equation:

1x1 + 2x2 + 1x3 + 3x4 = 120.

Table 8.1 Raw material requirements for the goods Gi , i {1, 2, 3, 4}

288 Linear equations and inequalities

Here 1x1 is the amount of raw material R1 necessary for the production of good G1, 2x2 the amount of R1 for good G2, 1x3 the amount of R1 for good G3 and 3x4 is the amount of R1 required for the production of good G4. Since all 120 units of raw material R1 should be used, we have an equation. Similarly we obtain an equation for the consumption of the other two raw materials. Thus, we get the following system of three linear equations with four variables:

Moreover, we are interested only in solutions for which all values xi , i {1, 2, 3, 4}, are non-negative. Considering e.g. the production programme x1 = 40, x2 = 15, x3 = 20 and x4 = 10, we can easily check that all equations are satisfied, i.e. this production programme is feasible, and there exists at least one solution of this system of linear equations. In order to describe all feasible production programmes, we have to find all solutions satisfying the above equations such that all values xi , i {1, 2, 3, 4}, are non-negative.

Often, it is not desired that all raw materials should necessarily be fully used. In the latter case it is required only that for each raw material Ri the available amount is not exceeded. Then all equality signs in the above equations have to be replaced by an inequality sign of the form ≤, and we obtain a system of linear inequalities which is discussed in Chapter 8.2.

In the following, we discuss general methods for solving systems of linear equations. We answer the question of whether a system has a solution, whether an existing solution is uniquely determined and how the set of all solutions can be determined in the general case.

am1x1 + am2x2 + · · · + amnxn = bm

is called a system of linear equations, where x1, x2, . . . , xn are the unknowns or variables, a11, a12, . . . , amn are the coefficients and b1, b2, . . . , bm are called the right-hand sides.

As an abbreviation, system (8.1) can also be written in matrix representation:

Ax = b,

290 Linear equations and inequalities

8.1.2 Existence and uniqueness of a solution

Next, we investigate in which cases system (8.1) has a solution. To this end, we introduce the notion of the rank of a matrix. For this purpose, the following property is useful.

THEOREM 8.1 Let A be a matrix of order m × n. Then the maximum number of linearly independent column vectors of A coincides with the maximum number of linearly independent row vectors of A.

Definition 8.4 Let A be a matrix of order m×n. The rank of matrix A, written r(A), is the maximum number p of linearly independent column (or according to Theorem 8.1 equivalently, row) vectors in A. If A is any zero matrix, we set r(A) = 0.

As an obvious consequence of Definition 8.4, we obtain r(A) = p ≤ min{m, n}. The following theorem gives a first criterion to determine the rank of a matrix A.

THEOREM 8.2 The rank r(A) of matrix A is equal to the order of the largest minor of A that is different from zero.

We recall that a minor of a matrix A was defined as a determinant of a square submatrix of A. The above criterion can be used when transforming the determinant of a matrix A in such a way that the order of the largest minor can be easily obtained. Otherwise, it can be rather time-consuming to determine the rank of a matrix by applying Theorem 8.2. Consider the following two examples.

We obtain |A| = 0 which means that matrix A cannot have rank three. However, e.g. for the minor obtained from matrix A by deleting the last row and column, we get

i.e. there is a minor of order two which is different from zero, and thus matrix A has rank two.

Example 8.3 Let us consider the matrix

Linear equations and inequalities 291

with x R. We determine the rank of matrix A in dependence on the value of x R. Expanding |A| by the third row, we get

=−3 − (1 − x) + (1 − x)(1 − x) + 1

=(−4 + x) + (x2 − 2x + 2)

=x2 − x − 2.

We determine the roots of the equation x2 − x − 2 = 0 and obtain x1 = −1 and x2 = 2. Thus, we get |A| = 0 for the case x = −1 and x = 2, i.e. due to Theorem 8.2, we have r(A) = 3 for x R \ {−1, 2}. For x {−1, 2}, we obtain for the minor formed by rows 1 and 3 as well as columns 2 and 3

Thus, we obtain r(A) = 2 for the case x = −1 and for the case x = 2 because the order of the largest minor with a value different from zero is two.

Definition 8.5 We define the m × (n + 1) augmented matrix

as the coefficient matrix A expanded by an additional column containing vector b of the right-hand side.

Obviously, we have r(A) ≤ r(Ab) since matrix Ab contains an additional column vector in comparison with matrix A. Moreover, since the augmented matrix differs from the coefficient matrix A by exactly one column, there are only two cases possible: either r(Ab) = r(A) or r(Ab) = r(A) + 1.

Definition 8.6 If system (8.1) has at least one solution, it is said to be consistent. If

this system has no solution, it is said to be inconsistent.

Next, we present a necessary and sufficient condition for the case where a system of linear equations has at least one solution.

292 Linear equations and inequalities

THEOREM 8.3 System (8.1) is consistent if and only if the rank of the coefficient matrix A is equal to the rank of the augmented matrix Ab = (A | b), i.e.

system Ax = b is consistent r(A) = r(Ab).

Since for a homogeneous system of equations, the augmented matrix contains matrix A plus an additional zero (column) vector, the number of linearly independent (column or row) vectors of the augmented matrix is always equal to the number of linearly independent vectors of matrix A. This leads to the following corollary.

COROLLARY 8.1 A homogeneous system Ax = 0 is always consistent.

Indeed, for system Ax = 0 we have r(A) = r(Ab). We can note that a homogeneous system has at least the so-called trivial solution xT = (x1, x2, . . . , xn) = (0, 0, . . . , 0).

Next, we deal with the following question. If system (8.1) is consistent, when is the solution uniquely determined? An answer is given by the following theorem.

THEOREM 8.4 Consider the system Ax = b of linear equations, where A is a matrix of order m × n, and let this system be consistent. Then:

(1)If r(A) = r(Ab) = n, then solution x = (x1, x2, . . . , xn)T is uniquely determined.

(2)If r(A) = r(Ab) = p < n, then there exist infinitely many solutions. In this case, the set of solutions forms an (n − p)-dimensional vector space.

In case (2) of Theorem 8.4 we say that the set of solutions has dimension n−p. Let us consider part (2) of Theorem 8.4 in a bit more detail. In this case, we can select n − p variables that can be chosen freely. Having their values fixed, the remaining variables are uniquely determined. We denote the n − p arbitrarily chosen variables as free variables, and we say that the system of linear equations has n − p degrees of freedom.

8.1.3 Elementary transformation; solution procedures

Solution procedures for systems of linear equations transform the given system into a ‘system with easier structure’. The following theorem characterizes some transformations of a given system of linear equations such that the set of solutions does not change.

THEOREM 8.5 The set of solutions of system (8.1) does not change if one of the following transformations is applied:

(1)An equation is multiplied by a number λ = 0 or it is divided by a number λ = 0.

(2)Two equations are interchanged.

(3)A multiple of one equation is added to another equation.

Operations (1) to (3) are called elementary or equivalent transformations. By such elementary transformations, the rank r(A) of a matrix A does not change either (see also rules for evaluating determinants given in Chapter 7.3). Finally, we introduce a special form of a system of linear equations and a solution as follows.

Linear equations and inequalities 293

Definition 8.7 A system Ax = b of p = r(A) linear equations, where in each equation one variable occurs only in this equation and it has the coefficient +1, is called a system

of linear equations in canonical form. These eliminated variables are called basic variables (bv), while the remaining variables are called non-basic variables (nbv).

Hence the number of basic variables of a system of linear equations in canonical form is equal to the rank of matrix A. As a consequence of Definition 8.7, if a system of linear equations Ax = b is given in canonical form, the coefficient matrix A always contains an identity matrix. If r(A) = p = n, the identity matrix I is of order n × n, i.e. the system has the form

I xB = b,

where xB is the vector of the basic variables. (Note that columns might have been interchanged in matrix A to get the identity matrix, which means that the order of the variables in vector xB is different from that in vector x.) If r(A) = p < n, the order of the identity submatrix is p × p. In the latter case, the system can be written as

I xB + AN xN = b,

where xB is the p-vector of the basic variables, xN is the (n − p)-vector of the non-basic variables and AN is the submatrix of A formed by the column vectors belonging to the nonbasic variables. (Again column interchanges in matrix A might have been applied.) This canonical form, from which the general solution can easily be derived, is used in one of the solution procedures described in this subsection.

Definition 8.8 A solution x of a system of equations Ax = b in canonical form,

where each non-basic variable has the value zero, is called a basic solution.

Thus, if matrix A is of order p × n with r(A) = p < n, then at least n − p variables are equal to zero in a basic solution of the system Ax = b. The number of possible basic solutions of a given system of linear equations is determined by the number of different possibilities of choosing p basic variables. That is, one has to find among the column vectors of matrix A all

possibilities of p linearly independent vectors belonging to the p basic variables. There exist

at most pn basic solutions (see Chapter 1.3 on Combinatorics).

One method of solving systems of linear equations has already been discussed in Chapter 7.3, but remember that Cramer’s rule is applicable only in special cases. The usual methods of solving systems of linear equations apply elementary transformations mentioned in Theorem 8.5 to transform the given system into a form from which the solution can be easily obtained. The methods typically used transform the original system into either

(1)a canonical form according to Definition 8.7 ( pivoting procedure or Gauss–Jordan elimination) or into

(2)a ‘triangular’ or echelon form (Gaussian elimination).

294 Linear equations and inequalities

It is worth noting that the notation for both procedures in the literature is not used in a standard way; in particular Gaussian elimination is often also denoted as pivoting. The reason is that both procedures are variants of the same strategy: simplify the given system of linear equations in such a way that the solution can be easily obtained from the final form of the system. We now discuss both methods in detail.

Pivoting

First, we discuss the pivoting procedure. The transformation of the original system into a canonical form (possibly including less than m equations) is based on the following theorem and the remark given below.

THEOREM 8.6 Let Ax = b be a given system of m linear equations with n variables and r(A) = p < min{m, n}. Then the augmented matrix Ab = (A | b) can be transformed by applying Theorem 8.5 and column interchanges into the form

with bp+1 = 0 or bp+1 = 0.

It is easy to see that the matrix A given in (8.2) (and therefore also the original coefficient matrix A) have rank p. In terms of Theorem 8.2, this means that matrix A has a minor of order p whose value is different from zero. This can easily be seen by taking the identity submatrix obtained in the left upper part (printed in bold face) whose determinant is equal to one. However, there is no minor of a larger order than p whose value is different from zero. (If we add one row and one column, the value of the determinant is equal to zero since there is one row containing only zero entries.) Notice also that the first p rows in representation (8.2) describe a system of linear equations in canonical form.

Remark In the case when r(A) = p is not smaller than min{m, n}, we can transform matrix A into one of the three following forms (A | b ):

Each of the matrices in (8.2) to (8.5) contains an identity submatrix. The order of this identity matrix is the rank of the coefficient matrix A originally given. In the transformed matrix A , each column corresponds to some variable. Since column interchanges were allowed, we simply denote the variable belonging to the first column as the first basic variable xB1, the variable belonging to the second column as the second basic variable xB2 and so on. (If no column interchanges were applied, we have the natural numbering x1, x2, . . . , xp.) Accordingly, we denote the variables belonging to the columns which do not form the identity submatrix as the non-basic variables xN 1, xN 2, . . . , xN ,n−p.

We now discuss how the solutions can be found from the system in canonical form given by the first rows of matrices (8.2) to (8.5) (including the identity matrix).

In the cases defined by matrices (8.3) and (8.5), there always exists a solution (r(A) = r(Ab) = m). In particular, in the case of matrix (8.5) we have the unique solution

xB1 = b1, xB2 = b2, . . . , xBn = bn.

(Note that variables might have been interchanged.) In the case of matrix (8.3), we have n − m degrees of freedom, i.e. n − m variables belonging to columns m + 1, m + 2, . . . , n can be chosen arbitrarily.

In the case of matrix (8.4), there exists a unique solution

xB1 = b1, xB2 = b2, . . . , xBn = bn,

provided that bn+1 = 0, otherwise the system has no solution. In the case of p < min{m, n} considered in Theorem 8.6, the system is consistent if and only if bp+1 = 0, and the system of the first p equations obtained by the transformation of matrix (A | b) into (A | b ) is a

Since r(A) = r(A ) = p, there are n − p degrees of freedom, i.e. n − p variables can be chosen arbitrarily. We can rewrite the system in canonical form in terms of the basic variables

xNj arbitrary for j = 1, 2, . . . , n − p.

We emphasize that, if xB1, xB2, . . . , xBp are the basic variables, this means that the column vectors of matrix A belonging to these variables are linearly independent and form a basis of the vector space spanned by the corresponding p column vectors of matrix A. If we choose xNj = 0, j = 1, 2, . . . , n − p, we obtain the basic solution

xB1 = b1, xB2 = b2, . . . , xBp = bp.

Next, we discuss how we can transform the augmented matrix Ab = (A | b) into matrix (A | b ) by elementary transformations in a systematic way from which the canonical form can be established.

These formulas correspond to those which have been presented in Chapter 6.4 when considering the replacement of one vector in a basis by another vector of a vector space. Assume that variable xl should become the basic variable in the kth equation of the system. This can be done only if akl = 0. The element akl is called the pivot or pivot element. Accordingly, row k is called the pivot row, and column l is called the pivot column. The transformation formulas distinguish between the pivot row and all remaining rows. They are as follows.

Transformation formulas

Linear equations and inequalities 297

These transformations can be performed by using the following tableaus, where rows 1 to m in the first tableau give the initial system, and rows m + 1 up to 2m in the second tableau give the system of equations after the transformation. The column bv indicates the basic variables in the corresponding rows (i.e. after the pivoting transformation, xl is basic variable in the kth equation of the system, while we do not in general have basic variables in the other equations yet). The last column describes the operation which has to be done in order to get the corresponding row.

Here all elements in the pivot column (except the pivot element) are equal to zero in the new tableau. After the transformation, we have the basic variable xl in row m + k which occurs with coefficient +1 now only in equation k of the system and the transformed equation can be found now in row m + k of the second tableau. To illustrate the computations in all rows except the pivot row, consider the determination of element a22. We have to consider four elements: the number a22 on the original position, the element a2l of the second row standing in the pivot column, the element ak2 of the pivot row standing in the second column and the pivot element akl . (The corresponding elements are printed in bold face in the tableau.) These elements form a rectangle, and therefore the rule for computing the values in all rows except the pivot row in the new scheme is also known as the rectangle rule.

Example 8.4 Let us consider the following system of linear equations:

Applying the pivoting procedure, we get the following sequence of tableaus. (Hereafter, the pivot element is always printed in bold face, and the last column is added in order to describe

298 Linear equations and inequalities

the operation that yields the corresponding row):

We avoid the interchange of the column of x3 with the column of x4 to get formally the structure given in Theorem 8.6. In the first two pivoting steps, we have always chosen element one as the pivot element, which ensures that all above tableaus contain only integers. In the second tableau (i.e. rows 5 up to 8), we could already delete one row since rows 6 and 8 are identical (therefore, these row vectors are linearly dependent). From the last tableau we also see that the rank of the coefficient matrix is equal to three (since we have a 3×3 identity submatrix, which means that the largest minor with a value different from zero is of order three). Since x1, x2 and x4 are basic variables, we have found that the column vectors belonging to these variables (i.e. the first, second and fourth columns of A) are linearly independent and constitute a basis of the space generated by the column vectors of matrix A. From rows 13 to 16, we can rewrite the system in terms of the basic variables:

x1 = 5 − 3x3

x2 = −10 + 5x3 x4 = 0

(x3 arbitrary).

Setting now x3 = t with t R, we get the following set of solutions of the considered system:

x1 = 5 − 3t; x2 = −10 + 5t; x3 = t; x4 = 0; t R.

Since we know that one variable can be chosen arbitrarily, we have selected the non-basic variable x3 as the free variable. From the last tableau we see that we could not choose x4 as the free variable since it must have value zero in any solution of the given system of linear equations. However, we can easily see from the transformed system of linear equations, whether one or several variables are uniquely determined so that they cannot be taken as free variables.