Diss / (Springer Series in Information Sciences 25) S. Haykin, J. Litva, T. J. Shepherd (auth.), Professor Simon Haykin, Dr. John Litva, Dr. Terence J. Shepherd (eds.)-Radar Array Processing-Springer-Verlag

By considering the overall transformation, illustrated in Fig. S.26, from the set of vectors {tpin)} to the set {qin)}, it is possible to develop the relationship with the QR decomposition algorithm. We define the reverse transformation rp(n) as that which reconstitutes the input data from the orthogonal output vectors, i.e.,

(S.A.5)

Inspection of the MGS network reveals the fact that it is very similar to the frozen QR decomposition processor discussed in Sect. S.6.2. In particular, it is easy to show that the transformation matrix rp(n) is simply composed of the individual orthogonalization coefficients, and takes the form

It follows that the forward transformation is given by

(S.A.7)

The simple identities expressed by (S.A.5) and (S.A.7) are of particular interest since we notice that, whereas the Gram-Schmidt orthogonalization process naturally computes the rp matrix factorization, it is the inverse of this matrix which represents the effective transformation performed by the network.

Although (S.A.1-7) have been presented for order p, they are valid for any order j = 1,2, ... ,p, and, in particular, for order p - 1 we have

(S.A.8)

Returning now to the QR decomposition of X(n), we may multiply both sides of (S.29) by QH(n) to give

X(n) = QH(n) [R~n)]

(S.A.9)

where the vectors Ql(n),Q2(n), ... ,Qp-l(n) are orthonormal, since the matrix QH(n) is unitary. If the matrix R(n) is now factorized according to (S.42), i.e.,

232 T.J. Shepherd and J.G. McWhirter

so that R(n) is of unit upper triangular form, the decomposition of X(n) in (5.A.9) becomes

(5.A.ll)

which is identical in form to (5.A.8). Since this decomposition of X(n) is unique (as can be verified by construction using the MGS procedure), we have

it is clear that the MGS and QR decomposition algorithms both generate a triangular decomposition of the covariance matrix M(n) without the need to compute M(n) explicitly and are similar in many respects. The most striking difference between the two methods, however, is that the QR decomposition algorithm allows the triangular system to be updated row-by-row, whereas, the MGS algorithm operates rigidly on a column-by-column basis, and requires a full orthogonalization process to be performed as each new row of data is received. Ling et al. [5.2J have shown that the MGS procedure can be made recursive in time. By constructing time updates for the projection operations, they have developed an algorithm, (the Recursive Modified Gram-Schmidt or RMGS algorithm), the cell functions for which are very similar to those of the square-root-free QR decomposition algorithms; indeed, the algorithms can be made identical upon some simple redefinitions of variables. Further, the projection techniques employed by Ling et al. may be formulated in a more abstract fashion, after definition of a general data Hilbert space; such techniques have been used by Kalson and Yao [5.3J to derive related algorithms.

Appendix S.B Constraints with Leading Zeros

The constraint pre-processor technique described in Sect. 5.7 will obviously break down if the leading element c1 of the constraint vector c takes zero value. Similarly, the technique in Sect. 5.7.2 will break down if the principal N x N submatrix of C is singular. In this case, at least one of the leading diagonal elements of the transformed constraint matrix C' will be zero. This appendix is devoted to showing how the problem may be avoided in practice. We consider only the conventional Givens rotation algorithm since the argument may be

234 T.J. Shepherd and J.G. McWhirter

i.e., X(n) denotes the first k columns of the data matrix, and the remaining columns are denoted by

Consider first the problem of minimizing II e(n) II with respect to the k-element weight vector Hi assuming that wis fixed. Using the same notation here, for QR decomposition of the n x k matrix X(n), as that which was adopted for the n x (p - 1) matrix in Sect. S.4, it follows by analogy with (S.27) that the optimum weight vector Hi(n) is given by

R(n) Hi(n) + w1udn) + ... + Wp-kup-k(n) = 0, (S.B.6)

where ui(n) denotes the upper k elements of Q(n)Yi(n). Denoting the lower n - k elements of Q(n)Yi(n) by vi(n), it also follows by comparison with (S.29) that

(S.B.7)

and, by analogy with (S.83), it can be shown that the a posteriori residual is given by

The parameter x~k:/)(n) in (S.B.8) denotes the result of rotating the input data sample Xk+i(tn ) with each element of the vectorYi(n - 1). Accordingly, the scalar X~k:il)(n) will emerge on the appropriate clock cycle as the value of Xout generated by the bottom cell in the k + ith column of the trapezoidal array defined by the top k rows in Fig. S.27. Note that x~k:l)(n) is equivalent to (Xk+ 1(n) according to the notation in Sect. 5.6.3. The scalar 'l'k+ 1(n), which was also defined in Sect. 5.6.3, denotes the product of all cosine parameters computed by the top k boundary processors in the course ofeliminating the first k elements of the input vector x(tn ). It will, therefore, emerge as the value of'l'out produced by the kth boundary cell on the appropriate clock cycle.

In order to complete the overall least-squares minimization problem, it remains to determine the (p - k)-element vector of weights w(n) which minimizes the norm of the residual vector e(n) as expressed in (S.B.7) subject to the constraint in (S.B.1). Now, since the matrix S(n) denotes the lower n - k rows of

the unitary matrix Q(n}, it follows that S(n}SH(n} = I, and so, from (5.B.7) we have

(5.B.9)

The minimization with respect to wcan therefore be achieved by supplying the vectors VI (n), v2(n}, ... ,vp-k(n} as input data to a canonical least-squares processor of order p - k with a single pre-processor to apply the constraint vector c: but this is precisely the function of the bottom p - k + 1 rows in Fig. 5.27, and, furthermore, the elements of each vector vi(n} are precisely the scalars x~k:/)(n} which emerge from the top k rows and enter the constraint preprocessor. It follows that the bottom p - k + 1 rows in Fig. 5.27, treated as a separate least-squares processor, produce the output residual

(5.B.10)

where w(n} is the vector which minimizes II e(n} II. It can therefore be deduced from (5.B.8) that the a posteriori residual for the overall minimization problem is given by

This output will be produced directly by the complete array, provided that the product of cosine parameters yo(n) produced in the top k rows is supplied as input to the boundary processor in the (k + 1)th row in the normal way.

The discussion above, which relates to a single constraint with k leading zeros, can readily be generalized to apply to a general constraint matrix C for which the reduced trapezoidal matrix C' is captured on arbitrary rows within the canonical least-squares processor array. The general procedure may be summarized as follows: (1) Input the constraint matrix to the processor array with all rows operating in the adaptive mode. (2) Freeze those rows on which the reduced constraint matrix has been captured, leaving all others free to adapt.

(3) Input the data matrix in the usual way, ensuring that the appropriate product of rotation parameters is generated by the scalar which propagates along the boundary of the array.

Appendix 5.C Weighted Least-Squares and Hyperbolic Rotations

We have already mentioned in Sect. 5.6 that the square-root-free algorithm can be simply employed to execute weighted least-squares, and that this fact was originally noted by Gentleman [5.18]. In this appendix we demonstrate how weighted least-squares can be implemented also within the original square-root algorithm, and how negative weights can be employed for both real and complex data, thus providing the facility of hyperbolic rotations and data downdating.

236 T.J. Shepherd and J.G. McWhirter

The weighted least-squares problem may be stated as follows: Defining the canonical residual vector e in the normal way, viz.

compute the final element e(n, n) of e(n) for which the weighted norm-square eH(n)G(n)e(n) is a minimum with respect to wen), where the n x n weighting matrix G(n) is given by

and each weighting coefficient g(q) (q = 1,2, ... , n) is real. The matrix G(n) defines a metric in the appropriate n-dimensional vector space. A form of data weighting has, of course, already been included in the preceding least-squares formulation in the form of the exponential deweighting factor p. This constitutes a weighting function

p = 1, and concentrate upon weights g that depend only upon a fixed time. We first suppose that the {g(q)} are positive. In this case the weighted

problem can be converted to the unweighted problem by scaling the data, for, defining (eHGe) as (EG)2, we have

(5.C.4)

where

(5.C.5)

Thus, by performing unweighted least-squares on the data vectors [gl/2(1)xT(td, gl/2(1)y(tl)]' [gl/2(2)XT(t2)' gl/2(2)y(t2)], etc., one may obtain the weight vector wand weighted residual eG using methods described in Sects. 5.1-4. The required element of the vector e is then obtained from (5.C.5) as

(5.C.6)

These operations may clearly be performed by the systolic network described in Sect. 5.5, appended by multipliers to scale the data at the input and output. Alternatively, we may employ an algorithm for a network in which the weighting is applied internally; this will be seen to lead to a more flexible processor.

Assume, as in Sect. 5.5, that EG(n - 1) has been minimized already, and that the QR decomposition of the weighted data matrix G I /2(n - 1)X(n - 1) has been performed, yielding the triangular matrix RG(n - 1). The next row of data to be rotated into the matrix R will now be the weighted vector gl/2(n)XT(tn).

Consider the requisite complex orthogonal rotation taking place on the first row of RG(n - 1), the elements of which are denoted, as usual, by ri and rk ,

Xi and Xk represent elements ofthe raw input row data vector xT (t,,), and g(n) has been abbreviated to g. Now the update equation reads

ri

then (S.C.10a, b) become

(S.C.12a)

(S.C.12b)

The following observations may be made from inspection of the resulting equations. Explicit scaling of the input vector xT is no longer necessary; scaling is performed within the algorithm. The update equations for the elements of RG differ from the original (unit weight) updates by the inclusion of the factor g in (S.C.8) and (S.C.12a). No further computation involving the factor g need be made in each rotation, as x;. represents an element of the row vector to be rotated with the second row of RG(n - 1), and, as in the case of the original input vector, contains no overall multiplicative factor of g 1/2. Rotations in the first row therefore need only involve computation of the elements x;', and the scaling by g 1/2 can again be incorporated into the rotations in the second row, and so on for each row of RG(n - 1). The inverse scaling in (S.C.4) will be unnecessary, as the final residual is of the form of x;. defined without the gl/2 factor. Note that only the factor g appears in the algorithm - the scaling factor gl/2 never occurs. Finally, it can be seen that § and care not in general parameters of a unitary

Within the associated processor the weighting 9 must be applied within every cell. It is necessary to broadcast the value of 9 appropriate to a given data vector along each row of the array in turn. This requires knowledge of 9 from each boundary cell, and so 9 must also be passed diagonally from one boundary cell to the next, having been entered at the first boundary cell.

For future reference let us note that an internal cell of this modified algorithm and processor now effectively performs the transformation

In many instances it proves convenient to apply a negative weighting. For example, if we wish to perform data downdating by subtracting from the leastsquares sum a previously-added term of weight unity, the same data can be subsequently added with weight -1. In this instance the transformation fI( -1) constitutes a hyperbolic or "J-orthogonal" rotation. (For a discussion see, for example, Steinhardt [S.94]). Such rotations must be applied with caution, however, for the error criterion (EG)2 must remain positive for a leastsquares solution. Negative weighting may be imposed for real data by applying a totally imaginary scaling to the data, and treating the algorithm as if all data were real - that is, using the inner product eT e; the derivation of the modified algorithm and processor then goes through as for the positively weighted case.

For complex data the situation is more difficult. The factorization of G as applied in (S.C.4) no longer exists: there is no number z, real or complex, for which zz* is negative, and so the derivation above of the modified weighted least-squares algorithm is invalid. We must consider the situation in which the metric G is left intact - that is to say that the square-root of G is not involved.

We seek a transformation H which simultaneously triangularizes the data matrix X, and preserves the metric EG , viz.

Provided that F is nonnegative definite, (EG)2 is clearly minimized with respect to w when

(S.C.24)

Let us construct the matrix H in a recursive fashion. Suppose that at time tn - 1 we have such a matrix, H(n - 1), for which

where G'(n) is also diagonal. A product of rotations of the type f'(g) is clearly capable of the transformation (5.C.31); however, it can be shown using (5.C.1S) that the same product of rotations can only effect (5.C.32), leaving G'(n) diagonal, if the general form of G' is

in which I is the unit matrix. In fact, the transformation (5.C.32) preserves such a form, and thus the assumption of the nonnegativity of F is supported. It is also possible to show that the f'(g) rotation will initiate this form from time t1, provided that the matrix RG is initialized to zero. Information about all weights {g(q)} is then contained in RG , and there is no need to store the matrix G' in memory.

The conclusion of this analysis is that the rotation f'(g) may be used in a triangular processor to perform weighted least-squares minimization, even for negative weights and complex data.

Application of a weighting factor is far simpler in the square-root-free algorithms. Recalling the scaling of elements of both Rand xT performed in Sect. 5.4.4, and used in the derivation of the square-root-free algorithm, we may easily include the weighting factor as follows: