C++ For Mathematicians (2006) [eng]

For example, here is a header file named trace.h that provides a template procedure to calculate the trace of an Array2D matrix:

Program 13.3: A template to calculate the trace of an Array2D matrix.

1 #ifndef TRACE_H

2#define TRACE_H

3

4#include "tnt.h"

5

6template <class T>

7 T trace(const TNT::Array2D<T>& X) {

8T sum = T(0);

9 for (int k=0; k<X.dim1() && k<X.dim2(); k++) {

10sum += X[k][k];

11}

12return sum;

13}

14

15 #endif

The TNT package provides rudimentary array arithmetic. If A and B are arrays of the same type and same size (e.g., both Array2D<long> and both m × n) then the following operations are available.

In each case, the arithmetic is performed element by element. If A, B, and C are Array2D<double> objects, and if A and B are the same size (both m × n), then C=A*B; sets C to be an m ×n-array in which C[i][j] is A[i][j]*B[i][j]. It is not matrix multiplication.

Matrix multiplication is available via the matmult procedure. Matrix multiplication can only be performed on Array2D arrays of the appropriate shapes. The statement C = matmult(A,B); sets C to the appropriate size (if A is m ×n and B is n × p, then C is m × p) and sets the elements of C appropriately:

n−1

C[i][j] = ∑ A[i][k]*B[k][j] k=0

The TNT package does not provide scalar multiplication of arrays, nor does it provide matrix–vector (Array2D times Array1D) multiplication. However, it is not difficult to create template procedures to perform these tasks. For example, here is a template for scalar–vector multiplication.

#include "tnt.h" using namespace TNT; template <class T>

void scalar_vector_multiply(T s,

const Array1D<T>& vec, Array1D<T>& ans) {

ans = Array1D<T>(vec.dim1()); // resize ans for (int i=0; i<vec.dim1(); ++i) {

ans[i] = s*vec[i];

}

}

More advanced linear algebra functions for real matrices are provided by the JAMA package. This package provides the following classes:

•JAMA::Cholesky — Given a real, symmetric, positive definite matrix A, find a lower triangular matrix L so that A = LLT . Header file: jama_cholesky.h.

•JAMA::Eigenvalue — Given a real, square matrix A find the eigenvalues and eigenvectors of A. If these are complex, the real and imaginary parts of the eigenvalues are accessed separately. (This class does not use the complex<> types.) Header file: jama_eig.h.

•JAMA::LU — Given a real, m ×n-matrix (with m ≥ n) A, find a lower triangular matrix L and an upper triangular matrix U so that LU is a (row permutation of) A. Header file: jama_lu.h.

•JAMA::QR — Given a real, m×n-matrix (with m ≥ n) A, find an m×n orthogonal matrix Q and an n ×n upper triangular matrix R so that A = QR. Header file: jama_qr.h.

•JAMA::SVD — Given a real, m × n matrix A (with m ≥ n), find an m × n orthogonal matrix U, an n × n diagonal matrix Σ, and an n × n orthogonal matrix V so that A = UΣV T . Header file: jama_svd.h.

The design philosophy for all these classes is the same. To find, say, the eigenvalues of a matrix A, we do not use a procedure to which A is passed. Instead, we create a JAMA::Eigenvalue object like this:

#include "tnt.h" #include "jama_eig.h" using namespace TNT; using namespace JAMA;

int main() { Array2D<double> A(10,10);

// assign values to the elements of A

....

Eigenvalue<double> eigs(A);

....

}

This code loads the necessary headers and sets up a matrix A. We then create an Eigenvalue object named eigs. The matrix A is passed to the constructor.

The eigenvalues and eigenvectors are now embedded inside the Eigenvalue object named eigs. To extract the information we use one of the access methods provided by the Eigenvalue class:

•eigs.getRealEigenvalues() returns an Array1D object containing the real parts of the matrix’s eigenvalues.

•eigs.getImagEigenvalues() returns an Array1D object containing the imaginary parts of the eigenvalues.

•eigs.getV() returns an Array2D (matrix) whose columns contain the eigenvectors of the matrix. (Refer to the documentation to see how Eigenvalue handles complex eigenvectors.)

There are other linear algebraic entities one might like to compute including the rank or determinant of a matrix, or the solution to the linear system Ax = b. The JAMA package provides tools for doing this. To find them, one needs to browse the documentation for the various classes (Cholesky, Eigenvalue, etc.), but it’s not too hard to guess where these might lie. Some examples:

•To find the determinant of a matrix: It’s easy to find the determinant of a matrix from its LU-factorization. Naturally enough, the LU class includes a det() method. For example, if A is a square, Array2D<double> matrix, the following code finds its determinant,

LU alu(A);

cout << "The determinant of A is " << alu.det() << endl;

•To solve a linear system Ax = b: Again, this is often found through the LUfactorization, and the LU class contains two solve() methods: one that solves Ax = b and another that solves AX = B (where x and b are vectors, and X and B are matrices).

The QR class also provides solve() methods; if the system is overdetermined, this solve gives a least-squares solution.

In the special case that A is symmetric and positive definite, a solution to the linear system can also be found through the Cholesky factorization. The class Cholesky has a method named is_spd() to check if a matrix is symmetric and positive definite, and a pair of solve() methods for solving linear systems.

Generally, it is numerically unwise to solve Ax = b by inverting A and calculating A−1b. However, if we wish to find the inverse of a square matrix A, we can create an identity matrix I and solve AX = I.

See the documentation for details.

•To find the rank of a matrix: The rank of A is the number of nonzero singular values. So, to find the rank of a matrix, use the SVD class’s rank() method:

Array2D<double> A;

...

SVD sing_vals(A);

cout << "The rank of A is " << sing_vals.rank() << endl;

The calculation of the rank of a matrix is difficult if the matrix is ill conditioned. To check a matrix’s condition number, use the cond() method, also found in the SVD class.

We illustrate the use of the TNT and JAMA packages with the following program. Recall that a Hilbert matrix is an n × n-matrix whose i, j-entry is 1/(i + j − 1) (in the usual notation in which the upper left corner contains the 1,1-entry). Hilbert matrices are invertible, but notoriously ill conditioned. Hence, finding their inverses is numerically unstable. The following program creates a Hilbert matrix (whose size is specified by the user), finds its inverse from its LU-factorization, calculates its eigenvalues, and (because Hilbert matrices are symmetric and positive definite) finds its Cholesky factorization.

Program 13.4: A program to illustrate the TNT and JAMA packages with calculations on a Hilbert matrix.

1#include "tnt.h"

2 #include "jama_lu.h"

3#include "jama_eig.h"

4 #include "jama_cholesky.h"

5 #include <iostream>

6 using namespace std;

7 using namespace TNT;

8using namespace JAMA;

9

10// A program to demonstrate the use of TNT and JAMA packages for

11// linear algebra work.

12

13int main() {

14// Get the size of the matrix from the user

15cout << "Enter Hilbert matrix size --> ";

16int n;

17cin >> n;

18

19if (n < 2) {

20cerr << "Please enter a value greater than 1" << endl;

21return 1;

22}

23

24// H holds the n-by-n Hilbert matrix and eye holds the n-by-n

25// identity matrix.

26Array2D<double> H(n,n);

27Array2D<double> eye(n,n);

28

29// Fill in the entries in H

30for (int i=0; i<n; i++) {

31for (int j=0; j<n; j++) {

32H[i][j] =1./(i+j+1);

33}

34}

35

36// Set up identity matrix

37for (int i=0; i<n; i++) eye[i][i] = 1;

38

39// Print out H

40cout << "H = " << endl;

41cout << H << endl;

42

43// Use the LU decomposition’s solve method to find the inverse of H.

44LU<double> HLU(H);

45Array2D<double> Hinv = HLU.solve(eye);

46cout << "H inverse = " << endl;

47cout << Hinv<< endl;

48

49// Use the LU decomposition of H to calculate its determinant

50cout << "The determinant of H is " << HLU.det() << endl;

51

52// Use the Eigenvalue class to find the eigenvalues of H

53Eigenvalue<double> Heig(H);

54cout << "The eigenvalues of H are " << endl;

55Array1D<double> eig_vals;

56Heig.getRealEigenvalues(eig_vals);

57for (int i=0; i<n; i++) cout << eig_vals[i] << " ";

58cout << endl << endl;

59

60// Use the Cholesky class to find a matrix C so that C*C’ = H

61Cholesky<double> Hchol(H);

62Array2D<double> C = Hchol.getL();

63cout << "The Cholesky matrix for H is " << endl;

64cout << C << endl;

65

66// Construct C’s transpose

67Array2D<double> Ctrans(n,n);

68for (int i=0; i<n; i++) {

69for (int j=0; j<n; j++) {

70Ctrans[i][j] = C[j][i];

71}

72}

73

74// Check that C*C’ gives H

75cout << "C * C’ = " << endl;

76cout << matmult(C,Ctrans) << endl;

H=

5 5

1 0.5 0.333333 0.25 0.2

0.5 0.333333 0.25 0.2 0.166667

0.333333 0.25 0.2 0.166667 0.142857

0.25 0.2 0.166667 0.142857 0.125

0.2 0.166667 0.142857 0.125 0.111111

Hinverse =

5 5

called Matrix and its constructor takes two arguments giving the number of rows and columns. Here is an example,

Matrix A(2,3);

Now A is a 2 ×3-matrix. It is also possible to declare a matrix without specifying its size, for example, Matrix A;.

Elements of a matrix are specified using the standard mathematical convention: the element in row i and column j of a matrix A is A(i,j). Here, i (respectively, j) is at least 1 and at most the number of rows (respectively, columns) of A. The expression A(i,j) may appear on either side of an assignment. (It is also possible to have newmat use the C++ multidimensional array notation A[i][j] in which case the subscripts begin at zero, but this is not recommended.)

The Matrix constructor does not specify values for its elements; it is the programmer’s responsibility to handle that. So, for example, to create a 10 × 5-matrix of zeros, do this:

Matrix Z(10,5);

Z = 0.;

In addition to Matrix, the newmat package provides several other classes for holding matrices including the following,

SquareMatrix,

UpperTriangularMatrix,

LowerTriangularMatrix,

SymmetricMatrix,

DiagonalMatrix,

IdentityMatrix,

and various banded matrices. For vectors, the package provides ColumnVector and RowVector. These alternatives use the computer’s memory efficiently. In addition, some procedures require specific types of matrices. For example, the EigenValues procedure may only be used on symmetric matrices and the eigenvalues are returned in a diagonal matrix:

The SymmetricMatrix class is clever about assigning values to elements. Consider this code.

#include <iostream> #include "newmat.h" #include "newmatio.h" using namespace std;

int main() { SymmetricMatrix A(3); A(1,2) = 3;

A(3,2) = 2;

cout << A << endl;

Because A is symmetric, setting A(i,j) to some value automatically sets A(j,i) to the same value as well.

The newmat packages provide a rich assortment of matrix operations. Here is a brief description of some of them.

Considerable care was taken in designing these operations. For example, to solve the linear system Ax = b, the statement x = A.i() * b; does not invert the matrix, but finds the solution by a better method. However, if A and B are n×n-matrices and x is an n-vector, the expression A*(B*x) evaluates much more quickly than (A*B)*x. If you type the ambiguous A*B*x it is not clear in what order the computer does the calculation.

Scalar–matrix calculations (including scalar–vector) behave as you might expect. If s is type double and A is type Matrix, then A+s (or s+A) gives a new matrix whose i, j entry is A(i,j)+s. The statement A+=s; adds the value in s to all the elements in A. The operations -, *, and / behave analogously.

The nrows() and ncols() methods give the number of rows and columns, respectively, of a matrix. There is a variety of methods for modifying the shape (and type) of a matrix including ReSize(), AsColumn(), AsRow(), AsDiagonal(), and so on. These are spelled out in the documentation.

The newmat package provides a rich assortment of procedures for important linear algebraic problems including: Cholesky factorization of symmetric matrices, QRfactorization, singular value decomposition, symmetric matrix eigenvalue decomposition, fast Fourier (and other trigonometric) transforms, determinant, trace, various matrix norms, dot product of vectors, and so on.

To illustrate some of these, here is a program that finds the determinant, trace, eigenvalues, and inverse of a Hilbert matrix (compare with Program 13.4).

Program 13.5: A program to illustrate the newmat package with calculations on a Hilbert matrix.

1 #include <iostream>

2#include "newmat.h"

3 #include "newmatio.h"

4#include "newmatap.h"

5

6using namespace std;

7

8int main() {

9 cout << "Enter size of Hilbert matrix --> ";

10int n;

11cin >> n;

12

13SymmetricMatrix H(n);

14for (int i=1; i<=n; i++) {

15for (int j=i; j<=n; j++) {

16H(i,j) = 1. / double(i+j-1);

17}

18}

19

20 cout << "The Hilbert matrix is " << endl << H << endl;

21

22 cout << "Its det is " << H.Determinant() << endl;

23

24 cout << "Its inverse is " << endl << H.i() << endl;

25

26DiagonalMatrix D;

27EigenValues(H,D);

28

29cout << "The eigenvalues of the Hilbert matrix are" << endl;

30cout << D << endl;

31

32cout << "Trace calculated two ways: " << H.Trace() << " and "

33<< D.Trace() << endl;

Its det is 3.7493e-12 Its inverse is

25.000000 -300.000000 1050.000000 -1400.000000 630.000000 -300.000000 4800.000000 -18900.000000 26880.000000 -12600.000000