// Pseudo-algorithm for Newton-Raphson for systems

initialise(v); do {

//Non-linear iterations formulate_non_linear_system(v); make_total_differential(v);

do {

// Linear iterations: update_linear_system_variables(v);

}

while((linear_system_has_not_converged(v)); update_non_linear_system_after_linear_convergence(v);

}

while((non_linear_system_has_not_converged(v));

17. Iteration methods for linear systems

As noted above, each non-linear iteration requires the solving of a linear system of equations, and a substantial part of the total computing time is spent in solving this system. Through the years a great deal of effort has been put into development of methods that solve linear systems more efficiently, and it is fair to say that a major reason Eclipse quickly became so popular was its linear solver, which made it possible to do real-size three dimensional reservoir simulations with implicit solvers that permitted large time steps. In this chapter we will describe the technique (developed by Appleyard, Cheshire and others in the 1980s), after first looking at fundamental methods that preceded it.

Notation:

Boldface capitals will denote (n x n) matrices.

If A is a matrix, aij will be the element of A in column i, row j.

Boldface small letters are used for vectors.

The problem to solve is always

If A is tridiagonal, the problem (55) can easily be solved exactly by the Thomas algorithm. Apart from that special case, iteration techniques are the only applicable candidates for solving (55), and are what we will concentrate on.

Equation (54) expanded becomes,

Direct, simple approach

Update all xi by previous iteration values. Working from row 1 and downwards, we get (with superscript denoting the iteration counter)

Start with some initial value (“guess”) for x, called x0. Then for k=1,2,3,...

Continue until convergence, i.e. the system has been solved to required precision. The method (57) has very poor convergence (and is probably not used at all).

93

The Gauss-Seidel method

A simple improvement to scheme (57) is to use the updated x-variables as soon as they become available. This approach is actually easier to implement.

The difference from Equation (57) is small but important,

This scheme, called Gauss-Seidel, has much better convergence properties than (57).

Accelerators – the point SOR method

Equation (58) still has much room for improvement. Imagine that the update performed on a single iteration by Equation (58) changes the solution vector in the correct manner (“pushes it in the right direction”), but e.g. not enough. So if we could “push it a little further” the system would obviously (?) converge in fewer iterations. Such arguments are the foundation for many acceleration techniques. We first look at one called point SOR, where SOR means “successive over-relaxation”.

The scheme is,

xik +1 = xik +ω( yik +1 − xik )

So first a temporary y-variable is found exactly as in (58), and then the x-variable is updated by “pushing” the update a little further than the Gauss-Seidel scheme, by a relaxation parameter ω. Note that if ω = 1, point SOR is identical to Gauss-Seidel. Whether point SOR really is an improvement over Gauss-Seidel depends entirely on the relaxation parameter. No general rules for defining ω can be given, except that the optimal ω is between 1 and 2. With a lucky or good choice of ω the system can converge very quickly, while other values can actually give a performance degradation.

In conclusion, point SOR can be very efficient if a good relaxation parameter is found, but the method is problem dependent, and not very robust. (I.e. it doesn’t solve all systems).

The point SOR method has seen many improvements, e.g. block SOR and different versions of line SOR. Mostly the methods are efficient, especially those which are tailored to a specific matrix structure, but the efficiency is always dependent on the relaxation parameter, which may be a disadvantage. We will not discuss the other SOR methods here, but go directly to the relaxation method used in Eclipse.

Conjugate Gradients – ORTHOMIN

Motivation

In the point SOR it was loosely referenced to “pushed in the right direction”. If the problem is one, two, or three-dimensional we should have no difficulty in visualising the direction from the current solution vector to the final one. But the problems we encounter are n-dimensional, with n normally very large. So which direction to move the current vector in, and criteria for if we are moving in a sensible direction or not can be hard to formulate. These questions are directly addressed in the conjugate gradient method, which combined with ORTHOMIN (orthogonal minimisation) is a support pillar in the linear solver in Eclipse.

The orthomin procedure defines a sequence of “search vectors”, q0, q1, q2,... Each q defines a “search direction” in the n-dimensional solution space, such that

•When a new q-vector is constructed, it should be orthogonal to all previously used q-vectors.

•Each qk should optimally be the best search direction available at the present stage of the process

Two q-vectors are orthogonal if Aqk Aq j = 0 when k ≠ j.

94

As usual we start with an initial guess x0. Then at each iteration, x is updated by,

(Note the similarity to point SOR, Equation (59).)

The relaxation parameter ωk is tried optimized, but note that in contrast to the SOR methods, ωk here is updated at each iteration.

The residual rk+1 is a measure for the current error, and is zero if xk+1 is an exact solution to the system:

We see that the solution found in (63) is the only solution to the minimisation problem. Moreover, it must be a minimum since rk +1 2 → ∞ when ω → ±∞.

Construction of the q-vectors

At iteration step k, the next q, qk+1, is constructed from the most recent residual and a linear combination of all previous q-vectors,

j =0

where αkj are parameters to be determined.

Since Aqk +1 Aq j = 0 for all j = 0,1,...,k, we get k equations for determining the unknown αkj :

The orthomin algorithm has then been established:

95

Set initial guess x0, and r = b – Ax, q0 = r. Set iteration counter k = 0.

while( not converged) {

Set ωk by Equation (63) Update x by x = x + ωkqk Update r by r = r - ωkAqk If not converged {

Update α by Equation (66) Update q by Equation (64)

}

Increase k

}

The orthomin procedure can be shown to converge in maximum n iterations (where n was the matrix dimension). This is however a theoretical limit, and much too large to be of interest for typical problems. Experience has shown that the method converges in maximum a few tens of iterations. As a rule of thumb, if orthomin hasn’t converged in about 50 iterations, it is recommended to reduce the time step. Also, for orthomin to function according to the theory it is necessary to store all previous q- vectors. Since each vector is n-dimensional (i.e. large), there will normally be a memory limitation to how many q-vectors that can be stored. If this maximum has been reached, the procedure will discard vectors from the beginning to create necessary storage. But then the search direction that was discarded will most likely be the optimal search direction available. With a (small) finite number of q- vectors stored there is hence a risk that the procedure will be running in loops.

But we shouldn’t stress these problems too much – by experience the method generally has excellent performance.

Preconditioning

Another or additional method to accelerate solving of linear systems is the so-called preconditioning. For Eclipse (and other simulators), ORTHOMIN alone is not sufficient to solve the linear equations with needed efficiency – preconditioning is also needed, and for Eclipse is at least as important as the iteration method to acheive speed-up.

Basically, the argument is, can the original problem be replaced by an equivalent problem that is easier to solve? Problems where A is a diagonal matrix are trivially solved, and the objective of preconditioning is to change the system so that the coefficient matrix becomes “close to diagonal”.

Our problem was A x = b . Multiplying this equation by the identity matrix I =B-1B changes nothing:

Bx is a vector of similar complexity as x, so almost no extra work is required to compute Bx.

If AB-1 ≈ I the system (67) will be easy to solve (i.e. it converges very quickly if an iteration method is used). Obviously if this is going to work, the effort to define and invert B must be substantially smaller than solving the original problem, hence ideally,

•B ≈ A

•B is easily inverted.

But if B ≈ A, inverting B is as difficult as inverting A, and we are no better off. So the challenge is to construct an effective preconditioner B without too much extra work.

Before we describe how this is done in Eclipse we will look at how preconditioning affects orthomin.

96