Nonlinear Equation Solution Methods—759
EViews also performs a crude trial-and-error search to determine the scale factor a for Marquardt and quadratic hill-climbing methods.
Derivative free methods
Other optimization routines do not require the computation of derivatives. The grid search is a leading example. Grid search simply computes the objective function on a grid of parameter values and chooses the parameters with the highest values. Grid search is computationally costly, especially for multi-parameter models.
EViews uses (a version of) grid search for the exponential smoothing routine.
Nonlinear Equation Solution Methods
When solving a nonlinear equation system, EViews first analyzes the system to determine if the system can be separated into two or more blocks of equations which can be solved sequentially rather than simultaneously. Technically, this is done by using a graph representation of the equation system where each variable is a vertex and each equation provides a set of edges. A well known algorithm from graph theory is then used to find the strongly connected components of the directed graph.
Once the blocks have been determined, each block is solved for in turn. If the block contains no simultaneity, each equation in the block is simply evaluated once to obtain values for each of the variables.
If a block contains simultaneity, the equations in that block are solved by either a GaussSeidel or Newton method, depending on how the solver options have been set.
Gauss-Seidel
By default, EViews uses the Gauss-Seidel method when solving systems of nonlinear equations. Suppose the system of equations is given by:
x1 = f1(x1, x2, º, xN, z)
x2 = f2(x1, x2, º, xN, z)
(B.4)
M
xN = fN(x1, x2, º, xN, z)
where x are the endogenous variables and z are the exogenous variables.
The problem is to find a fixed point such that x = f(x, z). Gauss-Seidel employs an iterative updating rule of the form:
x(i + 1) = f(x(i), z). |
(B.5) |
References—761
system rather than the true derivatives of the equation system when calculating the Newton step. That is, at each iteration, Broyden's method takes a step:
x |
t + 1 |
= x |
t |
– J |
–1F(x , z ) |
(B.10) |
|
|
t |
t |
|
where Jt is the current approximation to the matrix of derivatives of the equation system.
As well as updating the value of x at each iteration, Broyden's method also updates the existing Jacobian approximation, Jt , at each iteration based on the difference between the observed change in the residuals of the equation system and the change in the residuals predicted by a linear approximation to the equation system based on the current Jacobian approximation.
In particular, Broyden's method uses the following equation to update J :
Jt + 1 = Jt + |
(F(xt + 1, z ) – F(xt, z ) – JtDx)Dx¢ |
(B.11) |
|
-------------------------------------------------------------------------------------------Dx¢Dx - |
|||
|
|
where Dx = xt + 1 – xt . This update has a number of desirable properties (see Chapter 8 of Dennis and Schnabel (1983) for details).
In EViews, the Jacobian approximation is initialized by taking the true derivatives of the equation system at the starting values of x . The updating procedure given above is repeated until changes in x between periods become smaller than a specified tolerance. In some cases the method may stall before reaching a solution, in which case a fresh set of derivatives of the equation system is taken at the current values of x , and the updating is continued using these derivatives as the new Jacobian approximation.
Broyden's method shares many of the properties of Newton's method including the fact that it is not dependent on the ordering of equations in the system and that it will generally converge quickly in the vicinity of a solution. In comparison to Newton's method, Broyden's method will typically take less time to perform each iteration, but may take more iterations to converge to a solution. In most cases Broyden's method will take less overall time to solve a system than Newton's method, but the relative performance will depend on the structure of the derivatives of the equation system.
References
Amemiya, Takeshi (1983). “Nonlinear Regression Models,” Chapter 6 in Z. Griliches and M. D. Intriligator (eds.), Handbook of Econometrics, Volume 1, Amsterdam: Elsevier Science Publishers B.V.
Dennis, J. E. and R. B. Schnabel (1983). “Secant Methods for Systems of Nonlinear Equations,” Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, London.
Kincaid, David, and Ward Cheney (1996). Numerical Analysis, 2nd edition, Pacific Grove, CA: Brooks/ Cole Publishing Company.
Press, W. H., S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery (1992). Numerical Recipes in C, 2nd edition, Cambridge University Press.
