Appendix D. Gradients and Derivatives

964—Appendix D. Gradients and Derivatives

˜

where P and P are the projection matrices corresponding to the expressions for the estimators in Chapter 16, “Additional Regression Methods”, beginning on page 461, and where l is the log likelihood contribution function.

Note that the expressions for the regression gradients are adjusted accordingly in the presence of ARMA error terms.

Gradient Summary

To view the summary of the gradients, select View/Gradients and Derivatives/Gradient Summary, or View/Gradients/Summary. EViews will display a summary table showing the sum, mean, and Newton direction associated with the gradients. Here is an example table from a nonlinear least squares estimation equation:

Gradients of the objective function at estimated parameters

Equation: EQ1 Method: Least Squares

Specification: Y = C(1)*EXP(-C(2)*X) + C(3)*EXP( -((X -C(4))^2) / C(5)^2 ) + C(6)*EXP( -((X-C(7))^2) /

C(8)^2 )

Computed using analytic derivatives

Gradients—965

There are several things to note about this table. The first line of the table indicates that the gradients have been computed at estimated parameters. If you ask for a gradient view for an estimation object that has not been successfully estimated, EViews will compute the gradients at the current parameter values and will note this in the table. This behavior allows you to diagnose unsuccessful estimation problems using the gradient values.

Second, you will note that EViews informs you that the gradients were computed using analytic derivatives. EViews will also inform you if the specification is linear, if the derivatives were computed numerically, or if EViews used a mixture of analytic and numeric techniques. We remind you that all MA coefficient derivatives are computed numerically.

Lastly, there is a table showing the sum and mean of the gradients as well as a column labeled “Newton Dir.”. The column reports the non-Marquardt adjusted Newton direction used in first-derivative iterative estimation procedures (see “First Derivative Methods” on page 957).

In the example above, all of the values are “close” to zero. While one might expect these values always to be close to zero when evaluated at the estimated parameters, there are a number of reasons why this will not always be the case. First, note that the sum and mean values are highly scale variant so that changes in the scale of the dependent and independent variables may lead to marked changes in these values. Second, you should bear in mind that while the Newton direction is related to the terms used in the optimization procedures, EViews’ test for convergence does not directly use the Newton direction. Third, some of the iteration options for system estimation do not iterate coefficients or weights fully to convergence. Lastly, you should note that the values of these gradients are sensitive to the accuracy of any numeric differentiation.

Gradient Table and Graph

There are a number of situations in which you may wish to examine the individual contributions to the gradient vector. For example, one common source of singularity in nonlinear estimation is the presence of very small combined with very large gradients at a given set of coefficient values.

EViews allows you to examine your gradients in two ways: as a spreadsheet of values, or as line graphs, with each set of coefficient gradients plotted in a separate graph. Using these tools, you can examine your data for observations with outlier values for the gradients.

Gradient Series

You can save the individual gradient values in series using the Make Gradient Group procedure. EViews will create a new group containing series with names of the form GRAD## where ## is the next available name.

966—Appendix D. Gradients and Derivatives

Note that when you store the gradients, EViews will fill the series for the full workfile range. If you view the series, make sure to set the workfile sample to the sample used in estimation if you want to reproduce the table displayed in the gradient views.

Application to LM Tests

The gradient series are perhaps most useful for carrying out Lagrange multiplier tests for nonlinear models by running what is known as artificial regressions (Davidson and MacKinnon 1993, Chapter 6). A generic artificial regression for hypothesis testing takes the form of regressing:

where u are the estimated residuals under the restricted (null) model, and β are the esti-

mated coefficients. The Z are a set of “misspecification indicators” which correspond to departures from the null hypothesis.

An example program (“GALLANT2.PRG”) for performing an LM auxiliary regression test is provided in your EViews installation directory.

Gradient Availability

The gradient views are currently available for the equation, logl, sspace and system objects. The views are not, however, currently available for equations estimated by GMM or ARMA equations specified by expression.

Derivatives

EViews employs a variety of rules for computing the derivatives used by iterative estimation procedures. These rules, and the user-defined settings that control derivative taking, are described in detail in “Derivative Computation Options” on page 954.

In addition, EViews provides both object views and object procedures which allow you to examine the effects of those choices, and the results of derivative taking. These views and procedures provide you with quick and easy access to derivatives of your user-specified functions.

It is worth noting that these views and procedures are not available for all estimation techniques. For example, the derivative views are currently not available for binary models since only a limited set of specifications are allowed.

Derivative Description

The Derivative Description view provides a quick summary of the derivatives used in estimation.

968—Appendix D. Gradients and Derivatives

Here, EViews reports that it attempted to use analytic derivatives, but that it was forced to use a numeric derivative for C(2) (since it has not yet been taught the derivative of the @DNORM function).

If we set the estimation option so that we only compute fast numeric derivatives, the view would change to

Derivatives of the equation specification

Equation: EQ1

Method: Least Squares

Specification: RESID = Y - (C(1)*(1 - EXP(-C(2)*X)))

Computed using fast numeric derivatives

to reflect the different method of taking derivatives.

If your specification contains autoregressive terms, EViews will only compute the derivatives with respect to the regression part of the equation. The presence of the AR components is, however, noted in the description view.

Derivatives of the equation specification

Equation: EQ1

Method: Least Squares

Specification: [AR(1)=C(3)] = Y - (C(1)*(1 - EXP(-C(2)*X)))

Computed using analytic derivatives

C(1) -1 + exp(-c(2) * x)

C(2) -c(1) * x * exp(-c(2) * x)

*Note: derivative expressions do not account for AR components

Recall that the derivatives of the objective function with respect to the AR components are always computed analytically using the derivatives of the regression specification, and the lags of these values.

One word of caution about derivative expressions. For many equation specifications, analytic derivative expressions will be quite long. In some cases, the analytic derivatives will be longer than the space allotted to them in the table output. You will be able to identify these cases by the trailing “...” in the expression.

To see the entire expression, you will have to create a table object and then resize the appropriate column. Simply click on the Freeze button on the toolbar to create a table object, and then highlight the column of interest. Click on Width on the table toolbar and enter in a larger number.