480—Chapter 16. Additional Regression Methods

Note that EViews augments the instrument list by adding lagged dependent and regressor variables. Note however, that each MA term involves an infinite number of AR terms. Clearly, it is impossible to add an infinite number of lags to the instrument list, so that EViews performs an ad hoc approximation by adding a truncated set of instruments involving the MA order and an additional lag. If for example, you have an MA(5), EViews will add lagged instruments corresponding to lags 5 and 6.

Nonlinear Least Squares

where f is a general function of the explanatory variables xt and the parameters β . Least squares estimation chooses the parameter values that minimize the sum of squared residuals:

We say that a model is linear in parameters if the derivatives of f with respect to the parameters do not depend upon β ; if the derivatives are functions of β , we say that the model is nonlinear in parameters.

It is easy to see that this model is linear in its parameters, implying that it can be estimated using ordinary least squares.

has derivatives that depend upon the elements of β . There is no way to rearrange the terms in this model so that ordinary least squares can be used to minimize the sum-of- squared residuals. We must use nonlinear least squares techniques to estimate the parameters of the model.

Nonlinear least squares minimizes the sum-of-squared residuals with respect to the choice of parameters β . While there is no closed form solution for the parameter estimates, the estimates satisfy the first-order conditions:

482—Chapter 16. Additional Regression Methods

It is worth noting that EViews implicitly adds an additive disturbance to your specification. For example, the input

y = (c(1)*x + c(2)*z + 4)^2

is interpreted as yt = ( c( 1 ) xt + c( 2 ) zt + 4 )2 + t , and EViews will minimize:

t

If you wish, the equation specification may be given by a simple expression that does not include a dependent variable. For example, the input,

(c(1)*x + c(2)*z + 4)^2

is interpreted by EViews as −( c( 1) xt + c( 2) zt + 4 )2 = t , and EViews will minimize:

t

While EViews will estimate the parameters of this last specification, the equation cannot be used for forecasting and cannot be included in a model. This restriction also holds for any equation that includes coefficients to the left of the equal sign. For example, if you specify,

x + c(1)*y = z^c(2)

EViews will find the values of C(1) and C(2) that minimize the sum of squares of the implicit equation:

The estimated equation cannot be used in forecasting or included in a model, since there is no dependent variable.

Estimation Options

Starting Values. Iterative estimation procedures require starting values for the coefficients of the model. There are no general rules for selecting starting values for parameters. The closer to the true values the better, so if you have reasonable guesses for parameter values, these can be useful. In some cases, you can obtain good starting values by estimating a restricted version of the model using least squares. In general, however, you will have to experiment in order to find starting values.

EViews uses the values in the coefficient vector at the time you begin the estimation procedure as starting values for the iterative procedure. It is easy to examine and change these coefficient starting values.

484—Chapter 16. Additional Regression Methods

Output from NLS

Once your model has been estimated, EViews displays an equation output screen showing the results of the nonlinear least squares procedure. Below is the output from a regression of LOG(CS) on C, and the Box-Cox transform of GDP:

Dependent Variable: LOG(CS)

Method: Least Squares

Date: 10/15/97 Time: 11:51

Sample(adjusted): 1947:1 1995:1

Included observations: 193 after adjusting endpoints

Convergence achieved after 80 iterations

LOG(CS)= C(1)+C(2)*(GDP^C(3)-1)/C(3)

If the estimation procedure has converged, EViews will report this fact, along with the number of iterations that were required. If the iterative procedure did not converge, EViews will report “Convergence not achieved after” followed by the number of iterations attempted.

Below the line describing convergence, EViews will repeat the nonlinear specification so that you can easily interpret the estimated coefficients of your model.

EViews provides you with all of the usual summary statistics for regression models. Provided that your model has converged, the standard statistical results and tests are asymptotically valid.

Weighted NLS

Weights can be used in nonlinear estimation in a manner analogous to weighted linear least squares. To estimate an equation using weighted nonlinear least squares, enter your specification, press the Options button and click on the Weighted LS/TSLS option. Fill in the blank after Weight: with the name of the weight series and then estimate the equation.

EViews minimizes the sum of the weighted squared residuals:

t

Nonlinear Least Squares—485

with respect to the parameters β , where wt are the values of the weight series and W is the matrix of weights. The first-order conditions are given by,

and the covariance estimate is computed as:

NLS with AR errors

EViews will estimate nonlinear regression models with autoregressive error terms. Simply select Object/New Object.../Equation… or Quick/Estimate Equation… and specify your model using EViews expressions, followed by an additive term describing the AR correction enclosed in square brackets. The AR term should consist of a coefficient assignment for each AR term, separated by commas. For example, if you wish to estimate,

you should enter the specification:

cs = c(1) + gdp^c(2) + [ar(1)=c(3), ar(2)=c(4)]

See “How EViews Estimates AR Models” on page 500 for additional details. EViews does not currently estimate nonlinear models with MA errors, nor does it estimate weighted models with AR terms—if you add AR terms to a weighted nonlinear model, the weighting series will be ignored.

Nonlinear TSLS

Nonlinear two-stage least squares refers to an instrumental variables procedure for estimating nonlinear regression models involving functions of endogenous and exogenous variables and parameters. Suppose we have the usual nonlinear regression model:

where β is a k -dimensional vector of parameters, and xt contains both exogenous and endogenous variables. In matrix form, if we have m ≥ k instruments zt , nonlinear twostage least squares minimizes:

with respect to the choice of β .

While there is no closed form solution for the parameter estimates, the parameter estimates satisfy the first-order conditions:

486—Chapter 16. Additional Regression Methods

with estimated covariance given by:

How to Estimate Nonlinear TSLS in EViews

EViews performs the estimation procedure in a single step so that you don’t have to perform the separate stages yourself. Simply select Object/New Object.../Equation… or Quick/Estimate Equation… Choose TSLS from the Method: combo box, enter your nonlinear specification and the list of instruments. Click OK.

With nonlinear two-stage least squares estimation, you have a great deal of flexibility with your choice of instruments. Intuitively, you want instruments that are correlated with

G( β) . Since G is nonlinear, you may begin to think about using more than just the exogenous and predetermined variables as instruments. Various nonlinear functions of these variables, for example, cross-products and powers, may also be valid instruments. One should be aware, however, of the possible finite sample biases resulting from using too many instruments.

Weighted Nonlinear Two-stage Least Squares

Weights can be used in nonlinear two-stage least squares estimation. Simply add weighting to your nonlinear TSLS specification above by pressing the Options button, selecting Weighted LS/TSLS option, and entering the name of the weight series.

The objective function for weighted TSLS is,

S( β) = ( y − f( X, β) ) ′W′ WZ( Z′W′WZ)−1Z′W′W( y − f( X, β) ) . (16.43)

The reported standard errors are based on the covariance matrix estimate given by:

where b ≡ bWTSNLLS . Note that if you add AR or MA terms to a weighted specification, the weighting series will be ignored.

Nonlinear Two-stage Least Squares with AR errors

While we will not go into much detail here, note that EViews can estimate non-linear TSLS models where there are autoregressive error terms. EViews does not currently estimate nonlinear models with MA errors.

To estimate your model, simply open your equation specification window, and enter your nonlinear specification, including all AR terms, and provide your instrument list. For example, you could enter the regression specification:

Nonlinear Least Squares—487

cs = exp(c(1) + gdp^c(2)) + [ar(1)=c(3)]

with the instrument list:

c gov

EViews will transform the nonlinear regression model as described in “Estimating AR Models” on page 497, and then estimate nonlinear TSLS on the transformed specification using the instruments C and GOV. For nonlinear models with AR errors, EViews uses a GaussNewton algorithm. See “Optimization Algorithms” on page 956 for further details.

Solving Estimation Problems

EViews may not be able to estimate your nonlinear equation on the first attempt. Sometimes, the nonlinear least squares procedure will stop immediately. Other times, EViews may stop estimation after several iterations without achieving convergence. EViews might even report that it cannot improve the sums-of-squares. While there are no specific rules on how to proceed if you encounter these estimation problems, there are a few general areas you might want to examine.

Starting Values

If you experience problems with the very first iteration of a nonlinear procedure, the problem is almost certainly related to starting values. See the discussion above for how to examine and change your starting values.

Model Identification

If EViews goes through a number of iterations and then reports that it encounters a “Near Singular Matrix”, you should check to make certain that your model is identified. Models are said to be non-identified if there are multiple sets of coefficients which identically yield the minimized sum-of-squares value. If this condition holds, it is impossible to choose between the coefficients on the basis of the minimum sum-of-squares criterion.

is not identified, since any coefficient pair ( β1, β2) is indistinguishable from the pair ( − β1, − β2) in terms of the sum-of-squared residuals.

For a thorough discussion of identification of nonlinear least squares models, see Davidson and MacKinnon (1993, Sections 2.3, 5.2 and 6.3).

Convergence Criterion

EViews may report that it is unable to improve the sums-of-squares. This result may be evidence of non-identification or model misspecification. Alternatively, it may be the result of