Chapter 22. The Log Likelihood (LogL) Object
EViews contains customized procedures which help solve the majority of the estimation problems that you might encounter. On occasion, however, you may come across an estimation specification which is not included among these specialized routines. This specification may be an extension of an existing procedure, or it could be an entirely new class of problem.
Fortunately, EViews provides you with tools to estimate a wide variety of specifications through the log likelihood (logl) object. The logl object provides you with a general, openended tool for estimating a broad class of specifications by maximizing a likelihood function with respect to parameters.
When working with a log likelihood object, you will use EViews’ series generation capabilities to describe the log likelihood contribution of each observation in your sample as a function of unknown parameters. You may supply analytical derivatives of the likelihood for one or more parameters, or you can simply let EViews calculate numeric derivatives automatically. EViews will search for the parameter values that maximize the specified likelihood function, and will provide estimated standard errors for these parameter estimates.
In this chapter, we provide an overview and describe the general features of the logl object. We also give examples of specifications which may be estimated using the object. The examples include: multinomial logit, unconditional maximum likelihood AR(1) estimation, Box-Cox regression, disequilibrium switching models, least squares with multiplicative heteroskedasticity, probit specifications with heteroskedasticity, probit with grouped data, nested logit, zero-altered Poisson models, Heckman sample selection models, Weibull hazard models, GARCH(1,1) with t-distributed errors, GARCH with coefficient restrictions, EGARCH with a generalized error distribution, and multivariate GARCH.
Overview
Most of the work in estimating a model using the logl object is in creating the text specification which will be used to evaluate the likelihood function.
If you are familiar with the process of generating series in EViews, you should find it easy to work with the logl specification, since the likelihood specification is merely a list of series assignment statements which are evaluated iteratively during the course of the maximization procedure. All you need to do is write down a set of statements which, when evaluated, will describe a series containing the contributions of each observation to the log likelihood function.
672—Chapter 22. The Log Likelihood (LogL) Object
To take a simple example, suppose you believe that your data are generated by the conditional heteroskedasticity regression model:
yt = β1 + β2xt + β3zt + t
(22.1)
t N( 0, σ2zαt )
where x , y , and z are the observed series (data) and β1, β2, β3, σ, α are the parameters of the model. The log likelihood function (the log of the density of the observed data) for a sample of T observations can be written as:
l( β, α, σ) = |
− |
T |
( log ( 2π) + log σ |
2 |
) − |
α |
T |
|
|
|
|
T ( yt−β1−β2xt− β3zt)2 |
--2 |
|
--- |
Σ log ( zt) − |
|
Σ |
------------------------------------------------ |
α |
|
|
|
|
|
|
|
|
2 t = 1 |
|
|
|
t = 1 |
|
σ2zt |
|
|
T |
|
|
yt− β1−β2xt−β3zt |
|
1 |
|
|
|
|
(22.2) |
= |
|
|
− |
log ( σ |
2 |
α |
|
Σ |
|
log φ |
------------------------------------------- |
α ⁄ 2 |
|
|
2-- |
|
zt ) |
|
|
t = 1 |
|
|
σzt |
|
|
|
|
|
|
|
|
|
where φ is the standard normal density function.
Note that we can write the log likelihood function as a sum of the log likelihood contributions for each observation t :
T
l( β, α, σ) = Σ lt( β, α, σ)
t = 1
where the individual contributions are given by:
lt( β, α, σ) = |
yt− β1−β2xt−β3zt |
− |
1 |
log ( σ |
2 |
α |
) |
log φ |
-------------------------------------------α ⁄ 2 |
|
2-- |
|
zt |
|
|
|
|
|
|
|
|
|
|
σz t |
|
|
|
|
|
|
|
Suppose that you know the true parameter values of the model, and you wish to generate a series in EViews which contains the contributions for each observation. To do this, you could assign the known values of the parameters to the elements C(1) to C(5) of the coefficient vector, and then execute the following list of assignment statements as commands or in an EViews program:
series res = y - c(1) - c(2)*x - c(3)*z
series var = c(4) * z^c(5)
series logl1 = log(@dnorm(res/@sqrt(var))) - log(var)/2
The first two statements describe series which will contain intermediate results used in the calculations. The first statement creates the residual series, RES, and the second statement creates the variance series, VAR. The series LOGL1 contains the set of log likelihood contributions for each observation.
Specification—673
Now suppose instead that you do not know the true parameter values of the model, and would like to estimate them from the data. The maximum likelihood estimates of the parameters are defined as the set of parameter values which produce the largest value of the likelihood function evaluated across all the observations in the sample.
The logl object makes finding these maximum likelihood estimates easy. Simply create a new log likelihood object, input the assignment statements above into the logl specification view, then ask EViews to estimate the specification.
In entering the assignment statements, you need only make two minor changes to the text above. First, the series keyword must be removed from the beginning of each line (since the likelihood specification implicitly assumes it is present). Second, an extra line must be added to the specification which identifies the name of the series in which the likelihood contributions will be contained. Thus, you should enter the following into your log likelihood object:
@logl |
logl1 |
res |
= |
y |
- c(1) - c(2)*x - c(3)*z |
var |
= |
c(4) * z^c(5) |
logl1 |
= |
log(@dnorm(res/@sqrt(var))) - log(var)/2 |
The first line in the log likelihood specification, @logl logl1, tells EViews that the series LOGL1 should be used to store the likelihood contributions. The remaining lines describe the computation of the intermediate results, and the actual likelihood contributions.
When you tell EViews to estimate the parameters of this model, it will execute the assignment statements in the specification repeatedly for different parameter values, using an iterative algorithm to search for the set of values that maximize the sum of the log likelihood contributions. When EViews can no longer improve the overall likelihood, it will stop iterating and will report final parameter values and estimated standard errors in the estimation output.
The remainder of this chapter discusses the rules for specification, estimation and testing using the likelihood object in greater detail.
Specification
To create a likelihood object, choose Object/New Object…/LogL or type “logl” in the command window. The likelihood window will open with a blank specification view. The specification view is a text window into which you enter a list of statements which describe your statistical model, and in which you set options which control various aspects of the estimation procedure.
674—Chapter 22. The Log Likelihood (LogL) Object
Specifying the Likelihood
As described in the overview above, the core of the likelihood specification is a set of assignment statements which, when evaluated, generate a series containing the log likelihood contribution of each observation in the sample. There can be as many or as few of these assignment statements as you wish.
Each likelihood specification must contain a control statement which provides the name of the series which is used to contain the likelihood contributions. The format of this statement is:
@logl series_name
where series_name is the name of the series which will contain the contributions. This control statement may appear anywhere in the logl specification.
Whenever the specification is evaluated, whether for estimation or for carrying out a View or Proc, each assignment statement will be evaluated at the current parameter values, and the results stored in a series with the specified name. If the series does not exist, it will be created automatically. If the series already exists, EViews will use the existing series for storage, and will overwrite the data contained in the series.
If you would like to remove one or more of the series used in the specification after evaluation, you can use the @temp statement, as in:
@temp series_name1 series_name2
This statement tells EViews to delete any series in the list after evaluation of the specification is completed. Deleting these series may be useful if your logl creates a lot of intermediate results, and you do not want the series containing these results to clutter your workfile.
Parameter Names
In the example above, we used the coefficients C(1) to C(5) as names for our unknown parameters. More generally, any element of a named coefficient vector which appears in the specification will be treated as a parameter to be estimated.
In the conditional heteroskedasticity example, you might choose to use coefficients from three different coefficient vectors: one vector for the mean equation, one for the variance equation, and one for the variance parameters. You would first create three named coefficient vectors by the commands:
coef(3) beta
coef(1) scale
coef(1) alpha
Specification—675
You could then write the likelihood specification as:
@logl logl1
res = y - beta(1) - beta(2)*x - beta(3)*z
var = scale(1)*z^alpha(1)
logl1 = log(@dnorm(res/@sqrt(var))) - log(var)/2
Since all elements of named coefficient vectors in the specification will be treated as parameters, you should make certain that all coefficients really do affect the value of one or more of the likelihood contributions. If a parameter has no effect upon the likelihood, you will experience a singularity error when you attempt to estimate the parameters.
Note that all objects other than coefficient elements will be considered fixed and will not be updated during estimation. For example, suppose that SIGMA is a named scalar in your workfile. Then if you redefine the subexpression for VAR as:
var = sigma*z^alpha(1)
EViews will not estimate SIGMA. The value of SIGMA will remain fixed at its value at the start of estimation.
Order of Evaluation
The logl specification contains one or more assignment statements which generate the series containing the likelihood contributions. EViews always evaluates from top to bottom when executing these assignment statements, so expressions which are used in subsequent calculations should always be placed first.
EViews must also iterate through the observations in the sample. Since EViews iterates through both the equations in the specification and the observations in the sample, you will need to specify the order in which the evaluation of observations and equations occurs.
By default, EViews evaluates the specification by observation so that all of the assignment statements are evaluated for the first observation, then for the second observation, and so on across all the observations in the estimation sample. This is the correct order for recursive models where the likelihood of an observation depends on previously observed (lagged) values, as in AR or ARCH models.
You can change the order of evaluation so EViews evaluates the specification by equation, so the first assignment statement is evaluated for all the observations, then the second assignment statement is evaluated for all the observations, and so on for each of the assignment statements in the specification. This is the correct order for models where aggregate statistics from intermediate series are used as input to subsequent calculations.
676—Chapter 22. The Log Likelihood (LogL) Object
You can explicitly select which method of evaluation you would like by adding a statement to the likelihood specification. To force evaluation by equation, simply add a line containing the keyword “@byeqn”. To explicitly state that you require evaluation by observation, the “@byobs” keyword can be used. If no keyword is provided, @byobs is assumed.
In the conditional heteroskedasticity example above, it does not matter whether the assignment statements are evaluated by equation (line by line) or by observation, since the results do not depend upon the order of evaluation.
However, if the specification has a recursive structure, or if the specification requires the calculation of aggregate statistics based on intermediate series, you must select the appropriate evaluation order if the calculations are to be carried out correctly.
As an example of the @byeqn statement, consider the following specification:
@logl robust1
@byeqn
res1 = y-c(1)-c(2)*x
delta = @abs(res1)/6/@median(@abs(res1))
weight = (delta<1)*(1-delta^2)^2
robust1 = -(weight*res1^2)
This specification performs robust regression by downweighting outlier residuals at each iteration. The assignment statement for DELTA computes the median of the absolute value of the residuals in each iteration, and this is used as a reference point for forming a weighting function for outliers. The @byeqn statement instructs EViews to compute all residuals RES1 at a given iteration before computing the median of those residuals when calculating the DELTA series.
Analytic Derivatives
By default, when maximizing the likelihood and forming estimates of the standard errors, EViews computes numeric derivatives of the likelihood function with respect to the parameters. If you would like to specify an analytic expression for one or more of the derivatives, you may use the @deriv statement. The @deriv statement has the form:
@deriv pname1 sname1 pname2 sname2 …
where pname is a parameter in the model and sname is the name of the corresponding derivative series generated by the specification.
For example, consider the following likelihood object that specifies a multinomial logit model:
' multinomial logit with 3 outcomes
@logl logl1
Specification—677
xb2 = b2(1)+b2(2)*x1+b2(3)*x2 xb3 = b3(1)+b3(2)*x1+b3(3)*x2 denom = 1+exp(xb2)+exp(xb3)
'derivatives wrt the 2nd outcome params @deriv b2(1) grad21 b2(2) grad22 b2(3) grad23 grad21 = d2-exp(xb2)/denom
grad22 = grad21*x1 grad23 = grad21*x2
'derivatives wrt the 3rd outcome params @deriv b3(1) grad31 b3(2) grad32 b3(3) grad33 grad31 = d3-exp(xb3)/denom
grad32 = grad31*x1 grad33 = grad31*x2
'specify log likelihood
logl1 = d2*xb2+d3*xb3-log(1+exp(xb2)+exp(xb3))
See Greene (1997), Chapter 19.7.1 for a discussion of multinomial logit models. There are three possible outcomes, and the parameters of the three regressors (X1, X2 and the constant) are normalized relative to the first outcome. The analytic derivatives are particularly simple for the multinomial logit model and the two @deriv statements in the specification instruct EViews to use the expressions for GRAD21, GRAD22, GRAD23, GRAD31, GRAD32, and GRAD33, instead of computing numeric derivatives.
When working with analytic derivatives, you may wish to check the validity of your expressions for the derivatives by comparing them with numerically computed derivatives. EViews provides you with tools which will perform this comparison at the current values of parameters or at the specified starting values. See the discussion of the Check Derivatives view of the likelihood object in “Check Derivatives” on page 683.
Derivative Step Sizes
If analytic derivatives are not specified for any of your parameters, EViews numerically evaluates the derivatives of the likelihood function for those parameters. The step sizes
used in computing the derivatives are controlled by two parameters: r (relative step size) |
and m (minimum step size). Let θ(i) denote the value of the parameter θ |
at iteration i . |
Then the step size at iteration i + 1 is determined by: |
|
s(i + 1) = max(rθ(i), m) |
(22.5) |
The two-sided numeric derivative is evaluated as:
678—Chapter 22. The Log Likelihood (LogL) Object
f( θ(i) + s(i + 1)) − f( θ(i) − s(i + 1) ) |
(22.6) |
----------------------------------------------------------------------------------2s(i + 1) |
|
|
|
The one-sided numeric derivative is evaluated as: |
|
|
f( θ(i) + s(i + 1) ) − f( θ |
(i)) |
(22.7) |
------------------------------------------------------------s(i + 1) |
- |
|
|
where f is the likelihood function. Two-sided derivatives are more accurate, but require roughly twice as many evaluations of the likelihood function and so take about twice as long to evaluate.
The @derivstep statement can be used to control the step size and method used to evaluate the derivative at each iteration. The @derivstep keyword should be followed by sets of three arguments: the name of the parameter to be set (or the keyword @all), the relative step size, and the minimum step size.
The default setting is (approximately):
@derivstep(1) @all 1.49e-8 1e-10
where “1” in the parentheses indicates that one-sided numeric derivatives should be used and @all indicates that the following setting applies to all of the parameters. The first number following @all is the relative step size and the second number is the minimum step size. The default relative step size is set to the square root of machine epsilon
( 1.49 × 10−8) and the minimum step size is set to m = 10−10 .
The step size can be set separately for each parameter in a single or in multiple @derivstep statements. The evaluation method option specified in parentheses is a global option; it cannot be specified separately for each parameter.
For example, if you include the line:
@derivstep(2) c(2) 1e-7 1e-10
the relative step size for coefficient C(2) will be increased to m = 10−7 and a two-sided derivative will be used to evaluate the derivative. In a more complex example,
@derivstep(2) @all 1.49e-8 1e-10 c(2) 1e-7 1e-10 c(3) 1e-5 1e-8
computes two-sided derivatives using the default step sizes for all coefficients except C(2) and C(3). The values for these latter coefficients are specified directly.
