Chapter 9. Advanced Workfiles

208—Chapter 9. Advanced Workfiles

Unlike regular frequency data, Irregular frequency data do not arrive in a precisely regular pattern. An important example of irregular data is found in stock and bond prices, where the presence of missing days due to holidays and other market closures means that the data do not follow a regular daily (7- or 5-day) frequency.

The most important characteristic of regular data is that there are no structural gaps in the data—all observations in the specified frequency exist, even if there are missing values that are not observed. Alternatively, irregular data allow for gaps between successive observations in the given regular frequency. This is a subtle distinction, but has important consequences for lag processing.

The distinction is best illustrated by an example. Suppose that we are working with a daily calendar and that we have two kinds of data: data on bond prices (BOND), and data on temperature in Los Angeles in Farenheit (TEMP):

Notice that in this example, the bond price is not available on 12/21 and 12/25 (since the market was closed), and that the temperature reading was not available on 12/23 (due to equipment malfunction).

Typically, we would view the TEMP series as following a 7-day regular daily frequency with a missing value for 12/23. The key feature of this interpretation is that the day 12/23 exists, even though a temperature reading was not taken on that day. Most importantly, this interpretation implies that the lagged value of TEMP on 12/24 (the previous day’s TEMP value) is NA.

In contrast, most analysts would view BOND prices as following an irregular daily frequency in which days involving market closures do not exist. Under this interpretation, we would remove weekends and holidays from the calendar so that the bond data would be given by:

210—Chapter 9. Advanced Workfiles

comparing data for observation labeled “Arizona” and “Wyoming” is much easier than comparing data for observations “3” and “50”.

One last comment about the ordering of observations in cross-section data. While we can (and will) define the lag observation to be that “preceding” a given observation, such a definition is sensitive to the arbitrary ordering of our data, and may not be meaningful. If, as in our example, we order our states alphabetically, the first lag of “Arkansas” is taken from the “Arizona” observation, while if we order our observations by population, the lag of “Arkansas” will be the data for “Utah”.

Panel Data

Some data involve observations that possess both cross-section (group) and within-cross- section (cell) identifiers. We will term these to be panel data. Many of the previously encountered data structures may be viewed as a trivial case of panel data involving a single cross-section.

To extend our earlier example, suppose that instead of observing the cross-section state tax data for a single year, we observe these rates for several years. We may then treat an observation on any single tax rate as having two identifiers: a single identifier for STATE (the group ID), and an identifier for the YEAR (the cell ID). The data for two of our states, “Kansas” and “Kentucky” might look like the following:

We emphasize again that identifiers must uniquely determine the observation. A corollary of this requirement is that the cell IDs uniquely identify observations within a group. Note that requiring cell IDs to be unique within a group does not imply that the cell IDs are unique. In fact, cell ID values are usually repeated across groups; for example, a given YEAR value appears in many states since the tax rates are generally observed in the same years.

If we observe repeated values in the cell identifiers within any one group, we must either use a different cell identifier, or we must redefine our notion of a group. Suppose, for example, that Kansas changed its tax rate several times during 2002:

212—Chapter 9. Advanced Workfiles

Lags, Leads, and Panel Structured Data

Following convention, the observations in our panel dataset are always stacked by crosssection. We first collect the observations by cross-section and sort the cell IDs within each cross-section. We then stack the cross sections on top of one another, with the data for the first cross-section followed by the data for the second cross-section, the second followed by the third, and so on.

The primary impact of this data arrangement is its effect on lag processing. There are two fundamental principles of lag processing in panel data structures:

•First, lags and leads do not cross group boundaries, so that they never use data from a different group.

•Second, lags and leads taken within a cross-section are defined over the sorted values of the cell ID. This implies that lags of an observation are always associated with lower value of the cell ID, and leads always involve a higher value (the first lag observation has the next lowest value and the first lead has the next highest value).

Let us return to our original example with STATE as the group ID and YEAR as the cell ID, and consider the values of TAXRATE, TAXRATE(-1), and TAXRATE(1). Applying the two rules for panel lag processing, we have:

Note in particular, that the lags and leads of TAXRATE do not cross the group boundaries; the value of TAXRATE(-1) for Kentucky in 2001 is an NA since the previous value is from Kansas, and the value TAXRATE(1) for Kansas in 2003 is NA is the next value is from Kentucky.

Next, consider an example where we have invalid IDs since there are duplicate YEAR values for Kansas. Recall that there are two possible solutions to this problem: (1) creating a new cell ID, or (2) redefining our groups. Here, we see why the choice between using a new cell ID or subdividing groups has important implications for lag processing. First, we may simply create a new cell ID that enumerates the observations in each state (CELL_ID1). If we use CELL_ID1 as the cell identifier, we have:

214—Chapter 9. Advanced Workfiles

you wish your lags and leads to span all of the observations in the original groupings, you should create a new cell ID to be used with the original group identifier.

Types of Panel Data

Panel data may be characterized in a variety of ways. For purposes of creating panel workfiles in EViews, there are several concepts that are of particular interest.

Dated vs. Undated Panels

We characterize panel data as dated or undated on the basis of the cell ID. When the cell ID follows a frequency, we have a dated panel of the given frequency. If, for example, our cell IDs are defined by a variable like YEAR, we say we have an annual panel. Similarly, if the cell IDs are quarterly or daily identifiers, we say we have a quarterly or daily panel.

Alternatively, an undated panel uses group specific default integers as cell IDs; by default the cell IDs in each group are usually given by the default integers (1, 2, ...).

Regular vs. Irregular Dated Panels

Dated panels follow a regular or an irregular frequency. A panel is said to be a regular frequency panel if the cell IDs for every group follow a regular frequency. If one or more groups have cell ID values which do not follow a regular frequency, the panel is said to be an irregular frequency panel.

One can convert an irregular frequency panel into a regular frequency panel by adding observations to remove gaps in the calendar for all cross-sections. Note that this procedure is a form of internal balancing (see “Balanced vs. Unbalanced Panels” below) which uses the calendar to determine which observations to add instead of using the set of cell IDs found in the data.

See “Regular and Irregular Frequency Data” on page 207 for a general discussion of these topics.

Balanced vs. Unbalanced Panels

If every group in a panel has an identical set of cell ID values, we say that the panel is fully balanced. All other panel datasets are said to be unbalanced.

In the simplest form of balanced panel data, every cross-section follows the same regular frequency, with the same start and end dates—for example, data with 10 cross-sections, each with annual data from 1960 to 2002. Slightly more complex is the case where every cross-section has an identical set of irregular cell IDs. In this case, we say that the panel is balanced, but irregular.

We may balance a panel by adding observations to the unbalanced data. The procedure is quite simple—for each cross-section or group, we add observations corresponding to cell IDs that are not in the current group, but appear elsewhere in the data. By adding observa-

216—Chapter 9. Advanced Workfiles

between “1987” and “2001”. This set is given by {“1993”, “1994”, “1995”, “1996”}. Since “1993”, “1995”, and “1996” do not appear for individual 2, observations with these three cell IDs will be added for individual 2.

The result of this internal balancing is an expanded, internally balanced panel dataset containing:

We have marked the five added observations with an asterisk, and arranged the data so that the cell IDs line up where possible. Observations that are not present in the dataset are marked as “—”. Notice that the effect of the internal balancing is to fill in the missing cell IDs in the central portion of the data.

It is worth a digression to note here that an alternative form of internal balancing is to add observations to remove all gaps in the calendar between the starts and ends. This method of balancing, which converts the data from an irregular to a regular panel, uses the calendar to determine which observations to add instead of using the set of observed cell IDs found. If we are balancing the expanded dataset, we would add observations with the cell IDs for missing years: {“1986”, “1988”, “1990”, “1991”} for individual 1, and {“1988”, “1990”, “1991”, “1998”, “1999”, “2000”} for individual 2.

Lastly, we consider the effects of choosing to balance the starts or balance the ends of our data. In the former case, we ensure that every cross-section adds observations corresponding to observed cell IDs that come before the current starting cell ID. In this case, balancing the starts means adding an observation with ID “1985” to group 2. Similarly, balancing the ends ensures that we add, to every cross-section, observations corresponding to observed cell IDs that follow the cross-section end cell ID. In this case, balancing the ends involves adding observations with cell IDs “1997” and “2001” to group 1.

218—Chapter 9. Advanced Workfiles

Applying a Structure to a Workfile

To structure an existing workfile, select

Proc/“Structure/Resize Current Page...” in the main workfile window, or doubleclick on the portion of the window displaying the current range (“Range:”).

Selecting a Workfile Type

EViews opens the Workfile structure dialog. The basic structure of the dialog is quite similar to the Workfile create dialog (“Creating a Workfile” on page 50). On the

left-hand side is a combo box where you will select a structure type.

Clicking on the structure type combo box brings up several choices. As before, you may choose between the Unstructured/Undated, and Dated - regular frequency types. There are, however, several new options. In the place of Balanced Panel, you have the option to select from Dated - specified by date series, Dated Panel, Undated with ID series, or Undated Panel.

Workfile Structure Settings

As you select different workfile structure types, the right-hand side of the dialog changes to show relevant settings and options for the selected type. For example, if you select the Dated - regular frequency type, you will be prompted to enter information about the frequency of your data and date information; if you select an Undated Panel, you will be prompted for information about identifiers and the handling of balancing operations.

Dated - Regular Frequency

Given an existing workfile, the simplest method for defining a regular frequency structured workfile is to select Dated - regular frequency in the structure type combo box. The right side of the dialog changes to reflect your choice, prompting you to describe your data structure.

220—Chapter 9. Advanced Workfiles

Here we have changed the End date from “2011:08” to “2011:12”, thereby instructing EViews to add 4 observations to the end of the workfile. If you select OK, EViews will inform you that it will add 4 observations and prompt you to continue. If you proceed, EViews will resize the workfile to your specification.

Dated - specified by date series

The second approach to structuring your workfile is to provide the name of a series containing the dates (or series than may be interpreted as dates) to be used as observation identifiers. Select Dated - specified by date series in the combo box, and fill out the remainder of the dialog.

The first thing you must do is enter the name of one or more Date series that describe the unique date identifiers.

The series may contain EViews date values (a true date series), or the single or multiple series may contain numeric or string representations of unique dates. In the latter case, EViews will create a single date series containing the date values associ-

ated with the numeric or string representations. This new series, which will be given a name of the form DATEID##, will be used as the identifier series.

On the right side of the dialog, you will specify additional information about your workfile structure. In the first combo box, you will choose one of the standard EViews workfile frequencies (annual, quarterly, monthly, etc.). As shown in the image, there is an additional (default) option, Auto detect, where EViews attempts to detect the frequency of your data from the values in the specified series. In most cases you should use the default; if, however, you choose to override the auto-detection, EViews will associate the date values in the series with observations in the specified frequency.

222—Chapter 9. Advanced Workfiles

The resulting (structured) workfile window shown here indicates that we have a 10 observation irregular annual frequency workfile that ranges from an earliest date of 1960 to the latest date of 19706

Since the series B contained only text representations of dates, EViews has created a new series DATEID containing date values corresponding to those in B. DATEID is locked and can-

not be altered, renamed, or deleted so long as it is used to structure the workfile.

Here, we show a group containing the original series B, the new series DATEID, and the lag of B, B(-1). There are a few things to note. First, the observation identifiers are no longer integers, but instead are values taken from the identifier series DATEID. The formatting of the observation labels will use the display formatting present in the ID series. If you wish to change the appearance of the labels, you should set the display format for DATEID (see “Display Formats” on page 89).

Second, since we have sorted the contents of the workfile by the ID series, the values in B and DATEID are ordered by date. Third, the lagged values of series use the irregular calendar defined by DATEID—for example, the lag of the 1965 value is given by 1961.

Alternately, we could have chosen to restructure with the Insert empty obs checkbox selected, thus ensuring that we have a regular frequency workfile.

To see the effect of this option, we may reopen the Workfile structure dialog by double clicking on the “Range:” string near the top of the workfile window, selecting the Insert empty obs option, and then clicking on OK. EViews will inform

224—Chapter 9. Advanced Workfiles

Dated Panel Basics

We begin by considering the Grunfeld data that have been described in a number of places (see, for example, Baltagi,

The Econometric Analysis of Panel Data, from which this version of the data has been taken). The data measure R&D expenditure and other economic measures for 10 firms for the years 1935 to 1954. These 200 observations form a balanced panel dataset. We begin by reading the data into an unstructured, 200 observation workfile.

To structure the panel for these data, we call up the Workfile structure dialog, select Dated Panel as our structure type, and enter the name of the Cross-section ID series representing firm number, FN, along with the Date series (cell ID) representing the year, YR. If we leave the remaining settings at their default values, EViews will auto detect the frequency of the panel, setting the start and end dates on the basis of the values in the YR series, and will add any observations necessary so that the data between the starts and ends is balanced.

When you click on OK to accept these settings, EViews creates a DATEID series, sorts the data by ID and DATEID, locks the two series, and applies the structure. The auto detecting of the date frequency and endpoints yields an annual (balanced) panel beginning in 1935 and ending in 1954.

The basic information about this structure is displayed at the top of the work-

file window. There are a total of 200 observations representing a balanced panel of 10 cross-sections with data from 1935 to 1954.

226—Chapter 9. Advanced Workfiles

In addition to choosing whether to Balance starts and Balance ends, you may choose, at most, one of the two options Balance between starts and ends, and

Insert obs to remove date gaps so date follows regular frequency.

If balancing between starts and ends, the balancing procedure will use the observed cell IDs (in this case, the years encoded in

DATEID for all cross-sections) between a given start and end date. All cross-sections will share the same possibly irregular calendar for observations between their starts and ends. If you also elect to insert observations to remove date gaps, EViews balances each crosssection between starts and ends using every date in the calendar for the given frequency. In the latter case, all cross-sections share the same regular calendar for observations between their starts and ends.

Selecting all three options, Balance starts, Balance ends and Balance between starts and ends, ensures a balanced panel workfile. If we substitute the option Insert obs to remove date gaps so date follows regular frequency for Balance between starts and ends, we further guarantee that the data follow a regular frequency.

In partly or fully balancing the panel workfile, EViews will add observa-

tions as necessary, and update the corresponding data in the identifier series. All other variables will have their values for these observations set to NA. Here, we see that EViews has added data for the two identifier series FN and DATEID while the ordinary series YR values associated with the added observations are missing.

228—Chapter 9. Advanced Workfiles

To see the two primary effects of structuring cross-section workfiles, we again examine the values of STATE and the variable X. Notice that the data have been sorted (in ascending order) by the value of STATE and that the observation identifiers in the left-hand border now use the values of STATE.

Note that as with irregular structured workfiles, the observation labels will adopt the characteristics

of the classifier series display format. If you wish to change the appearance of the observation labels, you should set the spreadsheet display format for STATE (see “Changing the Spreadsheet Display” on page 88).

Undated Panels

To apply an undated panel structure to your workfile, you must specify one or more Crosssection ID series that identify members of a given group. First, select Undated Panel from the combo box, and then enter the names of your identifier series. You may optionally instruct EViews to balance between the starts and ends, the starts, or the ends of your data.

As an example, we consider the Harrison and Rubinfeld data on house prices for 506 observations located in 92 towns and cities in the harbor area near New Bedford, MA (Harrison and Rubinfeld 1978; Gilley and Pace 1996).

The group identifiers for these data are given by the series TOWNID, in which the town for a given observation is coded

from 1 to 92. Observations within a town are not further identified, so there is no cell ID within the data. Here we specify only the group identifier TOWNID.

230—Chapter 9. Advanced Workfiles

Common Structuring Errors

In most settings, you should find that the process of structuring your workfile is relatively straightforward. It is possible, however, to provide EViews with identifier information that contains errors so that it is inconsistent with the desired workfile structure. In these cases, EViews will either error, or issue a warning and offer a possible solution. Some common errors warrant additional discussion.

Non-unique identifiers

The most important characteristic of observation identifiers is that they uniquely identify every observation. If you attempt to structure a workfile with identifiers that are not unique, EViews will warn you of this fact, will offer to create a new cell ID, and will prompt you to proceed. If you choose to proceed, EViews will then prompt you to create a panel workfile structure using both the originally specified ID(s) and the new cell ID to identify the observations. We have seen an example of this behavior in our discussion of the undated panel workfile type (“Undated Panels” on page 228).

In some cases, however, this behavior is not desired. If EViews reports that your date IDs are not unique, you might choose to go back and either modify or correct the original ID values, or specify an alternate frequency in which the identifiers are unique. For example, the date string identifier values “1/1/2002” and “2/1/2002” are not unique in a quarterly workfile, but are unique in a monthly workfile.

Invalid date identifiers

When defining dated workfile structures, EViews requires that you enter the name or names of series containing date information. This date information may be in the form of valid EViews date values, or it may be provided in numbers or strings which EViews will attempt to interpret as valid date values. In the latter case, EViews will attempt to create a new series containing the interpreted date values.

If EViews is unable to translate your date information into date values, it will issue an error indicating that the date series has invalid values or that it is unable to interpret your date specification. You must either edit your date series, or structure your workfile as an undated workfile with an ID series.

In cases where your date information is valid, but contains values that correspond to unlikely dates, EViews will inform you of this fact and prompt you to continue. Suppose, for example, that you have a series that contains 4-digit year identifiers (“1981”, “1982”, etc.), but also has one value that is coded as a 2-digit year (“80”). If you attempt to use this series as your date series, EViews will warn you that it appears to be an integer series and will ask you if you wish to recode the data as integer dates. If your proceed, EViews will alter the values in your series and create an integer dated (i.e., not time dated) workfile, which may not be what you anticipated.