EH_ch1_4

stable equilibrium point of dθ/dτ = h(θ).

We remark that in the cases not covered (where there are roots with zero real parts, but no root with a positive real part), more refined techniques are required to determine stability.

The stochastic approximation results can be stated as follows:

Although the above statements appear fairly straightforward, the precise theorems are complex in detail. There are two reasons for this. First, there are vari-

ous ways to formalize the positive convergence result (when ¯ is a locally stable

θ

equilibrium point of the ODE). In certain cases, when there is a unique solution

and the ODE is globally stable, it can be shown that under the SRA, → ¯ with

θt θ probability 1 from any starting point. When there are multiple equilibria, however, such a strong result will not be possible, and indeed there may be multiple stable equilibria. In this case, if one artificially constrains θt to an appropri-

ate neighborhood of a locally stable equilibrium ¯ (using a so-called “projec-

θ

tion facility”), one can still obtain convergence with probability 1. Alternatively, without this device, one can, for example, show convergence with positive probability from appropriate starting points. The different ways of expressing local

stability of ¯ under the SRA are fully discussed in Chapter 6.

θ

Second, a careful statement is required of the technical assumptions under which the convergence conditions obtain. There are three broad classes of assumptions:

(i)regularity assumptions on Q,

(ii)conditions on the rate at which γt → 0,

(iii)assumptions on the properties of the stochastic process followed by the state variable Xt .

For condition (ii) on the gain sequence, a standard assumption is that γt = ∞ and γt2 < ∞. This is satisfied in particular by γt = t−1.

The precise statement of the conditions (i)–(iii) depends on the precise version of the stability or instability result, and in some cases, there are alternative sets of assumptions. Again, we will discuss these issues fully in Chapters 6 and 7. Finally, we remark that the formal instability result does not cover the

Having shown that the system can be placed in standard SRA form, the next step is to compute the associated ODE. To do this we fix the value of θ in Q(t, θt−1, Xt ) and compute the expectation over Xt . Fixing the value of θ means fixing the values of φ and S, so that we have

h(θ) = vec(hφ (φ, S), hS (φ, S)), but it is easier to continue to work directly with the separate vector and matrix functions hφ (φ, S) and hS (φ, S). Since

Ezt−1ηt = 0, and limt→∞ t/(t + 1) = 1, we obtain

hφ (φ, S) = S−1M(T (φ) − φ), hS (φ, S) = M − S.

We have therefore arrived at the associated ODE

This system is recursive and the second set of equations is a globally stable system with S → M from any starting point. It follows that S−1M → I from any starting point, provided S is invertible along the path, and hence that the stability of the differential equations (2.17)–(2.18) is determined entirely by the stability of the smaller dimension system

There are technical details required to establish this equivalence, since one must show that the possibility of a noninvertible S can be sidestepped. The technical arguments on this point are given in Chapter 6.

Figure 2.1.

A general definition of E-stability is a straightforward extension of the example in this chapter. Starting with an economic model, we consider its REE solutions. Assume that any particular solution can be described as a stochastic

parameters of an ARMA process or of a VAR or the mean values at the different points in a k-cycle. Under adaptive learning the agents are assumed not to

will in each case set up the problem as an SRA in order to examine the stability

of motion φ to the implied actual law of motion T (φ). [Note that REEs corre-

expanded set of parameters. Indeed, it is useful to introduce some terminology

REE is strongly E-stable. As we will see, weak and strong E-stability govern whether the corresponding adaptive learning rules are stable. By expanding the dimension of φ appropriately, one can also allow for heterogeneous expectations across agents and determine whether allowing for heterogeneity alters the stability conditions for convergence of adaptive learning. In an analogous way, the concept of E-stability can also be adapted to determine conditions for the convergence of underparameterized learning rules by reducing the dimension of φ.

One could also consider structural heterogeneity, i.e., models in which individual agents respond differently to expectations. An open question is how E-stability can be adapted to such environments. Throughout the book we implicitly make the assumption of structural homogeneity, so that models such as equation (2.1) arise from a world in which the characteristics of individual agents are identical.

There is one more important conceptual issue that arises in connection with E-stability. In some papers, e.g., DeCanio (1979), Bray (1982), Evans (1983), and Evans (1985), iterations of the T -map are considered and the stability of an REE under these iterations is determined.8 Formally, this version of E-stability

In order to have a clear terminology, we will refer to the notion of stability determined by equation (2.22) as iterative E-stability, reserving the unmodified phrase “E-stability” for stability under the differential equation. Thus an REE

8Iterations of the T -map were also considered in Lucas (1978). The term “expectational stability,” and the distinction between weak and strong (iterative) E-stability, were introduced in Evans (1983) and Evans (1985).

immediately that iterative E-stability is a stricter condition than E-stability. For example, in the model (2.1) the E-stability condition, as we have shown, is that α < 1, while the iterative E-stability condition is |α| < 1.

We have seen that E-stability provides the condition for stability under adaptive learning rules such as least squares. What, therefore, do we learn from iterative E-stability? One rationale given in the papers cited was that it gave the condition for stability if agents kept fixed their perceived law of motion until a large amount of data was collected, revising their estimates infrequently rather than at each point in time. An alternative “eductive” rationale, also suggested in these papers, was that it described a process of learning taking place in “mental time,” as each agent considered the possible forecasts of other agents, iteratively eliminating those that correspond to dominated strategies. Using the game-theoretic notions of rationalizability and common knowledge, this approach was developed in Guesnerie (1992). It is discussed in Section 15.4 of Chapter 15. It has also been recently shown that iterative E-stability plays an important role for learning dynamics based on finite-memory rules in stochastic frameworks (see Honkapohja and Mitra, 1999).

Since this book focuses on adaptive learning, the appropriate stability conditions required are in almost all cases the E-stability conditions provided by the differential equation formulation.

2.10 Discussion of the Literature

In addition to the cobweb model, which we discussed throughout this chapter, the overlapping generations model and various linear models were the most frequently used frameworks in the early literature on learning.

Lucas (1986) is an early analysis of the stability of steady states under learning in an OG model. Grandmont (1985) considered the existence of deterministic cycles for the basic OG model. He also examined learning using the generalizations of adaptive expectations to nonlinear finite-memory forecast functions. Guesnerie and Woodford (1991) proposed a generalization to adaptive expectations by allowing possible convergence to deterministic cycles. Convergence of learning to sunspot equilibria in the basic OG model was first discovered by Woodford (1990).

Linear models more general than the cobweb model were considered under learning in the early literature. As already noted, Marcet and Sargent (1989c)

proposed a general stochastic framework and the use of stochastic approximation techniques for the analysis of adaptive learning. Their paper includes several applications to well-known models. Margaritis (1987) applied Ljung’s method to the model of Bray (1982). Grandmont and Laroque (1991) examined learning in a deterministic linear model with a lagged endogenous variable for classes of finite-memory rules. Evans and Honkapohja (1994b) considered extensions of adaptive learning to stochastic linear models with multiple equilibria.

The more recent literature on adaptive learning will be referenced in the appropriate parts of this book. See also the surveys Evans and Honkapohja (1995a, 1999). The comments below provide references to approaches and literature that will not be covered in detail in later sections.

For Bayesian learning the first papers include Turnovsky (1969), Townsend (1978), Townsend (1983), and McLennan (1984). Bray and Kreps (1987) discussed rational learning and compared it to adaptive approaches. Nyarko (1991) showed in a monopoly model that Bayesian learning may fail to converge if the true parameters are outside the set of possible prior beliefs. Papers studying the implications of Bayesian learning include Feldman (1987b), Feldman (1987a), Vives (1993), Jun and Vives (1996), Bertocchi and Yong (1996), and Nyarko (1997). The collection Kurz (1997) contains central papers on a related notion of rational beliefs.

The study of finite-memory learning rules in nonstochastic models was initiated in Fuchs (1977), Fuchs (1979), Fuchs and Laroque (1976), and Tillmann (1983), and it was extended in Grandmont (1985) and Grandmont and Laroque (1986). These models can be viewed as a generalization of adaptive expectations. We remark that the finite-memory learning rules cannot converge to an REE in stochastic models, as noted by Evans and Honkapohja (1995c) and studied further by Honkapohja and Mitra (1999). Further references of expectations formation and learning in nonstochastic models are given in Section 7.2 of Chapter 7.

Learning in games has been subject to extensive work in recent years. Surveys are given in Marimon (1997) and Fudenberg and Levine (1998). Carton (1999) applied these techniques to the cobweb model. Kirman (1995) reviewed the closely related literature on learning in oligopoly models. Another related recent topic is social learning, see, e.g., Ellison and Fudenberg (1995) and Gale (1996).