Diss / 10

expectation estimate, the limits of integration in (12.38) must be t − T and t, respectively. Then the parameter estimation is defined as

The weight integration can be done by the linear filter with corresponding impulse response. For this purpose, we introduce the function

and substitute this function into (12.43) instead of υ(t) introducing a new variable t − τ = z. Then (12.43) can be transformed to the following form:

The integrals in (12.45) are the output responses of the linear filter with the impulse response h(t) given by (12.44) when the filter input is excited by x(t) and s(t), respectively.

The mathematical expectation of estimate

that is, the estimate of the maximum likelihood of the mathematical expectation of stochastic process is both the conditionally and unconditionally unbiased estimate. The conditional variance of the mathematical expectation estimate can be presented in the following form:

that is, the variance of estimate is unconditional. Since, according to (12.38), the integration of Gaussian stochastic process is a linear operation, the estimate EE is subjected to the Gaussian distribution.

Let the analyzed stochastic process be a stationary process and possess the correlation function given by (12.13). Substituting instead of the function υ(t) its value from (12.17) into (12.47) and integrating with delta functions, we obtain

where p is a ratio between the time required to analyze the stochastic process and the correlation interval of the same stochastic process. In doing so, according to (12.38), the formula for the optimal estimate takes the following form:

If p 1, we have

Formulae (12.48) and (12.49) can be obtained without determination of the pdf functional. For this purpose, the value defined by the following equation:

can be considered as the estimate. Here h(t) is the weight function defined based on the condition of unbiasedness of the estimate that is equivalent to

Transform the formula for the variance of estimate into a convenient form. For this purpose, introduce new variables in the double integral, namely,

and change the order of integration. Taking into consideration that R(τ) = R(−τ), we obtain

As was shown in Ref. [1], a definition of optimal form of the weight function h(t) is reduced to a solution of the integral Wiener–Hopf equation

where Varmin{E} is the minimal estimate variance, jointly with the condition given by (12.52). However, the solution of (12.56) is complicated.

Define the formula for an optimal estimate of mathematical expectation of the stationary stochastic process possessing the correlation function given by (12.14) and weight function given by (12.18). Substituting (12.18) into the formula for mathematical expectation estimate of the stochastic process defined as (12.38) and calculating the corresponding integrals, the following is obtained:

In doing so, the variance of the mathematical expectation estimate is defined as

If α ω0 and ω0T  1, the formula for the mathematical expectation estimate of the stationary Gaussian stochastic process transforms to the well-known formula of the mathematical expectation definition of the ergodic stochastic process given by (12.42), and the variance of the mathematical expectation estimate is defined as

At ω1 = 0 (ω0 = α), the correlation function given by (12.14), can be transformed into the following form

by limiting process. In particular, the given correlation function corresponds to the stationary stochastic process at the output of two RC circuits connected in series when the “white” noise excites the input. In this case, the formulae for the mathematical expectation estimate and variance take the following form:

Relationships between the definition of the estimate and the estimate variance of the mathematical expectation of the stochastic processes with other types of correlation functions can be defined analogously.

As we assumed before, the a priori domain of definition of the mathematical expectation is not limited. Thus, we consider a domain of possible values of the mathematical expectation as a function of the mathematical expectation estimate. Let the a priori domain of definition of the mathematical expectation be limited both by the upper bound and by the lower bound, that is,

In the considered case, the mathematical expectation estimate Ê cannot be outside the considered interval given by (12.63), even though it is defined as a position of the absolute maximum of the likelihood functional logarithm (12.9). The likelihood functional logarithm reaches its maximum at E = EE. As a result, when EE ≤ EL the likelihood functional logarithm becomes a monotonically decreasing function within the limits of the interval [EL, EU] and reaches its maximum value at E = EL. If EE ≥ EU, the likelihood functional logarithm becomes a monotonically increasing function within the limits of the interval [EL, EU] and, consequently, reaches its maximum value at E = EU. Thus, in the case of the limited a priori domain of definition of the mathematical expectation, the estimate of mathematical expectation of stochastic process can be presented in the following form:

Taking into consideration the last relationship, the structure of optimal device for the mathematical expectation estimate determination in the case of the limited a priori domain of mathematical expectation definition can be obtained by the addition of a linear limiter with the following characteristic:

to the circuit shown in Figure 12.1. Using the well-known relationships [2] to transform the Gaussian random variable pdf of by a nonlinear inertialess system with the chain characteristic g(z), we can define the conditional pdf of the mathematical expectation estimate as follows:

if the limiting process is carried out at EL → −∞ and EU → ∞, the conditional dispersion of the maximum likelihood estimate of stochastic process mathematical expectation coincides with the variance of estimate given by (12.47). If the true value of the mathematical expectation coincides with one of two bounds of the a priori domain of possible values of the mathematical expectation, then the following approximation is true:

that is, the dispersion of estimate is twice as less compared to the unlimited a priori domain case. With increasing variance of the maximum likelihood estimate of stochastic process mathematical expectation Var(EE | E0) → ∞, the conditional dispersion of the maximum likelihood estimate of the stochastic process mathematical expectation tends to approach the finite value since PL = PU = 0.5

whereas the dispersion of the maximum likelihood estimate of stochastic process mathematical expectation within the unlimited a priori domain of possible values of the maximum likelihood estimate of stochastic process mathematical expectation is increased without limit as Var(EE | E0) → ∞. It is important to note that although the bias and dispersion of the maximum likelihood estimate of stochastic process mathematical expectation are defined as the conditional values, they are nevertheless independent of the true value of the mathematical expectation E0 and are the unconditional estimates simultaneously.

Determine the unconditional bias and dispersion of maximum likelihood estimate of stochastic process mathematical expectation in the case of the limited a priori domain of possible estimate values. For this purpose, it is necessary to average the conditional characteristics given by (12.69) and (12.72) with respect to possible values of estimated parameter, assuming that the a priori pdf of estimated parameter is uniform within the limits of the interval [EL, EU]. In this case, we observe that the unconditional estimate is unbiased, and the unconditional dispersion is determined in the following form:

At the same time, it is not difficult to see that at small values of the variance, that is, Var(EE | E0) → 0, the unconditional dispersion transforms into a dispersion of the estimate obtained under the unlimited a priori domain of possible values, D(Ê) → Var(EE | E0). Otherwise, at high values of variance, that is, Var(EE | E0) → ∞, the dispersion of the estimate given by (12.47) increases without limit and the unconditional dispersion given by (12.76) has a limit equal to the average square of the a priori domain of possible values of the estimate, that is, (EU − EL)2/3.

12.3  BAYESIAN ESTIMATE OF MATHEMATICAL EXPECTATION: QUADRATIC LOSS FUNCTION

As before, we analyze the realization x(t) of stochastic process given by (12.2). The a posteriori pdf of estimated stochastic process parameter E can be presented in the following form:

where

pprior(E) is the a priori pdf of estimated stochastic process parameter υ(t) is the solution of the integral equation given by (12.8)

In accordance with the definition given in Section 11.4, the Bayesian estimate γE is the estimate minimizing the unconditional average risk given by (11.29) at the given loss function. As applied to the quadratic loss function defined as

the average risk coincides with the dispersion of estimate. In doing so, the Bayesian estimate γE is obtained based on minimization of the a posteriori risk at each fixed realization of observed data

To define the estimate characteristics, that is, the bias and dispersion, it is necessary to determine two first moments of the random variable γE. However, in the case of the arbitrary a priori pdf of estimated stochastic process parameter E, it is impossible to determine these moments in a general form. In accordance with this statement, we consider the discussed problem for the case of a priori Gaussian pdf of estimated parameter; that is, we assume [5]

where Eprior and Varprior(E) are the a priori values of the mathematical expectation and variance of the mathematical expectation estimate. Substituting (12.80) into the formula defining the Bayesian

estimate and carrying out the integration, we obtain

It is not difficult to note that if Varprior(E) → ∞, the a priori pdf of estimate is approximated by the uniform pdf of the estimate and the estimate becomes the maximum likelihood estimate (12.38).