28.Ordinary Least Squares (ols). The Gauss – Markov Theorem.

Gauss-Markov theorem

In Simple or Multiple Linear Regression, the model parameters are most often calculated by the Least Squares method. The main advantage of this method is its mathematical simplicity, which allows an easy identification of the statistical properties of the calculated estimators, in particular their bias (which is 0) and their covariance matrix. But there is no a priori reason to believe that these estimators are particularly good (low Mean Square Error of the parameters and of model predictions).

The Gauss-Markov theorem is here to somewhat soften our worries about the quality of the Least Squares estimator of the vector of the model parameters.

1) E(Ui)=0 for all observations. Expected value of the disturbance term in any observation should be “0”.

2) σ_i^2=σ_2^2 Population variance of Ui constant for all observations. One of the tasks of the regression analysis is to estimate the standard deviation of the disturbance term.

3) Ui is distributed independently of Uj. It means that there should be no systematic association between the values of the disturbance term in any 2 observations.

4) U is distributed independently of the explanatory variables. The population covariance between the explanatory variable and the disturbance term is “0”

30.Heteroscedasticity. Possible Causes of Heteroscedasticity. The Goldfeld–Quandt Test.

In statistics, sequence (последовательность) of random variables is heteroscedastic (H), if the random variables have different variances. In contrast, a sequence of random variables is called homoscedastic if it has constant variance.

Heteroscedasticity does not cause ordinary least squares coefficient estimates (B1, B2) to be biased (смещенный), although it can cause ordinary least squares estimates of the variance (and, thus, standard errors) of the coefficients to be biased, possibly above or below the true or population variance. Thus, regression analysis using heteroscedastic data will still provide an unbiased estimate for the relationship between the predicted variable and the outcome, but standard errors and therefore conclusions obtained from data analysis may be biased. Biased standard errors lead to biased conclusions, so results of hypothesis tests are possibly wrong.

Shortly, if H is present the OLS estimates are wrong, and the standard error of the regression coefficient will be wrong.

Possible Causes of H. H is likely to be a problem when the values of the variables in the sample vary substantially in different observations. There may be a case that the variations in the omitted variables and the measurement errors that are responsible for the disturbance term will be relatively small when Y and X are small, and large when Y and X are large.

The Goldfeld–Quandt Test is the most common test for H. If assume that the standard deviation of the probability distribution of the disturbance term in observation “i” is proportional to the size of it is assumed that the disturbance term is normally distributed and satisfy other Gauss-Markov conditions.

If , the model is HOMOscedsstic, and we CAN use OLS method, for all observations U is constant and satisfies 2d Gauss-Markov condition.

If we know standard deviation for each observation we can eliminate H by dividing each observation by its value of standard deviation.

ANDREY?!