Processing math: 100%

NOTES ON STATISTICS, PROBABILITY and MATHEMATICS


Variance - Covariance in Time Series:


AUTOREGRESSIVE ORDER ONE (AR (1)) PROCESSES:


The general formula for an AR(1) process is Xy=ρXt1+ϵt with ϵtiid(0,σ2).

The variance of Xt will be given by:

Var[Xt]=ρ2Var[Xt1]+Var[ϵt] because Var[aX]=a2Var[X].

The condition of stationarity implies that Var[Xt]=Var[Xt1]. Therefore,

Var[Xt]=ρ2Var[Xt]+σ2, since Var[ϵt]=σ2. It follows then that…

(1ρ2)Var[Xt]=σ2. Therefore:

Var[Xt]=σ21ρ2


To see the covariance of an AR(1) time series with itself lagged, we can backsubstitute in the Xy=ρXt1+ϵt equation:

Xy=ρXt1+ϵt=ρ[ρXt2+ϵt1]+ϵt=ρ2Xt2+ρϵt1+ϵt Or, expressed differently, Xt+h=ρhXt+h1i=0ρiϵt+hi.

Therefore the covariance of Xt with itself lagged into the future, Xt+h will be:

Cov(Xt,Xt+h)=Cov(Xt,ρhXt+h1i=0ϵt+h1). But there is no covariance between Xt and this very last term. Hence,

Cov(Xt,Xt+h)=Cov(Xt,ρhXt)

Cov(Xt,Xt+h)=ρhCov(Xt,Xt)

Cov(Xt,Xt+h)=ρhVar(Xt)

and resorting to Eq. 1:

Cov(Xt,Xt+h)=ρhσ21ρ2

|ρ|<1


Now we can calculate the correlation:

Corr.(Xt,Xt+h)=Cov(Xt,Xt+h)Var(Xt)

Corr.(Xt,Xt+h)=ρh

This implies that the correlogram (ACF) will give progressively (exponentially decaying) spikes each one corresponding to ρstep.

The partial autocorrelation function (PACF) will be used to distinguish AR(1) from AR(2). It allows us to see the residual correlation after a certain number of lags after removing more proximate lags. So in an AR(1), after the first lag the PACF will not be significant. On the other hand, in an AR(2) process, the second lag will still be significant, because of the relationship Xt=ρ1Xt1+ρ2Xt2.


MOVING AVERAGE ORDER ONE (MA (1)) PROCESSES:

They are of the form Xt=ϵt+θϵt1 with ϵiid(0,σ2). The variance of Xt will therefore be given by:

Var(Xt)=Var(ϵt+θϵt1)

Var(Xt)=Var(ϵt)+θ2Var(ϵt1)

Var(Xt)=σ2+θ2σ2=σ2(1+θ2)


As for the covariance:

Cov(Xt,Xt1)=Cov(ϵt+θϵt1,ϵt1+θϵt2). Since the errors are independent, the Cov of ϵt, ϵt1 and ϵt2 is zero.

Cov(Xt,Xt1)=θCov(ϵt1,ϵt1)

Cov(Xt,Xt1)=θσ2

Notice that this common element in the covariance expression (ϵt1) only exist if the lag is 1. For instance,

Cov(Xt,Xt+h)=Cov(ϵt+ϵt+h,ϵt+h+θϵt+h1)

Only in the case of h=1, we’ll end up dealing with…

Cov(Xt,Xt+h)=Cov(ϵt,θϵt)

Cov(Xt,Xt+h)=θCov(ϵt,ϵt)=θσ2. Otherwise the covariance is zero.


Having the variance and covariance, the correlation will be:

For h=1, Corr.(Xt,Xt+h)=θ1+θ2

If h>1, the correlation is zero. This explains why in a MA(1) process, the only statistically significant spike in the ACF is the first one (aside from the spike of 1 of the correlation with itself included in the ACF).


Home Page

NOTE: These are tentative notes on different topics for personal use - expect mistakes and misunderstandings.