Method of moments
NOTES ON STATISTICS, PROBABILITY and MATHEMATICS
Method of Moments:
The R implementation can be found in the gmm package.
Once decided that a sample comes from a normal distribution, the parameters \(\mu\) and \(\sigma\) can be estimated by either the method of moments or the generalized method of moments.
In the method of moments the two first moments of the population:
\(\mathbb E[X]=\mu\) and \(\mathbb E[(x-\mu)^2]= \sigma^2\)
can be estimated with a system of two equations and two unknowns:
\(\mathbb E[X-\hat\mu]=\frac{1}{n}\sum_{i=1}^n x_i - \hat \mu = \mu\)
and
\(\mathbb E[(X- \hat \sigma)^2] = \frac{1}{n}\sum_{i=1}^n (x_i-\hat\mu)^2=\sigma^2\)
However, more moments can be included in a weighted fashion:
The third moment (skewness) is:
\(\mathbb E[(x-\mu)^3]\), which it is zero in the population.
This is estimated in the sample by \(\frac{1}{n}\sum_{i=1}^n (x_i-\hat\mu)^3=0.\)
The fourth moment (kurtosis) is:
\(\mathbb E[(x-\mu)^4]\), which is \(3\sigma^4.\)
This is estimated in the sample by \(\frac{1}{n}\sum_{i=1}^n (x_i-\hat\mu)^4 = 3\sigma^4.\)
At this point we look at cost functions for each one of these moment estimators:
\(\hat g_1=\frac{1}{n}\sum_{i=1}^n x_i-\hat\mu\)
\(\hat g_2=\frac{1}{n}\sum_{i=1}^n (x_i-\hat\mu)^2-\sigma^2\)
\(\hat g_3=\frac{1}{n}\sum_{i=1}^n (x_i-\hat\mu)^3\)
\(\hat g_4=\frac{1}{n}\sum_{i=1}^n (x_i-\hat\mu)^4-\sigma^2-3\sigma^4\)
We place these estimated cost functions into a vector:
\(\hat g=\begin{bmatrix}\hat g_1\\ \hat g_2\\ \hat q_3\\ \hat g_4\end{bmatrix}\)
and we calculate the cost function:
\[\hat g\quad \begin{bmatrix}\text{WEIGHTS}\end{bmatrix}\quad\hat g\]
NOTE: These are tentative notes on different topics for personal use - expect mistakes and misunderstandings.