NOTES ON STATISTICS, PROBABILITY and MATHEMATICS

Proof that \(\chi^2_{(1\,\text{df})}\) models \(X^2\) with \(X\sim N(0,1)\):


[What follows was first written in a post in SE Mathematics.]

Let’s say that \(X \sim N(0,1)\) and that \(Y=X^2\) and find the density of \(Y\) by using the \(\text{cdf}\) method:

\(p(Y \leq y) = p(X^2 \leq y)= p(-\sqrt{y} \leq x \leq \sqrt{y})\). The problem is that we cannot integrate in close form the density of the normal distribution. But we can express it:

\[ F_X(y) = F_X(\sqrt{y})- F_X(-\sqrt[]{y}).\] Taking the derivative:

\[ f_X(y)= F_X'(\sqrt{y})\,\frac{1}{2\sqrt{y}}+ F_X'(\sqrt{-y})\,\frac{1}{2\sqrt{y}}.\]

Since the values of the normal \(pdf\) are symmetrical:

\(\large f_X(y)= F_X'(\sqrt{y})\,\frac{1}{\sqrt{y}}\). Equating this to the \(pdf\) of the normal (now the \(x\) in the \(pdf\) will be \(\sqrt{y}\) to be plugged into the \(\large e^{-\frac{x^2}{2}}\) part of the normal \(pdf\)); and remembering to in include \(\large \frac{1}{\sqrt{y}}\) at the end:

\[\large f_X(y)= F_X'(\sqrt[]{y})\,\frac{1}{\sqrt[]{y}}= \frac{1}{\sqrt{2\pi}}\,e^{-\frac{y}{2}}\, \frac{1}{\sqrt[]{y}}=\frac{1}{\sqrt{2\pi}}\,e^{-\frac{y}{2}}\, y^{\frac{1}{2}- 1}\]

Compare to the pdf of the chi square:

\[\Large f_X(x)= \frac{1}{2^{\nu/2}\Gamma(\frac{\nu}{2})}e^{\frac{-x}{2}}x^{\frac{\nu}{2}-1}\]

Since \(\Gamma(1/2)=\sqrt{\pi}\), for \(1\) df, we have derived exactly the \(pdf\) of the chi square.



There is a [youtube][1] video dealing with the proof that the sums of the squares of normally distributed \(n\) random errors, each one distributed as \(\sim \chi^2(1\text{ df})\) follows a chi square distribution with \(n\) degrees of freedom:

So if \(X_1, \ldots, X_n\) are \(\sim \chi^2 \,\small(1\text{ df})\), then \(X_1 +\cdots+X_n \sim \chi^2\,\small(n\text{ df})\).

PROOF THAT \(X_1, \ldots, X_n\) ARE \(\sim \chi^2_{(1\,\text{df})}\), then \(X_1 +\cdots+X_n \sim \chi^2_{(n\,\text{df})}\):



In probability theory and statistics, the chi-squared distribution (also chi-square or \(\chi^2\)-distribution) with \(k\) degrees of freedom is the distribution of a sum of the squares of \(k\) independent standard normal random variables.

Proof through the \(cdf\):

\(p\,(X_1 +\cdots+X_n \leq x)= p\,(Y_1^2 +\cdots+Y_n^2 \leq x)\), where \(X_i =Y _i^2\), and \(Y \sim N(0,1)\).

\(Y_1^2 +\cdots+Y_n^2 \leq x\) is an \(n\)-dimensional sphere with radius \(\,\sqrt{x}\): \(Y_1^2 +\cdots+Y_n^2 \leq \sqrt[]{x^2}\).

So the probability is:

\(p\,(X_1 +\cdots+X_n \leq x) = \displaystyle \int\cdots\int_{(Y_1^2 +\cdots+Y_n^2 \leq x)}\large f_{\small(Y_1,\ldots,Y_n)}\large(y_1,\ldots,y_n)\, dy_1,\ldots,dy_n\) with \(f\) being the joint density. Remembering that they are iid rv’s and that \(Y\) is normal:



\[p(X_1 +\cdots+X_n \leq x) = \displaystyle \int\cdots\int_{(Y_1^2 +\cdots+Y_n^2 \leq x)} \bigg(\frac{1}{2\pi}\bigg)^n e^{-\bigg(\frac{y_1^2}{2}+\cdots + \frac{y_n^2}{2}\bigg) } dy_1\cdots dy_n\]



Changing to spherical coordinates it will be something like:

\(dy_1\cdots dy_n=dr\,r\,d\theta_1\,\ldots\,r\,d\theta_n \cdot g(\theta_1\,\ldots\,\theta_n)\) with \(g\) being some function of the angles.



And, \[\small p(X_1 +\cdots+X_n \leq x) = \displaystyle \int_{r=0}^{\sqrt{x}} \int_{\theta_1=\alpha_1}^{\beta_1} \ldots\int_{\theta_{n-1}=\alpha_{n-1}}^{\beta_{n-1}} \bigg(\frac{1}{2\pi}\bigg)^{n/2}\, e^{-(\frac{r^2}{2}) } \,r^{n-1} \,g(\theta_1\,\ldots\,\theta_{n-1})\, dr\,d\theta_1\cdots \,d\theta_n.\]



There is no \(r\) dependency in the \(\theta\) integration. The integration of the \(\theta\)’s gets absorbed into a constant that depends on \(n\), \(C_n\):



\[\Large p(X_1 +\cdots+X_n \leq x) = C_n \cdot \displaystyle \int_{r=0}^{\sqrt{x}} e^{-\bigg(\frac{r^2}{2}\bigg) } r^{n-1} \, dr.\]



This is the distribution (\(cdf\)), and to get the density (\(pdf\)) we have to take derivatives:



Using Leibniz rule:


Given \(F(\alpha)=\int_{a(\alpha)}^{b(\alpha)}\,f(x)\,dx\)

\(\frac{d}{d\alpha}\,F(\alpha)=f(b(\alpha))\frac{d}{d\alpha}b(\alpha)\,-\,f(a(\alpha))\frac{d}{d\alpha}b(\alpha)\)

So,


\(\frac{d}{dx}\left[C_n \displaystyle \int_{r=0}^{\sqrt{x}} e^{-\bigg(\frac{r^2}{2}\bigg) } r^{n-1} dr\right] =C_n \, e^{-(\frac{(\sqrt{x})^2}{2})}\,(\sqrt{x})^{n-1}\,1/2(x)^{-1/2}\,-\,C_n\,e^{-0^2/2}\,0^{n-1}\,0\)

\(\large =C_n \frac{1}{2\sqrt{x}}e^{-x/2}x^{(n-1)/2} =\large C_n \frac{1}{2}\frac{1}{\sqrt{x}}\,e^{-x/2}\,x^{1/2(n-1)} =\large C_n e^{-x/2}\,x^{\frac{n-1}{2}}\,\frac{1}{2}\,x^{-\frac{1}{2}}= \large \frac{C_n}{2}\cdot e^{-x/2}\,x^{\frac{n}{2}-1}\)

which is a chi square with n degrees of freedom if we equate \(\frac{C_n}{2}=\frac{1}{2^{n/2}\Gamma(\frac{n}{2})}\) given that the pdf of chi-square is:

\[\displaystyle f_X(x)= \frac{1}{2^{\nu/2}\,\Gamma(\frac{\nu}{2})}\,e^{-x/2}\,x^{\frac{\nu}{2}-1}\] for \(x\leq 0\). Otherwise, \(0\).

Geometrically, this is equivalent to the surface area of the \(n\)-dimensional sphere.

Home Page

NOTE: These are tentative notes on different topics for personal use - expect mistakes and misunderstandings.