NOTES ON STATISTICS, PROBABILITY and MATHEMATICS

Probability Generating Functions:

From Wikipedia:

In probability theory, the probability generating function of a discrete random variable is a power series representation (the generating function) of the probability mass function of the random variable. Probability generating functions are often employed for their succinct description of the sequence of probabilities \(\Pr(X = i)\) in the probability mass function for a random variable X, and to make available the well-developed theory of power series with non-negative coefficients.

If \(X\) is a discrete random variable taking values in the non-negative integers \(\{0,1, \dots\}\), then the probability generating function of \(X\) is defined as:

\[\color{blue}{\displaystyle G(z)=\mathbb{E} \left(z^{X}\right)=\sum_{x=0}^{\infty }p(x)\;z^{x}}\]

where \(p\) is the probability mass function of \(X\). Note that the subscripted notations \(G_X\) and \(p_X\) are often used to emphasize that these pertain to a particular random variable \(X\), and to its distribution.

CHARACTERISTICS:

From this video:

  1. IT GIVES YOU PROBABILITIES by differentiating:

\[\color{blue}{\large p_i = \left. \frac{1}{i!}\quad\frac{d^i \, G(z)}{dx^i} \right|_{z=0}=\frac{1}{i!} \;G^{(i)}\;(0)}\]

  1. \(G\,(1)=1\) because \[\displaystyle\sum_{i=0}^\infty p_i \; 1^i=1\]

  2. First differential

\[G^{(1)}(z) =\frac{d}{dz}\mathbb E\left[z^X\right]=\mathbb E\left[X\,z^{X-1}\right]\]

  1. The first differential evaluated at \(1\) gives you the mean: \[G^{(1)}(1) =\left.\mathbb E\left[X\,z^{X-1}\right]\right|_{z=1}=\mathbb E\left[X\quad1^{X-1}\right]= \mathbb E[X].\]

  2. The second derivative evaluated at \(1\) is the factorial momment, and is NOT the variance, because the second term is not squared.

\[\begin{align}G^{(2)}\;(1) &=\frac{d^2}{dz^2}\; \left.\mathbb E\left[z^X\right]\right|_{z=1}\\[2ex]&=\mathbb E\left[X\;(X-1)\;z^{X-2}\right]\\[2ex]&=\mathbb E\left[X\;(X-1)\right]\\[2ex]&=\mathbb E\left [X^2-X\right ]\\[2ex]&=\mathbb E\left[X^2\right] - \mathbb E\left[X\right]\end{align}\]

  1. Generalizing, then, the \(i\)-th derivative evaluated at \(1\) is the \(i\)-th factorial moment:

\[G^{(i)}\;(1)= \mathbb E\left[X\;(X-1)\;\cdots\;(X-i+1)\right]\]

  1. To get the variance,

\[\begin{align}\sigma^2 &= \mathbb E\left[X^2\right]-\mathbb E\left[X\right]^2 \\[2ex] &=G^{(2)}\;(1)+G^{(1)}\;(1)-\left[G^{(1)}\;(1)\right]^2 \end{align}\]

  1. We can get raw moments by differentiating the pgf and multipling it by \(z\):

\[\mathbb E\left[X^i\right]= \left. \left( z\;\left(\frac{d}{dz}\right)^i \; G(z)\right)\right|_{z=1}\]

PGF FOR BERNOULLI DISTRIBUTION

\[f(i,p)= p^i\;(1-p)^{(1-i)} \text{ for }i\in \{0,1\}\]

The PGF is:

\[\begin{align}G(z)&=\sum_{i=0}^\infty p_i\;z^i\\[2ex]&= p_0\;z^0+p_1\;z^1\\[2ex] &= p^0\;(1-p)^{(1-0)}\times z^0 + p^1\;(1-p)^{(1-1)}\times z^1\\[2ex]&= (1-p)+p^1\times z^1\\[2ex]&= (1-p)+pz\\[2ex]&=\color{red}{q + pz}\end{align}\]

Differentiating and evaluating at zero it gives probabilities:

\[p_i = \left.\frac{1}{i!}\frac{d^iG(z)}{dz^i}\right|_{z=0}=\frac{1}{i!}G^{(i)}(0)\]

so

\[\color{red}{p_1= \frac{1}{1!}G^{(1)}(0)=\left.\frac{1}{1!}\frac{d}{dz}(q+pz)\right|_{z=0}=\;p}\]

and

\[\color{red}{p_0= \frac{1}{0!}G^{(0)}(0)=\left.\frac{1}{0!}(q+pz)\right|_{z=0}=\;q}\]

Home Page

NOTE: These are tentative notes on different topics for personal use - expect mistakes and misunderstandings.