5.2. Basic Statistics#
5.2.1. Statistic#
Let \( \{ X_1, X_2, ... X_n \} \) be a Random Sample set and \(\{x_1, x_2, ... x_n \} \) be the observed sample, the
A statistic \(T_n = T(X_1, X_2, ... X_n) \), is a function of a sample used to estimate the value of a parameter or something else (quantile, means etc)
Properties of statistic’s:-
Unbiased, a statistic whose expectation value is equal to the target parameter
Sample Mean Statistic (iid):
\[
\bar{X} = \frac{1}{n} \sum_{i = 1}^{n} X_i
\]
is called the mean statistic. It is an unbiased statistic as it has
\[
\mathbb{E}[\bar{X}] = \mu_{X} \; \; , \; \; Var(\bar{X}) = \frac{\sigma_{X}^2}{n}
\]
Sample Variance Statistic (iid):
\[
S^2 = \frac{1}{n-1} \sum_{i=1}^{n} (X_i - \bar{X})^2
\]
is called the variance statistics, it an unbiased statistic for the variance of RV X.
Proof of non bias in \(S^2\)
(5.5)#\[\begin{align}
\mathbb{E}[S^2] &= \frac{1}{n-1} \mathbb{E}[\sum_{i=1}^{n} (X_i - \bar{X})^2 ] \\
&= \frac{1}{n-1} \mathbb{E}[\sum_{i=1}^{n} X_i^2 + n\bar{X}^2 - 2 \bar{X} \sum_{i=1}^{n}X_i ] \\
&= \frac{1}{n-1}[ \sum_{i=1}^{n} \mathbb{E}[X_i^2] - n \mathbb{E}[\bar{X}^2]] \\
\end{align}\]
using
\[
\; \; \mathbb{E}[X_i^2] = var(X) + \mu_{X} \; \; , \; \; \mathbb{E}[\bar{X}^2] = Var(\bar{X}^2) + \mu_{\bar{X}}^2
\]
we get
(5.6)#\[\begin{align}
\mathbb{E}[S^2] &= \frac{1}{n-1}[ n\sigma^2 + n\mu^2 - n \frac{1}{n} \sigma^2 - n \mu^2 ] \\
&= \sigma^2 \\
\end{align}\]