Topics Covered

On this page

1. 1. Standard Normal Distribution
2. 2. Population, Sample, Parameter, Statistic
   • Sampling distribution
   • Parameters and statistics in symbols
   • Standard errors
   • What “exact” sampling distribution means
3. 3. Chi-square (χ²) Distribution
   • Deriving the density through the MGF
   • Cumulants, skewness and kurtosis
   • Mode
   • Additive property, proved
   • Three results from the density
   • Distribution of the sample variance
4. 4. Student's t-Distribution
   • Student's \(t\) and Fisher's \(t\)
   • Deriving the density
   • Moments
5. 5. F-Distribution (Snedecor's F)
   • Deriving the density
   • Moments
   • Mode
   • Reciprocal property, proved
6. 6. Relationships among Sampling Distributions
   • Proof that \(t^2 \sim F_{1,n}\)
7. 7. Limiting Cases of the Sampling Distributions
   • The limits proved
8. Key Take-aways

1. Standard Normal Distribution

MGF, Mean & Variance

CDF

\(\Phi(z) = P(Z \le z) = \int_{-\infty}^{z}\phi(u)\,du\). Tabulated; no closed form.

Area Property (Empirical Rule)

• \(P(-1 \le Z \le 1) = 0.6827\) (68.27 %)
• \(P(-2 \le Z \le 2) = 0.9545\) (95.45 %)
• \(P(-3 \le Z \le 3) = 0.9973\) (99.73 %)

Worked Examples

2. Population, Sample, Parameter, Statistic

Sampling Distribution

Examples:

• If \(X_i \sim N(\mu, \sigma^2)\): \(\bar X \sim N(\mu, \sigma^2/n)\).
• For any population (finite variance) and large \(n\), CLT gives \(\bar X \approx N(\mu, \sigma^2/n)\).
• SE of mean \(= \sigma/\sqrt n\).

Parameters and statistics in symbols

Write the population units as \(X_1, X_2, \ldots, X_N\) and the sample units as \(x_1, x_2, \ldots, x_n\).

Quantity	Population (parameter)	Sample (statistic)
Mean	\(\mu = \frac{1}{N}\sum_{i=1}^{N} X_i\)	\(\bar x = \frac{1}{n}\sum_{i=1}^{n} x_i\)
Variance	\(\sigma^2 = \frac{1}{N}\sum_{i=1}^{N} (X_i - \mu)^2\)	\(s^2 = \frac{1}{n}\sum_{i=1}^{n} (x_i - \bar x)^2\)
Standard deviation	\(\sigma = \sqrt{\sigma^2}\)	\(s = \sqrt{s^2}\)
Proportion	\(P = X/N\)	\(p = x/n\)

Here \(X\) and \(x\) in the last row count the units with the attribute. This unit also uses the sample variance with divisor \(n - 1\),

\[ S^2 = \frac{1}{n-1}\sum_{i=1}^{n} (x_i - \bar x)^2, \qquad \text{so that} \qquad n s^2 = (n-1) S^2 . \]

A statistic is random because the sample is drawn at random: a different sample gives a different value. That is why every statistic has a sampling distribution.

Standard errors

The standard error (S.E.) of a statistic is the standard deviation of its sampling distribution. For a random sample (with replacement, or from a large population):

The formulas for differences need the two samples to be independent: the variance of a difference of independent statistics is the sum of their variances.

What “exact” sampling distribution means

For large samples the central limit theorem gives an approximate normal distribution for many statistics. For a sample from a normal population some statistics have a distribution that can be written down exactly for every sample size, however small. These are the exact sampling distributions:

1. the chi-square (\(\chi^2\)) distribution,
2. Student's and Fisher's \(t\) distribution,
3. Snedecor's \(F\) distribution,
4. Fisher's \(z\) distribution (\(z = \tfrac12 \log_e F\)).

Because they are exact, they are the basis of the small sample tests. Sections 3 to 5 study the first three.

3. Chi-square (χ²) Distribution

PDF

Properties

• Mean = \(n\); Variance = \(2n\).
• MGF: \(M(t) = (1 - 2t)^{-n/2}, t < 1/2\).
• Additive: \(\chi^2_{n_1} + \chi^2_{n_2} \sim \chi^2_{n_1 + n_2}\) (independent).
• For large \(n\): \(\chi^2_n \approx N(n, 2n)\) (or use \(\sqrt{2\chi^2}\) approximation).
• Always non-negative; right-skewed (skewness \(\sqrt{8/n}\)).

Deriving the \(\chi^2\) density through the MGF

Setting. Let \(X_1, \ldots, X_n\) be independent with \(X_i \sim N(\mu_i, \sigma_i^2)\), and put \(z_i = (X_i - \mu_i)/\sigma_i\). Each \(z_i \sim N(0, 1)\), the \(z_i\) are independent, and \(\chi^2 = \sum z_i^2\).

1. MGF of one \(z^2\). By definition, for \(t < \tfrac12\), \[ M_{z^2}(t) = E\big(e^{tz^2}\big) = \int_{-\infty}^{\infty} e^{tz^2}\,\frac{1}{\sqrt{2\pi}}e^{-z^2/2}\,dz = \frac{2}{\sqrt{2\pi}}\int_0^{\infty} e^{-\frac{(1-2t)}{2}z^2}\,dz , \] using that the integrand is even.
2. Substitute \(y = z^2\), so \(dz = dy/(2\sqrt y)\) and \(y\) runs from 0 to \(\infty\): \[ M_{z^2}(t) = \frac{1}{\sqrt{2\pi}}\int_0^{\infty} e^{-\frac{(1-2t)}{2}y}\,y^{\frac12 - 1}\,dy = \frac{1}{\sqrt{2\pi}}\cdot\frac{\Gamma(\frac12)}{\big(\frac{1-2t}{2}\big)^{1/2}} = (1 - 2t)^{-1/2}, \] by the gamma integral \(\int_0^\infty e^{-ay}y^{k-1}dy = \Gamma(k)/a^k\) and \(\Gamma(\frac12) = \sqrt\pi\). The condition \(t < \frac12\) keeps \(a = (1-2t)/2\) positive.
3. Multiply. The MGF of a sum of independent variables is the product of their MGFs: \[ M_{\chi^2}(t) = \prod_{i=1}^{n} (1 - 2t)^{-1/2} = (1 - 2t)^{-n/2} . \]
4. Recognise it. \((1 - t/a)^{-\lambda}\) is the MGF of the gamma density \(\frac{a^\lambda}{\Gamma(\lambda)}e^{-ax}x^{\lambda-1}\). Here \(a = \frac12\) and \(\lambda = \frac n2\). By the uniqueness theorem of MGFs, \(\chi^2\) has the density stated above, \(\frac{(1/2)^{n/2}}{\Gamma(n/2)}e^{-\chi^2/2}(\chi^2)^{n/2-1}\). \(\blacksquare\)

Cumulants, skewness and kurtosis

The cumulant generating function is \(K(t) = \log M(t) = -\frac n2 \log(1 - 2t)\). Use the series \(-\log(1 - x) = x + \frac{x^2}{2} + \frac{x^3}{3} + \cdots\) with \(x = 2t\):

\[ K(t) = \frac n2\Big[2t + \frac{(2t)^2}{2} + \frac{(2t)^3}{3} + \frac{(2t)^4}{4} + \cdots\Big] = nt + n t^2 + \frac{4n}{3}t^3 + 2n t^4 + \cdots \]

The \(r\)-th cumulant \(\kappa_r\) is the coefficient of \(t^r/r!\), so multiply each coefficient by \(r!\):

\[ \kappa_1 = n, \qquad \kappa_2 = 2n, \qquad \kappa_3 = 8n, \qquad \kappa_4 = 48n . \]

• Mean \(= \kappa_1 = n\); variance \(\mu_2 = \kappa_2 = 2n\); \(\mu_3 = \kappa_3 = 8n\).
• \(\mu_4 = \kappa_4 + 3\kappa_2^2 = 48n + 12n^2 = 12n(n + 4)\).
• Skewness: \(\beta_1 = \dfrac{\mu_3^2}{\mu_2^3} = \dfrac{64n^2}{8n^3} = \dfrac{8}{n}\). Since \(\mu_3 > 0\), the distribution is positively skewed.
• Kurtosis: \(\beta_2 = \dfrac{\mu_4}{\mu_2^2} = \dfrac{12n(n+4)}{4n^2} = 3 + \dfrac{12}{n} > 3\), so it is leptokurtic.

Both \(\beta_1 \to 0\) and \(\beta_2 \to 3\) as \(n \to \infty\), the values for a normal curve, which agrees with the limiting result of section 7.

Characteristic function. The same gamma integral with \(it\) in place of \(t\) gives \(\phi(t) = E(e^{it\chi^2}) = (1 - 2it)^{-n/2}\). Unlike the MGF, it exists for every real \(t\).

Mode of \(\chi^2\)

Up to a constant, \(\log f(x) = \big(\frac n2 - 1\big)\log x - \frac x2\). Setting the derivative to zero,

\[ \frac{d}{dx}\log f(x) = \frac{n/2 - 1}{x} - \frac12 = 0 \quad\Longrightarrow\quad x = n - 2 . \]

The second derivative, \(-(n/2 - 1)/x^2\), is negative for \(n > 2\), so the mode is \(n - 2\) when \(n > 2\). For \(n = 1\) and \(n = 2\) the density decreases from \(x = 0\), so the mode is at 0.

Additive (reproductive) property, proved

Let \(\chi_1^2, \ldots, \chi_k^2\) be independent with \(n_1, \ldots, n_k\) degrees of freedom. The MGF of their sum is the product of their MGFs:

\[ M_{\chi_1^2 + \cdots + \chi_k^2}(t) = (1-2t)^{-n_1/2}\cdots(1-2t)^{-n_k/2} = (1 - 2t)^{-(n_1 + \cdots + n_k)/2}, \]

which is the MGF of \(\chi^2\) with \(n_1 + \cdots + n_k\) degrees of freedom. By uniqueness, \(\chi_1^2 + \cdots + \chi_k^2 \sim \chi^2_{n_1 + \cdots + n_k}\).

Applications

1. Test for population variance.
2. Goodness of fit test.
3. Test of independence in contingency tables.
4. Inference about variance of a normal population: \((n-1)S^2/\sigma^2 = ns^2/\sigma^2 \sim \chi^2_{n-1}\).
5. Testing the homogeneity of several independent estimates of a population variance.
6. Testing the homogeneity of several independent estimates of a population correlation coefficient (through Fisher's \(z\)-transformation).

Three results from the \(\chi^2\) density

Proof. \(x = 2y\), so \(|J| = |dx/dy| = 2\). Then \(f_Y(y) = f_X(2y)\cdot 2 = \dfrac{e^{-y}(2y)^{n/2-1}\cdot 2}{2^{n/2}\Gamma(n/2)} = \dfrac{e^{-y}y^{n/2-1}}{\Gamma(n/2)}\), since \(2^{n/2-1}\cdot 2 = 2^{n/2}\). \(\blacksquare\)

1. Transformation. Put \(u = x/y\), \(v = y\), so \(x = uv\), \(y = v\), and \(J = \begin{vmatrix} v & u \\ 0 & 1 \end{vmatrix} = v\).
2. Joint density (independence lets us multiply): \[ f(u, v) = f_X(uv)\,f_Y(v)\,v = \frac{u^{n_1/2-1}\,v^{(n_1+n_2)/2 - 1}\,e^{-(1+u)v/2}}{2^{(n_1+n_2)/2}\,\Gamma(\frac{n_1}{2})\Gamma(\frac{n_2}{2})} . \]
3. Integrate out \(v\) with the gamma integral, \(a = (1+u)/2\), \(k = (n_1+n_2)/2\): \[ f(u) = \frac{u^{n_1/2-1}}{2^{(n_1+n_2)/2}\Gamma(\frac{n_1}{2})\Gamma(\frac{n_2}{2})}\cdot\frac{\Gamma(\frac{n_1+n_2}{2})\,2^{(n_1+n_2)/2}}{(1+u)^{(n_1+n_2)/2}} , \] and \(\Gamma(\frac{n_1}{2})\Gamma(\frac{n_2}{2})/\Gamma(\frac{n_1+n_2}{2}) = B(\frac{n_1}{2}, \frac{n_2}{2})\) gives the result. \(\blacksquare\)

1. Transformation. \(u = x/(x+y)\), \(v = x + y\), so \(x = uv\), \(y = (1-u)v\), and \(J = \begin{vmatrix} v & u \\ -v & 1-u \end{vmatrix} = v(1-u) + uv = v\).
2. Joint density. The exponents add: \(e^{-uv/2}e^{-(1-u)v/2} = e^{-v/2}\), so \[ f(u, v) = \frac{u^{n_1/2-1}(1-u)^{n_2/2-1}\,v^{(n_1+n_2)/2-1}e^{-v/2}}{2^{(n_1+n_2)/2}\Gamma(\frac{n_1}{2})\Gamma(\frac{n_2}{2})} . \]
3. Integrate out \(v\): \(\int_0^\infty v^{(n_1+n_2)/2-1}e^{-v/2}dv = \Gamma(\frac{n_1+n_2}{2})\,2^{(n_1+n_2)/2}\), which leaves the stated density.
4. Range. \(x \ge 0\) and \(y \ge 0\) give \(uv \ge 0\) and \((1-u)v \ge 0\) with \(v > 0\), so \(0 \le u \le 1\). \(\blacksquare\)

Distribution of the sample variance

1. Split the sum of squares. Add and subtract \(\bar x\): \[ \sum (x_i - \mu)^2 = \sum (x_i - \bar x)^2 + n(\bar x - \mu)^2 + 2(\bar x - \mu)\sum (x_i - \bar x) , \] and the last sum is 0, because the deviations from the mean add to zero.
2. Divide by \(\sigma^2\) and name the three pieces: \[ \underbrace{\sum \Big(\frac{x_i - \mu}{\sigma}\Big)^2}_{V} = \underbrace{\frac{ns^2}{\sigma^2}}_{W} + \underbrace{\Big(\frac{\bar x - \mu}{\sigma/\sqrt n}\Big)^2}_{Z^2} . \]
3. Identify \(V\) and \(Z^2\). Each \((x_i - \mu)/\sigma \sim N(0, 1)\), independently, so \(V \sim \chi^2_n\). Since \(\bar x \sim N(\mu, \sigma^2/n)\), the ratio \((\bar x - \mu)/(\sigma/\sqrt n) \sim N(0, 1)\), so \(Z^2 \sim \chi^2_1\).
4. Use independence. For a normal sample, \(\bar x\) and \(s^2\) are independent (proved by the Helmert transformation in Distribution Theory, Unit 3), so \(W\) and \(Z^2\) are independent and \(M_V(t) = M_W(t)\,M_{Z^2}(t)\).
5. Solve for \(M_W\): \[ M_W(t) = \frac{(1 - 2t)^{-n/2}}{(1 - 2t)^{-1/2}} = (1 - 2t)^{-(n-1)/2}, \] the MGF of \(\chi^2_{n-1}\). By uniqueness, \(W = ns^2/\sigma^2 \sim \chi^2_{n-1}\). One degree of freedom is lost because the deviations \(x_i - \bar x\) satisfy one linear restriction. \(\blacksquare\)

Density of \(s^2\). Put \(y = ns^2/\sigma^2\), so \(|dy/ds^2| = n/\sigma^2\), and use the \(\chi^2_{n-1}\) density of \(y\):

\[ f(s^2) = \frac{\big(\frac{n}{2\sigma^2}\big)^{(n-1)/2}}{\Gamma\big(\frac{n-1}{2}\big)}\,e^{-ns^2/(2\sigma^2)}\,(s^2)^{(n-3)/2}, \qquad 0 \le s^2 < \infty . \]

Mean and variance of \(s^2\). From \(E(W) = n - 1\) and \(\text{Var}(W) = 2(n-1)\), with \(W = ns^2/\sigma^2\):

\[ E(s^2) = \frac{n-1}{n}\sigma^2 = \Big(1 - \frac1n\Big)\sigma^2, \qquad \text{Var}(s^2) = \frac{2(n-1)}{n^2}\sigma^4 = \frac{2\sigma^4}{n}\Big(1 - \frac1n\Big) . \]

So \(s^2\) slightly underestimates \(\sigma^2\) (which is why \(S^2\), with divisor \(n - 1\), is used for estimation), and for large \(n\), \(E(s^2) \approx \sigma^2\) and \(\text{Var}(s^2) \approx 2\sigma^4/n\).

4. Student's t-Distribution

PDF

Properties

• Symmetric about 0; bell-shaped, heavier tails than normal.
• Mean = 0 (for \(n > 1\)); Variance = \(n/(n-2)\) (for \(n > 2\)).
• As \(n \to \infty\): \(t_n \to N(0,1)\).
• Skewness 0; kurtosis \(\beta_2 > 3\) for \(n > 4\).

Student's \(t\) and Fisher's \(t\)

• Fisher's \(t\) is the definition above: \(t = \xi\big/\sqrt{\chi^2/n} \sim t_n\), with \(\xi \sim N(0, 1)\) independent of \(\chi^2 \sim \chi^2_n\).
• Student's \(t\) (W. S. Gosset, 1908) is the statistic for a sample of \(n\) from \(N(\mu, \sigma^2)\): \[ t = \frac{\bar x - \mu}{S/\sqrt n} \sim t_{n-1}, \qquad S^2 = \frac{1}{n-1}\sum (x_i - \bar x)^2 . \]

Student's \(t\) is a particular case of Fisher's. Take \(\xi = \dfrac{\bar x - \mu}{\sigma/\sqrt n} \sim N(0,1)\) and \(\chi^2 = \dfrac{ns^2}{\sigma^2} \sim \chi^2_{n-1}\), which are independent. Then

\[ \frac{\xi}{\sqrt{\chi^2/(n-1)}} = \frac{(\bar x - \mu)\sqrt n/\sigma}{\sqrt{ns^2/(\sigma^2(n-1))}} = \frac{\bar x - \mu}{\sqrt{ns^2/(n(n-1))}} = \frac{\bar x - \mu}{S/\sqrt n}, \]

using \(ns^2 = (n-1)S^2\). The unknown \(\sigma\) cancels, which is what makes \(t\) usable in practice. So Student's \(t\) has Fisher's \(t\) distribution with \(n - 1\) degrees of freedom.

Deriving the \(t\) density

1. Start from the independent densities \(f(\xi) = \frac{1}{\sqrt{2\pi}}e^{-\xi^2/2}\) and \(f(u) = \frac{1}{2^{n/2}\Gamma(n/2)}e^{-u/2}u^{n/2-1}\), where \(u = \chi^2\).
2. Transformation. \(t = \xi/\sqrt{u/n}\) and \(u = u\), so \(\xi = t\sqrt{u/n}\) and \(J = \begin{vmatrix} \sqrt{u/n} & \frac{t}{2\sqrt{nu}} \\ 0 & 1 \end{vmatrix} = \sqrt{u/n}\).
3. Joint density. \(f(t, u) = f\big(t\sqrt{u/n}\big)\,f(u)\,\sqrt{u/n}\); collecting the powers of \(u\) and the exponentials, \[ f(t, u) = \frac{1}{\sqrt{2\pi n}\;2^{n/2}\,\Gamma(\frac n2)}\;u^{\frac{n+1}{2}-1}\,e^{-\frac u2\left(1 + \frac{t^2}{n}\right)} . \]
4. Integrate out \(u\) with the gamma integral, \(a = \frac12(1 + t^2/n)\), \(k = \frac{n+1}{2}\): \[ f(t) = \frac{\Gamma(\frac{n+1}{2})\,2^{(n+1)/2}}{\sqrt{2\pi n}\;2^{n/2}\,\Gamma(\frac n2)}\Big(1 + \frac{t^2}{n}\Big)^{-\frac{n+1}{2}} = \frac{\Gamma(\frac{n+1}{2})}{\sqrt{n\pi}\;\Gamma(\frac n2)}\Big(1 + \frac{t^2}{n}\Big)^{-\frac{n+1}{2}} . \]
5. Write it with the beta function. \(B(\frac12, \frac n2) = \Gamma(\frac12)\Gamma(\frac n2)/\Gamma(\frac{n+1}{2})\) and \(\Gamma(\frac12) = \sqrt\pi\), so the constant is \(1/(\sqrt n\,B(\frac12, \frac n2))\), as stated above. \(\blacksquare\)

Another route. \(t^2/n = \xi^2/\chi^2\) is a ratio of independent \(\chi^2_1\) and \(\chi^2_n\) variates, so by Result 2 of section 3 it is a beta variate of the second kind, \(\beta_2(\frac12, \frac n2)\). Changing variable from \(t^2/n\) to \(t\) needs care: each value of \(t^2\) comes from two values, \(t\) and \(-t\), and the density is shared equally between them. That halving cancels the factor 2 in \(|d(t^2/n)/dt| = 2|t|/n\) and gives the same density.

Moments of \(t\)

• Odd moments. \(t^{2r+1}f(t)\) is an odd function, so \(\mu'_{2r+1} = 0\) whenever the moment exists (for \(2r + 1 < n\)). With \(r = 0\): the mean is 0 for \(n > 1\).
• Even moments. Using the even integrand and the substitution \(1 + t^2/n = 1/y\), which turns the integral into a beta integral, \[ \mu'_{2r} = \frac{n^r}{B(\frac12, \frac n2)}\int_0^1 y^{\frac n2 - r - 1}(1-y)^{r - \frac12}\,dy = n^r\,\frac{\Gamma(\frac n2 - r)\,\Gamma(r + \frac12)}{\Gamma(\frac12)\,\Gamma(\frac n2)}, \qquad n > 2r . \]
• \(r = 1\): using \(\Gamma(\frac32) = \frac12\Gamma(\frac12)\) and \(\Gamma(\frac n2) = (\frac n2 - 1)\Gamma(\frac n2 - 1)\), \(\mu_2 = \dfrac{n}{n-2}\) for \(n > 2\).
• \(r = 2\): \(\mu_4 = \dfrac{3n^2}{(n-2)(n-4)}\) for \(n > 4\).
• \(\beta_1 = 0\) (symmetric), and \(\beta_2 = \dfrac{\mu_4}{\mu_2^2} = \dfrac{3(n-2)}{n-4} > 3\) for \(n > 4\): leptokurtic.

No MGF. Only the moments of order less than \(n\) exist, and \(E(e^{st})\) is infinite for every \(s \ne 0\), because \(e^{st}\) grows faster than any power of \(t\) while the density falls only like \(|t|^{-(n+1)}\). So the \(t\) distribution has no moment generating function.

Applications

1. Inference about mean when SD is unknown (single mean, paired samples, two means with equal variances).
2. Confidence intervals: \(\bar x \pm t_{\alpha/2, n-1}\, S/\sqrt n\).
3. Tests of significance of a regression coefficient.
4. Tests of significance of an observed sample correlation coefficient and of a partial correlation coefficient. (A multiple correlation coefficient is tested with \(F\), section 5.)

5. F-Distribution (Snedecor's F)

PDF

Properties

• \(F > 0\); right-skewed.
• Mean \(= \dfrac{n_2}{n_2 - 2}\) for \(n_2 > 2\).
• Variance \(= \dfrac{2 n_2^2 (n_1 + n_2 - 2)}{n_1(n_2 - 2)^2(n_2 - 4)}\) for \(n_2 > 4\).
• Reciprocal property: if \(F \sim F_{n_1,n_2}\) then \(1/F \sim F_{n_2, n_1}\).
• \(F_{1, n_2} = t_{n_2}^2\) (the square of t with \(n_2\) df).

Deriving the \(F\) density

1. Start from independent \(X \sim \chi^2_{n_1}\) and \(Y \sim \chi^2_{n_2}\), and \(F = \dfrac{X/n_1}{Y/n_2}\).
2. Transformation. \(F\) and \(u = y\), so \(x = \frac{n_1}{n_2}Fu\), \(y = u\), and \(J = \begin{vmatrix} \frac{n_1}{n_2}u & \frac{n_1}{n_2}F \\ 0 & 1 \end{vmatrix} = \frac{n_1}{n_2}u\).
3. Joint density. \(f(F, u) = f_X\big(\frac{n_1}{n_2}Fu\big)\,f_Y(u)\,\frac{n_1}{n_2}u\), which collects to \[ f(F, u) = \frac{(n_1/n_2)^{n_1/2}\,F^{n_1/2-1}}{2^{(n_1+n_2)/2}\,\Gamma(\frac{n_1}{2})\Gamma(\frac{n_2}{2})}\;u^{\frac{n_1+n_2}{2}-1}\,e^{-\frac u2\left(1 + \frac{n_1}{n_2}F\right)} . \]
4. Integrate out \(u\) with the gamma integral, \(a = \frac12(1 + \frac{n_1}{n_2}F)\), \(k = \frac{n_1+n_2}{2}\). The powers of 2 cancel and the gammas form \(B(\frac{n_1}{2}, \frac{n_2}{2})\), leaving the density stated above. \(\blacksquare\)

Moments of \(F\)

Substitute \(y = \frac{n_1}{n_2}F\) in \(\mu'_r = \int_0^\infty F^r f(F)\,dF\). The integral becomes \(\big(\frac{n_2}{n_1}\big)^r \int_0^\infty \frac{y^{n_1/2 + r - 1}}{(1+y)^{(n_1+n_2)/2}}\,dy\big/B(\frac{n_1}{2}, \frac{n_2}{2})\), a beta integral of the second kind:

\[ \mu'_r = \Big(\frac{n_2}{n_1}\Big)^r \frac{B(\frac{n_1}{2} + r, \frac{n_2}{2} - r)}{B(\frac{n_1}{2}, \frac{n_2}{2})} = \Big(\frac{n_2}{n_1}\Big)^r \frac{\Gamma(\frac{n_1}{2} + r)\,\Gamma(\frac{n_2}{2} - r)}{\Gamma(\frac{n_1}{2})\,\Gamma(\frac{n_2}{2})}, \qquad n_2 > 2r . \]

• \(r = 1\): \(\Gamma(\frac{n_1}{2} + 1) = \frac{n_1}{2}\Gamma(\frac{n_1}{2})\) and \(\Gamma(\frac{n_2}{2}) = (\frac{n_2}{2} - 1)\Gamma(\frac{n_2}{2} - 1)\), so the mean is \(\dfrac{n_2}{n_1}\cdot\dfrac{n_1/2}{n_2/2 - 1} = \dfrac{n_2}{n_2 - 2}\), for \(n_2 > 2\).
• \(r = 2\): \(\mu'_2 = \dfrac{n_2^2(n_1 + 2)}{n_1(n_2 - 2)(n_2 - 4)}\), for \(n_2 > 4\).
• Variance: \(\mu'_2 - \mu_1'^2 = \dfrac{2n_2^2(n_1 + n_2 - 2)}{n_1(n_2 - 2)^2(n_2 - 4)}\), for \(n_2 > 4\).

Mode of \(F\)

Up to a constant, \(\log f = \big(\frac{n_1}{2} - 1\big)\log F - \frac{n_1+n_2}{2}\log\big(1 + \frac{n_1}{n_2}F\big)\). Setting the derivative to zero and multiplying through by \(2F(1 + n_1F/n_2)\):

\[ (n_1 - 2)\Big(1 + \frac{n_1}{n_2}F\Big) = (n_1 + n_2)\frac{n_1}{n_2}F \quad\Longrightarrow\quad F = \frac{n_2(n_1 - 2)}{n_1(n_2 + 2)}, \qquad n_1 > 2 . \]

Both factors \(\frac{n_1-2}{n_1}\) and \(\frac{n_2}{n_2+2}\) are below 1, so the mode is always less than 1, while the mean \(\frac{n_2}{n_2-2}\) is above 1: the distribution is positively skewed.

Reciprocal property, proved

If \(F = \dfrac{X/n_1}{Y/n_2} \sim F_{n_1, n_2}\), then \(\dfrac1F = \dfrac{Y/n_2}{X/n_1}\) is again a ratio of independent \(\chi^2\) variates, each divided by its degrees of freedom, now with \(Y\) on top. So \(1/F \sim F_{n_2, n_1}\). This is why tables give only upper percentage points: the lower point is \(F_{1-\alpha}(n_1, n_2) = 1/F_{\alpha}(n_2, n_1)\).

Applications

1. Test of equality of two population variances.
2. One-way and two-way ANOVA.
3. Test of overall significance in regression.
4. Testing the linearity of regression, and the significance of an observed multiple correlation coefficient or correlation ratio.

6. Relationships among Sampling Distributions

• Standard Normal: \(Z = (X - \mu)/\sigma\).
• \(\chi^2_n = \sum Z_i^2\) (sum of \(n\) squared standard normals).
• \(t_n = Z/\sqrt{\chi^2_n / n}\).
• \(F_{n_1, n_2} = (\chi^2_{n_1}/n_1)/(\chi^2_{n_2}/n_2)\).
• \(t_n^2 = F_{1, n}\).
• For large df: \(t \to Z\); \((\chi^2 - n)/\sqrt{2n} \to Z\).

Proof that \(t^2 \sim F_{1, n}\)

Write \(t = \xi/\sqrt{\chi^2/n}\). Squaring,

\[ t^2 = \frac{\xi^2/1}{\chi^2/n} . \]

The numerator \(\xi^2\) is the square of a standard normal variate, so \(\xi^2 \sim \chi^2_1\); it is independent of \(\chi^2 \sim \chi^2_n\). So \(t^2\) is a ratio of independent \(\chi^2\) variates, each divided by its degrees of freedom: \(t^2 \sim F_{1, n}\). In tables, \(t_{\alpha/2}(n)^2 = F_{\alpha}(1, n)\); for example \(2.228^2 = 4.96 = F_{0.05}(1, 10)\).

7. Limiting Cases of the Sampling Distributions

The limits proved from the generating function and the densities

\(\chi^2 \to\) normal, through the CGF. Let \(z = (\chi^2 - n)/\sqrt{2n}\). Then \(M_z(t) = e^{-tn/\sqrt{2n}}M_{\chi^2}(t/\sqrt{2n}) = e^{-t\sqrt{n/2}}\big(1 - t\sqrt{2/n}\big)^{-n/2}\), and

\[ K_z(t) = -t\sqrt{\tfrac n2} + \frac n2\Big[t\sqrt{\tfrac 2n} + \frac{t^2}{2}\cdot\frac 2n + \frac{t^3}{3}\Big(\frac 2n\Big)^{3/2} + \cdots\Big] = \frac{t^2}{2} + \frac{t^3}{3}\sqrt{\frac 2n} + \cdots \]

The first two terms cancel, and every term after \(t^2/2\) has a power of \(\sqrt n\) in its denominator, so \(K_z(t) \to t^2/2\), the CGF of \(N(0, 1)\). By the continuity theorem for MGFs, \(z\) tends in distribution to \(N(0, 1)\).

Proof. Stirling's formula, \(\Gamma(m) \approx \sqrt{2\pi}\,m^{m - 1/2}e^{-m}\), gives

\[ \frac{\Gamma(n+K)}{\Gamma(n)\,n^K} \approx \frac{(n+K)^{n+K-\frac12}e^{-K}}{n^{n-\frac12}\,n^K} = \Big(1 + \frac Kn\Big)^{n}\Big(1 + \frac Kn\Big)^{K - \frac12}e^{-K} \to e^K\cdot 1\cdot e^{-K} = 1, \]

using \((1 + K/n)^n \to e^K\) and \((1 + K/n)^{c} \to 1\) for a fixed exponent \(c\). (With an exponent that grows with \(n\) the second limit fails: \((1 + 1/n)^n \to e\), not 1.) \(\blacksquare\)

\(t \to\) normal, through the density. In \(f(t) = \dfrac{\Gamma(\frac{n+1}{2})}{\sqrt{n\pi}\,\Gamma(\frac n2)}\big(1 + \frac{t^2}{n}\big)^{-n/2}\big(1 + \frac{t^2}{n}\big)^{-1/2}\):

• by the lemma with \(n/2\) and \(K = \frac12\), \(\Gamma(\frac n2 + \frac12)/\Gamma(\frac n2) \approx (\frac n2)^{1/2}\), so the constant tends to \((\frac n2)^{1/2}/\sqrt{n\pi} = 1/\sqrt{2\pi}\);
• \(\big(1 + \frac{t^2}{n}\big)^{-n/2} = \Big[\big(1 + \frac{t^2}{n}\big)^{n}\Big]^{-1/2} \to (e^{t^2})^{-1/2} = e^{-t^2/2}\);
• \(\big(1 + \frac{t^2}{n}\big)^{-1/2} \to 1\).

So \(f(t) \to \frac{1}{\sqrt{2\pi}}e^{-t^2/2}\), the standard normal density.

\(n_1F \to \chi^2_{n_1}\), through the density. In the \(F\) density, by the lemma with \(n_2/2\) and \(K = n_1/2\), \(\Gamma(\frac{n_1+n_2}{2})/\Gamma(\frac{n_2}{2}) \approx (\frac{n_2}{2})^{n_1/2}\), so \((n_1/n_2)^{n_1/2}(n_2/2)^{n_1/2} = (n_1/2)^{n_1/2}\). Also \(\big(1 + \frac{n_1F}{n_2}\big)^{-n_2/2} \to e^{-n_1F/2}\) and \(\big(1 + \frac{n_1F}{n_2}\big)^{-n_1/2} \to 1\). Hence

\[ f(F) \to \frac{(n_1/2)^{n_1/2}}{\Gamma(n_1/2)}F^{n_1/2-1}e^{-n_1F/2} . \]

Now put \(x = n_1F\), with \(|dF/dx| = 1/n_1\): the density of \(x\) is \(\frac{(1/2)^{n_1/2}}{\Gamma(n_1/2)}e^{-x/2}x^{n_1/2-1}\), the \(\chi^2_{n_1}\) density.

Key Take-aways

• Standard Normal: mean 0, variance 1; central role in inference.
• Sampling distribution describes how a statistic varies sample-to-sample.
• \(\chi^2_n\): Gamma\((n/2, 1/2)\); used for variance and goodness-of-fit tests.
• \(t_n\): bell-shaped, heavy tails; used when \(\sigma\) is unknown.
• \(F_{n_1, n_2}\): ratio of variances; used in ANOVA & regression.
• Together \(Z, t, F, \chi^2\) form the foundation of classical statistical inference.
• \(\chi^2_n\): \(\kappa_r = n2^{r-1}(r-1)!\), so \(\beta_1 = 8/n\), \(\beta_2 = 3 + 12/n\); mode \(n - 2\); \(X/Y \sim \beta_2(\frac{n_1}{2}, \frac{n_2}{2})\) and \(X/(X+Y) \sim \beta_1(\frac{n_1}{2}, \frac{n_2}{2})\).
• For a normal sample \(ns^2/\sigma^2 = (n-1)S^2/\sigma^2 \sim \chi^2_{n-1}\), so \(E(s^2) = (1 - \frac1n)\sigma^2\) and \(\text{Var}(s^2) = 2(n-1)\sigma^4/n^2\).
• Student's \(t = (\bar x - \mu)/(S/\sqrt n)\) is Fisher's \(t\) with \(n - 1\) df; \(\mu_2 = n/(n-2)\), \(\beta_2 = 3(n-2)/(n-4)\); no MGF.
• \(F\): \(\mu'_r = (n_2/n_1)^r\,\Gamma(\frac{n_1}{2} + r)\Gamma(\frac{n_2}{2} - r)/[\Gamma(\frac{n_1}{2})\Gamma(\frac{n_2}{2})]\); mode \(n_2(n_1-2)/(n_1(n_2+2)) < 1\); \(1/F \sim F_{n_2, n_1}\).
• Limiting cases: \(t_n \to N(0,1)\) and \((\chi^2_n - n)/\sqrt{2n} \to N(0,1)\) as df \(\to \infty\); \(n_1 F_{n_1,n_2} \to \chi^2_{n_1}\) as \(n_2 \to \infty\).