Skip to the content

Topics Covered

Discrete Uniform Bernoulli Binomial MGF, CF, CGF, PGF Additive Property Skewness & Kurtosis Recurrence Relation Limiting Form to Normal
On this page
  1. 1. Discrete Uniform Distribution
  2. 2. Bernoulli Distribution
  3. 3. Binomial Distribution
  4. 4. Multinomial Distribution
  5. Key Take-aways from Unit 1

1. Discrete Uniform Distribution

DEFINITION

A random variable \(X\) is said to follow a discrete uniform distribution on \(\{1,2,\ldots,N\}\) if each of the \(N\) values is equally likely:

\[ P(X = x) = \dfrac{1}{N}, \quad x = 1, 2, \ldots, N. \]

Mean and Variance

\[ E(X) \;=\; \dfrac{N+1}{2}, \qquad \text{Var}(X) \;=\; \dfrac{N^2 - 1}{12}. \]

Derivation: \(\sum_{x=1}^{N} x = N(N+1)/2\) gives \(E(X) = (N+1)/2\). \(\sum_{x=1}^{N} x^2 = N(N+1)(2N+1)/6\), so

\(E(X^2) = (N+1)(2N+1)/6\) and Var \(= E(X^2) - [E(X)]^2 = (N^2-1)/12\).

MGF

\[ M_X(t) \;=\; \dfrac{1}{N}\sum_{x=1}^{N} e^{tx} \;=\; \dfrac{e^t(e^{Nt}-1)}{N(e^t-1)}, \quad t \ne 0. \]
EXAMPLE 1

A fair die: \(N=6\), \(P(X=x) = 1/6\) for \(x=1,\ldots,6\).
Mean = 7/2 = 3.5; Variance = (36-1)/12 = 35/12 ≈ 2.917.

EXAMPLE 2

A random integer is selected from {1,…,10}. Mean = 5.5, Variance = 99/12 = 8.25.
\(P(X \le 4) = 4/10 = 0.4\).

2. Bernoulli Distribution

DEFINITION

A random variable \(X\) follows a Bernoulli distribution with parameter \(p\) (\(0 < p < 1\)) if

\[ P(X = 1) = p, \quad P(X = 0) = q, \quad q = 1 - p. \]

It models a single trial with two outcomes ("success" / "failure"). Notation: \(X \sim B(1,p)\) or Bernoulli\((p)\).

Mean and Variance

\[ E(X) \;=\; p, \qquad \text{Var}(X) \;=\; pq. \]

MGF, CF, CGF, PGF

\[ M_X(t) = q + p e^t, \quad \phi_X(t) = q + p e^{it}, \quad K_X(t) = \ln(q + p e^t), \quad P_X(s) = q + p s. \]
EXAMPLE 1

Flipping a fair coin once with \(X = 1\) for head: \(p = 0.5\), Mean = 0.5, Variance = 0.25.

EXAMPLE 2

Defective bolt rate is 2 %. Let \(X = 1\) if a randomly chosen bolt is defective. \(E(X) = 0.02,\; \text{Var}(X) = 0.0196\).

3. Binomial Distribution

3.1 Definition & PMF

DEFINITION

If \(n\) independent Bernoulli\((p)\) trials are performed, the number of successes \(X\) follows a Binomial distribution \(B(n, p)\) with PMF

\[ P(X = x) \;=\; \binom{n}{x} p^x q^{n-x}, \quad x = 0, 1, \ldots, n. \]

Conditions for Binomial model:

  1. Fixed number of trials \(n\).
  2. Each trial has only two outcomes: success / failure.
  3. Probability of success \(p\) is constant across trials.
  4. Trials are independent.

3.2 Moments via MGF

\[ M_X(t) \;=\; E(e^{tX}) \;=\; \sum_{x=0}^{n}\binom{n}{x}(pe^t)^x q^{n-x} \;=\; (q + p e^t)^n. \]

\(M'_X(0) = np = E(X)\) (mean).

\(M''_X(0) = n(n-1)p^2 + np \Rightarrow E(X^2) = n^2p^2 - np^2 + np\).

Hence \(\text{Var}(X) = E(X^2) - [E(X)]^2 = (n^2p^2 - np^2 + np) - n^2p^2 = np - np^2 = np(1-p) = npq.\)

MEAN & VARIANCE \[ E(X) = np, \qquad \text{Var}(X) = npq. \]

3.3 Other Generating Functions

\[ \phi_X(t) = (q + p e^{it})^n, \quad K_X(t) = n\ln(q + p e^t), \quad P_X(s) = (q + p s)^n. \]

3.4 Skewness and Kurtosis

\[ \mu_3 = npq(q - p), \qquad \mu_4 = 3n^2p^2q^2 + npq(1 - 6pq). \] \[ \gamma_1 = \dfrac{q - p}{\sqrt{npq}}, \qquad \beta_2 = 3 + \dfrac{1 - 6pq}{npq}. \]

Hence \(\gamma_2 = \beta_2 - 3 = (1-6pq)/(npq)\).

If \(p < 0.5\): right-skewed; \(p > 0.5\): left-skewed; \(p = 0.5\): symmetric.

3.5 Additive Property

If \(X \sim B(n_1, p)\) and \(Y \sim B(n_2, p)\) are independent (same \(p\)), then

\[ X + Y \sim B(n_1 + n_2,\; p). \]

The Binomial is not additive across different \(p\) values.

3.6 Recurrence Relation for Probabilities

\[ P(X = x+1) \;=\; \dfrac{n - x}{x + 1} \cdot \dfrac{p}{q} \cdot P(X = x). \]

Starting from \(P(X = 0) = q^n\), all subsequent probabilities can be generated rapidly.

x → 0 to 10 B(10, 0.3) — right-skewed x → 0 to 10 B(10, 0.5) — symmetric
Fig 1.1 — Shape of Binomial PMF for p = 0.3 (right-skewed) vs p = 0.5 (symmetric)

3.7 Limiting Case: Binomial → Normal

DE MOIVRE–LAPLACE

For large \(n\) (with \(np\) and \(nq\) both moderately large):

\[ \dfrac{X - np}{\sqrt{npq}} \;\xrightarrow{d}\; N(0, 1). \]

Rule of thumb: normal approximation is good when \(np \ge 5\) and \(nq \ge 5\).

3.8 Worked Examples

EXAMPLE 1

10 % of items produced by a machine are defective. In a sample of 5, find the probability that (i) exactly 2 are defective, (ii) at most 1 is defective.

\(n = 5, p = 0.1, q = 0.9\).

(i) \(P(X = 2) = \binom{5}{2}(0.1)^2(0.9)^3 = 10(0.01)(0.729) = 0.0729\).

(ii) \(P(X \le 1) = (0.9)^5 + 5(0.1)(0.9)^4 = 0.5905 + 0.3281 = 0.9185\).

EXAMPLE 2 (Mean & Variance)

An exam has 20 multiple-choice questions, each with 4 options. A student guesses every answer. Find the expected score and SD.

\(n = 20,\; p = 1/4,\; q = 3/4\).

Mean = \(np = 5\); Variance = \(npq = 3.75\); SD = 1.936.

4. Multinomial Distribution

DEFINITION

The multinomial distribution generalises the binomial to more than two outcomes. In \(n\) independent trials, each trial results in exactly one of \(k\) mutually exclusive categories with probabilities \(p_1, p_2, \dots, p_k\) (\(\sum p_i = 1\)). If \(X_i\) is the number of trials falling in category \(i\), then

\[ P(X_1 = x_1, \dots, X_k = x_k) = \dfrac{n!}{x_1!\,x_2!\cdots x_k!}\, p_1^{x_1} p_2^{x_2}\cdots p_k^{x_k}, \qquad \sum_{i=1}^{k} x_i = n. \]

With \(k = 2\) this reduces to the binomial. Each marginal is binomial: \(X_i \sim B(n, p_i)\).

MOMENTS \[ E(X_i) = n p_i, \qquad \text{Var}(X_i) = n p_i (1 - p_i), \qquad \text{Cov}(X_i, X_j) = -\,n p_i p_j \;\;(i \neq j). \]

The covariance is negative: since the counts must add to \(n\), a large count in one category forces smaller counts elsewhere.

EXAMPLE 1

A fair die is rolled 12 times. The probability that each face appears exactly twice is

\(P = \dfrac{12!}{(2!)^{6}}\left(\dfrac{1}{6}\right)^{12} = 7\,484\,400 \times \dfrac{1}{6^{12}} = 0.00344\).

EXAMPLE 2

Items are Good, Minor-defect or Major-defect with probabilities \(0.5, 0.3, 0.2\). In a sample of 10, the probability of exactly 5 Good, 3 Minor and 2 Major is

\(P = \dfrac{10!}{5!\,3!\,2!}(0.5)^5(0.3)^3(0.2)^2 = 2520 \times 0.03125 \times 0.027 \times 0.04 = 0.0851\).

Multinomial as a distribution over k categories (Example 2) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 probability p_i p = 0.5 E(X_i)=np_i=5 Good p = 0.3 E(X_i)=np_i=3 Minor defect p = 0.2 E(X_i)=np_i=2 Major defect n = 10 trials — probabilities sum to 1; expected counts sum to n. The outcome (5, 3, 2) equals the expected counts here.
Fig 4.1 — A multinomial is fixed by its category probabilities \(p_i\) (bars). Over \(n=10\) trials each category has expected count \(E(X_i)=np_i\), and the negative covariance means a high count in one category pushes the others down. The specific outcome (5, 3, 2) in Example 2 happens to match these expected counts.

Key Take-aways from Unit 1

← Course Home Next: Unit 2 — Poisson →