Skip to the content

Topics Covered

Definition PMF Mean & Variance MGF / CF / CGF / PGF Additive Property Skewness & Kurtosis Recurrence Normal Limit
On this page
  1. 1. Definition & PMF
  2. 2. MGF, Mean and Variance
  3. 3. Other Generating Functions
  4. 4. Additive Property
  5. 5. Skewness & Kurtosis
  6. 6. Recurrence Relation
  7. 7. Limiting Form: Negative Binomial → Normal
  8. 8. Limiting Form: Negative Binomial → Poisson
  9. Key Take-aways from Unit 3

1. Definition & PMF

DEFINITION (Number of failures before \(r\) successes)

Let \(X\) = number of failures preceding the \(r\)-th success in a sequence of independent Bernoulli\((p)\) trials. \(X\) follows the Negative Binomial distribution NB\((r, p)\) with PMF

\[ P(X = x) \;=\; \binom{x + r - 1}{x} p^r q^x, \quad x = 0, 1, 2, \ldots \]

where \(q = 1 - p\). Some textbooks define \(Y = X + r\) = total number of trials; the two forms differ only by a shift.

Validity: \(\sum_{x=0}^{\infty} \binom{x+r-1}{x} q^x = (1-q)^{-r} = p^{-r}\), hence \(\sum P(X=x) = p^r \cdot p^{-r} = 1\) ✓.

Special case: Geometric distribution

When \(r = 1\), NB\((1, p)\) reduces to the Geometric distribution (Unit 4).

2. MGF, Mean and Variance

MGF \[ M_X(t) \;=\; \sum_{x=0}^{\infty} \binom{x+r-1}{x} (qe^t)^x p^r = \dfrac{p^r}{(1 - q e^t)^r}, \quad q e^t < 1. \]

Differentiating once and setting \(t=0\):

\(M'_X(0) = \dfrac{rq}{p} = E(X)\).

Differentiating again: \(E(X^2) = \dfrac{rq(1+rq)}{p^2}\).

MEAN & VARIANCE \[ E(X) = \dfrac{rq}{p}, \qquad \text{Var}(X) = \dfrac{rq}{p^2}. \]

Note that Var \(=\) Mean / \(p\), so variance > mean (over-dispersed compared to Poisson).

3. Other Generating Functions

\[ \phi_X(t) = \dfrac{p^r}{(1 - q e^{it})^r}, \qquad K_X(t) = r\ln p - r \ln(1 - q e^t), \] \[ P_X(s) = \dfrac{p^r}{(1 - qs)^r}. \]

4. Additive Property

If \(X_1 \sim \text{NB}(r_1, p)\) and \(X_2 \sim \text{NB}(r_2, p)\) are independent (same \(p\)),

\[ X_1 + X_2 \;\sim\; \text{NB}(r_1 + r_2,\; p). \]

Proof. Again we work with the moment generating function.

  1. MGF of each variable. For NB\((r, p)\) the MGF is \(M_X(t) = \dfrac{p^{r}}{(1-qe^t)^{r}}\) (with \(q = 1-p\)). Hence \(M_{X_1}(t) = \dfrac{p^{r_1}}{(1-qe^t)^{r_1}}\) and \(M_{X_2}(t) = \dfrac{p^{r_2}}{(1-qe^t)^{r_2}}\). Note both share the same \(p\).
  2. MGF of the sum. By independence the MGF of the sum is the product: \[ M_{X_1+X_2}(t) = M_{X_1}(t)\,M_{X_2}(t) = \dfrac{p^{r_1}}{(1-qe^t)^{r_1}}\cdot\dfrac{p^{r_2}}{(1-qe^t)^{r_2}}. \]
  3. Add the exponents. Multiplying powers with the same base adds their exponents: \[ M_{X_1+X_2}(t) = \dfrac{p^{\,r_1+r_2}}{(1-qe^t)^{\,r_1+r_2}}. \]
  4. Identify the distribution. This is the MGF of NB\((r_1+r_2,\, p)\); by the uniqueness of the MGF, \(X_1+X_2 \sim \text{NB}(r_1+r_2,\, p)\). (Had the two \(p\)'s differed, step 3 could not combine the denominators, which is why the same \(p\) is required.) \(\blacksquare\)

5. Skewness & Kurtosis

\[ \mu_3 = \dfrac{rq(1 + q)}{p^3}, \qquad \mu_4 = \dfrac{rq(p^2 + 6q + 3rq)}{p^4}. \] \[ \gamma_1 = \dfrac{1 + q}{\sqrt{rq}}, \qquad \beta_2 = 3 + \dfrac{p^2 + 6q}{rq}. \]

Always positively skewed and leptokurtic; both decrease toward 0 as \(r\) grows.

x → number of failures NB(r = 2, p = 0.4) Strongly right-skewed x → number of failures NB(r = 8, p = 0.4) Approaches normal shape
Fig 3.1 — Negative binomial PMF (number of failures before the \(r\)-th success). As \(r\) increases the skew \(\gamma_1 = (1+q)/\sqrt{rq}\) shrinks and the distribution becomes bell-shaped, consistent with the normal limit in Section 7.

6. Recurrence Relation

\[ P(X = x + 1) \;=\; \dfrac{r + x}{x + 1} \cdot q \cdot P(X = x), \qquad P(X = 0) = p^r. \]
EXAMPLE 1

A salesman closes a deal with probability \(p = 0.4\) each call. Find probability that the third sale (\(r=3\)) comes on the 7th call (i.e., 4 failures before).

\(X = 4\). \(P(X=4) = \binom{6}{4}(0.4)^3(0.6)^4 = 15 \times 0.064 \times 0.1296 = 0.1244\).

EXAMPLE 2

A coin (head probability 0.5) is tossed until 5 heads appear. What is the expected number of tails?

\(r = 5,\; p = 0.5,\; q = 0.5\). \(E(X) = rq/p = 5\). Variance = \(rq/p^2 = 10\).

7. Limiting Form: Negative Binomial → Normal

THEOREM

For large \(r\), the standardized variable

\[ Z \;=\; \dfrac{X - rq/p}{\sqrt{rq/p^2}} \;\xrightarrow{d}\; N(0, 1). \]

This follows from the Central Limit Theorem since \(X\) can be expressed as a sum of \(r\) i.i.d. geometric variables.

8. Limiting Form: Negative Binomial → Poisson

THEOREM

Let \(X \sim\) NB\((r, p)\) (number of failures before the \(r\)-th success). As \(r \to \infty\) and \(p \to 1\) (so that \(q = 1 - p \to 0\)) in such a way that \(rq \to \lambda\), a finite constant, the distribution of \(X\) tends to the Poisson distribution with mean \(\lambda\):

\[ P(X = x) \;\longrightarrow\; \dfrac{e^{-\lambda}\lambda^{x}}{x!}, \qquad x = 0, 1, 2, \dots \]

(Since \(p \to 1\), the mean \(rq/p \to \lambda\) as well.)

PROOF (via PGF)

The probability generating function of NB\((r, p)\) is

\[ G_X(s) = \left(\dfrac{p}{1 - qs}\right)^{r} = \left(\dfrac{1 - q}{1 - qs}\right)^{r}. \]

Taking logarithms and expanding for small \(q\),

\[ \ln G_X(s) = r\big[\ln(1 - q) - \ln(1 - qs)\big] = r\big[(-q) - (-qs)\big] + O(rq^{2}) = rq(s - 1) + O(rq^{2}). \]

As \(r \to \infty,\ q \to 0\) with \(rq \to \lambda\) (hence \(rq^{2} \to 0\)),

\[ \ln G_X(s) \to \lambda(s - 1), \qquad\text{so}\qquad G_X(s) \to e^{\lambda(s - 1)}, \]

which is the PGF of Poisson\((\lambda)\). By the uniqueness of generating functions, \(X \xrightarrow{d} \text{Poisson}(\lambda)\). \(\blacksquare\)

NUMERICAL CHECK

For \(r = 200,\ p = 0.985\ (q = 0.015,\ rq = 3)\), the NB probabilities for \(x = 0, 1, 2, 3, 4\) are \(0.049, 0.146, 0.220, 0.222, 0.169\) — matching the Poisson\((3)\) values \(0.050, 0.149, 0.224, 0.224, 0.168\) very closely.

Key Take-aways from Unit 3

← Previous: Unit 2 Next: Unit 4 — Geometric →