Skip to the content

Topics Covered

Concurrent Deviation Probable Error Coefficient of Determination Multiple Correlation Partial Correlation Intra-class Correlation Correlation Ratio
On this page
  1. 1. Coefficient of Concurrent Deviation
  2. 2. Probable Error of \(r\)
  3. 3. Coefficient of Determination
  4. 4. Multiple Correlation Coefficient (Three Variables)
  5. 5. Partial Correlation Coefficient
  6. 6. Intra-class Correlation
  7. 7. Correlation Ratio
  8. Key Take-aways

1. Coefficient of Concurrent Deviation

DEFINITION

This is a quick, simple measure of correlation based only on the direction of changes in the two series. For each successive pair, mark the change with "+" (increase), "−" (decrease) or "0".

A pair of corresponding deviations is called concurrent if both have the same sign.

\[ r_c \;=\; \pm \sqrt{ \pm \dfrac{2c - n}{n} }, \]

where \(c\) is the number of pairs of concurrent deviations and \(n\) is the number of pairs of deviations (i.e., one less than number of observations). The sign inside & outside the square root is taken positive if \(2c - n > 0\), negative otherwise.

Properties

EXAMPLE 1

Series of 6 observations of \(X\) and \(Y\). Direction of changes (5 differences each): X gives +,+,−,+,−; Y gives +,−,−,+,−. Concurrent (same sign) pairs: positions 1 (+,+), 3 (−,−), 4 (+,+), 5 (−,−) ⇒ \(c = 4,\; n = 5\).

\(r_c = \sqrt{(2 \cdot 4 - 5)/5} = \sqrt{3/5} = +0.775\).

EXAMPLE 2

If \(c = 1, n = 5\) (mostly opposite directions): \(2c - n = -3 < 0\), so

\(r_c = -\sqrt{|-3|/5} = -\sqrt{0.6} = -0.775\).

2. Probable Error of \(r\)

DEFINITION

The probable error (P.E.) of the correlation coefficient measures its reliability:

\[ \text{P.E.}(r) \;=\; 0.6745 \cdot \dfrac{1 - r^2}{\sqrt{n}}. \]

The constant 0.6745 corresponds to 50 % confidence under normal theory.

Properties

  1. If \(|r| < \text{P.E.}\): correlation is not significant.
  2. If \(|r| > 6 \cdot \text{P.E.}\): correlation is highly significant.
  3. The limits for population \(\rho\) are \(r \pm \text{P.E.}\) (a 50% interval, by the definition of P.E.) and \(r \pm 3\,\text{P.E.}\), within which \(\rho\) is "practically certain" to lie. For a 95% interval use \(r \pm 1.96\,\text{S.E.}\), which is about \(r \pm 2.9\,\text{P.E.}\), since \(\text{P.E.} = 0.6745\,\text{S.E.}\)
  4. P.E. decreases with \(n\) (more data ⇒ tighter estimate) and with \(|r|\) (stronger correlation ⇒ smaller error).
EXAMPLE 1

For \(r = 0.8, n = 25\): \(\text{P.E.} = 0.6745(1 - 0.64)/5 = 0.6745(0.36)/5 = 0.0486\).

Since \(|r| = 0.8 \gg 6(0.0486) = 0.29\), correlation is highly significant.

EXAMPLE 2

For \(r = 0.2, n = 16\): \(\text{P.E.} = 0.6745 (0.96)/4 = 0.162\).

Since \(|r| = 0.2 \approx \text{P.E.}\), correlation is not significant.

3. Coefficient of Determination

DEFINITION

The coefficient of determination is the square of the correlation coefficient, \(r^2\). It represents the proportion of variation in \(Y\) explained by the linear relationship with \(X\).

\[ r^2 \;=\; \dfrac{\text{Explained variation}}{\text{Total variation}}, \qquad 1 - r^2 = \text{Coefficient of non-determination}. \]
EXAMPLE 1

If \(r = 0.9\), then \(r^2 = 0.81\): about 81 % of variation in \(Y\) is explained by \(X\); 19 % is unexplained.

EXAMPLE 2

For \(r = 0.5\), only 25 % of variability is explained — a lot of variation remains.

4. Multiple Correlation Coefficient (Three Variables)

DEFINITION

For three variables \(X_1, X_2, X_3\), the multiple correlation coefficient of \(X_1\) on \(X_2\) and \(X_3\) measures the linear relationship between \(X_1\) and the joint behaviour of \(X_2, X_3\). Notation: \(R_{1.23}\).

\[ R_{1.23}^2 \;=\; \dfrac{r_{12}^2 + r_{13}^2 - 2\, r_{12}\, r_{13}\, r_{23}}{1 - r_{23}^2}. \]

Properties

  1. \(0 \le R_{1.23} \le 1\) (always non-negative).
  2. \(R_{1.23} = 0\) iff \(r_{12} = r_{13} = 0\) (only when both simple correlations vanish).
  3. \(R_{1.23} = 1\) iff \(X_1\) is a perfect linear function of \(X_2, X_3\).
  4. \(R_{1.23} \ge \max(|r_{12}|, |r_{13}|)\) — adding a regressor can only increase the multiple correlation.
EXAMPLE 1

Given \(r_{12} = 0.7,\; r_{13} = 0.6,\; r_{23} = 0.4\):

\(R_{1.23}^2 = (0.49 + 0.36 - 2(0.7)(0.6)(0.4))/(1 - 0.16) = (0.85 - 0.336)/0.84 = 0.514/0.84 = 0.612\).

\(R_{1.23} = 0.782\). About 61.2 % of variation in \(X_1\) is explained jointly by \(X_2, X_3\).

EXAMPLE 2

If \(r_{12} = r_{13} = 0.5\) and \(r_{23} = 0\): the predictors are uncorrelated, so

\(R_{1.23}^2 = (0.25 + 0.25 - 0)/1 = 0.5\). \(R_{1.23} = 0.707\).

5. Partial Correlation Coefficient

DEFINITION

The partial correlation between \(X_1\) and \(X_2\) keeping \(X_3\) constant measures the correlation after removing the influence of \(X_3\). Notation: \(r_{12.3}\).

\[ r_{12.3} \;=\; \dfrac{r_{12} - r_{13}\, r_{23}}{\sqrt{(1 - r_{13}^2)(1 - r_{23}^2)}}. \]

Similarly:

\[ r_{13.2} \;=\; \dfrac{r_{13} - r_{12}\, r_{23}}{\sqrt{(1 - r_{12}^2)(1 - r_{23}^2)}}, \quad r_{23.1} \;=\; \dfrac{r_{23} - r_{12}\, r_{13}}{\sqrt{(1 - r_{12}^2)(1 - r_{13}^2)}}. \]

Properties

  1. \(-1 \le r_{12.3} \le 1\).
  2. If the third variable has no effect (\(r_{13} = r_{23} = 0\)), \(r_{12.3} = r_{12}\).
  3. Partial correlation may differ in sign from simple correlation when there is confounding.
EXAMPLE 1

Continuing previous example: \(r_{12} = 0.7,\; r_{13} = 0.6,\; r_{23} = 0.4\).

\(r_{12.3} = (0.7 - 0.6 \cdot 0.4)/\sqrt{(1 - 0.36)(1 - 0.16)} = (0.7 - 0.24)/\sqrt{0.64 \cdot 0.84} = 0.46/\sqrt{0.5376} = 0.46/0.7332 = 0.628\).

So after removing the effect of \(X_3\), \(X_1\) and \(X_2\) still have a moderate-to-strong positive correlation.

EXAMPLE 2

If \(r_{12} = 0.4,\; r_{13} = 0.8,\; r_{23} = 0.5\): \(r_{12.3} = (0.4 - 0.4)/\sqrt{0.36 \cdot 0.75} = 0\).

Apparent correlation between \(X_1, X_2\) is fully explained by \(X_3\) — a classic spurious correlation.

6. Intra-class Correlation

DEFINITION

Intra-class correlation coefficient (\(r_I\)) measures the correlation among members of the same family / class / group when the order within a class is irrelevant.

Suppose data are grouped into \(k\) classes, each of size \(p\). Let \(\sigma^2\) be the total variance and \(\sigma_b^2\) the variance between class means. Then

\[ r_I \;=\; \dfrac{p\, \sigma_b^2 - \sigma^2}{(p - 1)\, \sigma^2}. \]

Properties

EXAMPLE

For 5 families of 4 children each — total variance 25, between-family variance 16: \(p = 4\).

\(r_I = (4 \cdot 16 - 25)/(3 \cdot 25) = (64 - 25)/75 = 39/75 = 0.52\).

7. Correlation Ratio

DEFINITION

The correlation ratio of \(Y\) on \(X\), denoted \(\eta_{yx}\), measures both linear and non-linear association:

\[ \eta_{yx}^2 \;=\; \dfrac{\text{Variance of group means of }Y}{\text{Total variance of }Y} \;=\; \dfrac{\sigma_b^2}{\sigma_y^2}. \]

Properties

EXAMPLE

Group \(Y\) by values of \(X\). If between-group variance is 80 and total variance of \(Y\) is 100, then \(\eta_{yx}^2 = 0.8\), \(\eta_{yx} = 0.894\).

If for the same data \(r = 0.7\), then \(\eta - r = 0.194\) — a sizeable gap, suggesting the relation is not linear (whether it is significant needs a test, e.g. the \(F\) test of \(\eta^2 - r^2\)).

Key Take-aways

← Previous: Unit 2 Next: Unit 4 — Regression →