When data are recorded on two variables simultaneously — say \(X\) (independent) and \(Y\) (dependent) — they are called bivariate data: \((x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)\).
Curve fitting is the process of finding a mathematical curve that best describes the relationship between \(X\) and \(Y\).
Once a curve is fitted, it can be used to estimate future values (interpolation / extrapolation) and to understand the trend in the data.
Among all possible curves \(Y = f(X)\) of a given form, the best-fit curve is the one for which the sum of squares of the residuals (vertical distances between observed and predicted values) is a minimum:
\[ S \;=\; \sum_{i=1}^{n}\bigl(y_i - f(x_i)\bigr)^2 \;=\; \min. \]Setting \(\partial S/\partial(\text{parameter}) = 0\) for each parameter gives a system of normal equations whose solution provides the optimal parameter values.
For each observation the fitted curve gives an estimated value \(\hat y_i = f(x_i)\). The difference \(e_i = y_i - \hat y_i\) is the residual. Positive and negative residuals would cancel in a plain sum, so the principle squares them first. The method was published by A.-M. Legendre in 1805; C. F. Gauss, who had used it earlier, gave its probability justification.
For the straight line, \(E(a, b) = \sum (y_i - a - bx_i)^2\). Setting the first partial derivatives to zero finds a stationary point; the second derivatives show it is a minimum:
\[ \frac{\partial^2 E}{\partial a^2} = 2n, \qquad \frac{\partial^2 E}{\partial b^2} = 2\sum x_i^2, \qquad \frac{\partial^2 E}{\partial a\,\partial b} = 2\sum x_i . \]The test for a minimum needs \(\frac{\partial^2 E}{\partial a^2} > 0\) and
\[ \frac{\partial^2 E}{\partial a^2}\cdot\frac{\partial^2 E}{\partial b^2} - \Big(\frac{\partial^2 E}{\partial a\,\partial b}\Big)^2 = 4\Big[n\sum x_i^2 - \big(\textstyle\sum x_i\big)^2\Big] = 4n\sum (x_i - \bar x)^2 > 0, \]which holds whenever the \(x_i\) are not all equal. Since \(E\) is a sum of squares it has no maximum, and the single stationary point is the least value. The same holds for every curve whose parameters enter linearly.
Suppose we wish to fit \(Y = a_0 + a_1 X + a_2 X^2 + \cdots + a_k X^k\) to bivariate data. The least-squares normal equations are:
This is a system of \(k + 1\) linear equations in the \(k + 1\) unknowns \(a_0, a_1, \ldots, a_k\).
Form: \(Y = a + bX\).
Solving for \(a\) and \(b\):
\[ b \;=\; \dfrac{n \sum xy - \sum x \sum y}{n \sum x^2 - (\sum x)^2}, \qquad a \;=\; \bar y - b \bar x. \]The first normal equation also says \(\sum (y_i - \hat y_i) = 0\): the residuals of a least-squares line add to zero. That gives a quick arithmetic check on any fit.
Fit a straight line to the data:
| x | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|
| y | 3 | 7 | 13 | 21 | 31 |
\(\sum x = 15,\; \sum y = 75,\; \sum xy = 295,\; \sum x^2 = 55,\; n = 5\).
\(b = (5 \cdot 295 - 15 \cdot 75)/(5 \cdot 55 - 225) = (1475 - 1125)/(275 - 225) = 350/50 = 7\).
\(a = \bar y - b\bar x = 15 - 7 \cdot 3 = -6\). Fitted line: \(Y = -6 + 7X\).
Sales (₹ '000) of a company in 5 years are 12, 15, 18, 22, 25. Fit a linear trend (use \(x = 1, 2, 3, 4, 5\)).
\(\sum x = 15, \sum y = 92, \sum xy = 309, \sum x^2 = 55\).
\(b = (5 \cdot 309 - 15 \cdot 92)/50 = (1545 - 1380)/50 = 165/50 = 3.3\).
\(a = 18.4 - 3.3(3) = 8.5\). Trend line: \(\hat y = 8.5 + 3.3 x\). Predicted Year-6 sales \(= 8.5 + 3.3(6) = 28.3\), i.e. ₹28 300.
Form: \(Y = a + bX + cX^2\).
With \(E = \sum [y_i - (a + bx_i + cx_i^2)]^2\), each partial derivative brings down the factor that multiplies its parameter: \(-1\) for \(a\), \(-x_i\) for \(b\), and \(-x_i^2\) for \(c\):
\[ \frac{\partial E}{\partial a} = -2\sum [y_i - (a + bx_i + cx_i^2)] = 0, \] \[ \frac{\partial E}{\partial b} = -2\sum x_i[y_i - (a + bx_i + cx_i^2)] = 0, \] \[ \frac{\partial E}{\partial c} = -2\sum x_i^2[y_i - (a + bx_i + cx_i^2)] = 0 . \]Multiplying out and moving the unknowns to the right gives the three normal equations above. The pattern is the general rule: multiply the equation of the curve by each term that carries a parameter (\(1, x, x^2\)) and add over the data.
Fit parabola to data:
| x | 0 | 1 | 2 | 3 | 4 |
|---|---|---|---|---|---|
| y | 1 | 1.8 | 1.3 | 2.5 | 6.3 |
\(\sum x = 10\), \(\sum y = 12.9\), \(\sum x^2 = 30\), \(\sum x^3 = 100\), \(\sum x^4 = 354\), \(\sum xy = 37.1\), \(\sum x^2 y = 130.3\).
Solve the 3×3 system:
Solution (by elimination): \(a \approx 1.42,\; b \approx -1.07,\; c \approx 0.55\).
Fitted parabola: \(Y \approx 1.42 - 1.07 X + 0.55 X^2\). Worked Problem 4 below carries out the
elimination step by step.
For data with \(x = -2, -1, 0, 1, 2\) (centred origin), normal equations simplify because \(\sum x = \sum x^3 = 0\). The parabola \(Y = a + bx + cx^2\) reduces to:
This trick (centred \(x\)) is widely used in time-series trend fitting.
Take logarithms: \(\log Y = \log a + X \log b\). Let \(Y' = \log Y, A = \log a, B = \log b\). Then \(Y' = A + B X\) — fit a straight line.
Then \(a = \text{antilog}(A),\; b = \text{antilog}(B)\).
Same idea: \(\ln Y = \ln a + bX\). Fit straight line in \((x, \ln y)\).
With common logarithms. \(\log_{10} Y = \log_{10} a + (b\log_{10} e)X\). Put \(A = \log_{10} a\) and \(B = b\log_{10} e\); fit \(\log_{10} y = A + Bx\) as a straight line, then
\[ a = \text{antilog}(A), \qquad b = \frac{B}{\log_{10} e} = \frac{B}{0.4343} . \]Both routes give the same \(a\) and \(b\). Natural logarithms are the shorter route when a calculator is at hand.
For curves fitted through logarithms (sections 6 and 7) the least-squares principle is applied to \(\log y\), not to \(y\). The fit minimises \(\sum (\log y_i - \log \hat y_i)^2\), so it controls relative errors, \(y_i/\hat y_i\), rather than the residuals \(y_i - \hat y_i\). It is the standard textbook method and is easy to compute, but the curve is not the least-squares curve in the original units. Always compare the fitted values with the data.
Fit \(Y = a b^X\) to:
| x | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| y | 2 | 4 | 8 | 16 |
\(\log y\) values: 0.301, 0.602, 0.903, 1.204. \(\sum x = 10,\; \sum \log y = 3.010,\; \sum x \log y = 9.030,\; \sum x^2 = 30\).
\(B = (4 \cdot 9.030 - 10 \cdot 3.010)/(4 \cdot 30 - 100)\) \(= (36.12 - 30.10)/20 = 0.301\); \(A = 3.010/4 - 0.301 \cdot 2.5\) \(= 0.7525 - 0.7525 = 0\).
\(\Rightarrow a = 10^0 = 1,\; b = 10^{0.301} = 2\). Fitted: \(Y = 1 \cdot 2^X = 2^X\). ✓
Population (in lakhs) of a city for 4 years:
| Year (x) | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| Pop (y) | 10 | 11.6 | 13.5 | 15.7 |
Fit \(y = ae^{bx}\). Step 1. \(\ln y\) = 2.3026, 2.4510, 2.6027, 2.7537, so \(\sum \ln y = 10.1099\), \(\sum x\ln y = 26.0273\), with \(\sum x = 10\), \(\sum x^2 = 30\), \(n = 4\).
Step 2. \(b = \dfrac{4(26.0273) - 10(10.1099)}{4(30) - 10^2} = \dfrac{3.0102}{20} = 0.1505\), and \(\ln a = (10.1099 - 10 \times 0.1505)/4 = 2.151\), so \(a = e^{2.151} = 8.60\).
Step 3. \(\hat y = 8.60\,e^{0.1505x}\). Each year multiplies the population by \(e^{0.1505} = 1.162\), a growth of about 16.2% a year; the fitted values 9.99, 11.61, 13.50, 15.69 match the data.
Form: \(Y = a X^b\). Take logs: \(\log Y = \log a + b \log X\). Let \(X' = \log X,\; Y' = \log Y\):
Apply linear least squares to \((X', Y')\).
Fit \(Y = a X^b\) to:
| x | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|
| y | 1 | 4 | 9 | 16 | 25 |
\(\log x\): 0, 0.301, 0.477, 0.602, 0.699; \(\log y\): 0, 0.602, 0.954, 1.204, 1.398.
Sums: \(\sum X' = 2.0792\), \(\sum Y' = 4.1584\), \(\sum X'^2 = 1.1693\), \(\sum X'Y' = 2.3386\), \(n = 5\). Then \(b = \dfrac{5(2.3386) - 2.0792(4.1584)}{5(1.1693) - 2.0792^2} = \dfrac{3.0469}{1.5234} = 2.000\) and \(A = (4.1584 - 2 \times 2.0792)/5 = 0\), so \(a = 10^0 = 1\) and \(Y = X^2\). ✓ (Every \(Y'\) is exactly \(2X'\), so the fit is exact.)
Allometric relation: weight \(W\) and length \(L\) of fish often follow \(W = aL^b\). Logs and least squares yield estimates of \(a\) and \(b\) (typically \(b \approx 3\) for isometric growth).
Eight problems in the textbook's order, grouped by the curve fitted. Each one sets up the normal equations, builds the table of sums, solves the equations step by step and checks the result. Every figure was recomputed; where the textbook prints a different figure, the correct one is used and the difference is noted.
Fit a straight line \(y = a + bx\) to:
| \(x\) | \(y\) | \(x^2\) | \(xy\) |
|---|---|---|---|
| 1 | 1 | 1 | 1 |
| 3 | 2 | 9 | 6 |
| 4 | 4 | 16 | 16 |
| 6 | 4 | 36 | 24 |
| 8 | 5 | 64 | 40 |
| 9 | 7 | 81 | 63 |
| 11 | 8 | 121 | 88 |
| 14 | 9 | 196 | 126 |
| 56 | 40 | 524 | 364 |
Normal equations with \(n = 8\), \(\sum x = 56\):
\[ 40 = 8a + 56b \quad\text{(i)}, \qquad 364 = 56a + 524b \quad\text{(ii)} . \]Eliminate \(a\). Multiply (i) by 7: \(280 = 56a + 392b\). Subtract from (ii): \(84 = 132b\), so \(b = 84/132 = 7/11 = 0.6364\).
Back-substitute. From (i), \(8a = 40 - 56 \times \frac{7}{11} = \frac{48}{11}\), so \(a = 6/11 = 0.5455\).
Result. \(\hat y = 0.5455 + 0.6364x\). Check: \(\bar x = 7\), \(\bar y = 5\), and \(0.5455 + 0.6364 \times 7 = 5.000\): the line passes through \((\bar x, \bar y)\), as it must.
Note. The textbook prints \(a = 0.5452\); the exact value is \(6/11 = 0.5455\).
Fit \(y = a + bx\) to:
| \(x\) | \(y\) | \(x^2\) | \(xy\) |
|---|---|---|---|
| 2 | 10 | 4 | 20 |
| 4 | 14 | 16 | 56 |
| 6 | 19 | 36 | 114 |
| 8 | 25 | 64 | 200 |
| 10 | 31 | 100 | 310 |
| 12 | 36 | 144 | 432 |
| 42 | 135 | 364 | 1132 |
Normal equations (\(n = 6\), \(\sum x = 42\)):
\[ 135 = 6a + 42b \quad\text{(i)}, \qquad 1132 = 42a + 364b \quad\text{(ii)} . \]Multiply (i) by 7: \(945 = 42a + 294b\). Subtract from (ii): \(187 = 70b\), so \(b = 2.6714\). Then \(6a = 135 - 42 \times 2.6714 = 22.8\), so \(a = 3.8\).
Result. \(\hat y = 3.8 + 2.6714x\) (the textbook rounds \(b\) to 2.67). Check: at \(\bar x = 7\), \(3.8 + 18.7 = 22.5 = \bar y\).
Fit \(y = a + bx\) and estimate \(y\) at \(x = 7\):
| \(x\) | \(y\) | \(x^2\) | \(xy\) |
|---|---|---|---|
| 1 | 18 | 1 | 18 |
| 2 | 51 | 4 | 102 |
| 3 | 90 | 9 | 270 |
| 4 | 120 | 16 | 480 |
| 5 | 140 | 25 | 700 |
| 6 | 150 | 36 | 900 |
| 21 | 569 | 91 | 2470 |
Normal equations: \(569 = 6a + 21b\) (i) and \(2470 = 21a + 91b\) (ii).
Multiply (i) by 3.5: \(1991.5 = 21a + 73.5b\). Subtract from (ii): \(478.5 = 17.5b\), so \(b = 27.3429\). Then \(6a = 569 - 21 \times 27.3429 = -5.2\), so \(a = -0.8667\).
Result. \(\hat y = -0.8667 + 27.3429x\), and at \(x = 7\): \(\hat y = -0.8667 + 27.3429 \times 7 = 190.53\).
Note. The textbook rounds \(b\) to 27.34 before multiplying by 7 and gets 190.5133; with \(b\) to four decimals the estimate is 190.53.
A caution. \(x = 7\) lies outside the data, and the data are levelling off (120, 140, 150). The residuals \(-8.5, -2.8, 8.8, 11.5, 4.2, -13.2\) run negative, positive, negative: the points lie below the line at both ends and above it in the middle, the sign of a curve. A parabola fitted to the same data, \(\hat y = -32.7 + 51.218x - 3.411x^2\), has a residual sum of squares of 47.0 against the line's 481.3, and estimates 158.7 at \(x = 7\). Fig 1.2 shows both.
Fit \(y = a + bx + cx^2\) to the data of Example 1 above:
| \(x\) | \(y\) | \(x^2\) | \(x^3\) | \(x^4\) | \(xy\) | \(x^2y\) |
|---|---|---|---|---|---|---|
| 0 | 1 | 0 | 0 | 0 | 0 | 0 |
| 1 | 1.8 | 1 | 1 | 1 | 1.8 | 1.8 |
| 2 | 1.3 | 4 | 8 | 16 | 2.6 | 5.2 |
| 3 | 2.5 | 9 | 27 | 81 | 7.5 | 22.5 |
| 4 | 6.3 | 16 | 64 | 256 | 25.2 | 100.8 |
| 10 | 12.9 | 30 | 100 | 354 | 37.1 | 130.3 |
Result. \(\hat y = 1.42 - 1.07x + 0.55x^2\) (exact, since the data have one decimal place).
Note. In the textbook the last \(x\) of the data table is printed as 6 and its \(x^2\) as 36; the working uses \(x = 4\) and \(x^2 = 16\), which give the totals 30, 100 and 354.
Fit a parabola to the resistance \(y\) of a train at speed \(x\) km/hr, and estimate the resistance at 140 km/hr.
| Speed \(x\) | 20 | 40 | 60 | 80 | 100 | 120 |
|---|---|---|---|---|---|---|
| Resistance \(y\) | 5.5 | 9.1 | 14.9 | 22.8 | 33.3 | 46.0 |
Direct sums. \(\sum x = 420\), \(\sum x^2 = 36\,400\), \(\sum x^3 = 3\,528\,000\), \(\sum x^4 = 364\,000\,000\), \(\sum y = 131.6\), \(\sum xy = 12\,042\), \(\sum x^2y = 1\,211\,720\), giving
\[ 131.6 = 6a + 420b + 36\,400c, \qquad 12\,042 = 420a + 36\,400b + 3\,528\,000c, \] \[ 1\,211\,720 = 36\,400a + 3\,528\,000b + 364\,000\,000c . \]An easier route: code the speeds. With \(u = (x - 70)/10\) the speeds become \(u = -5, -3, -1, 1, 3, 5\), so \(\sum u = \sum u^3 = 0\):
| \(u\) | −5 | −3 | −1 | 1 | 3 | 5 | Total |
|---|---|---|---|---|---|---|---|
| \(uy\) | −27.5 | −27.3 | −14.9 | 22.8 | 99.9 | 230 | 283 |
| \(u^2y\) | 137.5 | 81.9 | 14.9 | 22.8 | 299.7 | 1150 | 1706.8 |
With \(\sum u^2 = 70\) and \(\sum u^4 = 1414\), the normal equations for \(y = A + Bu + Cu^2\) are
\[ 131.6 = 6A + 70C, \qquad 283 = 70B, \qquad 1706.8 = 70A + 1414C . \]Back to \(x\). Substituting \(u = (x - 70)/10\) and collecting powers of \(x\): \(a = A - 7B + 49C = 4.35\), \(b = B/10 - 1.4C = 0.002411\), \(c = C/100 = 0.0028705\). So
\[ \hat y = 4.35 + 0.002411x + 0.0028705x^2 , \]which reproduces the data closely (5.55, 9.04, 14.83, 22.91, 33.30, 45.98).
Estimate at 140 km/hr (\(u = 7\)): \(A + 7B + 49C = 18.5844 + 28.3 + 14.0656 = 60.95\).
Notes. The textbook rounds \(c\) to 0.0028 and gets 59.566. Because \(c\) multiplies \(140^2 = 19\,600\), dropping its third significant figure costs \(0.0000705 \times 19\,600 = 1.38\); the correct estimate is 60.95. Its table also prints the first \(xy\) as 10 (it is \(20 \times 5.5 = 110\); the total 12 042 uses 110), and its first normal equation has \(364\,000c\) for \(36\,400c\). 140 km/hr is beyond the fastest speed observed, so the estimate assumes the parabola continues.
Fit \(y = ax^b\) to:
| \(x\) | \(y\) | \(X = \log x\) | \(Y = \log y\) | \(X^2\) | \(XY\) |
|---|---|---|---|---|---|
| 1 | 1200 | 0 | 3.0792 | 0 | 0 |
| 2 | 900 | 0.3010 | 2.9542 | 0.0906 | 0.8892 |
| 3 | 600 | 0.4771 | 2.7782 | 0.2276 | 1.3255 |
| 4 | 200 | 0.6021 | 2.3010 | 0.3625 | 1.3854 |
| 5 | 110 | 0.6990 | 2.0414 | 0.4886 | 1.4269 |
| 6 | 50 | 0.7782 | 1.6990 | 0.6056 | 1.3222 |
| Total | 2.8574 | 14.8530 | 1.7749 | 6.3492 |
Normal equations for \(Y = A + bX\), \(A = \log a\):
\[ 14.8530 = 6A + 2.8574b, \qquad 6.3492 = 2.8574A + 1.7749b . \]Solve. \(b = \dfrac{6(6.3492) - 2.8574(14.8530)}{6(1.7749) - 2.8574^2} = \dfrac{38.0952 - 42.4410}{10.6494 - 8.1647} = \dfrac{-4.3458}{2.4847} = -1.7490\), and \(A = (14.8530 + 1.7490 \times 2.8574)/6 = 3.3084\).
Result. \(a = \text{antilog}(3.3084) \approx 2034\), so \(\hat y = 2034\,x^{-1.749}\).
Note. The textbook's table has \(X^2 = 0.0910\) for \(x = 2\); \(0.3010^2 = 0.0906\). With its total 1.7753 it gets \(A = 3.3076\), \(b = -1.7473\) and \(a = 2031\); the corrected sums give \(2034\,x^{-1.749}\).
How good is the fit? The fitted values are 2034, 605, 298, 180, 122 and 89 against 1200, 900, 600, 200, 110 and 50: too high at both ends and too low in the middle (Fig 1.3). The calculation is right, but a power law does not describe these data well. Plotting \(\log y\) against \(\log x\) first, to see whether the points lie near a line, would have shown that.
Fit \(y = ab^x\) to:
| \(x\) | \(y\) | \(Y = \log y\) | \(x^2\) | \(xY\) |
|---|---|---|---|---|
| 1 | 1.0 | 0 | 1 | 0 |
| 2 | 1.2 | 0.0792 | 4 | 0.1584 |
| 3 | 1.8 | 0.2553 | 9 | 0.7659 |
| 4 | 2.5 | 0.3979 | 16 | 1.5916 |
| 5 | 3.6 | 0.5563 | 25 | 2.7815 |
| 6 | 4.7 | 0.6721 | 36 | 4.0326 |
| 7 | 6.6 | 0.8195 | 49 | 5.7365 |
| 8 | 9.1 | 0.9590 | 64 | 7.6720 |
| 36 | 3.7393 | 204 | 22.7385 |
Normal equations for \(Y = A + Bx\), \(A = \log a\), \(B = \log b\): \(3.7393 = 8A + 36B\) and \(22.7385 = 36A + 204B\).
Solve. \(B = \dfrac{8(22.7385) - 36(3.7393)}{8(204) - 36^2} = \dfrac{47.2932}{336} = 0.14075\), and \(A = (3.7393 - 36 \times 0.14075)/8 = -0.1660\).
Result. \(a = \text{antilog}(-0.1660) = 0.682\) and \(b = \text{antilog}(0.14075) = 1.383\), so \(\hat y = 0.682\,(1.383)^x\): \(y\) grows by about 38% for each unit of \(x\). The fitted values (0.94, 1.30, 1.80, 2.49, 3.45, 4.77, 6.60, 9.12) follow the data closely.
Note. The textbook prints \(B\) as “01408” (it is 0.1408), and gets \(A = -0.1662\) by rounding \(B\) to four decimals before substituting; to four decimals \(A = -0.1660\). Its \(a\) and \(b\) are right.
Fit \(y = ae^{bx}\) to:
| \(x\) | \(y\) | \(Y = \log y\) | \(x^2\) | \(xY\) |
|---|---|---|---|---|
| 1 | 1.6 | 0.2041 | 1 | 0.2041 |
| 2 | 4.5 | 0.6532 | 4 | 1.3064 |
| 3 | 13.8 | 1.1399 | 9 | 3.4197 |
| 4 | 40.2 | 1.6042 | 16 | 6.4168 |
| 5 | 125 | 2.0969 | 25 | 10.4845 |
| 6 | 300 | 2.4771 | 36 | 14.8626 |
| 21 | 8.1754 | 91 | 36.6941 |
Normal equations for \(Y = A + Bx\), with \(A = \log a\) and \(B = b\log_{10} e\):
\[ 8.1754 = 6A + 21B, \qquad 36.6941 = 21A + 91B . \]Solve. \(B = \dfrac{6(36.6941) - 21(8.1754)}{6(91) - 21^2} = \dfrac{220.1646 - 171.6834}{105} = \dfrac{48.4812}{105} = 0.46173\), and \(A = (8.1754 - 21 \times 0.46173)/6 = -0.2535\).
Back-transform. \(a = \text{antilog}(-0.2535) = 0.558\) and \(b = B/\log_{10} e = 0.46173/0.43429 = 1.063\). So \(\hat y = 0.558\,e^{1.063x}\): the fitted values are 1.6, 4.7, 13.5, 39.2, 113.5 and 328.8. Natural logarithms give the same \(a\) and \(b\) directly.
Note. The textbook's second normal equation is printed as \(36.6941 = 21 + 91B\); the 21 multiplies \(A\).