Polynomial Approximation and Interpolation

Suppose the scatterplot of the data $(t_{i}, y_{i})$ looks like this:
Pasted image 20231025055222.png

We can fit a line to the data, but it does not really make sense. A parabola $y = α + β t + γ t^{2}$ seems to be a better model. In general, suppose we want to fit a polynomial of degree $n$ to the data

y (t) = α_{0} + α_{1} t + \dots + α_{n} t^{n}, α_{i} \in R

The $i$ th residual: $e_{i} = y_{i} - y (t_{i}) = y_{i} - (α_{0} + α_{1} t_{i} + \dots + α_{n} t_{i}^{n})$ , $i = 1, \dots, m$ .

[\begin{matrix} e_{1} \\ ⋮ \\ e_{m} \end{matrix}] = [\begin{matrix} y_{1} \\ ⋮ \\ y_{m} \end{matrix}] - [\begin{matrix} 1 & t_{1} & \dots & t_{1}^{n} \\ 1 & t_{2} & \dots & t_{2}^{n} \\ ⋮ & ⋮ & \dots & ⋮ \\ 1 & t_{m} & \dots & t_{m}^{n} \end{matrix}] [\begin{matrix} α_{0} \\ ⋮ \\ α_{n} \end{matrix}]

$A$ is called a Vaudermoude matrix (French mathematician that did not introduce the Vandermonds matrix) $A \in M_{m \times (n + 1)}$ .

Consider a special case: $m = n + 1$ (# measurements = # coefficients) $\Rightarrow$ $A$ is square and, if $A$ is nonsingular, we can find $x$ such that $e = 0$ . In other words, we can solve $A x = y$ exactly, i.e. find a polynomial that fits the data ${(t_{i}, y_{i})}$ exactly. This polynomial is called interpolating polynomial.

Lemma: If $t_{1}, \dots, t_{n + 1}$ are all distinct $(t_{i} \neq t_{j})$ $\Rightarrow$ the $(n + 1) \times (n + 1)$ Vandermond matrix is nonsingular.

Remark: Textbook gives a proof based on an LU decomposition. But the statement is very intuitive if you think about it geometrically.