Chapter 7 Approximation Theory
Least Square Approximation
The general problem of fitting the best least squares line to a collection of data involves minimizing with respect to the parameters and . For a minimum to occur, it it necessary that
These equations simplify to the normal equations:
The solution to this system of equations is
The general problem of approximating a set of data, , with an algebraic polynomial of degree using the least squares procedure is handled in a similar manner. It requires choosing the constants to minimize the least squares error
As in the linear case, for to be minimized, it is necessary that for each . Thus, for each ,
This gives normal equations in the unknowns, ,
It is helpful to write the equations as follows:
It can be shown that the normal equations have a unique solution provided that the , are distinct.
Occasionally it is appropriate to assume that the data are exponentially related. This requires the approximating function to be of the form
for some constants and . Considering the logarithm of the approximating equations:
, in the case of Eq. (1)
and , in the case of Eq. (2).
In either case, a linear problem now appears and solutions for and can be obtained by appropriately modifying the normal equations (1) and (2).
In fitting a function to data points , a linear combination of any known functions, including polynomials, may be used:
where are prescribed functions, are undetermined coefficients, and is the total number of prescribed functions. By fitting Eq. (3) to each data point, an over-determined equation is written as
, and , where .
R. L. Burden and J. D. Faires, Numerical Analysis, PWS,
S. Nakamura, Numerical
Analysis and Graphic Visualization with MATLAB,