Chapter 7  Approximation Theory


Least Square Approximation



   The general problem of fitting the best least squares line to a collection of data  involves minimizing  with respect to the parameters  and . For a minimum to occur, it it necessary that


and .

These equations simplify to the normal equations:

 and .

The solution to this system of equations is






   The general problem of approximating a set of data, , with an algebraic polynomial  of degree  using the least squares procedure is handled in a similar manner. It requires choosing the constants  to minimize the least squares error

As in the linear case, for  to be minimized, it is necessary that  for each . Thus, for each ,


This gives  normal equations in the  unknowns, ,

,         .

It is helpful to write the equations as follows:

It can be shown that the normal equations have a unique solution provided that the ,  are distinct.


   Occasionally it is appropriate to assume that the data are exponentially related. This requires the approximating function to be of the form




for some constants  and . Considering the logarithm of the approximating equations:

,            in the case of Eq. (1)

and ,       in the case of Eq. (2).

In either case, a linear problem now appears and solutions for  and  can be obtained by appropriately modifying the normal equations (1) and (2).


   In fitting a function to data points , a linear combination of any known functions, including polynomials, may be used:


where  are prescribed functions,  are undetermined coefficients, and  is the total number of prescribed functions. By fitting Eq. (3) to each data point, an over-determined equation is written as




, and  , where .




【1】         R. L. Burden and J. D. Faires, Numerical Analysis, PWS, Boston, 1993.

【2】         S. Nakamura, Numerical Analysis and Graphic Visualization with MATLAB, Prentice-Hall, New Jersey, 1996.