Chapter 7  Approximation Theory

Least Square Approximation

The general problem of fitting the best least squares line to a collection of data  involves minimizing  with respect to the parameters  and . For a minimum to occur, it it necessary that

,

and .

These equations simplify to the normal equations:

and .

The solution to this system of equations is

,

and

.

The general problem of approximating a set of data, , with an algebraic polynomial  of degree  using the least squares procedure is handled in a similar manner. It requires choosing the constants  to minimize the least squares error

As in the linear case, for  to be minimized, it is necessary that  for each . Thus, for each ,

.

This gives  normal equations in the  unknowns, ,

,         .

It is helpful to write the equations as follows:

It can be shown that the normal equations have a unique solution provided that the ,  are distinct.

Occasionally it is appropriate to assume that the data are exponentially related. This requires the approximating function to be of the form

(1)

or

(2)

for some constants  and . Considering the logarithm of the approximating equations:

,            in the case of Eq. (1)

and ,       in the case of Eq. (2).

In either case, a linear problem now appears and solutions for  and  can be obtained by appropriately modifying the normal equations (1) and (2).

In fitting a function to data points , a linear combination of any known functions, including polynomials, may be used:

(3)

where  are prescribed functions,  are undetermined coefficients, and  is the total number of prescribed functions. By fitting Eq. (3) to each data point, an over-determined equation is written as

,

with

,

, and  , where .

References:

【1】         R. L. Burden and J. D. Faires, Numerical Analysis, PWS, Boston, 1993.

【2】         S. Nakamura, Numerical Analysis and Graphic Visualization with MATLAB, Prentice-Hall, New Jersey, 1996.