Chapter 8 Numerical Solutions of Nonlinear Systems of Equations
The method of Steepest Descent determines a local minimum for a multivariable function of the form . The connection between the minimization of a function from to and the solution of a system of nonlinear equations is due to the fact that a system of the form
has a solution at precisely when the function defined by
has the minimal value zero.
The method of Steepest Descent for finding a local minimum for an arbitrary function from into can be intuitively described as follows:
i. Evaluate at an initial approximation ;
ii. Determine a direction from that results in a decrease in the value of ;
iii. Move an appropriate distance in this direction and call the new vector ;
iv. Repeat steps i through iii with by .
An appropriate choice for is
, for some constant .
To determine an appropriate choice of , we consider the single-variable function
The value of that minimize is what we need.
R. L. Burden and J. D. Faires, Numerical Analysis, PWS,