On the worst case performance of the steepest descent algorithm for quadratic functions.

The existing choices for the step lengths used by the classical steepest descent algorithm for minimizing a convex quadratic function require in the worst case $$ \mathcal{{O}}(C\log (1/\varepsilon )) $$ iterations to achieve a precision $$ \varepsilon $$ , where C is the Hessian condition number. We show how to construct a sequence of step lengths with which the algorithm stops in $$ \mathcal{{O}}(\sqrt{C}\log (1/\varepsilon )) $$ iterations, with a bound almost exactly equal to that of the Conjugate Gradient method. [ABSTRACT FROM AUTHOR]