Minimizing sequence for an operator

A sequence of elements $ \{ z _ {n} \} $, $ z _ {n} \in Z $, minimizing a continuous functional $ I [ z ] $, $ z \in Z $:

$$ I [ z _ {n} ] \rightarrow \inf _ {z \in Z } I[ z ] ,\ n \rightarrow \infty . $$

Minimization problems for functionals are usually divided into two groups. To the first group belong problems of finding the minimal value of a functional, for which it is irrelevant at which element $ z $ the minimum is attained. In this case the values of the functional associated with any minimizing sequence may be used as an approximate solution. The other group of problems concerns the search for an element $ z ^ {*} $ at which the functional $ I [ z ] $ attains its least value:

$$ \tag{1 } \inf _ {z \in Z } I [ z ] = I [ z ^ {*} ] = I ^ {*} . $$

In this connection, there are minimizing sequences which do not converge to an element $ z ^ {*} $.

Let the minimization problem (1) have a unique solution $ z ^ {*} $ and let $ \{ z _ {n} \} $ be a minimizing sequence, that is, a sequence such that

$$ \tag{2 } \lim\limits _ {n \rightarrow \infty } I [ z _ {n} ] = I ^ {*} . $$

The minimization problem (1) is called stable if every minimizing sequence (2) converges to the element $ z ^ {*} \in Z $.

In the solution of stable problems a minimizing sequence is obtained by constructing a sequence of iterates so that relative to $ z _ {n} $( the $ n $- th iterate) a "direction" $ y _ {n} $ is found, and an element

$$ z _ {n+} 1 = z _ {n} - f ( \rho _ {n} ) y _ {n} $$

is chosen from the set of elements $ z _ {n} - f ( \rho ) y _ {n} $ minimizing the function $ I [ z _ {n} - f ( \rho ) y _ {n} ] $ of the variable $ \rho $.

Methods for constructing minimizing sequences for stable problems (1) are divided into three families. In the first, derivatives are not used; these are the direct methods. The second family uses the first derivatives of the functional; these methods are usually called descent methods. The third group of methods consists of algorithms which use second derivatives of the functional.

In non-stable minimization problems for functionals, regularization methods are applied to construct a sequence $ \{ z _ {n} \} $ converging to the element $ z ^ {*} $.

References

[1]	A.N. Tikhonov, V.I. [V.I. Arsenin] Arsenine, "Solution of ill-posed problems" , Winston (1977) (Translated from Russian)
[2]	J. Céa, "Optimisation. Théorie et algorithmes" , Dunod (1971)