Parameter, method of variation of the

A method for approximately solving non-linear (and linear) functional and operator equations $ y = P( x) $, $ x \in X $, $ y \in Y $, in Banach spaces, and also for qualitatively investigating them. The method of variation of the parameter consists in the following: The equation $ P( x) = 0 $, where the operator $ P( x) $ is continuously Fréchet-differentiable up to the required order (cf. Fréchet derivative), or a certain non-linear functional $ \Phi ( x) $ connected with the solution of this equation, is generalized by introducing an auxiliary numerical (or, in general, functional) parameter $ \lambda $ taking values in a finite or infinite interval $ \lambda _ {0} \leq \lambda \leq \lambda ^ \star $ to $ F( x, \lambda ) = 0 $, where $ F( x, \lambda ), $ $ x \in X $, $ \lambda _ {0} \leq \lambda \leq \lambda ^ \star $, is an operator with values in $ Y $ such that $ P( x) = 0 $ is obtained for $ \lambda = \lambda ^ \star $: $ F( x, \lambda ^ \star ) = P( x) $, and the equation $ F( x, \lambda _ {0} ) = 0 $ is either easy to solve or a solution $ x _ {0} $ of it is known. Here it is assumed that $ F( x, \lambda ) $ is continuously Fréchet-differentiable in $ x $ and $ \lambda $, that is, the partial derivatives $ F _ {x} ( x, \lambda ) $ and $ F _ \lambda ( x, \lambda ) $ exist and are continuous, and that the operator $ \Gamma ( x, \lambda ) = [ F _ {x} ( x, \lambda )] ^ {-} 1 $ from $ Y $ into $ X $ exists and is continuous. To construct a solution $ x( \lambda ) $ of the equation $ F( x, \lambda ) = 0 $ on the whole interval $ \lambda _ {0} \leq \lambda \leq \lambda ^ \star $ one sets up the corresponding differential problem (Cauchy problem) under the assumption that $ x( \lambda ) $ is a continuously-differentiable function with values in $ X $, defined by the equation

$$ \tag{1 } F _ {x} ( x, \lambda ) \frac{dx}{d \lambda } + F _ \lambda ( x,\ \lambda ) = 0,\ \ x( \lambda _ {0} ) = x _ {0} , $$

or

$$ \tag{2 } \frac{dx}{d \lambda } = - \Gamma ( x, \lambda ) F _ \lambda ( x, \lambda ) ,\ \ x( \lambda _ {0} ) = x _ {0} . $$

The interval $ [ \lambda _ {0} , \lambda ^ \star ] $ is partitioned by points $ \lambda _ {0} < \lambda _ {1} < \dots < \lambda _ {n} = \lambda ^ \star $ into finer subintervals of lengths $ h _ {k} = \lambda _ {k} - \lambda _ {k-} 1 $, $ k = 1 \dots n $, and to the Cauchy problem (2) or (1) one applies a method of numerical integration of ordinary differential equations with step $ h _ {k} $( or several such methods). As a result, for constructing a solution $ x( \lambda ) $ of $ F( x, \lambda ) = 0 $ one obtains methods of variation of the parameter of corresponding types. The resulting value $ x( \lambda ^ \star ) $ is a solution of $ P( x) = 0 $.

The solution at every step of linear problems in $ dx/d \lambda $ of the form (1), the inversion of the linear operators $ F _ {x} ( x, \lambda ) $ in (2) or the successive approximation of the inverse operation $ \Gamma ( x, \lambda ) $ proceed by various methods or once again by variation of the parameter.

The steps $ h _ {k} $ are chosen by various means, for example, from the condition of minimizing the norm of the discrepancy $ \| P( x _ {k+} 1 ) \| $ as a function of, generally speaking, several variables. Here a joint choice of $ h _ {k} $ and the free parameters of a method of numerical integration is also effective, for example, the Runge–Kutta method of order of accuracy $ s $, the use of the roots of Chebyshev and related polynomials, etc.

The Cauchy problem (2) is not only a means for determining an approximate solution of the equation in question, but also for proving the existence of the solution itself. A number of distinct ways of introducing the parameter $ \lambda $ have been studied. As the numerical parameter $ \lambda $ one may also use one of the natural parameters inherent in the problem.

Depending on the way of introducing $ \lambda $, the method of variation of the parameter is a direct or an iterative method. A joint application of a direct and an iterative method is said to give a combined method. For example, the iterative method of improved Euler–Cauchy type with step $ h _ {k} = 1 $( for $ F( x, \lambda ) = P( x) - ( 1- \lambda ) P( x _ {0} ) $, $ \lambda _ {0} = 0 $ and $ \lambda ^ \star = 1 $) is of the third order of accuracy and has the following form:

$$ x _ {i+} 1 = x _ {i} - \frac{1}{2} [ \Gamma ( x _ {i} ) + \Gamma ( x _ {i} - \Gamma ( x _ {i} ) P( x _ {i} ))] P( x _ {i} ), $$

$$ i = 0, 1 , . . . ; \ \Gamma ( x) = [ P ^ \prime ( x)] ^ {-} 1 . $$

Every method of numerical integration gives rise to an iterative method of variation of the parameter of a higher order of accuracy, and without the need to bring in the derivatives of $ P( x) $ of an order higher than the first.

The use of methods of numerical integration in direct methods of variation of the parameter in conjunction with a correction of the results after each step by an iterative method of variation of the parameter (a combined method) is one of the most effective methods for solving non-linear equations.

For a broad class of problems the method of variation of the parameter has been worked-out sufficiently well. Originally it was proposed for systems of algebraic and transcendental equations, integral equations, ordinary and partial differential equations, and later for the solution of more general non-linear and operator equations. Conditions have been studied under which the solvability of the equation $ P( x) = 0 $ is guaranteed, as well as the possibility of constructing a solution of it by integrating the Cauchy problem (2) over $ [ \lambda _ {0} , \lambda ^ \star ] $ and establishing the domains of its location. Convergence conditions and error estimates have also been studied, as well as problems of applying a method of variation of the parameter for the inversion and pseudo-inversion of linear operators, for the construction of pseudo-solutions (and solutions) of linear functional equations with minimal deviation in norm (in a given space) from the initial value, for the summation of operator series and the construction of certain classes of projections, for the determination of initial approximations for iteration processes, for the solution of operator differential equations and problems in linear algebra, or for proving the solvability of non-linear systems connected with variational problems and the construction of solutions of them, for the minimization of functionals and in many others. Other research concerns extensive classes of effective modifications of the method of variation of the parameter, among them successive approximation of the inverse operator $ \Gamma ( x, \lambda ) $ or $ \Gamma ( x) $. Also, broad classes of branching problems and non-linear problems on eigen values have been studied.

$$ $$

(However, the case of branching may be blocked by other ways of introducing a parameter $ \lambda $ or of an additional parameter $ \tau $.) The method of variation of the parameter has also been treated as a method of "gradient" type, and also without the assumption that $ \Gamma ( x) $ exists.

See also Continuation method (to a parametrized family); Continuation method (to a parametrized family, for non-linear operators).

References

[1]	D.F. Davidenko, "On a new numerical solution method for systems of nonlinear equations" Dokl. Akad. Nauk SSSR , 88 : 4 (1953) pp. 601–602 (In Russian)
[2]	D.F. Davidenko, "Application of the method of variation of parameters to the theory of nonlinear functional equations" Ukrain Mat. Zh. , 7 (1955) pp. 18–28 (In Russian)
[3]	M.K. Gavurin, "Nonlinear functional equations and continuous analogues of iteration methods" Izv. Vuz. Mat. , 5 (1958) pp. 18–31 (In Russian)
[4]	B.T. Polyak, "Gradient methods for solving equations and inequalities" Zh. Vychisl. Mat. i Mat. Fiz. , 4 : 6 (1964) pp. 995–1005 (In Russian)
[5]	D.F. Davidenko, "An application of the method of variation of parameters to the construction of iterative formulas of increased accuracy for numerical solutions of nonlinear integral equations" Soviet Math. Dokl. , 6 : 3 (1965) pp. 702–706 Dokl. Akad. Nauk SSSR , 162 : 3 (1965) pp. 499–502
[6]	S.G. Mikhlin, "Numerical realization of variational methods" , Moscow (1966) (In Russian)
[7]	H. Kleinmichel, "Stetige Analoga und Iterationsverfahren für Gleichungen in Banachräume" Math. Nachr. , 37 : 5–6 (1968) pp. 313–343
[8]	M.A. Krasnosel'skii, G.M. Vainikko, P.P. Zabreiko, et al., "Approximate solution of operator equations" , Wolters-Noordhoff (1972) (Translated from Russian)
[9]	D.K. Lika, R.A. Shafiev, "Continuous analogues of certain iteration methods for solving equations" Izv. Akad. Nauk MoldavSSR Ser. Fiz-Tekhn. i Mat. Nauk , 2 (1970) pp. 13–18 (In Russian)
[10]	E.P., et al. Zhidkov, Fizika Element. Chast. i Atomn. Yadra , 4 : 1 (1973) pp. 127–166
[11]	D.F. Davidenko, "The iteration method of parameter variation for the inversion of linear operations" Zh. Vychisl. Mat. i Mat. Fiz. , 15 : 1 (1975) pp. 30–47, 274 (In Russian)
[12]	J.M. Ortega, W.C. Rheinboldt, "Iteration methods for the solution of non-linear systems of equations in many unknowns" , Acad. Press (1970)
[13]	H. Keller, , Methods of numerical and applied mathematics , 2 , Novosibirsk (1977) pp. 6–36 (In Russian)
[14]	D.F. Davidenko, , Mathematical programming and related problems. Computational methods , Moscow (1976) pp. 187–212 (In Russian)
[15]	Yu.V. Kolyada, V.P. Sigorskii, Kibernetika : 3 (1980) pp. 24–28

Comments

The general idea of the method of variation of the parameter as described above is to continuously transform (homotope) a (complicated) model problem $ F $ into an easy model $ H $ and then to go back taking a solution of $ H $ along. Hence also the names homotopy method, continuation method and path following method for these and similar methods. The basic idea goes back to H. Poincaré. Bifurcation points may be met, so that this area of numerical analysis has much interrelation with numerical methods for the calculation of bifurcations. For a recent survey of the literature, available software and algorithms in this field cf. [a1].

References

[a1]	K. Ulrich, "State of the art in numerical methods for continuation and bifurcation problems with applications in continuum mechanics. A survey and comparative study" , Preprint Lab. Nac. Comp. Cien. , Rio de Janeiro, Brazil (1990)
[a2]	E.L. Allgower, K. Georg, "Continuation methods for numerically solving systems of equations" , Springer (1987)