Talk:Ill-posed problems

This is being TeXed by --Jjg 12:29, 22 April 2012 (CEST)

incorrectly-posed problems, improperly-posed problems

Problems for which at least one of the conditions below, which characterize well-posed problems, is violated. The problem of determining a solution in a metric space (with metric ) from "initial data" in a metric space (with metric ) is said to be well-posed on the pair of spaces if: a) for every there exists a solution ; b) the solution is uniquely determined; and c) the problem is stable on the spaces , i.e.: For every there is a such that for any it follows from that , where and .

The concept of a well-posed problem is due to J. Hadamard (1923), who took the point of view that every mathematical problem corresponding to some physical or technological problem must be well-posed. In fact, what physical interpretation can a solution have if an arbitrary small change in the data can lead to large changes in the solution? Moreover, it would be difficult to apply approximation methods to such problems. This put the expediency of studying ill-posed problems in doubt.

However, this point of view, which is natural when applied to certain time-depended phenomena, cannot be extended to all problems. The following problems are unstable in the metric of , and therefore ill-posed: the solution of integral equations of the first kind; differentiation of functions known only approximately; numerical summation of Fourier series when their coefficients are known approximately in the metric of ; the Cauchy problem for the Laplace equation; the problem of analytic continuation of functions; and the inverse problem in gravimetry. Other ill-posed problems are the solution of systems of linear algebraic equations when the system is ill-conditioned; the minimization of functionals having non-convergent minimizing sequences; various problems in linear programming and optimal control; design of optimal systems and optimization of constructions (synthesis problems for antennas and other physical systems); and various other control problems described by differential equations (in particular, differential games). Various physical and technological questions lead to the problems listed (see [TiArAr]).

A broad class of so-called inverse problems that arise in physics, technology and other branches of science, in particular, problems of data processing of physical experiments, belongs to the class of ill-posed problems. Let be a characteristic quantity of the phenomenon (or object) to be studied. In a physical experiment the quantity is frequently inaccessible to direct measurement, but what is measured is a certain transform (also called outcome). For the interpretation of the results it is necessary to determine from , that is, to solve the equation

(1)

Problems of solving an equation (1) are often called pattern recognition problems. Problems leading to the minimization of functionals (design of antennas and other systems or constructions, problems of optimal control and many others) are also called synthesis problems.

Suppose that in a mathematical model for some physical experiments the object to be studied (the phenomenon) is characterized by an element (a function, a vector) belonging to a set of possible solutions in a metric space . Suppose that is inaccessible to direct measurement and that what is measured is a transform, , , where is the image of under the operator . Evidently, , where is the operator inverse to . Since is obtained by measurement, it is known only approximately. Let be this approximate value. Under these conditions the question can only be that of finding a "solution" of the equation

(2)

approximating .

In many cases the operator is such that its inverse is not continuous, for example, when is a completely-continuous operator in a Hilbert space, in particular an integral operator of the form

Under these conditions one cannot take, following classical ideas, an exact solution of (2), that is, the element , as an approximate "solution" to . In fact: a) such a solution need not exist on , since need not belong to ; and b) such a solution, if it exists, need not be stable under small changes of (due to the fact that is not continuous) and, consequently, need not have a physical interpretation. The problem (2) then is ill-posed.

Numerical methods for solving ill-posed problems.

For ill-posed problems of the form (1) the question arises: What is meant by an approximate solution? Clearly, it should be so defined that it is stable under small changes of the original information. A second question is: What algorithms are there for the construction of such solutions? Answers to these basic questions were given by A.N. Tikhonov (see [Ti], [Ti2]).

The selection method. In some cases an approximate solution of (1) can be found by the selection method. It consists of the following: From the class of possible solutions one selects an element for which approximates the right-hand side of (1) with required accuracy. For the desired approximate solution one takes the element . The question arises: When is this method applicable, that is, when does

imply that

where as ? This holds under the conditions that the solution of (1) is unique and that is compact (see [Ti3]). On the basis of these arguments one has formulated the concept (or the condition) of being Tikhonov well-posed, also called conditionally well-posed (see [LaLa]). As applied to (1), a problem is said to be conditionally well-posed if it is known that for the exact value of the right-hand side there exists a unique solution of (1) belonging to a given compact set . In this case is continuous on , and if instead of an element is known such that and , then as an approximate solution of (1) with right-hand side one can take . As , tends to .

In many cases the approximately known right-hand side does not belong to . Under these conditions equation (1) does not have a classical solution. As an approximate solution one takes then a generalized solution, a so-called quasi-solution (see [Iv]). A quasi-solution of (1) on is an element that minimizes for a given the functional on (see [Iv2]). If is compact, then a quasi-solution exist for any , and if in addition , then a quasi-solution coincides with the classical (exact) solution of (1). The existence of quasi-solutions is guaranteed only when the set of possible solutions is compact.

The regularization method. For a number of applied problems leading to (1) a typical situation is that the set of possible solutions is not compact, the operator is not continuous on , and changes of the right-hand side of (1) connected with the approximate character can cause the solution to go out of . Such problems are called essentially ill-posed. An approach has been worked out to solve ill-posed problems that makes it possible to construct numerical methods that approximate solutions of essentially ill-posed problems of the form (1) which are stable under small changes of the data. In this context, both the right-hand side and the operator should be among the data.

In what follows, for simplicity of exposition it is assumed that the operator is known exactly. At the basis of the approach lies the concept of a regularizing operator (see [Ti2], [TiArAr]). An operator from to is said to be a regularizing operator for the equation (in a neighbourhood of ) if it has the following properties: 1) there exists a such that the operator is defined for every , , and for any such that ; and 2) for every there exists a such that implies , where .

Sometimes it is convenient to use another definition of a regularizing operator, comprising the previous one. An operator from to , depending on a parameter , is said to be a regularizing operator (or regularization operator) for the equation (in a neighbourhood of ) if it has the following properties: 1) there exists a such that is defined for every and any for which ; and 2) there exists a function of such that for any there is a such that if and , then , where . In this definition it is not assumed that the operator is globally single-valued.

If , then as an approximate solution of (1) with an approximately known right-hand side one can take the element obtained by means of the regularizing operator , where is compatible with the error of the initial data (see [Ti], [Ti2], [TiArAr]). This is said to be a regularized solution of (1). The numerical parameter is called the regularization parameter. As , the regularized approximate solution tends (in the metric of ) to the exact solution .

Thus, the task of finding approximate solutions of (1) that are stable under small changes of the right-hand side reduces to: a) finding a regularizing operator; and b) determining the regularization parameter from additional information on the problem, for example, the size of the error with which the right-hand side is given.

The construction of regularizing operators. It is assumed that the equation has a unique solution . Suppose that instead of the equation is solved and that . Since , the approximate solution of is looked for in the class of elements such that . This is the set of possible solutions. As an approximate solution one cannot take an arbitrary element from , since such a "solution" is not unique and is, generally speaking, not continuous in . As a selection principle for the possible solutions ensuring that one obtains an element (or elements) from depending continuously on a nd tending to as , one uses the so-called variational principle (see [Ti]). Let be a continuous non-negative functional defined on a subset of that is everywhere-dense in and is such that: a) ; and b) for every the set of elements in for which , is compact in . Functionals having these properties are said to be stabilizing functionals for problem (1). Let be a stabilizing functional defined on a subset of . ( can be the whole of .) Among the elements of one looks for one (or several) that minimize(s) on . The existence of such an element can be proved (see [TiArAr]). It can be regarded as the result of applying a certain operator to the right-hand side of the equation , that is, . Then is a regularizing operator for equation (1). In practice the search for can be carried out in the following manner: under mild additional restrictions on (quasi-monotonicity of , see [TiArAr]) it can be proved that is attained on elements for which . An element is a solution to the problem of minimizing given , that is, a solution of a problem of conditional extrema, which can be solved using Lagrange's multiplier method and minimization of the functional

For any one can prove that there is an element minimizing . The parameter is determined from the condition . If there is an for which , then the original variational problem is equivalent to that of minimizing , which can be solved by various methods on a computer (for example, by solving the corresponding Euler equation for ). The element minimizing can be regarded as the result of applying to the right-hand side of the equation a certain operator depending on , that is, in which is determined by the discrepancy relation . Then is a regularizing operator for (1). Equivalence of the original variational problem with that of finding the minimum of holds, for example, for linear operators . For non-linear operators this need not be the case (see [GoLeYa]).

The so-called smoothing functional can be introduced formally, without connecting it with a conditional extremum problem for the functional , and for an element minimizing it sought on the set . This poses the problem of finding the regularization parameter as a function of , , such that the operator determining the element is regularizing for (1). Under certain conditions (for example, when it is known that and is a linear operator) such a function exists and can be found from the relation . There are also other methods for finding .

Let be a class of non-negative non-decreasing continuous functions on , a solution of (1) with right-hand side , and a continuous operator from to . For any positive number and functions and from such that and , there exists a such that for and it follows from that , where for all for which .

Methods for finding the regularization parameter depend on the additional information available on the problem. If the error of the right-hand side of the equation for is known, say , then in accordance with the preceding it is natural to determine by the discrepancy, that is, from the relation .

The function is monotone and semi-continuous for every . If is a linear operator, a Hilbert space and a strictly-convex functional (for example, quadratic), then the element is unique and is a single-valued function. Under these conditions, for every positive number , where , there is an such that (see [TiArAr]).

However, for a non-linear operator the equation may have no solution (see [GoLeYa]).

The regularization method is closely connected with the construction of splines (cf. Spline). For example, the problem of finding a function with piecewise-continuous second-order derivative on that minimizes the functional and takes given values on a grid , is equivalent to the construction of a spline of the second degree.

A regularizing operator can be constructed by spectral methods (see [TiArAr], [GoLeYa]), by means of the classical integral transforms in the case of equations of convolution type (see [Ar], [TiArAr]), by the method of quasi-mappings (see [LaLi]), or by the iteration method (see [Kr]). Necessary and sufficient conditions for the existence of a regularizing operator are known (see [Vi]).

Next, suppose that not only the right-hand side of (1) but also the operator is given approximately, so that instead of the exact initial data one has , where

Under these conditions the procedure for obtaining an approximate solution is the same, only instead of one has to consider the functional

and the parameter can be determined, for example, from the relation (see [TiArAr])

If (1) has an infinite set of solutions, one introduces the concept of a normal solution. Suppose that is a normed space. Then one can take, for example, a solution for which the deviation in norm from a given element is minimal, that is,

An approximation to a normal solution that is stable under small changes in the right-hand side of (1) can be found by the regularization method described above. The class of problems with infinitely many solutions includes degenerate systems of linear algebraic equations. So-called badly-conditioned systems of linear algebraic equations can be regarded as systems obtained from degenerate ones when the operator is replaced by its approximation . As a normal solution of a corresponding degenerate system one can take a solution of minimal norm . In the smoothing functional one can take for the functional . Approximate solutions of badly-conditioned systems can also be found by the regularization method with (see [TiArAr]).

Similar methods can be used to solve a Fredholm integral equation of the second kind in the spectrum, that is, when the parameter of the equation is equal to one of the eigen values of the kernel.

Instability problems in the minimization of functionals.

A number of problems important in practice leads to the minimization of functionals . One distinguishes two types of such problems. In the first class one has to find a minimal (or maximal) value of the functional. Many problems in the design of optimal systems or constructions fall in this class. For such problems it is irrelevant on what elements the required minimum is attained. Therefore, as approximate solutions of such problems one can take the values of the functional on any minimizing sequence .

In the second type of problems one has to find elements on which the minimum of is attained. They are called problems of minimizing over the argument. E.g., the minimizing sequences may be divergent. In these problems one cannot take as approximate solutions the elements of minimizing sequences. Such problems are called unstable or ill-posed. These include, for example, problems of optimal control, in which the function to be optimized (the object function) depends only on the phase variables.

Suppose that is a continuous functional on a metric space and that there is an element minimizing . A minimizing sequence of is called regularizing if there is a compact set in containing . If the minimization problem for has a unique solution , then a regularizing minimizing sequence converges to , and under these conditions it is sufficient to exhibit algorithms for the construction of regularizing minimizing sequences. This can be done by using stabilizing functionals .

Let be a stabilizing functional defined on a set , let and let . Frequently, instead of one takes its -approximation relative to , that is, a functional such that for every ,

Then for any the problem of minimizing the functional

over the argument is stable.

Let and be null-sequences such that for every , and let be a sequence of elements minimizing . This is a regularizing minimizing sequence for the functional (see [TiArAr]), consequently, it converges as to an element . As approximate solutions of the problems one can then take the elements .

Similarly approximate solutions of ill-posed problems in optimal control can be constructed.

In applications ill-posed problems often occur where the initial data contain random errors. For the construction of approximate solutions to such classes both deterministic and probability approaches are possible (see [TiArAr], [LaVa]).

References

[Ar]	V.Ya. Arsenin, "On a method for obtaining approximate solutions to convolution integral equations of the first kind" Proc. Steklov Inst. Math., 133 (1977) pp. 31–48 Trudy Mat. Inst. Steklov., 133 (1973) pp. 33–51
[Ba]	A.B. Bakushinskii, "A general method for constructing regularizing algorithms for a linear ill-posed equation in Hilbert space" USSR Comp. Math. Math. Phys., 7 : 3 (1968) pp. 279–287 Zh. Vychisl. Mat. i Mat. Fiz., 7 : 3 (1967) pp. 672–677
[GoLeYa]	A.V. Goncharskii, A.S. Leonov, A.G. Yagoda, "On the residual principle for solving nonlinear ill-posed problems" Soviet Math. Dokl., 15 (1974) pp. 166–168 Dokl. Akad. Nauk SSSR, 214 : 3 (1974) pp. 499–500
[Iv]	V.K. Ivanov, "On ill-posed problems" Mat. Sb., 61 : 2 (1963) pp. 211–223 (In Russian)
[Iv2]	V.K. Ivanov, "On linear problems which are not well-posed" Soviet Math. Dokl., 3 (1962) pp. 981–983 Dokl. Akad. Nauk SSSR, 145 : 2 (1962) pp. 270–272
[Kr]	A.V. Kryanev, "The solution of incorrectly posed problems by methods of successive approximations" Soviet Math. Dokl., 14 (1973) pp. 673–676 Dokl. Akad. Nauk SSSR, 210 : 1 pp. 20–22
[LaLa]	M.M. [M.A. Lavrent'ev] Lavrentiev, "Some improperly posed problems of mathematical physics", Springer (1967) (Translated from Russian)
[LaLi]	R. Lattes, J.L. Lions, "Méthode de quasi-réversibilité et applications", Dunod (1967)
[LaVa]	M.M. Lavrent'ev, V.G. Vasil'ev, "The posing of certain improper problems of mathematical physics" Sib. Math. J., 7 : 3 (1966) pp. 450–463 Sibirsk. Mat. Zh., 7 : 3 (1966) pp. 559–576
[Ti]	A.N. Tikhonov, "Solution of incorrectly formulated problems and the regularization method" Soviet Math. Dokl., 4 (1963) pp. 1035–1038 Dokl. Akad. Nauk SSSR, 151 : 3 (1963) pp. 501–504
[Ti2]	A.N. Tikhonov, "Regularization of incorrectly posed problems" Soviet Math. Dokl., 4 (1963) pp. 1624–1627 Dokl. Akad. Nauk SSSR, 153 : 1 (1963) pp. 49–52
[Ti3]	A.N. Tikhonov, "On stability of inverse problems" Dokl. Akad. Nauk SSSR, 39 : 5 (1943) pp. 176–179 (In Russian)
[Ti4]	A.N. Tikhonov, "On the stability of the functional optimization problem" USSR Comp. Math. Math. Phys., 6 : 4 (1966) pp. 28–33 Zh. Vychisl. Mat. i Mat. Fiz., 6 : 4 (1966) pp. 631–634
[TiArAr]	A.N. Tikhonov, V.I. [V.I. Arsenin] Arsenine, "Solution of ill-posed problems", Winston (1977) (Translated from Russian)
[Vi]	V.A. Vinokurov, "On the regularization of discontinuous mappings" USSR Comp. Math. Math. Phys., 11 : 5 (1971) pp. 1–21 Zh. Vychisl. Mat. i Mat. Fiz., 11 : 5 (1971) pp. 1097–1112

Comments

The idea of conditional well-posedness was also found by B.L. Phillips [Ph]; the expression "Tikhonov well-posed" is not used in the West.

Other problems that lead to ill-posed problems in the sense described above are the Dirichlet problem for the wave equation, the non-characteristic Cauchy problem for the heat equation, the initial boundary value problem for the backward heat equation, inverse scattering problems ([CoKr]), identification of parameters (coefficients) in partial differential equations from over-specified data ([Ba2], [EnGr]), and computerized tomography ([Na2]).

If is a bounded linear operator between Hilbert spaces, then, as also mentioned above, regularization operators can be constructed via spectral theory: If as , then under mild assumptions, is a regularization operator (cf. [Gr]); for choices of the regularization parameter leading to optimal convergence rates for such methods see [EnGf]. For , the resulting method is called Tikhonov regularization: The regularized solution is defined via . A variant of this method in Hilbert scales has been developed in [Na] with parameter choice rules given in [Ne]. The parameter choice rule discussed in the article given by is called the discrepancy principle ([Mo]).

References

[Ba2]	J. Baumeister, "Stable solution of inverse problems", Vieweg (1986)
[Ba3]	H.P. Baltes (ed.), Inverse source problems in optics, Springer (1978)
[Ba4]	H.P. Baltes (ed.), Inverse scattering problems in optics, Springer (1980)
[BaGi]	G. Backus, F. Gilbert, "The resolving power of gross earth data" Geophys. J. R. Astr. Soc., 16 (1968)
[BeBlSt]	J.V. Beck, B. Blackwell, C.R. StClair, "Inverse heat conduction: ill posed problems", Wiley (1985)
[BoJo]	W.M. Boerner, A.K. (eds.) Jordan, "Inverse methods in electromagnetics" IEEE Trans. Antennas Propag., 2 (1981)
[Ca]	J.R. Cann on, "The one-dimensional heat equation", Addison-Wesley (1984)
[CaSt]	A. Carasso, A.P. Stone, "Improperly posed boundary value problems", Pitman (1975)
[Co]	A.M. Cormak, "Representation of a function by its line integrals with some radiological applications" J. Appl. Phys., 34 (1963)
[Co2]	L. Colin, "Mathematics of profile inversion", Proc. Workshop Ames Res. Center, June 12–16, 1971, TM-X-62.150, NASA
[CoKr]	D.L. Colton, R. Kress, "Integral equation methods in scattering theory", Wiley (1983)
[EnGf]	H.W. Engl, H. Gfrerer, "A posteriori parameter choice for general regularization methods for solving linear ill-posed problems" Appl. Num. Math., 4 (1988) pp. 395–417
[EnGr]	H.W. Engl (ed.) C.W. Groetsch (ed.), Inverse and ill-posed problems, Acad. Press (1987)
[Gr]	C.W. Groetsch, "The theory of Tikhonov regularization for Fredholm equations of the first kind", Pitman (1984)
[Gr2]	C.W. Groetsch, "The theory of Tikhonov regularization for Fredholm equations of the first kind", Pitman (1984)
[He]	G.T. Herman (ed.), Image reconstruction from projections, Springer (1979)
[HeNa]	G.T. Herman (ed.) F. Natterer (ed.), Mathematical aspects of computerized tomography, Proc. Oberwolfach, February 10–16, 1980, Springer (1981)
[Jo]	F. John, "Continuous dependence on data for solutions of partial differential equations with a prescribed bound" Comm. Pure Appl. Math., 13 (1960) pp. 551–585
[Ka]	M. Kac, "Can one hear the shape of a drum?" Amer. Math. Monthly, 73 (1966) pp. 1–23
[LaRoSh]	M.H. Lavrent'ev, V.G. Romanov, S.P. Shishalskii, "Ill-posed problems of mathematical physics and analysis", Amer. Math. Soc. (1986) (Translated from Russian)
[Mo]	V.A. Morozov, "Methods for solving incorrectly posed problems", Springer (1984) (Translated from Russian)
[Na]	F. Natterer, "Error bounds for Tikhonov regularization in Hilbert scales" Applic. Analysis, 18 (1984) pp. 29–37
[Na2]	F. Natterer, "The mathematics of computerized tomography", Wiley (1986)
[Ne]	A. Neubauer, "An a-posteriori parameter choice for Tikhonov regularization in Hilbert scales leading to optimal convergence rates" SIAM J. Numer. Anal., 25 (1988) pp. 1313–1326
[Pa]	L.E. Payne, "Improperly posed problems in partial differential equations", SIAM (1975)
[Ph]	B.L. Phillips, "A technique for the numerical solution of certain integral equations of the first kind" J. ACM, 9 (1962) pp. 84–97