Talk:Ill-posed problems
This is being TeXed by --Jjg 12:29, 22 April 2012 (CEST)
incorrectly-posed problems, improperly-posed problems
Problems for which at least one of the conditions below, which characterize well-posed problems, is violated. The problem of determining a solution in a metric space
(with metric
) from "initial data"
in a metric space
(with metric
) is said to be well-posed on the pair of spaces
if: a) for every
there exists a solution
; b) the solution is uniquely determined; and c) the problem is stable on the spaces
, i.e.: For every
there is a
such that for any
it follows from
that
, where
and
.
The concept of a well-posed problem is due to J. Hadamard (1923), who took the point of view that every mathematical problem corresponding to some physical or technological problem must be well-posed. In fact, what physical interpretation can a solution have if an arbitrary small change in the data can lead to large changes in the solution? Moreover, it would be difficult to apply approximation methods to such problems. This put the expediency of studying ill-posed problems in doubt.
However, this point of view, which is natural when applied to certain time-depended phenomena, cannot be extended to all problems. The following problems are unstable in the metric of , and therefore ill-posed: the solution of integral equations of the first kind; differentiation of functions known only approximately; numerical summation of Fourier series when their coefficients are known approximately in the metric of
; the Cauchy problem for the Laplace equation; the problem of analytic continuation of functions; and the inverse problem in gravimetry. Other ill-posed problems are the solution of systems of linear algebraic equations when the system is ill-conditioned; the minimization of functionals having non-convergent minimizing sequences; various problems in linear programming and optimal control; design of optimal systems and optimization of constructions (synthesis problems for antennas and other physical systems); and various other control problems described by differential equations (in particular, differential games). Various physical and technological questions lead to the problems listed (see [TiArAr]).
A broad class of so-called inverse problems that arise in physics, technology and other branches of science, in particular, problems of data processing of physical experiments, belongs to the class of ill-posed problems. Let be a characteristic quantity of the phenomenon (or object) to be studied. In a physical experiment the quantity
is frequently inaccessible to direct measurement, but what is measured is a certain transform
(also called outcome). For the interpretation of the results it is necessary to determine
from
, that is, to solve the equation
![]() | (1) |
Problems of solving an equation (1) are often called pattern recognition problems. Problems leading to the minimization of functionals (design of antennas and other systems or constructions, problems of optimal control and many others) are also called synthesis problems.
Suppose that in a mathematical model for some physical experiments the object to be studied (the phenomenon) is characterized by an element (a function, a vector) belonging to a set
of possible solutions in a metric space
. Suppose that
is inaccessible to direct measurement and that what is measured is a transform,
,
, where
is the image of
under the operator
. Evidently,
, where
is the operator inverse to
. Since
is obtained by measurement, it is known only approximately. Let
be this approximate value. Under these conditions the question can only be that of finding a "solution" of the equation
![]() | (2) |
approximating .
In many cases the operator is such that its inverse
is not continuous, for example, when
is a completely-continuous operator in a Hilbert space, in particular an integral operator of the form
![]() |
Under these conditions one cannot take, following classical ideas, an exact solution of (2), that is, the element , as an approximate "solution" to
. In fact: a) such a solution need not exist on
, since
need not belong to
; and b) such a solution, if it exists, need not be stable under small changes of
(due to the fact that
is not continuous) and, consequently, need not have a physical interpretation. The problem (2) then is ill-posed.
Numerical methods for solving ill-posed problems.
For ill-posed problems of the form (1) the question arises: What is meant by an approximate solution? Clearly, it should be so defined that it is stable under small changes of the original information. A second question is: What algorithms are there for the construction of such solutions? Answers to these basic questions were given by A.N. Tikhonov (see [Ti], [Ti2]).
The selection method. In some cases an approximate solution of (1) can be found by the selection method. It consists of the following: From the class of possible solutions one selects an element
for which
approximates the right-hand side of (1) with required accuracy. For the desired approximate solution one takes the element
. The question arises: When is this method applicable, that is, when does
![]() |
imply that
![]() |
where as
? This holds under the conditions that the solution of (1) is unique and that
is compact (see [Ti3]). On the basis of these arguments one has formulated the concept (or the condition) of being Tikhonov well-posed, also called conditionally well-posed (see [LaLa]). As applied to (1), a problem is said to be conditionally well-posed if it is known that for the exact value of the right-hand side
there exists a unique solution
of (1) belonging to a given compact set
. In this case
is continuous on
, and if instead of
an element
is known such that
and
, then as an approximate solution of (1) with right-hand side
one can take
. As
,
tends to
.
In many cases the approximately known right-hand side does not belong to
. Under these conditions equation (1) does not have a classical solution. As an approximate solution one takes then a generalized solution, a so-called quasi-solution (see [Iv]). A quasi-solution of (1) on
is an element
that minimizes for a given
the functional
on
(see [Iv2]). If
is compact, then a quasi-solution exist for any
, and if in addition
, then a quasi-solution
coincides with the classical (exact) solution of (1). The existence of quasi-solutions is guaranteed only when the set
of possible solutions is compact.
The regularization method. For a number of applied problems leading to (1) a typical situation is that the set of possible solutions is not compact, the operator
is not continuous on
, and changes of the right-hand side of (1) connected with the approximate character can cause the solution to go out of
. Such problems are called essentially ill-posed. An approach has been worked out to solve ill-posed problems that makes it possible to construct numerical methods that approximate solutions of essentially ill-posed problems of the form (1) which are stable under small changes of the data. In this context, both the right-hand side
and the operator
should be among the data.
In what follows, for simplicity of exposition it is assumed that the operator is known exactly. At the basis of the approach lies the concept of a regularizing operator (see [Ti2], [TiArAr]). An operator
from
to
is said to be a regularizing operator for the equation
(in a neighbourhood of
) if it has the following properties: 1) there exists a
such that the operator
is defined for every
,
, and for any
such that
; and 2) for every
there exists a
such that
implies
, where
.
Sometimes it is convenient to use another definition of a regularizing operator, comprising the previous one. An operator from
to
, depending on a parameter
, is said to be a regularizing operator (or regularization operator) for the equation
(in a neighbourhood of
) if it has the following properties: 1) there exists a
such that
is defined for every
and any
for which
; and 2) there exists a function
of
such that for any
there is a
such that if
and
, then
, where
. In this definition it is not assumed that the operator
is globally single-valued.
If , then as an approximate solution of (1) with an approximately known right-hand side
one can take the element
obtained by means of the regularizing operator
, where
is compatible with the error of the initial data
(see [Ti], [Ti2], [TiArAr]). This is said to be a regularized solution of (1). The numerical parameter
is called the regularization parameter. As
, the regularized approximate solution
tends (in the metric of
) to the exact solution
.
Thus, the task of finding approximate solutions of (1) that are stable under small changes of the right-hand side reduces to: a) finding a regularizing operator; and b) determining the regularization parameter from additional information on the problem, for example, the size of the error with which the right-hand side
is given.
The construction of regularizing operators. It is assumed that the equation has a unique solution
. Suppose that instead of
the equation
is solved and that
. Since
, the approximate solution of
is looked for in the class
of elements
such that
. This
is the set of possible solutions. As an approximate solution one cannot take an arbitrary element
from
, since such a "solution" is not unique and is, generally speaking, not continuous in
. As a selection principle for the possible solutions ensuring that one obtains an element (or elements) from
depending continuously on
a
nd tending to
as
, one uses the so-called variational principle (see [Ti]). Let
be a continuous non-negative functional defined on a subset
of
that is everywhere-dense in
and is such that: a)
; and b) for every
the set of elements
in
for which
, is compact in
. Functionals having these properties are said to be stabilizing functionals for problem (1). Let
be a stabilizing functional defined on a subset
of
. (
can be the whole of
.) Among the elements of
one looks for one (or several) that minimize(s)
on
. The existence of such an element
can be proved (see [TiArAr]). It can be regarded as the result of applying a certain operator
to the right-hand side of the equation
, that is,
. Then
is a regularizing operator for equation (1). In practice the search for
can be carried out in the following manner: under mild additional restrictions on
(quasi-monotonicity of
, see [TiArAr]) it can be proved that
is attained on elements
for which
. An element
is a solution to the problem of minimizing
given
, that is, a solution of a problem of conditional extrema, which can be solved using Lagrange's multiplier method and minimization of the functional
![]() |
For any one can prove that there is an element
minimizing
. The parameter
is determined from the condition
. If there is an
for which
, then the original variational problem is equivalent to that of minimizing
, which can be solved by various methods on a computer (for example, by solving the corresponding Euler equation for
). The element
minimizing
can be regarded as the result of applying to the right-hand side of the equation
a certain operator
depending on
, that is,
in which
is determined by the discrepancy relation
. Then
is a regularizing operator for (1). Equivalence of the original variational problem with that of finding the minimum of
holds, for example, for linear operators
. For non-linear operators
this need not be the case (see [GoLeYa]).
The so-called smoothing functional can be introduced formally, without connecting it with a conditional extremum problem for the functional
, and for an element
minimizing it sought on the set
. This poses the problem of finding the regularization parameter
as a function of
,
, such that the operator
determining the element
is regularizing for (1). Under certain conditions (for example, when it is known that
and
is a linear operator) such a function exists and can be found from the relation
. There are also other methods for finding
.
Let be a class of non-negative non-decreasing continuous functions on
,
a solution of (1) with right-hand side
, and
a continuous operator from
to
. For any positive number
and functions
and
from
such that
and
, there exists a
such that for
and
it follows from
that
, where
for all
for which
.
Methods for finding the regularization parameter depend on the additional information available on the problem. If the error of the right-hand side of the equation for is known, say
, then in accordance with the preceding it is natural to determine
by the discrepancy, that is, from the relation
.
The function is monotone and semi-continuous for every
. If
is a linear operator,
a Hilbert space and
a strictly-convex functional (for example, quadratic), then the element
is unique and
is a single-valued function. Under these conditions, for every positive number
, where
, there is an
such that
(see [TiArAr]).
However, for a non-linear operator the equation
may have no solution (see [GoLeYa]).
The regularization method is closely connected with the construction of splines (cf. Spline). For example, the problem of finding a function with piecewise-continuous second-order derivative on
that minimizes the functional
and takes given values
on a grid
, is equivalent to the construction of a spline of the second degree.
A regularizing operator can be constructed by spectral methods (see [TiArAr], [GoLeYa]), by means of the classical integral transforms in the case of equations of convolution type (see [Ar], [TiArAr]), by the method of quasi-mappings (see [LaLi]), or by the iteration method (see [Kr]). Necessary and sufficient conditions for the existence of a regularizing operator are known (see [Vi]).
Next, suppose that not only the right-hand side of (1) but also the operator is given approximately, so that instead of the exact initial data
one has
, where
![]() |
![]() |
Under these conditions the procedure for obtaining an approximate solution is the same, only instead of one has to consider the functional
![]() |
and the parameter can be determined, for example, from the relation (see [TiArAr])
![]() |
If (1) has an infinite set of solutions, one introduces the concept of a normal solution. Suppose that is a normed space. Then one can take, for example, a solution
for which the deviation in norm from a given element
is minimal, that is,
![]() |
An approximation to a normal solution that is stable under small changes in the right-hand side of (1) can be found by the regularization method described above. The class of problems with infinitely many solutions includes degenerate systems of linear algebraic equations. So-called badly-conditioned systems of linear algebraic equations can be regarded as systems obtained from degenerate ones when the operator is replaced by its approximation
. As a normal solution of a corresponding degenerate system one can take a solution
of minimal norm
. In the smoothing functional one can take for
the functional
. Approximate solutions of badly-conditioned systems can also be found by the regularization method with
(see [TiArAr]).
Similar methods can be used to solve a Fredholm integral equation of the second kind in the spectrum, that is, when the parameter of the equation is equal to one of the eigen values of the kernel.
Instability problems in the minimization of functionals.
A number of problems important in practice leads to the minimization of functionals . One distinguishes two types of such problems. In the first class one has to find a minimal (or maximal) value of the functional. Many problems in the design of optimal systems or constructions fall in this class. For such problems it is irrelevant on what elements the required minimum is attained. Therefore, as approximate solutions of such problems one can take the values of the functional
on any minimizing sequence
.
In the second type of problems one has to find elements on which the minimum of
is attained. They are called problems of minimizing over the argument. E.g., the minimizing sequences may be divergent. In these problems one cannot take as approximate solutions the elements of minimizing sequences. Such problems are called unstable or ill-posed. These include, for example, problems of optimal control, in which the function to be optimized (the object function) depends only on the phase variables.
Suppose that is a continuous functional on a metric space
and that there is an element
minimizing
. A minimizing sequence
of
is called regularizing if there is a compact set
in
containing
. If the minimization problem for
has a unique solution
, then a regularizing minimizing sequence converges to
, and under these conditions it is sufficient to exhibit algorithms for the construction of regularizing minimizing sequences. This can be done by using stabilizing functionals
.
Let be a stabilizing functional defined on a set
, let
and let
. Frequently, instead of
one takes its
-approximation
relative to
, that is, a functional such that for every
,
![]() |
Then for any the problem of minimizing the functional
![]() |
over the argument is stable.
Let and
be null-sequences such that
for every
, and let
be a sequence of elements minimizing
. This is a regularizing minimizing sequence for the functional
(see [TiArAr]), consequently, it converges as
to an element
. As approximate solutions of the problems one can then take the elements
.
Similarly approximate solutions of ill-posed problems in optimal control can be constructed.
In applications ill-posed problems often occur where the initial data contain random errors. For the construction of approximate solutions to such classes both deterministic and probability approaches are possible (see [TiArAr], [LaVa]).
References
[Ar] | V.Ya. Arsenin, "On a method for obtaining approximate solutions to convolution integral equations of the first kind" Proc. Steklov Inst. Math., 133 (1977) pp. 31–48 Trudy Mat. Inst. Steklov., 133 (1973) pp. 33–51 |
[Ba] | A.B. Bakushinskii, "A general method for constructing regularizing algorithms for a linear ill-posed equation in Hilbert space" USSR Comp. Math. Math. Phys., 7 : 3 (1968) pp. 279–287 Zh. Vychisl. Mat. i Mat. Fiz., 7 : 3 (1967) pp. 672–677 |
[GoLeYa] | A.V. Goncharskii, A.S. Leonov, A.G. Yagoda, "On the residual principle for solving nonlinear ill-posed problems" Soviet Math. Dokl., 15 (1974) pp. 166–168 Dokl. Akad. Nauk SSSR, 214 : 3 (1974) pp. 499–500 |
[Iv] | V.K. Ivanov, "On ill-posed problems" Mat. Sb., 61 : 2 (1963) pp. 211–223 (In Russian) |
[Iv2] | V.K. Ivanov, "On linear problems which are not well-posed" Soviet Math. Dokl., 3 (1962) pp. 981–983 Dokl. Akad. Nauk SSSR, 145 : 2 (1962) pp. 270–272 |
[Kr] | A.V. Kryanev, "The solution of incorrectly posed problems by methods of successive approximations" Soviet Math. Dokl., 14 (1973) pp. 673–676 Dokl. Akad. Nauk SSSR, 210 : 1 pp. 20–22 |
[LaLa] | M.M. [M.A. Lavrent'ev] Lavrentiev, "Some improperly posed problems of mathematical physics", Springer (1967) (Translated from Russian) |
[LaLi] | R. Lattes, J.L. Lions, "Méthode de quasi-réversibilité et applications", Dunod (1967) |
[LaVa] | M.M. Lavrent'ev, V.G. Vasil'ev, "The posing of certain improper problems of mathematical physics" Sib. Math. J., 7 : 3 (1966) pp. 450–463 Sibirsk. Mat. Zh., 7 : 3 (1966) pp. 559–576 |
[Ti] | A.N. Tikhonov, "Solution of incorrectly formulated problems and the regularization method" Soviet Math. Dokl., 4 (1963) pp. 1035–1038 Dokl. Akad. Nauk SSSR, 151 : 3 (1963) pp. 501–504 |
[Ti2] | A.N. Tikhonov, "Regularization of incorrectly posed problems" Soviet Math. Dokl., 4 (1963) pp. 1624–1627 Dokl. Akad. Nauk SSSR, 153 : 1 (1963) pp. 49–52 |
[Ti3] | A.N. Tikhonov, "On stability of inverse problems" Dokl. Akad. Nauk SSSR, 39 : 5 (1943) pp. 176–179 (In Russian) |
[Ti4] | A.N. Tikhonov, "On the stability of the functional optimization problem" USSR Comp. Math. Math. Phys., 6 : 4 (1966) pp. 28–33 Zh. Vychisl. Mat. i Mat. Fiz., 6 : 4 (1966) pp. 631–634 |
[TiArAr] | A.N. Tikhonov, V.I. [V.I. Arsenin] Arsenine, "Solution of ill-posed problems", Winston (1977) (Translated from Russian) |
[Vi] | V.A. Vinokurov, "On the regularization of discontinuous mappings" USSR Comp. Math. Math. Phys., 11 : 5 (1971) pp. 1–21 Zh. Vychisl. Mat. i Mat. Fiz., 11 : 5 (1971) pp. 1097–1112 |
Comments
The idea of conditional well-posedness was also found by B.L. Phillips [Ph]; the expression "Tikhonov well-posed" is not used in the West.
Other problems that lead to ill-posed problems in the sense described above are the Dirichlet problem for the wave equation, the non-characteristic Cauchy problem for the heat equation, the initial boundary value problem for the backward heat equation, inverse scattering problems ([CoKr]), identification of parameters (coefficients) in partial differential equations from over-specified data ([Ba2], [EnGr]), and computerized tomography ([Na2]).
If is a bounded linear operator between Hilbert spaces, then, as also mentioned above, regularization operators can be constructed via spectral theory: If
as
, then under mild assumptions,
is a regularization operator (cf. [Gr]); for choices of the regularization parameter leading to optimal convergence rates for such methods see [EnGf]. For
, the resulting method is called Tikhonov regularization: The regularized solution
is defined via
. A variant of this method in Hilbert scales has been developed in [Na] with parameter choice rules given in [Ne]. The parameter choice rule discussed in the article given by
is called the discrepancy principle ([Mo]).
References
[Ba2] | J. Baumeister, "Stable solution of inverse problems", Vieweg (1986) |
[Ba3] | H.P. Baltes (ed.), Inverse source problems in optics, Springer (1978) |
[Ba4] | H.P. Baltes (ed.), Inverse scattering problems in optics, Springer (1980) |
[BaGi] | G. Backus, F. Gilbert, "The resolving power of gross earth data" Geophys. J. R. Astr. Soc., 16 (1968) |
[BeBlSt] | J.V. Beck, B. Blackwell, C.R. StClair, "Inverse heat conduction: ill posed problems", Wiley (1985) |
[BoJo] | W.M. Boerner, A.K. (eds.) Jordan, "Inverse methods in electromagnetics" IEEE Trans. Antennas Propag., 2 (1981) |
[Ca] | J.R. Cann on, "The one-dimensional heat equation", Addison-Wesley (1984) |
[CaSt] | A. Carasso, A.P. Stone, "Improperly posed boundary value problems", Pitman (1975) |
[Co] | A.M. Cormak, "Representation of a function by its line integrals with some radiological applications" J. Appl. Phys., 34 (1963) |
[Co2] | L. Colin, "Mathematics of profile inversion", Proc. Workshop Ames Res. Center, June 12–16, 1971, TM-X-62.150, NASA |
[CoKr] | D.L. Colton, R. Kress, "Integral equation methods in scattering theory", Wiley (1983) |
[EnGf] | H.W. Engl, H. Gfrerer, "A posteriori parameter choice for general regularization methods for solving linear ill-posed problems" Appl. Num. Math., 4 (1988) pp. 395–417 |
[EnGr] | H.W. Engl (ed.) C.W. Groetsch (ed.), Inverse and ill-posed problems, Acad. Press (1987) |
[Gr] | C.W. Groetsch, "The theory of Tikhonov regularization for Fredholm equations of the first kind", Pitman (1984) |
[Gr2] | C.W. Groetsch, "The theory of Tikhonov regularization for Fredholm equations of the first kind", Pitman (1984) |
[He] | G.T. Herman (ed.), Image reconstruction from projections, Springer (1979) |
[HeNa] | G.T. Herman (ed.) F. Natterer (ed.), Mathematical aspects of computerized tomography, Proc. Oberwolfach, February 10–16, 1980, Springer (1981) |
[Jo] | F. John, "Continuous dependence on data for solutions of partial differential equations with a prescribed bound" Comm. Pure Appl. Math., 13 (1960) pp. 551–585 |
[Ka] | M. Kac, "Can one hear the shape of a drum?" Amer. Math. Monthly, 73 (1966) pp. 1–23 |
[LaRoSh] | M.H. Lavrent'ev, V.G. Romanov, S.P. Shishalskii, "Ill-posed problems of mathematical physics and analysis", Amer. Math. Soc. (1986) (Translated from Russian) |
[Mo] | V.A. Morozov, "Methods for solving incorrectly posed problems", Springer (1984) (Translated from Russian) |
[Na] | F. Natterer, "Error bounds for Tikhonov regularization in Hilbert scales" Applic. Analysis, 18 (1984) pp. 29–37 |
[Na2] | F. Natterer, "The mathematics of computerized tomography", Wiley (1986) |
[Ne] | A. Neubauer, "An a-posteriori parameter choice for Tikhonov regularization in Hilbert scales leading to optimal convergence rates" SIAM J. Numer. Anal., 25 (1988) pp. 1313–1326 |
[Pa] | L.E. Payne, "Improperly posed problems in partial differential equations", SIAM (1975) |
[Ph] | B.L. Phillips, "A technique for the numerical solution of certain integral equations of the first kind" J. ACM, 9 (1962) pp. 84–97 |
Ill-posed problems. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Ill-posed_problems&oldid=25060