View source for Reproducing-kernel Hilbert space

<!--This article has been texified automatically. Since there was no Nroff source code for this article, 
the semi-automatic procedure described at https://encyclopediaofmath.org/wiki/User:Maximilian_Janisch/latexlist
was used.
If the TeX and formula formatting is correct, please remove this message and the {{TEX|semi-auto}} category.

Out of 180 formulas, 180 were replaced by TEX code.-->

{{TEX|semi-auto}}{{TEX|done}}
Let $H$ be a [[Hilbert space|Hilbert space]] of functions defined on an abstract set $E$.

Let $( f , g )$ denote the [[Inner product|inner product]] and let $\| f \| = ( f , f ) ^ { 1 / 2 }$ be the norm in $H$. The space $H$ is called a reproducing-kernel Hilbert space if there exists a function $K ( x , y )$, the [[Reproducing kernel|reproducing kernel]], on $E \times E$ such that:

1) $K ( x , y ) \in H$ for any $y \in E$;

2) $( f ( . ) , K (. , y ) ) = f ( y )$ for all $f \in H$ (the reproducing property). From this definition it follows that the value $f ( y )$ at a point $y \in E$ is a continuous [[Linear functional|linear functional]] in $H$:

\begin{equation*} | f ( y ) | \leq c ( y ) \| f \| , c ( y ) : = \| K (\, .\, , y ) \|. \end{equation*}

The converse is also true. The following theorem holds: A Hilbert space of functions on a set $E$ is a reproducing-kernel Hilbert space if and only if $| f ( y ) | \leq c ( y ) \| f \|$ for all $y \in E$.

By the [[Riesz theorem(2)|Riesz theorem]], the above assumption implies the existence of a linear functional $K (\ \cdot \ , y )$ such that $f ( y ) = ( f , K (\, .\, , y ) )$. By the construction, the kernel $K ( x , y )$ is the reproducing kernel for $H$.

An example of a construction of a reproducing-kernel Hilbert space is the rigged triple of Hilbert spaces $H _ { + } \subset H _ { 0 } \subset H _ { - }$, which is defined as follows [[#References|[a5]]] (cf. also [[Rigged Hilbert space|Rigged Hilbert space]]). Let $H _ { 0 }$ be a Hilbert space of functions, let $A > 0$ be a linear densely defined [[Self-adjoint operator|self-adjoint operator]] on $H _ { 0 }$, $A \varphi _ { j } = \lambda _ { j } \varphi _ { j }$ (the eigenvalues $\lambda_j > 0$ are counted according to their multiplicities) and assume that

\begin{equation*} \Lambda ^ { 2 } : = \sum _ { j = 1 } ^ { \infty } \lambda _ { j } < \infty , | \varphi _ { j } ( x ) | < c , \forall j , x. \end{equation*}

Define $H _ { - } \supset H _ { 0 }$ to be the Hilbert space with inner product $( u , v ) _ { - } = ( A ^ { 1 / 2 } u , A ^ { 1 / 2 } v ) _ { 0 }$. $H_-$ is the completion of $H _ { 0 }$ in the norm $\| u \| : = ( u , u ) ^ { 1 / 2 }_ { - }$. Let $H _ { + } \subset H _ { 0 }$ be the dual space to $H_-$ with respect to $H _ { 0 }$. Then the inner product in $H _ { + }$ is defined by the formula $( u , v ) _ { + } : = ( A ^ { - 1 / 2 } u , A ^ { - 1 / 2 } v ) _ { 0 },$ and $H _ { + } = R ( A ^ { 1 / 2 } )$, equipped with the inner product $( u , v )_+$, is a Hilbert space.

Define $B ( x , y ) = \sum _ { j = 1 } ^ { \infty } \lambda _ { j } \overline { \varphi _ { j } ( x ) } \varphi _ { j } ( y )$, where the overline stands for complex conjugation. For any $y$, one has $B ( x , y ) \in H _ { + }$. Indeed,

\begin{equation*} \| B ( x , y ) \| _ { + } \leq c \sum _ { j = 1 } ^ { \infty } \| \lambda _j \varphi_j ( x ) \| _ { + } = \end{equation*}

\begin{equation*} = c \sum _ { j = 1 } ^ { \infty } ( A \varphi _ { j } , \varphi _ { j } ) _ { 0 } = c \Lambda ^ { 2 } < \infty. \end{equation*}

Furthermore,

\begin{equation*} ( u , B ( x , y ) ) _ { + } = ( u , A ^ { - 1 } B ) = u ( y ), \end{equation*}

so that $B ( x , y )$ is the [[Reproducing kernel|reproducing kernel]] in $H _ { + }$. Moreover $| u ( y ) | \leq c ( y ) \| u \|_+$, where $c ( y ) > 0$ is a constant independent of $u \in H _ { + }$. Indeed, if $u _ { j } : = ( u , \varphi _ { j } ) _ { 0 }$ and $u \in H _ { + }$, then $u = A ^ { 1 / 2 } v$, $v \in H _ { 0 }$ $v _ { j } \lambda _ { j } ^ { 1 / 2 } = u _ { j }$, and $| u ( y ) | \leq \sum _ { j = 1 } ^ { \infty } | u _ { j } \varphi _ { j } ( y ) | \leq c \Lambda \| v \| = c \Lambda \| u \| _ { + }$.

Thus $H _ { + }$ is a reproducing kernel Hilbert space with the reproducing kernel $B ( x , y )$ defined above. If $K ( x , y )$ is a function on $E \times E$ such that

\begin{equation} \tag{a1} \sum _ { i ,\, j = 1 } ^ { n } K ( x _ { i } , x _ { j } ) t _ { j } \overline { t } _ { i } \geq 0 ,\, \forall t \in \mathbf{C} ^ { n } ,\, \forall x _ { i } \in E, \end{equation}

then one can define a [[Pre-Hilbert space|pre-Hilbert space]] $H ^ { 0 }$ of functions of the form

\begin{equation*} f ( x ) : = \sum _ { j = 1 } ^ { J } K ( x , y_j ) c_j , c_j =\text{const.} \end{equation*}

The inner product of two functions from $H ^ { 0 }$ is defined by

\begin{equation*} ( f , g ) : = \left( \sum _ { j = 1 } ^ { J } K ( x , y _ { j } ) c _ { j } , \sum _ { m = 1 } ^ { M } K ( x , z _ { m } ) \beta _ { m } \right) = \end{equation*}

\begin{equation*} = \sum _ { j , m } K ( z _ { m } , y _ { j } ) c _ { j } \overline { \beta _ { m } }. \end{equation*}

This definition makes sense because of (a1) and because of reproducing property 2). In particular, $( f , f ) \geq 0$, as follows from (a1), and if $( f , f ) = 0$ then $f = 0$, as follows from property 2).

Indeed,

\begin{equation*} ( f ( x ) , K ( x , y ) ) = \left( \sum _ { j = 1 } ^ { J } K ( x , y _ { j } ) c _ { j } , K ( x , y ) \right) = \end{equation*}

\begin{equation*} = \sum _ { j = 1 } ^ { J } K ( y , y _ { j } ) c _ { j } = f ( y ) , \forall y \in E. \end{equation*}

Thus, if $( f , f ) = 0$, then $\| f \| = 0$ and $| f ( y ) | \leq \| f \| \| K ( x , y ) \| = 0$, so $f ( y ) = 0$ as claimed.

Denote by $H$ the completion of $H ^ { 0 }$ in the norm $\| f \|$. Then $H$ is a reproducing-kernel Hilbert space and $K ( x , y )$ is its reproducing kernel.

A reproducing-kernel Hilbert space is uniquely defined by its reproducing kernel. Indeed, if $H _ { 1 }$ is another reproducing-kernel Hilbert space with the same reproducing kernel $K ( x , y )$, then $H ^ { 0 } \subset H _ { 1 }$ and $H ^ { 0 }$ is dense in $H _ { 1 }$: If $f \in H _ { 1 }$, $0 = ( f , K ( x , y ) ) _ { H _ { 1 } } = f ( y )$ for all $y \in E$, then $f \equiv 0$. Using this and the equality $( f , g ) _ { H _ { 1 } } = ( f , g ) _ { H }$ for all $f , g \in H ^ { 0 }$, one can check that $H \subset H _ { 1 }$ and vice versa, so $H = H _ { 1 }$, that is, $H$ and $H _ { 1 }$ consist of the same set of elements. Moreover, the norms in $H$ and $H _ { 1 }$ are equal. Indeed, take an arbitrary $f \in H _ { 1 }$ and a sequence $f _ { n } \in H ^ { 0 }$, $\| f _ { n } - f \| _ { 1 } \rightarrow 0$. Then

\begin{equation*} \| f \| _ { 1 } ^ { 2 } = \operatorname { lim } _ { n \rightarrow \infty } \| f _ { n } \| _ { 1 } ^ { 2 } = \end{equation*}

\begin{equation*} = \operatorname { lim } _ { n \rightarrow 0 } \left( \sum _ { j_n = 1 } ^ { J _ { n } } K ( x , y _ { j _n } ) c _ { j _n } , \sum _ { m_n = 1 } ^ { J _ { n } } K ( x , y _ { m_n } ) c _ { m_n } \right) _ { 1 } = \end{equation*}

\begin{equation*} = \sum _ { j _ { n } ,\, m _ { n } } ^ { J _ { n } } K ( y _ { m _ { n } } , y _ { j _ { n } } ) c _ { j _ { n } } \overline { c_{m _ { n }}} = \end{equation*}

\begin{equation*} = \operatorname { lim } _ { n \rightarrow \infty } ( f _ { n } , f _ { n } ) = \| f \| ^ { 2 }. \end{equation*}

Thus, the norms in $H _ { 1 }$ and $H$ are equal, as claimed, and so are the inner products (by the [[polarization identity]]).

Define a [[Linear operator|linear operator]] $L : {\cal H} \rightarrow H$, $D ( L ) = \mathcal{H}$, where $\mathcal{H} = L ^ { 2 } ( T , d m )$ and $H$ is the range $R ( L )$ of $L$, which will be equipped with the structure of a Hilbert space below:

\begin{equation} \tag{a2} f ( x ) = L F : = \int _ { T } F ( t ) \overline { h ( t , x ) } d m ( t ). \end{equation}

Here, $T$ is a domain in ${\bf R} ^ { n }$ and $m$ is a positive [[Measure|measure]] on $T$, $m ( T ) < \infty$, $h ( t , x ) \in \mathcal{H}$ for all $x \in E$, and it is assumed that $L$ is injective, that is, the system $\{ h ( t , x ) \}_{\forall x \in E}$ is total in $\mathcal{H}$ (cf. also [[Total set|Total set]]).

Define

\begin{equation} \tag{a3} K ( x , y ) : = \int _ { T } h ( t , y ) \overline { h ( t , x ) } d m ( t ) = \end{equation}

\begin{equation*} = ( h (\, . \,  , y ) , h (\, . \, , x ) ) _ { \mathcal{H} }. \end{equation*}

This kernel clearly satisfies condition (a1) and therefore is a reproducing kernel for the reproducing-kernel Hilbert space $H _ { K }$ which it generates. Clearly $K ( x , y ) \in H$ for all $y \in E$. If $f \in H$, that is, $f = L F$, $f \in \mathcal{H}$, then

\begin{equation*} ( f ( \cdot ) , K ( \cdot , y ) ) _ { H } = ( L F , K ( \cdot , y ) ) _ { H } = \end{equation*}

\begin{equation*} = ( ( F ( \cdot ) , h ( \cdot , x ) ) _ { \mathcal{H} } , ( h ( \text{..} , y ) , h ( \text{..} , x ) ) _ { \mathcal{H} } ) _ { H } = \end{equation*}

\begin{equation*} = ( F ( . ) , ( h ( .. , y ) , ( h (. , x ) , h ( .. , x ) ) _ { H } ) _ {\cal H } ) _ {\cal H } = \end{equation*}

\begin{equation*} = ( F ( \cdot ) , h ( \cdot , y ) ) _ { \mathcal{H} } = f ( y ), \end{equation*}

if one equips $H$ with the inner product such that $( f , g ) _ { H } = ( F , G ) _ { \mathcal{H} }$. This requirement is formally equivalent to the following one: $( h ( s , x ) , h ( t , x ) ) _ { H } = \delta _ { m } ( t - s )$, where $( h ( s , y ) , \delta _ { m } ( t - s ) ) _ { \mathcal{H} } = h ( t , y )$, so that the distributional kernel $\delta _ { m } ( t - s )$ is not the usual [[Delta-function|delta-function]], but the one which acts by the rule

\begin{equation*} \int _ { T } d m ( t ) F ( t ) \int _ { T } d m ( s ) G ( s ) \delta _ { m } ( t - s ) = \end{equation*}

\begin{equation*} = \int _ { T } d m ( t ) F ( t ) G ( t ), \end{equation*}

and formally one has $\int _ { T } d m ( s ) G ( s ) \delta _ { m } ( t - s ) = G ( t )$.

With the inner product $( f , g ) _ { H }$, the linear set $R ( L )$ becomes a Hilbert space:

\begin{equation} \tag{a4} ( f , g ) _ { H } = ( L F , L G ) _ { H } = \end{equation}

\begin{equation*} = \int_{T} \int _ { T } d m ( t ) d m ( s ) F ( t ) \overline { G ( s ) } ( h ( s , x ) , h ( t , x ) ) _ { H } = \end{equation*}

\begin{equation*} = \int _ { T } d m ( t ) F ( t ) \overline { G ( t ) } = ( F , G ) _ { \mathcal{H} }. \end{equation*}

Thus, this inner product makes $L$ an [[Isometric operator|isometric operator]] defined on all of $\mathcal{H}$ and makes $H = R ( L )$ a (complete) Hilbert space, namely $H = H _ { K }$, a reproducing-kernel Hilbert space. Since $L$ is assumed injective, it follows that $L ^ { - 1 }$ is defined on all of $R ( L ) = H$ and, since $H$ is complete in the norm $\|\, f \| = ( f , f ) ^ { 1 / 2 } _ { H }$, one concludes that $L ^ { - 1 }$ is continuous (by the Banach theorem). Consequently, $L$ is a co-isometry, that is, $L ^ { * } = L ^ { - 1 }$, where $L ^ { * }$ is the [[Adjoint operator|adjoint operator]] to $L$. If $L ^ { * } = L ^ { - 1 }$, then one can write an inversion formula for the linear transform $L$ similar to the well-known inversion formula for the [[Fourier transform|Fourier transform]]. Formally one has:

\begin{equation*} f ( x ) = ( F ( t ) , h ( t , x ) ) _ { \cal H } , ( f ( x ) , h ( s , x ) ) _ { H } = F ( s ). \end{equation*}

The space $H = H _ { K }$ is the reproducing-kernel Hilbert space generated by kernel (a3) which is the reproducing kernel for $H$. The above formal inversion formulas may be of practical interest if the norm in $H$ is a standard one. In this case the second formula should be suitably interpreted, since $F ( s )$ is defined at $m$-almost all $s$.

In [[#References|[a6]]] it is claimed that the characterization of the range of the linear operator $L$, defined in (a3), can be given as follows: $R ( L ) = H _ { K }$, where $H _ { K }$ is the reproducing-kernel Hilbert space generated by kernel (a3).

However, in fact such a characterization does not give, in general, practically useful necessary and sufficient conditions for $f ( x ) \in R ( L )$ because the norm in $H _ { K }$ is not defined in terms of standard norms such as Sobolev or Hölder ones (see [[#References|[a3]]], [[#References|[a4]]], [[#References|[a5]]]). However, when the norm in $H _ { K }$ is equivalent to a standard norm, the above characterization becomes efficient (see [[#References|[a3]]], [[#References|[a4]]], [[#References|[a5]]], and also [[#References|[a6]]]).

Many concrete examples of reproducing-kernel Hilbert spaces can be found in [[#References|[a1]]], [[#References|[a2]]] and [[#References|[a6]]].

The papers [[#References|[a1]]] and [[#References|[a7]]] are important in this area, the book [[#References|[a6]]] contains many references, while [[#References|[a2]]] is an earlier book important for the development of the theory of reproducing-kernel Hilbert spaces.

====References====
<table><tr><td valign="top">[a1]</td> <td valign="top">  N. Aronszajn,   "Theory of reproducing kernels"  ''Trans. Amer. Math. Soc.'' , '''68'''  (1950)  pp. 337–404</td></tr><tr><td valign="top">[a2]</td> <td valign="top">  S. Bergman,   "The kernel function and conformal mapping" , Amer. Math. Soc.  (1950)</td></tr><tr><td valign="top">[a3]</td> <td valign="top">  A.G. Ramm,   "On the theory of reproducing kernel Hilbert spaces"  ''J. Inverse Ill-Posed Probl.'' , '''6''' :  5  (1998)  pp. 515–520</td></tr><tr><td valign="top">[a4]</td> <td valign="top">  A.G. Ramm,   "On Saitoh's characterization of the range of linear transforms"  A.G. Ramm (ed.) , ''Inverse Problems, Tomography and Image Processing'' , Plenum  (1998)  pp. 125–128</td></tr><tr><td valign="top">[a5]</td> <td valign="top">  A.G. Ramm,   "Random fields estimation theory" , Longman/Wiley  (1990)</td></tr><tr><td valign="top">[a6]</td> <td valign="top">  S. Saitoh,   "Integral transforms, reproducing kernels and their applications" , ''Pitman Res. Notes'' , Longman  (1997)</td></tr><tr><td valign="top">[a7]</td> <td valign="top">  L. Schwartz,   "Sous-espaces hilbertiens d'espaces vectoriels topologiques et noyaux associès (noyaux reproduisants)"  ''J. Anal. Math.'' , '''13'''  (1964)  pp. 115–256</td></tr></table>