Reproducing kernel

Consider an abstract set $E$ and a linear set $F$ of functions $f : E \rightarrow \mathbf{C}$.

Assume that $F$ is equipped with an inner product $( f , g )$ and $F$ is complete with respect to the norm $\| f \| = ( f , f ) ^ { 1 / 2 }$. Then $F$ is a Hilbert space.

A function $K ( x , y )$, $x , y \in E$, is called a reproducing kernel of such a Hilbert space $H$ if and only if the following two conditions are satisfied:

i) for every fixed $y \in E$, the function $K ( x , y ) \in H$;

ii) $( f ( x ) , K ( x , y ) ) = f ( y )$, $\forall f \in H$.

This definition is given in [a1]; see also [a6].

Some properties of reproducing kernels are:

1) If a reproducing kernel $K ( x , y )$ exists, then it is unique.

2) A reproducing kernel $K ( x , y )$ exists if and only if $| f ( y ) | \leq c ( y ) \| f \|$, $\forall f \in H$, where $c ( y ) = \| K ( . , y ) \|$.

3) $K ( x , y )$ is a non-negative-definite kernel, that is,

\begin{equation*} \sum _ { i , j = 1 } ^ { n } K ( x _ { i } , x _ { j } ) t _ { j } \overline { t } _ { i } \geq 0 , \forall x _ { i } , y _ { j } \in E , \forall t \in {\bf C} ^ { n }, \end{equation*}

where the overbar stands for complex conjugation.

In particular, 3) implies:

\begin{equation*} K ( x , y ) = \overline { K ( y , x ) } , K ( x , x ) \geq 0, \end{equation*}

\begin{equation*} | K ( x , y ) | ^ { 2 } \leq K ( x , x ) K ( y , y ). \end{equation*}

Every non-negative-definite kernel $K ( x , y )$ generates a Hilbert space $H _ { K }$ for which $K ( x , y )$ is a reproducing kernel (see also Reproducing-kernel Hilbert space).

If $K ( x , y )$ is a reproducing kernel, then the operator $K f : = ( K f ) ( \cdot ) = ( f , K ( x , ) ) = f ( \cdot )$ is injective: $K f = 0$ implies $f = 0$, by reproducing property ii), and $K : H \rightarrow H$ is surjective (cf. also Injection; Surjection). Therefore the inverse operator $K ^ { - 1 }$ is defined on $R ( K ) = H$, and since $K f = f$, the operator $K$ is the identity operator on $H _ { K }$, and so is its inverse.

Examples of reproducing kernels.

Consider the Hilbert space $H$ of analytic functions (cf. Analytic function) in a bounded simply-connected domain $D$ of the complex $z$-plane. If $f ( z )$ is analytic in $D$, $z _ { 0 } \in D$, and the disc $D _ { z _ { 0 } , r } : = \{ z : | z - z _ { 0 } | \leq r \} \in D$, then

\begin{equation*} | f ( z _ { 0 } ) | ^ { 2 } \leq \frac { 1 } { \pi r ^ { 2 } } \int _ { D _ { z _ { 0 } , r } } | f ( \zeta ) | ^ { 2 } d x d y \leq \frac { 1 } { \pi r ^ { 2 } } ( f , f ) _ { L^2(D) }. \end{equation*}

Therefore $H$ is a reproducing-kernel Hilbert space. Its reproducing kernel $K _ { D } ( z , \zeta )$ is called the Bergman kernel (cf. also Bergman kernel function).

If $\{ \phi_j ( z ) \}$ is an orthonormal basis of $H$ (cf. also Orthogonal system; Basis), $\phi _ { j } \in H$, then $K _ { D } ( z , \zeta ) = \sum _ { j = 1 } ^ { \infty } \phi _ { j } ( z ) \overline { \phi _ { j } ( \zeta ) }$.

If $w = f ( z , z_0 )$ is the conformal mapping of $D$ onto the disc $| w | \leq \rho _ { D }$, such that $f ( z , z _ { 0 } ) = 0$, $f ^ { \prime } ( z _ { 0 } , z _ { 0 } ) = 1$, then [a2]:

\begin{equation*} f ( z , z_0 ) = \frac { 1 } { K _ { D } ( z_0 , z _ { 0 } ) } \int _ { z _ { 0 } } ^ { z } K _ { D } ( t , z _ { 0 } ) d t. \end{equation*}

Let $T$ be a domain in ${\bf R} ^ { n }$ and $h ( t , p ) \in L ^ { 2 } ( T , d m )$ for every $p \in E$. Here $m ( t ) > 0$ is a finite measure on $T$.

Define a linear mapping $L : L ^ { 2 } ( T , d m ) \rightarrow F$ by

\begin{equation} \tag{a1} f ( p ) = L g : = \int _ { T } g ( t ) \overline { h ( t , p ) } d m ( t ). \end{equation}

Define the kernel

\begin{equation} \tag{a2} K ( p , q ) : = \int _ { T } h ( t , q ) \overline { h ( t , p ) } d m ( t ) , p , q \in E. \end{equation}

This kernel is non-negative-definite:

\begin{equation*} \sum _ { i , j + 1 } ^ { n } K ( p _ { i } , p _ { j } ) \xi _ { j } \overline { \xi _ { i } } = \int _ { T } | \sum _ { j = 1 } ^ { n } \xi _ { j } h ( t , p _ { j } ) | ^ { 2 } d m ( t ) > 0 \end{equation*}

\begin{equation*} \xi \neq 0, \end{equation*}

provided that for any set $\{ p _ { 1 } , \dots , p _ { n } \} \in E$ the set of functions $\{ h ( t , p _ { j } ) \} _ { 1 \leq j \leq n}$ is linearly independent in $L ^ { 2 } ( T , d m )$ (cf. Linear independence).

In this case the kernel $K ( p , q )$ generates a uniquely determined reproducing-kernel Hilbert space $H _ { K }$ for which $K ( p , q )$ is the reproducing kernel.

In [a6] it is claimed that a convenient characterization of the range $R ( L )$ of the linear transformation (a1) is given by the formula $R ( L ) = H _ { K }$. In [a4] it is shown by examples that such a characterization is often useless in practice: in general the norm in $H _ { K }$ can not be described in terms of the standard Sobolev or Hölder norms, and the assumption in [a6] that $H _ { K }$ can be realized as $L ^ { 2 } ( E , d \mu )$ is not justified and is not correct, in general.

However, in [a6] there are some examples of characterizations of $H _ { K }$ for some special operators $L$ and in [a5] a characterization of the range of a wide class of multi-dimensional linear transforms, whose kernels are kernels of positive rational functions of self-adjoint elliptic operators, is given.

Reproducing kernels are discussed in [a5] for rigged triples of Hilbert spaces (cf. also Rigged Hilbert space). If $H _ { 0 }$ is a Hilbert space and $A > 0$ is a linear compact operator defined on all of $H$, then the closure of $H _ { 0 }$ in the norm $( A u , u ) ^ { 1 / 2 } = \| A ^ { 1 / 2 } u \|$ is a Hilbert space $H _ { - } \supset H _ { 0 }$. The space dual to $H_-$, with respect to $H _ { 0 }$, is denoted by $H _ { + }$, $H _ { + } \subset H _ { 0 } \subset H _ { - }$. The inner product in $H _ { + }$ is given by the formula $( u , v ) _ { + } = ( A ^ { - 1 / 2 } u , A ^ { - 1 / 2 } v ) _ { 0 }$. The space $H _ { + } = R ( A ^ { 1 / 2 } )$, equipped with this inner product, is a Hilbert space.

Let $A \varphi _ { j } = \lambda _ { j } \varphi _ { j }$, where the eigenvalues $\lambda_j$ are counted according to their multiplicities and $( \varphi_j , \varphi _ { m } ) _ { 0 } = \delta _ { j m }$, where $\delta _ { j m }$ is the Kronecker delta.

Assume that $| \varphi_j ( x ) | < c$ for all $j$ and all $x$, and $\Lambda ^ { 2 } : = \sum _ { j = 1 } ^ { \infty } \lambda _ { j } < \infty$.

Then $H _ { + }$ is a reproducing-kernel Hilbert space and its reproducing kernel is $K ( x , y ) = \sum _ { j = 1 } ^ { \infty } \lambda _ { j } \varphi _ { j } ( y ) \overline { \varphi _ { j } ( x ) }$.

To check that $K ( x , y )$ is indeed the reproducing kernel of $H _ { + }$, one calculates $( A ^ { - 1 / 2 } u , A ^ { - 1 / 2 } K ) _ { 0 } = ( u , A ^ { - 1 } K ) _ { 0 } = u ( y )$. Indeed, $A ^ { - 1 } K = I$ is the identity operator because $A u = \sum _ { j = 1 } ^ { \infty } \lambda _ { j } ( u , \varphi _ { j } ) \varphi _ { j } ( x )$, so that $K ( x , y )$ is the kernel of the operator $A$ in $H _ { 0 }$.

The value $u ( y )$ is a linear functional in $H _ { + }$, so that $H _ { + }$ is a reproducing-kernel Hilbert space. Indeed, if $u \in H _ { + }$, then $v : = A ^ { - 1 / 2 } u \in H _ { 0 }$. Therefore, denoting $v _ { j } : = ( v , \varphi _ { j } ) _ { 0 }$ and using the Cauchy inequality and Parseval equality one gets:

\begin{equation*} | u ( y ) | = \left| \sum _ { j = 1 } ^ { \infty } \lambda _ { j } ^ { 1 / 2 } v _ { j } \varphi _ { j } ( x ) \right| < c \Lambda \| v \| _ { 0 } = c \Lambda \| u \| _ { + }, \end{equation*}

as claimed.

From the representation of the inner product in the reproducing-kernel Hilbert space $H _ { + }$ by the formula $( u , v ) _ { + } = ( A ^ { - 1 / 2 } u , A ^ { - 1 / 2 } v ) _ { 0 }$ it is clear that, in general, the inner product in $H _ { + }$ is not an inner product in $L ^ { 2 } ( E , d \mu )$.

The inner product in $H _ { + }$ is of the form

\begin{equation*} ( u , v )_ + = \int _ { D } \int _ { D } B ( x , y ) u ( y ) \overline { v ( x ) } d y d x \;\text { if } H _ { 0 } = L ^ { 2 } ( D ), \end{equation*}

where the distributional kernel $B ( x , y ) = \sum _ { j = 1 } ^ { \infty } \lambda _ { j } ^ { - 1 } \varphi _ { j } ( x ) \overline { \varphi _ { j } ( y ) }$ acts on $u \in R ( A )$ by the formula $\int _ { D } B ( x , y ) u ( y ) d y = \sum _ { j = 1 } ^ { \infty } \lambda _ { j } ^ { - 1 } ( u , \varphi _ { j } ) _ { 0 } \varphi _ { j } ( x )$, where $( u , \varphi _ { j } ) _ { 0 } : = \int _ { D } u ( y ) \overline { \varphi _ { j } ( y ) } d y$ is the Fourier coefficient of $u$ (cf. also Fourier coefficients). If $u \in R ( A )$, then $u = A w$ for some $w \in H _ { 0 }$, and $( u , \varphi_j ) = \lambda _ { j } w _ { j }$. Thus, the series $\sum _ { j = 1 } ^ { \infty } \lambda _ { j } ^ { - 1 } ( u , \varphi_j ) _ { 0 } \varphi _ { j } ( x ) = \sum _ { j = 1 } ^ { \infty } w _ { j } \varphi _ { j } ( x ) = w ( x )$ converges in $H _ { 0 } = L ^ { 2 } ( D )$.

References

[a1]	N. Aronszajn, "Theory of reproducing kernels" Trans. Amer. Math. Soc. , 68 (1950) pp. 337–404
[a2]	S. Bergman, "The kernel function and conformal mapping" , Amer. Math. Soc. (1950)
[a3]	A.G. Ramm, "On the theory of reproducing kernel Hilbert spaces" J. Inverse Ill-Posed Probl. , 6 : 5 (1998) pp. 515–520
[a4]	A.G. Ramm, "On Saitoh's characterization of the range of linear transforms" A.G. Ramm (ed.) , Inverse Problems, Tomography and Image Processing , Plenum (1998) pp. 125–128
[a5]	A.G. Ramm, "Random fields estimation theory" , Longman/Wiley (1990)
[a6]	S. Saitoh, "Integral transforms, reproducing kernels and their applications" , Pitman Res. Notes , Longman (1997)
[a7]	L. Schwartz, "Sous-espaces hilbertiens d'espaces vectoriels topologique et noyaux associes" Anal. Math. , 13 (1964) pp. 115–256