Ridge function

plane wave

In its simplest form, a ridge function is a multivariate function

\begin{equation*} f : \mathbf{R} ^ { n } \rightarrow \mathbf{R} \end{equation*}

of the form

\begin{equation*} f ( x _ { 1 } , \dots , x _ { n } ) = g ( a _ { 1 } x _ { 1 } + \ldots + a _ { n } x _ { n } ) = g ( \mathbf{a}\cdot \mathbf{x} ), \end{equation*}

where $g : \mathbf{R} \rightarrow \mathbf{R}$ and ${\bf a} = ( a _ { 1 } , \dots , a _ { n } ) \in {\bf R} ^ { n } \backslash \{ 0 \}$. The vector $\mathbf{a} \in \mathbf{R} ^ { n } \backslash \{ 0 \}$ is generally called the direction. In other words, a ridge function is a multivariate function constant on the parallel hyperplanes $\mathbf{a} \cdot \mathbf{x} = c$, $c \in \mathbf R$.

Ridge functions appear in various areas and under various guises. In 1975, B.F. Logan and L.A. Shepp coined the name "ridge function" in their seminal paper [a6] in computerized tomography. In tomography, or at least in tomography as the theory was initially constructed in the early 1980s, ridge functions were basic. However, these functions have been considered for some time, but under the name of plane waves. See, for example, [a5] and [a1]. In general, linear combinations of ridge functions with fixed directions occur in the study of hyperbolic partial differential equations with constant coefficients.

Ridge functions and ridge function approximation are studied in statistics. There they often go under the name of projection pursuit, see e.g. [a3], [a4], [a2]. Projection pursuit algorithms approximate a function of $n$ variables by functions of the form

\begin{equation*} \sum _ { i = 1 } ^ { r } g_i ( \mathbf{a} ^ { i }. \mathbf{x} ), \end{equation*}

where the $\mathbf{a} ^ { i }$ and $g_i$ are the variables. The idea here is to "reduce dimension" and thus bypass the curse of dimension. The vector $\mathbf{a} ^ { i } \mathbf{x}$ is considered as a projection of $\mathbf{x}$. The directions $\mathbf{a}$ are chosen to "pick out the salient features" .

One of the popular models in the theory of neural nets is that of a multi-layer feedforward neural net with input, hidden and output layers (cf. also Neural network). The simplest case (which is that of one hidden layer, $r$ processing units and one output) considers, in mathematical terms, functions of the form

\begin{equation*} \sum _ { i = 1 } ^ { r } \alpha _ { i } \sigma ( \mathbf{w} ^ { i } \mathbf{x} + \theta _ { i } ) \end{equation*}

where $\sigma : \mathbf{R} \rightarrow \mathbf{R}$ is some given fixed univariate function. In this model, which is just one of many, one is in general permitted to vary over the $\mathbf{w} ^ { i }$ and $\theta _ { i }$, in order to approximate an unknown function. Note that for each $\theta \in \mathbf{R}$ and $\mathbf{w} \in \mathbf{R} ^ { n } \backslash \{ 0 \}$ the function

\begin{equation*} \sigma ( \mathbf{w}.\mathbf{v} + \theta ) \end{equation*}

is also a ridge function, see e.g. [a8] and references therein.

For a survey on some approximation-theoretic questions concerning ridge functions, see [a7] and references therein.

References

[a1]	R. Courant, D. Hilbert, "Methods of mathematical physics" , II , Interscience (1962)
[a2]	D.L. Donoho, I.M. Johnstone, "Projection-based approximation and a duality method with kernel methods" Ann. Statist. , 17 (1989) pp. 58–106
[a3]	J.H. Friedman, W. Stuetzle, "Projection pursuit regression" J. Amer. Statist. Assoc. , 76 (1981) pp. 817–823
[a4]	P.J. Huber, "Projection pursuit" Ann. Statist. , 13 (1985) pp. 435–475
[a5]	F. John, "Plane waves and spherical means applied to partial differential equations" , Interscience (1955)
[a6]	B.F. Logan, L.A. Shepp, "Optimal reconstruction of a function from its projections" Duke Math. J. , 42 (1975) pp. 645–659
[a7]	A. Pinkus, "Approximating by ridge functions" A. Le Méhauté (ed.) C. Rabut (ed.) L.L. Schumaker (ed.) , Surface Fitting and Multiresolution Methods , Vanderbilt Univ. Press (1997) pp. 279–292
[a8]	A. Pinkus, "Approximation theory of the MLP model in neural networks" Acta Numerica , 8 (1999) pp. 143–195