Gradient

One of the fundamental concepts in vector analysis and the theory of non-linear mappings.

The gradient of a scalar function $ f $ of a vector argument $ t = ( t ^ {1} \dots t ^ {n} ) $ from a Euclidean space $ E ^ {n} $ is the derivative of $ f $ with respect to the vector argument $ t $, i.e. the $ n $- dimensional vector with components $ \partial f / \partial t ^ {i} $, $ 1 \leq i \leq n $. The following notations exist for the gradient of $ f $ at $ t _ {0} $:

$$ \mathop{\rm grad} f ( t _ {0} ),\ \ \nabla f ( t _ {0} ),\ \ \frac{\partial f ( t _ {0} ) }{\partial t } ,\ \ f ^ { \prime } ( t _ {0} ) ,\ \ \left . \frac{\partial f }{\partial t } \right | _ {t _ {0} } . $$

The gradient is a covariant vector: the components of the gradient, computed in two different coordinate systems $ t = ( t ^ {1} \dots t ^ {n} ) $ and $ \tau = ( \tau ^ {1} \dots \tau ^ {n} ) $, are connected by the relations:

$$ \frac{\partial f }{\partial t ^ {i} } ( \tau ( t)) = \ \sum _ {j = 1 } ^ { n } \frac{\partial f ( \tau ) }{\partial \tau ^ {j} } \ \frac{\partial \tau ^ {j} }{\partial t ^ {i} } . $$

The vector $ f ^ { \prime } ( t _ {0} ) $, with its origin at $ t _ {0} $, points to the direction of fastest increase of $ f $, and is orthogonal to the level lines or surfaces of $ f $ passing through $ t _ {0} $.

The derivative of the function at $ t _ {0} $ in the direction of an arbitrary unit vector $ \mathbf N = ( N ^ {1} \dots N ^ {n} ) $ is equal to the projection of the gradient function onto this direction:

$$ \tag{1 } \frac{\partial f ( t _ {0} ) }{\partial \mathbf N } = \ ( f ^ { \prime } ( t _ {0} ), \mathbf N ) \equiv \ \sum _ {j = 1 } ^ { n } \frac{\partial f ( t _ {0} ) }{\partial t ^ {j} } N ^ {j} = | f ^ { \prime } ( t _ {0} ) | \cos \phi , $$

where $ \phi $ is the angle between $ \mathbf N $ and $ f ^ { \prime } ( t _ {0} ) $. The maximal directional derivative is attained if $ \phi = 0 $, i.e. in the direction of the gradient, and that maximum is equal to the length of the gradient.

The concept of a gradient is closely connected with the concept of the differential of a function. If $ f $ is differentiable at $ t _ {0} $, then, in a neighbourhood of that point,

$$ \tag{2 } f ( t) = f ( t _ {0} ) + ( f ^ { \prime } ( t _ {0} ),\ t - t _ {0} ) + o ( | t - t _ {0} | ), $$

i.e. $ df = ( f ^ { \prime } ( t _ {0} ), dt) $. The existence of the gradient of $ f $ at $ t _ {0} $ is not sufficient for formula (2) to be valid.

A point $ t _ {0} $ at which $ f ^ { \prime } ( t _ {0} ) = 0 $ is called a stationary (critical or extremal) point of $ f $. An example of such a point is a local extremal point of $ f $, and the system $ \partial f ( t _ {0} ) / \partial t ^ {i} = 0 $, $ 1 \leq i \leq n $, is employed to find an extremal point $ t _ {0} $.

The following formulas can be used to compute the value of the gradient:

$$ \mathop{\rm grad} ( \lambda f ) = \ \lambda \mathop{\rm grad} f,\ \ \lambda = \textrm{ const } , $$

$$ \mathop{\rm grad} ( f + g) = \mathop{\rm grad} f + \mathop{\rm grad} g, $$

$$ \mathop{\rm grad} ( fg) = g \mathop{\rm grad} f + f \mathop{\rm grad} g, $$

$$ \mathop{\rm grad} \left ( { \frac{f}{g} } \right ) = \frac{1}{g ^ {2} } ( g \mathop{\rm grad} f - f \mathop{\rm grad} g). $$

The gradient $ f ^ { \prime } ( t _ {0} ) $ is the derivative at $ t _ {0} $ with respect to volume of the vector function given by

$$ \Phi ( E) = \ \int\limits _ {t \in \partial E } f ( t) \mathbf M ds, $$

where $ E $ is a domain with boundary $ \partial E $, $ t _ {0} \in E $, $ ds $ is the area element of $ \partial E $, and $ \mathbf M $ is the unit vector of the outward normal to $ \partial E $. In other words,

$$ f ^ { \prime } ( t _ {0} ) = \ \lim\limits \frac{\Phi ( E) }{ \mathop{\rm vol} E } \ \textrm{ as } \ \ E \rightarrow t _ {0} . $$

Formulas (1), (2) and the properties of the gradient listed above indicate that the concept of a gradient is invariant with respect to the choice of a coordinate system.

In a curvilinear coordinate system $ x = ( x ^ {1} \dots x ^ {n} ) $, in which the square of the linear element is

$$ ds ^ {2} = \ \sum _ {i, j = 1 } ^ { n } g _ {ij} ( x) dx ^ {i} dx ^ {j} , $$

the components of the gradient of $ f $ with respect to the unit vectors tangent to coordinate lines at $ x $ are

$$ \sum _ {j = 1 } ^ { n } g ^ {ij} ( x) \frac{\partial f }{\partial x ^ {j} } ,\ \ 1 \leq i \leq n, $$

where the matrix $ \| g ^ {ij} \| $ is the inverse of the matrix $ \| g _ {ij} \| $.

The concept of a gradient for more general vector functions of a vector argument is introduced by means of equation (2). Thus, the gradient is a linear operator the effect of which on the increment $ t - t _ {0} $ of the argument is to yield the principal linear part of the increment $ f( t) - f( t _ {0} ) $ of the vector function $ f $. E.g., if $ f = ( f ^ { 1 } \dots f ^ { m } ) $ is an $ m $- dimensional vector function of the argument $ t = ( t ^ {1} \dots t ^ {n} ) $, then its gradient at a point $ t _ {0} $ is the Jacobi matrix $ J = J ( t _ {0} ) $ with components $ ( \partial f ^ { i } / \partial t ^ {j} ) ( t _ {0} ) $, $ 1 \leq i \leq m $, $ 1 \leq j \leq n $, and

$$ f ( t) = f ( t _ {0} ) + J ( t - t _ {0} ) + o ( t - t _ {0} ), $$

where $ o ( t - t _ {0} ) $ is an $ m $- dimensional vector of length $ o ( | t - t _ {0} | ) $. The matrix $ J $ is defined by the limit transition

$$ \tag{3 } \lim\limits _ {\rho \rightarrow 0 } \ \frac{f ( t _ {0} + \rho \tau ) - f ( t _ {0} ) } \rho = J \tau , $$

for any fixed $ n $- dimensional vector $ \tau $.

In an infinite-dimensional Hilbert space definition (3) is equivalent to the definition of differentiability according to Fréchet, the gradient then being identical with the Fréchet derivative.

If the values of $ f $ lie in an infinite-dimensional vector space, various types of limit transitions in (3) are possible (see, for example, Gâteaux derivative).

In the theory of tensor fields on a domain of an $ n $- dimensional affine space with a connection, the gradient serves to describe the principal linear part of increment of the tensor components under parallel displacement corresponding to the connection. The gradient of a tensor field

$$ f ( t) = \ \{ { f _ {j _ {1} \dots j _ {q} } ^ { i _ {1} \dots i _ {p} } ( t) } : { 1 \leq i _ \alpha , j _ \beta \leq n } \} $$

of type $ ( p, q) $ is the tensor of type $ ( p, q + 1 ) $ with components

$$ \{ { \nabla _ {k} f _ {j _ {1} \dots j _ {q} } ^ { i _ {1} \dots i _ {p} } ( t) } : { 1 \leq k, i _ \alpha , j _ \beta \leq n } \} , $$

where $ \nabla _ {k} $ is the operator of absolute (covariant) differentiation (cf. Covariant differentiation).

The concept of a gradient is widely employed in many problems in mathematics, mechanics and physics. Many physical fields can be regarded as gradient fields (cf. Potential field).

References

[1]	N.E. Kochin, "Vector calculus and fundamentals of tensor calculus" , Moscow (1965) (In Russian)
[2]	P.K. [P.K. Rashevskii] Rashewski, "Riemannsche Geometrie und Tensoranalyse" , Deutsch. Verlag Wissenschaft. (1959) (Translated from Russian)

References

[a1]	W. Fleming, "Functions of several variables" , Addison-Wesley (1965) MR0174675 Zbl 0136.34301