In flat spaces, we often identify the differential of a function with its gradient. However, when dealing with a more general setting, one can run into problems making sense out of such a definition.
For example, the gradient is a vector, and it should be possible to
think of vectors as infinitesimal displacements of points. In
, any infinitesimal displacement
must satisfy
. Thus,
may not always be a
vector, since it does not necessarily satisfy this equation. A gradient
should be an infinitesimal displacement that
points in the direction of the displacement which will give
the greatest increase in
.
If the tangent space has an inner product, though, one can find a
useful way to identify the uniquely with a tangent vector. Let
be a symmetric nondegenerate bilinear form on the
tangent space of
at
. Then one can define the
gradient,
, implicitly by
tangent
carries out this projection from
differentials to tangents (shown in Figure 9.7).
This operation is performed
by grad
to produce the gradient of the objective function.
![]() |