We've now laid the groundwork for a meaningful definition of the gradient in the setting of a constraint manifold. At this point, one could run off and try to do a steepest descent search to maximize one's objective functions. Trouble will arise, however, when one discovers that there is no sensible way to combine a point and a displacement to produce a new point because, for finite , violates the constraint equations and thus does not give a point on the manifold.
For any manifold, one updates by solving a set of differential equations of motion of the form
To see how these equations could be satisfied,
we take the infinitesimal constraint equation for ,
In the next subsection, we will have reason to consider
for .
For technical reasons, one usually requests that
The connection for the Stiefel manifold using two different inner products
(the Euclidean and the canonical) can be found in the work by Edelman,
Arias, and Smith (see [151,155,416]). The function connection
computes
in the template software.
Usually the solution of the equations of motions on a manifold are
very difficult to carry out. For the Stiefel manifold,
analytic solutions exist and can be found in the aforementioned
literature, though we have found very little performance
degradation between moving along paths via the equations of motion and
simply performing some orthogonalizing factorization on ,
as long as the displacements are small. The move
function
supports multiple methods of geodesic motion, depending on the degree
of approximation desired.