Introduction

This section increases the scope of the algebraic eigenvalue problem by focusing on the geometrical properties of the eigenspaces. We discuss a template that solves variational problems defined on sets of subspaces. Areas where such problems arise include electronic structures computation, eigenvalue regularization, control theory, signal processing, and graphics camera calibration. The underlying geometry also provides the numerical analyst theoretical insight into the common mathematical structure underlying eigenvalue algorithms (see [151,155,416]).

Suppose that one wishes to optimize a real-valued function $F(Y)$ over $Y$ such that $Y^*Y=I$, where $Y^*$ is either transpose or Hermitian transpose as appropriate. Our template is designed as a means to optimize such an $F$.

One simple case is optimization over square orthogonal (or unitary) matrices such as the least squares simultaneous diagonalization of symmetric matrices (also known as INDSCAL [107]) problem described later. Another simple case is optimization over the unit sphere, as in symmetric Rayleigh quotient minimization. In between, we have rectangular $n\times p$ ($n \ge p$) matrices $Y$ with orthonormal columns such as the orthogonal Procrustes problem.

Furthermore, some functions $F$ may have the symmetry property $F(Y)=F(YQ)$ for all orthogonal (unitary) $Q$ with a specified block-diagonal structure, which causes some search directions to have no effect on the value of $F$. For example, if $A$ is a symmetric matrix, $n \ge p$, and $F(Y)=\tr(Y^*AY)$, then $F(Y)=F(YQ)$ for all $p \times p$ orthogonal $Q$. More generally, some functions $F$ have the property that $F(Y)=F(YQ)$, where $Q$ is orthogonal (unitary) with the block-diagonal form

$$Q = \left[ \begin{array}{cccc} Q_1 & 0 & \cdots & 0 \cr 0 & ... ...dotfill & \dotfill \cr 0 & 0 & \cdots & Q_p \end{array}\right],$$

where the $Q_i$ are orthogonal (unitary) matrices. More complicated problems with block-diagonal orthogonal (unitary) symmetry can arise in eigenvalue regularization problems (an example of such a problem is the sample nearest-Jordan block problem solved in the examples).

This chapter is organized as follows. §9.4.2 discusses the basics of calling the template code. §9.4.3 discusses the objective functions and their derivatives for the example problems explored in this chapter. §9.4.4 contains sample runs of instances of the example problems. §9.4.5 explains the code structure and the places where a user may wish to make modifications. §9.4.6 will cover some of the basic mathematics concerning the geometry of the Stiefel manifold.