Singular Values and Singular Vectors

The square roots of the $n$ eigenvalues of $A^* A$ are the singular values of $A$. Since $A^* A$ is Hermitian and positive semidefinite, the singular values are real and nonnegative. This lets us write them in sorted order $0 \leq \sigma_n \leq \cdots \leq \sigma_1$.

The $n$ eigenvectors of $A^* A$ are called (right) singular vectors. We denote them by $v_1,\ldots,v_n$, where $v_i$ is the eigenvector for eigenvalue $\sigma_i^2$. The $m$ by $m$ matrix $AA^*$ is also Hermitian positive semidefinite. Its largest $n$ eigenvalues are identical to those of $AA^*$, and the rest are zero. The $m$ eigenvectors of $AA^*$ are called (left) singular vectors. We denote them by $u_1,\ldots,u_m$, where $u_1$ through $u_n$ are eigenvectors for eigenvalues $\sigma_1^2$ through $\sigma_n^2$, and $u_{n+1}$ through $u_{m}$ are eigenvectors for the zero eigenvalue. The singular vectors can be chosen to satisfy the identities $Av_i = \sigma_i u_i$ and $A^* u_i = \sigma_i v_i$ for $i=1,\ldots,n$, and $A^*u_i = 0$ for $i=n+1,\ldots,m$.

We may assume without loss of generality that each $\Vert u_i\Vert _2=1$ and $\Vert v_i\Vert _2=1$. The singular vectors are real if $A$ is real. Though the singular vectors may not be unique (e.g., any vector is a singular vector of the identity matrix), they may all be chosen to be orthogonal to one another: $u_i^*u_j = v_i^*v_j = 0$ if $i \neq j$. When a singular value is distinct from all the other singular values, its singular vectors are unique (up to multiplication by scalars).