The Spectral Theorems

Let $V$ be a finite-dimensional inner product space over $\mathbb K$ , and $T : V \to V$ be a linear transformation (or if you’d prefer, linear operator on $V$ ). Recall that $T$ is diagonalisable if and only if there exists a linearly independent set $K$ of eigenvectors of $T$ such that $\mathrm{span}(K) = V$ .

Not all operators are diagonalisable (though all operators over $\mathbb C$ will have a Jordan normal form, details here). But diagonalisable operators are convenient since we can compute $T^n$ with relative ease. What’s fascinating, however, is that in the case $\mathbb K = \mathbb C$ , if $T$ is normal, then $T$ is diagonalisable, i.e. there exists a basis $K$ of $V$ such that each vector in $K$ is an eigenvector of $T$ . Something nicer, in fact, happens in this case. It turns out that $K$ must be orthonormal. These two conditions turn out to be equivalent, and proving this surprising fact is our goal for this post.

Lemma 1. Multiplication in the subset of $\mathcal M_{n \times n}(\mathbb K)$ of diagonal matrices is commutative.

Now suppose $\mathbb K = \mathbb R$ or $\mathbb C$ .

Theorem 1. Suppose $c_T$ is a product of linear factors in $\mathbb K$ . $T$ is normal if and only if there exists an orthonormal (ordered) set $K$ of eigenvectors of $T$ such that $\mathrm{span}(K) = V$ .

Proof. We first prove the direction $(\Leftarrow)$ . Suppose there exists an orthonormal set $K$ of eigenvectors of $T$ such that $\mathrm{span}(K) = V$ . Then $T$ is similar to the matrix

$[T]_K := \begin{bmatrix} \lambda_1 & 0 & \cdots & 0 \\ 0 & \ddots & \ddots & \vdots \\ \vdots & \ddots & \ddots & 0 \\ 0 & \cdots & 0 & \lambda_n \end{bmatrix}.$

By definition of the conjugate-transpose,

$[T]_K^* := \begin{bmatrix} \bar{\lambda}_1 & 0 & \cdots & 0 \\ 0 & \ddots & \ddots & \vdots \\ \vdots & \ddots & \ddots & 0 \\ 0 & \cdots & 0 & \bar{\lambda}_n \end{bmatrix}.$

Hence, since both matrices are diagonal, $[T]_K [T]_K^* = [T]_K^* [T]_K$ . We leave it as an exercise to verify that $[T^*]_K = [T]_K^*$ . Hence, $[T \circ T^*]_K = [T^* \circ T]_K$ implies that $T \circ T^* = T^* \circ T$ , i.e. $T$ is normal.

For the direction $(\Rightarrow)$ , we will prove using induction on $\dim(V)$ . The case $n = 1$ is left as an obvious exercise. Assume that $T$ is a normal operator and suppose $\dim(V) = k+1$ . By hypothesis, the characteristic polynomial $c_T$ has at least one complex root $\lambda$ , which means that $T$ has at least one unit eigenvector $\mathbf u$ such that $T(\mathbf v) = \lambda \mathbf u$ .

Define the subspace $W := \mathrm{span}\{\mathbf u\}$ . Since $V$ is finite-dimensional, we can make the direct sum decomposition $V = W \oplus W^\perp$ . By considering dimensions,

$k+1 = \dim(V) = \dim(W) + \dim(W^{\perp}) = 1 + \dim(W^{\perp}),$

so that $\dim(W^{\perp}) = k$ . We claim that $S := T|_{W^{\perp}} : W^{\perp} \to W$ is a normal operator. Firstly, we need to check that $S (W^{\perp}) \subseteq W^{\perp}$ , i.e. that $S$ is $W^{\perp}$ -invariant, so that $S$ is indeed a linear operator. Secondly, we need to prove that $S \circ S^* = S^* \circ S$ , so that $S$ is indeed normal.

For the first claim, fix $\mathbf v \in W^{\perp}$ , so that $\langle \mathbf u, \mathbf v \rangle = 0$ . We need to check that $S(\mathbf v) \in W^{\perp}$ . Since $T$ is normal,

$\langle \mathbf u, S(\mathbf v) \rangle = \langle \mathbf u, T(\mathbf v) \rangle = \langle T^*(\mathbf u), \mathbf v\rangle = \langle \bar{\lambda}\mathbf u, \mathbf v\rangle = \bar{\lambda}\langle \mathbf u, \mathbf v\rangle = 0,$

so that we do have $S(\mathbf v) \in W^{\perp}$ , as required.

For the second claim, we first observe that for any $\mathbf v, \mathbf w \in W^{\perp}$ ,

$\langle \mathbf v, S^*(\mathbf w) \rangle = \langle S(\mathbf v), \mathbf w) = \langle T(\mathbf v), \mathbf w)\rangle = \langle \mathbf v, T^*(\mathbf w) \rangle.$

Hence, $S^*|_{W^{\perp}} = T^*|_{W^{\perp}}$ . By a similar argument, as before, for any $\mathbf v \in W^{\perp}$ ,

$\langle \mathbf u, S^*(\mathbf v) \rangle = \langle \mathbf u, T^*(\mathbf v) \rangle = \lambda \langle \mathbf u, \mathbf v \rangle = 0,$

so that $S^*$ is $W^{\perp}$ -invariant. Hence,

$S \circ S^* = T|_{W^{\perp}} \circ T^* |_{W^{\perp}} = (T \circ T^*)|_{W^{\perp}}.$

Therefore, $S$ is normal since $T$ is normal. Similarly, $c_S$ is a product of linear factors in $\mathbb K$ . By the induction hypothesis, there exists an orthonormal basis $K$ of eigenvectors of $S$ for $W^{\perp}$ . Then $\{\mathbf u\} \cup K$ forms an orthonormal basis for $W \oplus W^{\perp} = V$ , as required.

Corollary 1. $T$ is normal if and only if there exists an orthonormal basis $K$ of $V$ consisting of eigenvectors of $T$ such that $[T]_K$ is diagonal.

Corollary 2. The square matrix $\mathbf A : \mathbb C^n \to \mathbb C^n$ is normal if and only if there exists a unitary matrix $\mathbf P$ such that $\mathbf P^* \mathbf A \mathbf P$ is diagonal.

Theorem 2. Suppose $\mathbb K = \mathbb R$ . $T$ is self-adjoint if and only if there exists an orthonormal (ordered) set $K$ of eigenvectors of $T$ such that $\mathrm{span}(K) = V$ .

Proof. For the direction $(\Leftarrow)$ , the proof in Theorem 1 yields that

$[T^*]_K = [T]_K^* = [T]_K^{\mathrm T} = [T]_K,$

so that $T^* = T$ , as required.

For the direction $(\Rightarrow)$ , suppose $T$ is self-adjoint. Since $T$ is normal, by Theorem 1, it suffices to check that $c_T$ is a product of linear factors in $\lambda \in \mathbb R$ . Let $\lambda \in \mathbb C$ be any root of $c_T$ . Then there exists $\mathbf v \in V$ such that

$\bar{\lambda}\mathbf v =T^*(\mathbf v) = T(\mathbf v) = \lambda \mathbf v.$

Since $\mathbf v \neq \mathbf 0$ , we must have $\lambda = \bar{\lambda}$ , yielding $\lambda \in \mathbb R$ . Hence, $c_T$ can be written as a product of linear factors, as required.

Corollary 3. The square matrix $\mathbf A : \mathbb R^n \to \mathbb R^n$ is symmetric if and only if there exists an orthogonal matrix $\mathbf P$ such that $\mathbf P^{\mathrm T} \mathbf A \mathbf P$ is diagonal.

These climactic results are known as the spectral theorems, and we will revisit them in the following crucial result: the singular value decomposition. These results cap off a theoretical study in undergraduate linear algebra.

In subsequent posts, we will explore various applications of what we have learned in other areas of mathematics.

—Joel Kindiak, 18 Mar 25, 1812H

KindiakMath

The Spectral Theorems

Leave a comment Cancel reply

The Spectral Theorems

Share this:

Leave a comment Cancel reply