§5.4 The spectral theorem · A rough guide to linear algebra

Given an abstract vector space $V$ and a linear map $T : V \to V$, many of the important properties of $T$ are determined by the $k[x]$-module structure on $V$. The module structure is then almost characterized by the set of eigenvalues of $T$ (with multiplicity).

In the context of analysis, the set of eigenvalues is called the spectrum of a linear operator. Let us look at one example. Consider the space

$$ \begin{aligned} \mathcal{H} &= \lbrace \text{complex-valued } 1\text{-periodic functions on } \mathbb{R} \rbrace \\ &= \lbrace (f : \mathbb{R} \to \mathbb{C}) : f(x) = f(x+1) \text{ for all } x \in \mathbb{R} \rbrace . \end{aligned} $$

Well, this is only a vector space over $\mathbb{C}$ and we need to specify the inner product. We can define the inner product as

$$ \langle f, g \rangle = \int_{0}^{1} \bar{f}(x) g(x) dx. $$

Consider the differential operator

$$ T = \tfrac{d^2}{dx^2} : \mathcal{H} \to \mathcal{H}; \quad f \mapsto \frac{d^2 f}{dx^2}. $$

Technically, not all functions are differentiable, and hence the operator is not really defined. But let us overlook this issue. We have

$$ \langle f^{\prime\prime}, g \rangle = \int_{0}^{1} \bar{f}^{\prime\prime} g dx = -\int_{0}^{1} \bar{f}^\prime g^\prime dx = \int_{0}^{1} \bar{f} g^{\prime\prime} dx = \langle f, g^{\prime\prime} \rangle. $$

by integration by parts, and this shows that $T = \frac{d^2}{dx^2}$ is self-adjoint. What is the spectrum of this operator? To answer this question, we need to solve the differential equation

$$ \frac{d^2 f}{dx^2} = \lambda f $$

where $\lambda$ is a constant. It turns out that a solution exists only when $\lambda = -4 \pi^2 n^2$ and $f(x) = c e^{2 \pi i n x}$, with $n \in \mathbb{Z}$. This set $\lbrace 0, -4\pi^2, -4\pi^2, -16\pi^2, -16\pi^2, \ldots \rbrace $ is then the spectrum of $T$.

Before stating the spectral theorem, we state and prove a lemma that will be used in its proof.

Let $V$ be a finite-dimensional Hilbert space, and let $T : V \to V$ be a self-adjoint linear operator. If $\dim V \ge 1$, then there exists a nonzero $v \in V$ such that

$$ Tv = \pm \lVert T \rVert v. $$

Proof.

By the definition of $\lVert T \rVert$ (and Heine–Borel), there exists a $v \neq 0$ such that $\lVert Tv \rVert = \lVert T \rVert \lVert v \rVert$. Note that $T^2 v = (T^2 v - \lVert T \rVert^2 v) + \lVert T \rVert^2 v$. Observe that the inner product of the two summands is

$$ \langle T^2 v - \lVert T \rVert^2 v, \lVert T \rVert^2 v \rangle = \lVert T \rVert^2 \langle T^2 v, v \rangle - \lVert T \rVert^4 \langle v, v \rangle = \lVert T \rVert^2 \lVert Tv \rVert^2 - \lVert T \rVert^4 \lVert v \rVert^2 = 0 $$

because $\langle T^2 v, v \rangle = \langle Tv, Tv \rangle$. This implies that the length of $T^2 v$ can be expressed as

$$ \lVert T^2 v \rVert^2 = \lVert T^2 v - \lVert T \rVert^2 v \rVert^2 + \lVert T \rVert^4 \lVert v \rVert^2 \ge \lVert T \rVert^4 \lVert v \rVert^2. $$

On the other hand,

$$ \lVert T^2 v \rVert^2 \le \lVert T \rVert^2 \lVert Tv \rVert^2 \le \lVert T \rVert^4 \lVert v \rVert^2. $$

Therefore all the inequalities are equalities. In particular, $T^2 v - \lVert T \rVert^2 v = 0$.

If $Tv = -\lVert T \rVert v$, we are done. Hence assume that

$$ w = Tv + \lVert T \rVert v \neq 0. $$

Then

$$ Tw = T^2 v + \lVert T \rVert Tv = \lVert T \rVert Tv + \lVert T \rVert^2 v = \lVert T \rVert w $$

finishes the proof.

Without the self-adjointness assumption, find a counterexample to the above lemma.

Let $V$ be a nonzero finite-dimensional Hilbert space and $T : V \to V$ be a self-adjoint operator. Show that

$$ \lVert T \rVert = \max_{\lVert v \rVert \le 1} \lvert \langle Tv, v \rangle \rvert. $$

Moreover, show that if $\lvert \langle Tv, v \rangle \rvert = \lVert T \rVert \lVert v \rVert^2$ then $Tv = \pm \lVert T \rVert v$.

Let us now state and prove the spectral theorem.

Let $V$ be a finite-dimensional Hilbert space, and let $T : V \to V$ be a self-adjoint operator. Then there exist and orthonormal basis $v_1, v_2, \ldots, v_n \in V$ of eigenvectors, so that

$$ \langle v_i, v_j \rangle = \begin{cases} 1 & i = j \\ 0 & i \neq j, \end{cases} \qquad T v_i = \lambda_i v_i $$

where $\lambda_i$ are (not necessarily distinct) real numbers.

Proof.

The idea is to find one eigenvector at a time. We induct on $\dim V$. If $\dim V = 0$, the statement is trivial. If $\dim V \ge 1$, Lemma 5.4.1 finds a nonzero vector $v \in V$ such that $Tv = \lambda v$ where $\lambda = \pm \lVert T \rVert$.

Take the orthogonal complement of $v$,

$$ W = v^\perp = \lbrace w \in V : \langle w, v \rangle = 0 \rbrace \subseteq V. $$

Then for an arbitrary $w \in W$, we have

$$ \langle Tw, v \rangle = \langle w, Tv \rangle = \langle w, \lambda v \rangle = \lambda \langle w, v \rangle = 0. $$

This shows that $T$ sends a vector in $W$ to $W$. Hence we may restrict $T$ to a self-adjoint operator $T : W \to W$, where $\dim W = \dim V - 1$. By the induction hypothesis, we can find an orthonormal basis of $W$ consisting of eigenvectors. Together with $v / \lVert v \rVert$, they form an orthonormal basis of $V$ consisting of eigenvectors.

Let $A$ be an $n \times n$ Hermitian (resp. symmetric) matrix with complex (resp. real) entries. Show that there exists a unitary (resp. orthogonal) matrix $U$ (resp. $Q$) and a diagonal matrix $\Lambda$ such that

$$ A = U \Lambda U^{-1} \text{ (resp.~} Q \Lambda Q^{-1} \text{).} $$

Consider the symmetric real matrix

$$ A = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 3 & 4 \\ 3 & 4 & 5 \end{bmatrix}. $$

Find an orthogonal matrix $Q$ and a diagonal matrix $\Lambda$ such $A = Q \Lambda Q^{-1}$.

There is another way to state the spectral theorem. Consider the eigenspace

$$ V_\lambda = \ker(T - \lambda I) = \lbrace v \in V : Tv = \lambda v \rbrace \subseteq V. $$

Then $\dim V_\lambda > 0$ if and only if $\lambda$ is an eigenvalue of $T$.

Show that

$$ V = \bigoplus_{\lambda \in \mathbb{R}} V_\lambda, $$

where the spaces $V_\lambda$ are orthogonal to each other. (Translation: for each $v \in V$ there uniquely exist vectors $v_\lambda \in V_\lambda$, zero except for finitely many $\lambda$, such that $v = \sum_{\lambda}^{} v_\lambda$.)

Let $V$ be a finite-dimensional Hilbert space, and let $X, Y : V \to V$ be self-adjoint operators. Suppose that $X$ and $Y$ commute, i.e., $XY = YX$.

(a)
Consider the eigenspaces $V_\lambda = \ker(X - \lambda I)$ with respect to $X$ so that $V = \bigoplus_\lambda V_\lambda$. Show that $Y$ sends everything in $V_\lambda$ into $V_\lambda$, i.e., $Y$ restricts to $Y : V_\lambda \to V_\lambda$ for each $\lambda \in \mathbb{R}$.
(b)
Consider the simultaneous eigenspaces

$$ V_{\lambda,\kappa} = \ker (X - \lambda I) \cap \ker (Y - \kappa I) = \lbrace v \in V : X v = \lambda v, Y v = \kappa v \rbrace . $$

Show that $V = \bigoplus_{\lambda, \kappa} V_{\lambda, \kappa}$ where the spaces $V_{\lambda, \kappa}$ are all orthogonal to each other.
(c)
Show that there exists an orthonormal basis $v_1, \ldots, v_n$ of $V$ such that each $v_k$ is an eigenvector of both $X$ and $Y$.
(d)
If $X, Y$ are $n \times n$ Hermitian matrices and $XY = YX$, show that there exists a unitary matrix $U$ such that both $U^{-1} X U$ and $U^{-1} Y U$ are diagonal.

The spectral theorem applies only to self-adjoint operators. If the base field is $k = \mathbb{R}$, diagonalization by an orthogonal operator exists if and only if the operator is symmetric.

Let $V$ be a finite-dimensional Hilbert space over $k = \mathbb{R}$. Show that for a linear operator $T : V \to V$, the following are equivalent:

(a)
The operator $T$ is symmetric.
(b)
There exists an orthonormal basis $v_1, \ldots, v_n$ consisting of eigenvectors.
(c)
There exists an isometry $Q : \mathbb{R}^n \to V$ and a diagonal matrix $\Lambda : \mathbb{R}^n \to \mathbb{R}^n$ such that $T = Q \Lambda Q^{-1}$.

However for $k = \mathbb{C}$, there is a larger class of matrices that can be diagonalized by a unitary matrix. The reason is that self-adjoint operators can only have real eigenvalues. If we take a unitary matrix $U$, a diagonal matrix $\Lambda$ with non-real entries, and consider the linear operator $T = U \Lambda U^{-1}$, it will have an orthonormal set of eigenvectors but not necessarily self-adjoint.

Let $V$ be a finite-dimensional Hilbert space over $k = \mathbb{C}$. We say that a linear operator $T : V \to V$ is normal if $T^\dagger T = T T^\dagger$.

Let $V$ be a finite-dimensional Hilbert space over $k = \mathbb{C}$, and let $T : V \to V$ be a linear operator. The following are equivalent:

(i)
$T$ is normal.
(ii)
The Hermitian part $X = \frac{1}{2}(T + T^\dagger)$ and skew-Hermitian part $Y = \frac{1}{2}(T - T^\dagger)$ commutes, i.e., $XY = YX$.
(iii)
$\lVert Tv \rVert = \lVert T^\dagger v \rVert$ for all $v \in V$.
(iv)
$T$ is diagonalizable by a unitary matrix, i.e., there exists an isometry $U : k^n \to V$ and a diagonal matrix $\Lambda$ such that $T = U \Lambda U^{-1}$.

Proof.

(i) $\Leftrightarrow$ (ii) We compute

$$ XY - YX = \frac{1}{4} [ (T + T^\dagger) (T - T^\dagger) - (T - T^\dagger) (T + T^\dagger) ] = \frac{1}{2} (T^\dagger T - T T^\dagger). $$

So $XY = YX$ if and only if $T^\dagger T = T T^\dagger$ if and only if $T$ is normal.

(i) $\Leftrightarrow$ (iii) Define $S = T^\dagger T - T T^\dagger$, which is easily seen to be a self-adjoint operator. Then

$$ \lVert Tv \rVert^2 - \lVert T^\dagger v \rVert^2 = \langle Tv, Tv \rangle - \langle T^\dagger v, T^\dagger v \rangle = \langle v, (T^\dagger T - T T^\dagger) v \rangle = \langle v, Sv \rangle. $$

If $T$ is normal so that $S = 0$, then this is always zero. Conversely, suppose that $\langle v, Sv \rangle = 0$ for all $v \in V$. By Exercise 5.4.B, this implies that $\lVert S \rVert = 0$, that is, $S = 0$.

(ii) $\Rightarrow$ (iv) We note that $X^\dagger = \frac{1}{2} (T^\dagger + T) = X$, i.e., $X$ is Hermitian. On the other hand, $Y^\dagger = -Y$ shows that $iY$ is Hermitian. By Exercise 5.4.F, there exists a unitary matrix $U$ such that $U^{-1} X U$ and $U^{-1} (iY) U$ are both diagonal matrices. Then

$$ T = X + Y = U [ U^{-1} X U + (-i) (U^{-1} (iY) U) ] U^{-1}, $$

where $U^{-1} X U + (-i) (U^{-1} (iY) U)$ is a sum of diagonal matrices, hence diagonal.

(iv) $\Rightarrow$ (i) This can be verified directly. If $T = U \Lambda U^{-1}$ then

$$ T T^\dagger = (U \Lambda U^{-1}) (U \Lambda^\dagger U^{-1}) = U (\Lambda \Lambda^\dagger) U^{-1} = U (\Lambda^\dagger \Lambda) U^{-1} = T^\dagger T. $$

Here, $\Lambda^\dagger \Lambda = \Lambda \Lambda^\dagger$ because $\Lambda$ is a diagonal matrix.

Let $V$ be a finite-dimensional Hilbert space over $k = \mathbb{C}$. Suppose $T_1, T_2, \ldots, T_k$ are normal operators $V \to V$ such that $T_i T_j = T_j T_i$ for all $1 \le i, j \le k$. Show that there exists an isometry $U : \mathbb{C}^n \to V$ such that $U^{-1} T_i U$ is a diagonal matrix for all $1 \le i \le k$.